Brief Screening for Executive Dysfunction in Schizophrenia in a Rehabilitation Hospital

Cognitive dysfunction has been viewed as a core feature of schizophrenia since it was first conceptualized. Kraepelin used the term “dementia praecox” to describe the functional and intellectual deterioration as key disease features. Recent evidence suggests that 50%–70% of patients with schizophrenia have neurocognitive deficits^1,2 on measures of attention, learning and memory, problem-solving, language, and sensorimotor skill.^3–5 These deficits predate the onset of clinical symptoms, represent a decline from previous level of functioning,^6–9 persist over the disease course,^10–12 and are more closely linked to function than are clinical symptoms of schizophrenia.^13–17 In terms of evaluating strategies that may ultimately improve functional outcomes for patients, and directing appropriate allocation of health funding, developing well-validated and time-efficient tools to identify neurocognitive deficits at the bedside is paramount.

Studies designed to characterize neurocognitive dysfunction in schizophrenia vary in the tools they utilize, ranging from the comprehensive battery employed by Hallmayer and colleagues,¹⁸ for instance (including: National Adult Reading Test, Shipley Institute of Living Scale, Continuous Performance Task, FAS version of the Controlled Oral Word-Association Task, Rey Auditory Verbal Learning Test, Inspection Time Task) to very brief, albeit nonspecific, screening measures that are able to accommodate participants’ level of impairment in given clinical settings (e.g., the Mini-Mental State Exam), or to global rating measures from dementia rating scales (e.g., the Clinical Dementia Rating Scale).^19–23 More recently, instruments have been developed specifically for schizophrenia research, such as The Brief Assessment of Cognition in Schizophrenia,²⁴ which have the benefit of relatively brief administration (30 minutes), can be performed by a range of health-practitioners, and assesses those cognitive domains consistently impaired and related to outcome in schizophrenia. The majority of these tools, however, rely on patient performance in the isolation of the structured testing environment, and few, if any, tools are able to measure integrated executive functions of the patient in the context of the problem-solving demands of their everyday world (either subjectively or objectively).

The BRIEF–A (Behavior Rating Inventory of Executive Function–Adult version) is a 75-item questionnaire designed to assess executive functioning in daily life over the previous month;²⁵ it has both Self-Report (SR) and Informant Report (IR) versions, each taking approximately 15 minutes to complete. The BRIEF–A yields nine clinical scales: Inhibit, Shift, Emotional Control, Self-Monitor, Initiate, Working Memory, Plan/Organize, Task Monitor, and Organization of Materials. Two indexes are obtained by summing raw scale scores, the Metacognition Index (MI), and the Behavioral Regulation Index (BRI), which are summed to obtain a raw score for the Global Executive Composite (GEC). T-scores are based on comparison with a normative sample and, by definition, have a mean of 50 and standard deviation (SD) of 10; higher scores reflect a greater degree of executive dysfunction. A T-score of 65 represents 1.5 SDs above the mean, and is the recommended threshold for interpreting a score as abnormally elevated. BRIEF–A psychometric properties, including validity and reliability, have been reported previously; it has been evaluated in traumatic brain injury, attention-deficit/hyperactivity disorder, epilepsy, depression, mild cognitive impairment, and Alzheimer’s dementia.²⁵

Few published studies have evaluated the BRIEF or BRIEF–A in schizophrenia. In a study of psychosis prodrome in children age 12–18, Niendam and colleagues²⁶ utilized the BRIEF Parent Form and demonstrated that those at ultra-high-risk for psychosis had elevated BRIEF scores (particularly Working Memory). Using the BRIEF–A in 29 adults with schizophrenia, Kumbhani and colleagues²⁷ demonstrated that patients reported greater problems on Shift and Working Memory scales, and informant data showed significantly greater problems on all BRIEF–A scales (with the exception of Emotional Control), as compared with controls. In the same sample of patients, Garlinghouse and colleagues²⁸ demonstrated that poorer subjective working memory was associated with smaller left and right frontal lobe volumes.

The present study was designed to examine the utility of the BRIEF–A Informant Report in a long-stay inpatient population with schizophrenia. First, we aimed to characterize a BRIEF–A profile; second, we aimed to examine the relationship between BRIEF–A scores and performance on tools used to rate poor functioning or difficult behaviors; and third, we aimed to identify and interpret the factor structure of the BRIEF–A in this population.

Method

Setting and Participants

We recruited 112 inpatients without exclusion, with chronic schizophrenia, in a long-stay rehabilitation facility of a major psychiatric hospital in Perth, Western Australia. The diagnosis of schizophrenia was established by a consensus of psychiatrists according to ICD-10 criteria. The study was approved and undertaken within the clinical audit provisions of the NMAHS–MH (North Metropolitan Area Health Service–Mental Health).

Measures

A mental-health practitioner with frequent (i.e., more than twice per week) face-to-face contact with the patient completed the BRIEF–A IR Form. Instruments to assess patient functioning were also completed: the Global Assessment of Functioning (GAF), which rates symptoms and social functioning on a scale ranging from 10 to 100 (lower scores reflecting poorer function);²⁹ and the Social Behavior Schedule (SBS), a standard instrument describing the behavioral difficulties of patients likely to be dependent on psychiatric services for some time,³⁰ (higher scores reflecting greater difficulties); the SBSC is a composite score obtained by summing the raw scores of the first 25 items of the SBS.

Statistical Analysis

Means and standard deviations (SDs) of patient demographics and T-score means on each of the BRIEF–A scales were determined; T-scores for BRI, MI, and GEC were examined in relation to GAF and SBS scores.

We conducted an exploratory factor analysis (EFA), using a principal-component extraction method and varimax rotation of the 75 items. Before the analysis was run, the data were screened, and found to be interval-like; variable pairs appeared to be bivariate, normally distributed, and all cases were independent of one another. The Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett’s test of sphericity were then measured to ensure that the data were suitable for analysis. The EFA generated an initial and extraction principal-components solution; extraction communalities were inspected for any variables that appeared to be particularly low (<0.49), and these items were removed from further analysis. The total variance explained (derived from the initial, extraction, and rotation phases of the analysis) was inspected; using the Kaiser-Guttman retention criteria of eigenvalues greater than 1.0, together with inspecting the scree plot for the initial solution, the number of factors were chosen that best described the principal-components solution. These factors were inspected, and items that had significant loadings across a number of factors were removed from the analysis. The final analysis was performed on the remaining items; the rotated components matrix was inspected to determine the essence of each component, and the factor weights in excess of 0.65 were used to guide the interpretation and labeling of each factor.³¹

Confirmatory factor analysis (CFA) was then performed to determine how well the hypothesized theoretical structure obtained in the above analysis fitted the empirical data. Unlike EFA, in CFA, a model is constructed a priori in an attempt to account for (usually) the covariance matrix. CFA is often thought to be a stronger approach to the analysis of underlying factor structure. Before analysis in LISREL, the data were examined in PRELIS, where polychoric coefficients and an asymptotic covariance matrix were generated. Because of the relatively small sample size, maximum likelihood (ML) estimation was used. The assessment of the fit of various models was conducted using a scaled chi-square statistic,³² as this statistic corrects an inflated chi-square value due to nonnormality and is recommended for use in small samples.³³ Besides the chi-square statistic, which is sensitive to sample size, the fit was assessed using several fit indices: 1) GFI (goodness-of-fit index); 2) AGFI (adjusted goodness-of-fit index); 3) NFI (normed-fit index); 4) AIC (Akaike information criterion); and 5) the root mean-square error of approximation (RMSEA).

Analyses were performed using SPSS (Version 17) and LISREL (Version 8.52).

Results

Descriptive Statistics

The minimum age of participants was 18.5 years, with the maximum 79.2 years, with a mean of 44.50 (SD: 13.03); 27.7% of patients were women; the mean length of stay in the hospital was 1,204 days.

Table 1 provides the T-score means and SDs of the BRIEF–A scales, together with BRI, MI, and GEC T-scores. T-score means of several scales were abnormally elevated, together with T-scores for BRI, MI, and GEC; the distribution of T-scores for the GEC is shown in Figure 1.

Relationship With GAF and SBS

As evident in Figure 1, elevated T-scores on the BRIEF–A GEC were related to poorer functioning as measured by the GAF (r = −0.2535), and with increased behavioral difficulties as measured by the SBS (r=0.5931); similar trends were observed for both BRI and MI T-scores.

Exploratory Factor Analysis

The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.85; and Bartlett’s test of sphericity was significant (p <0.001), indicating that the data were suitable for analysis. Fourteen factors were initially identified, with a three-factor solution providing the clearest extraction, accounting for 53.0% of the total variance in the initial analysis (which is considered a robust solution, as it accounts for at least 50% of the variance³⁴). After exclusion of items with low communalities or significant cross-loading, a total of 18 items remained for further analysis. Principal-components analysis was performed, specifying a three-factor solution, with varimax rotation; these three factors accounted for 69.9% of the variance (Table 2). Factor 1 accounted for 41.9% of the variance and had 8 items (Table 2), items related to modulation of emotional responses; hence the factor was labeled, “emotional regulation.” Factor 2 accounted for 19.5% of the variance and had 7 items (Table 2), items related to holding information in mind for the purpose of completing a task, attention, goal-setting, and carrying out tasks in a systematic manner; hence, the factor was labeled “problem-solving.” Factor 3 accounted for 8.5% of the variance and had 3 items (Table 2), items related to keeping living areas in an orderly manner; hence, the factor was labeled “orderliness.”

Confirmatory Factor Analysis

Figure 2 presents the parameter estimates and goodness-of-fit indices for two structural models. Overall, the comparison of two competing models confirmed that the three-factor model was superior to the two-factor model (χ²[2]=124.3; p <0.001).

Discussion

Findings

The results of the study suggest that inpatients with schizophrenia in a long-stay rehabilitation hospital had abnormally elevated T-scores in the BRIEF–A. Increased BRIEF–A GEC T-scores were associated with poorer functioning and increased behavioral difficulties. The three-factor solution identified appeared to be robust and accounted for 69.9% of the variance; reducing the data from 75 to 18 BRIEF–A items, and relating to Emotional Regulation, Problem-Solving, and Orderliness.

Limitations of the Study Design

One of the major limitations is the small sample size, and relatively poor variables-to-cases ratio in the factor analysis; furthermore, no other performance-based measure of executive functioning (e.g., the Wisconsin Card-Sorting test) was undertaken. Replication of this study with a larger sample size, matched-control group, and including other measures of cognitive functioning is warranted; the addition of scales measuring negative and positive symptoms would also enable one to correlate these domains with factor scores.

This study did not utilize the SR Form; using both report forms would have allowed for comparison of performance between subjective and objective measures. Nonetheless, the SR and IR Forms have been demonstrated to correlate well with each other,^25,27 and utilizing an informant overcomes barriers that are likely to arise in the rehabilitation setting (see below). Furthermore, the BRIEF–A includes a Negativity scale: a high score on this scale (≥6) raises the possibility of an overly negative view of the rated individual (a potential source of bias in a long-stay inpatient setting). The mean Negativity score in our sample was 3.04 (SD: 2.87), suggesting that this particular bias in our study was limited.

There were no exclusion criteria, and it is highly likely that within the sample there were comorbid diagnoses such as mood, anxiety, substance use, and organic brain disorders; it is important to bear this in mind in the context of BRIEF–A performance. Despite this, we feel our sample is clinically representative of a “real life” population of patients with schizophrenia in a long-stay rehabilitation hospital and has implications for the relevance of the findings.

Interpretation of Findings

Our patients were reported to have difficulty in nearly all manifestations of their executive functions in daily life, consistent with the literature suggesting executive dysfunction in patients with schizophrenia,³⁵ and consistent with previous studies utilizing the BRIEF or BRIEF–A in schizophrenia.^26–28 This study adds weight to the potential utility of the BRIEF–A as a screening tool for executive dysfunction in everyday living for patients suffering from schizophrenia.

The BRIEF–A has several noteworthy merits in the long-stay inpatient setting: it can be administered by an informant in the day-to-day clinical setting of the ward environment, circumventing patient factors that can be a barrier to testing (i.e., the patient does not understand or cannot complete the task, or does not cooperate); little training is required for the informant, and the tool is brief to administer and score, making it ideal for time-constrained health-professionals; and, finally, it is able to measure integrated executive functions of the patient in the context of the problem-solving demands of their everyday world, and hence is a valuable tool in the rehabilitation setting.

Furthermore, we found that for inpatients in the long-stay rehabilitation setting, difficulty in daily life related to poor executive functioning; given that the ability to live and work independently is a key quality-of-life issue in chronic schizophrenia, the BRIEF–A may be helpful in this setting in formulating management plans that are focused on improving performance in patients’ real-world functioning.

The particular factor structure of the BRIEF–A in our patient sample warrants explanation. Factor 1 appeared to tap the patients’ ability to regulate their emotional responses, with poor control reflecting emotional lability or explosiveness. This could reflect a number of determinants in our sample, including treatment-setting (e.g., negativity bias by informants, as discounted above; patients’ responses to limitations imposed by the structure of the long-stay inpatient setting) or disease process (i.e., reflecting hypofrontality of schizophrenia; responses to the distressing positive psychotic symptoms), or a combination of these. As we had no measure of positive symptoms or other measure of executive functions, we were unable to further clarify these hypotheses.

Factor 2 appeared to tap poor performance on tasks essential to establishing and maintaining a cognitive set for problem-solving in daily-living activities. This factor could be construed as reflective of behavioral manifestations of executive functions (poor working memory, attention, sequencing, and set-shifting), may reflect an impact of the negative symptoms of schizophrenia, or a combination of these determinants; we had no measures of negative symptoms and so were unable to further investigate this relationship.

It has been proposed that there are two fundamental types of executive functions in the frontal lobes that, although subserved by different prefrontal areas, are closely related:^36–39emotional/motivational executive functions (relating to orbital and medial parts of the prefrontal cortex) which are responsible for coordinating cognition and emotion, and exemplified by the orbitofrontal and medial frontal syndrome; and metacognitive executive functions (relating to the dorsolateral prefrontal cortex), which includes problem-solving, abstracting, planning, and working memory, and exemplified by the dorsolateral syndrome. It may well be that the BRIEF–A has tapped these distinct executive functions in Factor 1 and Factor 2 in this sample.

The nature of the third factor, “orderliness” was intriguing, and there are a number of possible interpretations. The Organization of Materials scale of the BRIEF–A is relevant to the manner in which persons order the contents of their environment, and the third factor contained several items from this scale, reflecting, “messiness” of a patient’s living environment. At first glance, this factor might appear to be at odds with the structured environment of long-stay psychiatric rehabilitation wards. However, in the context of recent findings suggesting that executive dysfunction in schizophrenia is associated with obsessive-compulsive symptoms (with the direction of causality unclear), Factor 3 takes on a different meaning. Kumbhani and colleagues²⁷ found that more severe hoarding was related to poorer organization of materials in schizophrenia. The authors hypothesized that patients acquire and fail to discard possessions, causing clutter within the daily living environment and, hence, impaired functioning. Furthermore, hoarding has previously been found to be a common behavior exhibited by cognitively-impaired individuals in structured settings: in 133 consecutive dementia patients admitted to a psychogeriatric unit, Hwang and colleagues⁴⁰ reported that over 20% of patients exhibited hoarding behavior. Given that a unique factor encompassing orderliness in one’s daily living emerges from the BRIEF–A in our sample, the hypothesis by Kumbhani and colleagues seems plausible, and would have implications for treatment-planning in the rehabilitation setting. It may well be that different cognitive strategies need to be employed to address deficits of executive dysfunction as well as obsessive-compulsive symptoms (e.g., acquisition versus disposal strategies); in this setting, the BRIEF–A may be a useful tool in monitoring the response to such therapies.

Conclusion

The findings of the present study suggest that the BRIEF–A Informant Report is psychometrically robust in this population, and, in the absence of patient participation, it provides an estimate of executive functions that has clinical utility in a population that may otherwise be “beyond the reach” of formal neurocognitive testing. Furthermore, factor analysis has determined the core items of the BRIEF–A in this population, and, hence, there is potential to utilize these items to develop a more compact screening tool for executive dysfunction in patients with schizophrenia in the rehabilitation setting.