Cognitive Impairment and Predicting Response to Treatment in an Intensive Clinical Program for Post-9/11 Veterans With Posttraumatic Stress Disorder

A significant proportion of veterans who served in Operation Iraqi Freedom, Operation Enduring Freedom, and Operation New Dawn (OIF/OEF/OND) suffer mental health consequences associated with their deployment. A recent meta-analysis of 33 studies of these veterans estimated the rate of posttraumatic stress disorder (PTSD) at 23% (1). Veterans also suffer from depression (2), substance use disorders (3), chronic pain (4), psychosocial problems such as marital and family distress (5), financial difficulties (6), and unemployment (7). Most veterans benefit from routine outpatient psychiatric (8) and psychological care (9). Others require more innovative and intensive treatment (10–12).

Patient responses to psychological and psychiatric treatment are related to a variety of factors. Some evidence suggests that worse pretreatment cognitive functioning may predict or modify outcomes. In adults with depression, worse baseline executive functioning (e.g., switching ability) has been associated with worse response to psychotherapy (13), whereas lower word fluency has been associated with smaller responses to serotonin and serotonin/norepinephrine reuptake inhibitors (14). Lower pretreatment learning and memory scores have been associated with a smaller response to cognitive-behavioral therapy (CBT) (15–17). Lower intelligence (within the normal range) and lower education in women with PTSD following sexual assault have been associated with higher dropout from CBT but not with lower efficacy among completers (18). A recent study in patients with PTSD and comorbid severe mental illness found that worse cognitive performance was associated with less learning of information about PTSD but not with poorer participation in CBT, completion of homework, or clinical benefits (19). Lower baseline verbal learning performance has been associated with poorer response to treatment for PTSD nightmares (20). Given that a significant portion of veterans have a history of a traumatic brain injury (TBI), studies have examined whether patients with both PTSD and a history of TBI benefit from psychological treatment for PTSD (21–23). These studies did not find the effectiveness of CBT to be related to TBI diagnosis. However, the first two studies (21, 22) did not include direct measures of cognition, and the third study (23) did not report whether cognitive impairment was associated with treatment outcome. Thus, the extant, relatively sparse literature involves different study designs, different measures of cognitive performance, and different populations. This heterogeneity makes comparison between studies difficult and generalization about the association between cognitive impairment and the effectiveness of mental health treatment for PTSD nearly impossible.

The purpose of this study was to examine whether objectively measured pretreatment cognitive functioning predicts response to treatment in a new 2-week intensive clinical program for OEF/OIF/OND veterans and active duty service members with PTSD. Based on prior studies, we hypothesized that baseline cognitive impairment in general, and verbal memory impairment specifically, would be associated with less improvement in traumatic stress in response to treatment.

Methods

Participants

Home Base, a Red Sox Foundation and Massachusetts General Hospital program, offers a clinic devoted to the needs of veterans with serious and persistent mental health problems, primarily anxiety and depressive disorders. Participants in the present study were OIF/OEF/OND veterans and active duty service members who received trauma-focused CBT at Home Base as part of the 2-week intensive clinical program between June 2016 and August 2017. Medical records were reviewed by a team of health care providers (psychologists, physicians, and social workers) to ensure all patients had PTSD symptoms, typically in the moderate to severe range. Veterans were not eligible for this treatment program if they had active psychotic or manic symptoms, a suicide attempt or gesture within 90 days prior to admission, current substance use disorder severe enough to warrant detoxification, or serious medical conditions requiring a higher level of care and monitoring. Patients committed to being sober from drugs and alcohol in the days before treatment and the entirety of their treatment program. They completed a urine drug screen and alcohol breathalyzer on the first day of the treatment program as well as every morning of treatment, to ensure sobriety. A total of 145 patients attended the program. We excluded eight patients who did not meet DSM-5 criteria for PTSD and another 24 who lacked the primary outcome measure (i.e., posttreatment scores on the Posttraumatic Stress Disorder Checklist for DSM-5 [PCL-5]). The final study sample included 113 participants, who were between 25 and 60 years old (mean, 39.7 years [SD=8.5]), with between 10 and 20 years of education (mean, 14.5 years [SD=2.1]). No patient had less than a general equivalency diploma or a high school degree. Most participants were men (N=91 [80.5%]), Caucasian (N=87 [77.0%]) by race, non-Hispanic/Latino (N=102 [90.3%]) by ethnicity, and partnered (N=73 [64.6%]). Additional participant demographics are presented in Table 1. The demographic characteristics of those retained for analysis did not significantly differ from those excluded. Participants completed self-report measures prior to engaging in the 2-week intensive clinical program and again following treatment completion.

Measures

PCL-5.

The PCL-5 is a 20-item self-report questionnaire that is designed to be consistent with the DSM-5 definition for PTSD (24). Before treatment, participants indicated how bothered they were by each symptom over the past month on a scale from 0 (not at all) to 4 (extremely). After treatment, participants were asked to rate their symptoms over the past week. The questionnaire has demonstrated high internal consistency (Cronbach’s alpha=0.96), test-retest reliability (r=0.84), and convergent (construct) validity with the previously validated PCL-C (prior version of the scale; r=0.87) (25). PCL-5 scores of 31–33 have the highest efficiency for predicting a Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) diagnosis of PTSD, with sensitivity of 0.88, specificity of 0.69, positive predictive value of 0.81, and negative predictive value of 0.78 (25). In the present sample, Cronbach’s alpha was 0.88 before treatment and 0.94 after treatment.

Patient Health Quesionnaire-9 (PHQ-9).

The PHQ-9 is a nine-item self-report rating scale of depression symptoms based on the DSM-IV-TR (26). Participants respond to questions on a scale of 0 (not at all) to 3 (nearly every day) when thinking about their symptoms over the past 2 weeks. The total score ranges from 0 to 27, with higher scores indicating higher symptom reporting. Cronbach’s alpha has been reported as 0.89 (26). In the present sample, Cronbach’s alpha was 0.84 before treatment and 0.85 after treatment.

Self-Efficacy for Symptom Management Scale (SE-SMS).

The SE-SMS is a 13-item self-report scale developed to assess patients’ confidence in their ability to manage difficulties commonly associated with TBI exposure (27). Four items were extracted from this scale that specifically address the respondent’s confidence in their ability to 1) “keep any problems with concentration caused by your injury from interfering with the things that you want to do; 2) keep any problems with memory caused by your injury from interfering with the things that you want to do; 3) keep any problems with thinking caused by your injury from interfering with things that you want to do; and 4) compensate for any cognitive difficulties caused by your injury so that they don’t interfere with the things you want to do.” Participants were asked to consider the “present time” and respond to questions on a scale of 1 (not at all confident) to 10 (totally confident). The total score ranges from 4 to 40, with higher scores indicating greater confidence. Because the four items used were a subset of the measure items, published psychometrics are not available. In the present sample, Cronbach’s alpha was 0.91 before treatment and 0.96 after treatment.

Patient-Reported Outcomes Measurement Information System–Satisfaction With Participation in Social Roles and Activities–Short Form 8a (version 1.0) (PROMIS-S).

PROMIS-S is an eight-item self-report scale designed to assess satisfaction with one’s ability to perform daily tasks and routines (28). Participants are asked to consider the last seven days and respond to questions on a scale of 1 (not at all) to 5 (very much). The total possible score ranges from 8 to 40, with higher scores indicating greater satisfaction. Reliability has been shown to be greater than 0.90 for T-scores=30–70 (28). In the present sample, Cronbach’s alpha was 0.92 before treatment and 0.97 after treatment.

CNS Vital Signs.

CNS Vital Signs is a computer-based neuropsychological battery composed of seven measures testing verbal memory, visual memory, finger tapping speed, symbol digit coding, shifting attention, sustained attention, and response inhibition/interference. From these are calculated an overall neurocognition index and eleven domain scores: memory, verbal memory, visual memory, processing speed, motor speed, psychomotor speed, executive functioning, reaction time, simple attention, complex attention, and cognitive flexibility (29). Seven of these scores are based on single cognitive tests (i.e., executive functioning, reaction time, processing speed, verbal memory, visual memory, simple attention, and motor speed), five of which were considered the core cognitive domains for this study (i.e., executive functioning, reaction time, processing speed, verbal memory, and visual memory). The other four domain scores are composite scores derived from more than one test (i.e., memory, psychomotor speed, complex attention, and cognitive flexibility) (29). Normative reference values across the lifespan are age adjusted. The psychometric properties of CNS-VS are acceptable based on the normative data. Its test-retest reliability for the cognitive domain scores were adequate or better (Spearman’s rho=0.65–0.88), and correlations between CNS Vital Signs subtests and traditional neuropsychological measures showed convergent validity across most comparisons (see Table 4 in the normative study conducted by Gualtieri and Johnson) (30).

Assessment Procedures and Treatment Program

Participants completed self-report measures (PCL-5, PHQ-9, SE-SMS, and PROMIS-S) and the CNS Vital Signs testing battery within 2 days before treatment onset. On the final day of treatment (11 days after treatment began), participants repeated the same patient-reported outcome measures.

Over the course of their 2-week involvement, participants engaged in an array of multidisciplinary treatments targeting PTSD. As the core of their treatment, individual psychotherapy consisted of 55 minutes of prolonged exposure (PE) or cognitive processing therapy (CPT). Psychologists and patients worked collaboratively to choose PE or CPT. In addition, participants engaged in several daily psychotherapy groups that totaled about 45 hours over 2 weeks. Participants attended eight 55-minute sessions of skills groups based on dialectical behavioral therapy to improve their interpersonal skills and emotion regulation. They also attended six 75-minute sessions of Resilient Warrior, a program adapted from the Relaxation Response Resiliency Program (31, 32). They attended eight 55-minute Warrior Cognitive Health group sessions that focused on psychoeducation about factors that affect cognition and the implementation of new cognitive strategies. Participants also engaged in several complementary integrative health sessions, including art therapy, yoga, Tai Chi, fitness, and nutrition. Patients’ medications were managed by a psychiatrist, as needed, during their stay at the program. Participants with a history of substance abuse attended four 55-minute dual recovery group sessions to educate them about the relationship between substance use disorders and PTSD. See the article by Harvey et al. (33) for a comprehensive description of the 2-week intensive treatment program.

Analyses

Cognitive functioning was analyzed in two ways: first as a continuous variable and second as impaired or nonimpaired, dichotomized based on performance on the CNS Vital Signs. Cognitive impairment was defined as performance at or below the 5th percentile (i.e., standard score ≤76 or z score ≤–1.64) on at least two of the five CNS Vital Signs core cognitive domains (i.e., executive functioning, reaction time, processing speed, verbal memory, and visual memory). We chose this definition based on prior research illustrating that having two or more low index scores (out of five) is a low base rate finding, occurring in approximately 5%−8% of normative subjects (34–36).

We analyzed demographic differences in patients by impairment status using Student’s t tests (two-tailed) for continuous variables and Pearson’s chi-square tests for categorical variables. We created a correlation matrix between our predictor and outcome variables. Because the outcome variables were not normally distributed (Shapiro-Wilk test; all p values <0.05), we used Spearman’s correlations. Next, we used a mixed-model multivariate analysis of variance (MANOVA) for repeated measures to examine our four outcome variables: PCL5, PHQ-9, SE-SMS, PROMIS-S. Time (i.e., pretreatment, posttreatment) was the within-subjects factor and cognitive group status (i.e., cognitively impaired versus not cognitively impaired) was the between-groups factor. To ensure our findings were not due to differences between the cognitively impaired and not-impaired group, we added gender, years of education, and relationship status as covariates in the MANOVA. The threshold for statistical significance was set at a p value ≤0.05. All statistical analyses were conducted in SPSS (version 21).

Results

The overall sample characteristics are summarized in Table 1. Of the 113 participants, 49.6% met our definition of cognitive impairment (N=56). Patients in the cognitively impaired group were more likely to be men, unpartnered, and less educated (Table 2). Correlations between our cognitive predictor variables and change in our clinical outcome variables are presented in Table 3. There were no significant correlations between the two. For the five core cognitive domains, 38.9% performed at or below the 5th percentile in executive functioning (N=44), 45.1% in reaction time (N=51), 22.1% in processing speed (N=25), 48.7% in verbal memory (N=55), and 28.3% in visual memory (N=32). Approximately 1 out of 4 (25.7%) participants performed above the 5th percentile (i.e., standard score >76 or z score >–1.64) on all five core cognitive domains (N=29), whereas 24.8% performed at or below the 5th percentile (i.e., standard score ≤76 or z score ≤–1.64) on one core cognitive domain (N=28), 17.7% on two domains (N=20), 12.4% on three domains (N=14), 11.5% on four domains (N=13), and 8.0% on all five core cognitive domains (N=9).

Descriptive statistics for the four outcome measures, stratified by cognitive group and time point, are presented in Table 4. Before treatment, the veterans reported moderate to severe levels of traumatic stress and depression, low perceived cognitive self-efficacy, and low satisfaction with their participation in social roles (Table 4). In the mixed MANOVA, the interaction between cognitive impairment and time was not significant (F=0.83, df=4, 108, p=0.51), indicating that the changes in scores over time were not significantly different based on cognitive impairment group membership (Table 5). The multivariate main effect for time was statistically significant (F=36.75, df=4, 108, p<0.001). Univariate tests showed that all four dependent variables significantly improved from pre to post treatment (PCL-5: F=123.18, df=1, 111, p<0.001, ηp²=0.53; PHQ-9: F=91.48, df=1, 111, p<0.001, ηp²=0.45; SE-SMS: F=35.97, df=1, 111, p<0.001, ηp²=0.25; PROMIS-S: F=50.26, df=1, 111, p<0.001, ηp²=0.31). There was no significant multivariate main effect for cognitive impairment group membership (F=2.27, df=4, 108, p=0.07). Post hoc between-subjects analyses showed that the cognitively impaired group had lower PROMIS-S scores than the cognitively intact group regardless of time point (F=8.84, df=1, 111, p=0.004). There were no differences between groups on the other measures (PCL-5: F=0.71, df=1, 111, p=0.40; PHQ-9: F=0.78, df=1, 111, p=0.38; SE-SMS: F=0.98, df=1, 111, p=0.33).

In a mixed MANOVA where gender, education, and relationship status were used as covariates, the interaction between cognitive impairment and time was not significant (F=0.33, df=4, 105, p=0.86) (Table 6). The multivariate main effect for time was not significant (F=0.69, df=4, 105, p=0.60). None of the covariates had significant interactions with time (all p values >0.05). Between-subjects analyses showed that the cognitively impaired group had lower PROMIS-S scores than the cognitively intact group regardless of time point (F=6.60, df=1, 108, p=0.01). There were no group differences on the other measures (PCL-5: F=1.08, df=1, 108, p=0.30; PHQ-9: F=1.83, df=1, 108, p=0.18; SE-SMS: F=0.68, df=1, 108, p=0.41). None of the covariates had significant interactions with cognitive impairment (all p values >0.05).

Discussion

A large sample of military veterans with severe and persistent PTSD and comorbid conditions such as depression and substance abuse completed a 2-week multidisciplinary intensive clinical program comprising individual psychotherapy, group psychotherapy, psychoeducation and skills building groups, and complementary and alternative medicine treatments. The subjects reported statistically significant and large, clinically meaningful reductions in traumatic stress symptoms (ηp²=0.53) and depression (ηp²=0.45) following treatment. They also reported improvements in cognitive self-efficacy (ηp²=0.25) and satisfaction with their ability to participate in social roles (ηp²=0.31). We examined whether pretreatment neurocognitive test performance predicted response to treatment for PTSD. When analyzed as continuous variables, there were no significant correlations between pretreatment cognitive test scores and the outcome variables. A correlation is one type of effect size. As can be seen, none of the correlations were significant, and all were 0.10 or less. Approximately half of the veterans met our criteria for cognitive impairment on the computerized neurocognitive battery. Importantly, we did not find that baseline cognitive impairment was associated with any patient-reported outcomes.

The results of this study are different from what Wild and Gur found: that story recall immediately after it was presented was associated with improvement in PTSD severity score. Notably, their sample had PTSD from interpersonal trauma or a motor vehicle accident and consisted of 43% women. The current study sample was predominantly male; all were veterans, and most had combat trauma. Further, our index of verbal memory relied on a word list presented on a computer screen. Our study results also differ from the Haaland et al. (15) study that showed learning/memory predicted PTSD treatment response. That study sampled only female veterans and assessed learning/memory with a composite score derived from a list of words presented orally across a variety of conditions.

Our results are similar to a study that did not find an association between baseline cognitive performance and change in PTSD symptoms in response to CBT among women with sexual assault-related PTSD (18). Unlike that study, we measured cognitive performance objectively using a computerized assessment tool, CNS Vital Signs, which was administered at baseline and therefore was not influenced by treatment. The results of the current study are inconsistent with one previous study that found lower verbal memory scores predicted poorer response to treatment of PTSD nightmares (20). This discrepancy may be related to different outcome measures (i.e., overall PTSD score versus nightmare score), samples, treatments, cutoffs for cognitive impairment, as well as the presence of possible confounders. Our results are also consistent with studies that did not find an association between history of TBI and poorer treatment outcome (21–23). Although these latter studies did not measure cognitive abilities, a history of TBI and any possible cognitive impairment associated with it was not associated with treatment gains. Our study did objectively measure pretreatment cognitive functioning, and we did not find an association with traumatic stress symptom improvement. Further, we did not find evidence that cognitive impairment attenuated improvements in depressive symptoms, confidence in the ability to overcome cognitive difficulties, or satisfaction in the ability to complete everyday role responsibilities and tasks.

Because our study used a naturalistic treatment design in an intensive care program setting, it is affected by the usual limitations inherent in a nonrandomized and noncontrolled trial. Our naturalistic design did not allow us to measure other possible confounders (e.g., medications, drug and alcohol use, and socioeconomic status). All patients in the program were asked to abstain from alcohol, illicit substances, and nonprescribed medications before and during treatment. Patients completed alcohol breathalyzers and urine drug screens every day that they attended the treatment facility. This likely reduced but did not eliminate the possible effects of substance abuse on our findings. Our treatment program includes subprograms for PTSD and TBI (33). Only patients in the PTSD track were included in this study. Those in the TBI track do not undergo pretreatment testing with CNS Vital Signs; they undergo comprehensive neuropsychological evaluations. Many veterans in the PTSD track have a lifetime history of concussions in sports, daily life, and military service. A future study with a larger sample could be used to examine whether prior concussions are associated with treatment outcome. We also cannot claim the independence of our predictor from our outcome variable. PTSD can cause cognitive impairment, and several PCL-5 items constitute significant risks to attention (e.g., hypervigilance, poor sleep). Our cognitive predictors were measured only pretreatment. We believe that a reassessment after completion of treatment would have informed the relationship between our predictor and outcome variables. Another limitation is that measures of symptom validity and performance validity were not included, so it is unclear if patients were sufficiently engaged in cognitive testing—or whether symptom exaggeration influenced the outcome of this study. Future studies that measure cognitive performance and mental health outcomes repeatedly and simultaneously are needed. Because multiple treatments were delivered in the context of our multidisciplinary program, we cannot determine which treatment or combination of treatments led to the observed improvement in symptoms. Generalization of our results to other populations is limited. The results are likely generalizable to other similar intensive treatment programs for PTSD in OEF/OIF/OND populations, and they may even be generalizable to the outpatient setting, with the caveat that the dropout rate in our population is substantially lower than that in outpatients (37–39).

In conclusion, there are mixed findings in the literature on the relationship between pretreatment cognitive deficits and response to treatment in adults with PTSD (18–23) or depression (13, 14). We hypothesized that veterans with objectively measured cognitive impairment would have a worse response to treatment, but this was not the case. Veterans with and without cognitive impairment reported large improvements in symptoms and functioning after participating in this 2-week multidisciplinary intensive clinical program.