Can Computerized Cognitive Tests Assist in the Clinical Diagnosis of Attention-Deficit Hyperactivity Disorder?

The assessment and diagnosis of attention-deficit hyperactivity disorder (ADHD) is clinical. It is based primarily on behavioral symptoms gathered via clinical examination, patient history, and reports and questionnaires. In the clinical realm, the diagnostic procedure is often a complex one for a variety of reasons: difficulty in obtaining valid information, additional clinical conditions that present with similar symptoms, comorbid conditions, and patients who meet borderline criteria.^1,2 Since cognitive difficulties are known to be associated with behavioral symptoms, it is not surprising that professionals in the field are interested in finding objective tools to further validate the diagnosis.

Findings of impaired sustained attention functions were frequently reported^1,3 and are therefore considered by some to be central to the understanding of the symptoms of ADHD.⁴ Thus, continuous performance tests (CPT) that examine these functions are probably the most commonly-used computerized neurocognitive assessments aiding in the clinical diagnosis of ADHD. The Test of Variables of Attention (TOVA) is a commonly-used CPT, with an estimated sensitivity of approximately 85% and specificity of approximately 70%.⁴ Because of its lower specificity, it carries a risk of overdiagnosis and treatment of children who do not have ADHD.¹ One of the important issues related to assessing the sensitivity, specificity, and predictive value of a test is the population selected and the prevalence of the disorder in that population. Most studies used a healthy-control group^2,5 or a well-established other diagnosis such as a learning disorder or conduct disorder.^2,4 Whereas the study of core deficits in ADHD is based on differentiation from healthy volunteers, referrals differ from healthy volunteers in the sense that they are referred because of some difficulty. Thus, in clinical practice, the validity of the tests should be evaluated in relation to referrals and not to the asymptomatic population or another specific diagnosis. In a previous study,⁵ the CPT failed to differentiate between children with ADHD and children with subclinical levels of behavioral or cognitive problems, after excluding other diagnoses.

Some consider executive functions (EFs) as the core deficit in ADHD. Among the heterogeneous EFs, working memory, set-shift, response inhibition, and planning were frequently reported to be impaired in patients with ADHD.^4,6–9 Most of the studies that relate to difficulties in these EFs used a normal-control group. The use of various neurocognitive batteries examining a range of EFs is becoming more popular in clinical practice.¹⁰

Still, despite the well-replicated EF weakness in ADHD subjects, variability between studies has been noted.⁸ Although the literature relates to other possible core findings, including delay-aversion, low frustration-tolerance, and emotional lability, neurocognitive batteries assessing CPT and EF have gained popularity in clinical practice.

The aim of the current study is to estimate the potential benefit of commonly used EF and sustained-attention assessment in the establishment of the diagnosis of ADHD, in a clinical setting of ADHD.

METHOD

A group of 34 children and adolescents, 19 boys and 15 girls, between the ages of 7 and 17, participated in the study (mean age: 11.5 years). Study protocol and consent forms were approved by the Institutional Review Board. All of the participants were children who had been referred by a neurologist or a child psychiatrist for a neurocognitive evaluation in order to substantiate a possible diagnosis of ADHD. Criteria for exclusion were known diagnosis of mental retardation or major psychopathology (namely, major affective disorder, psychotic disorder, pervasive developmental disorder, substance abuse, posttraumatic stress disorder, obsessive-compulsive disorder, panic disorder). After parents signed the informed consent form (and minors signed an assent form), all patients underwent the following:

1. Questionnaires: The Swanson, Nolan, & Pelham Questionnaire (SNAP–IV), a well-validated DSM–IV based questionnaire,¹¹ was completed by one of the parents and, on a separate form, by a teacher; The Strengths and Difficulties Questionnaire (SDQ), a widely-used dimensional measure of child mental health,^12,13 was completed by parents and children.

2. Structured interview: MP, a research psychologist trained in ADHD evaluation, performed a clinical interview separately for both child and parents according to the Development and Well-Being Assessment (DAWBA). The DAWBA is constructed to generate ICD–10 and DSM–IV psychiatric diagnosis for minors.¹⁴

3. Clinical interview: YB, a certified child and adolescent psychiatrist, blinded to the result of the structured interview, performed a clinical interview of child and parents.

The clinical diagnosis of ADHD was based on consensus between the research team, blinded to results of the computerized test.

4. Neurocognitive tests: For testing sustained attention, we administered the TOVA.^1,5 Subtests of the CANTAB were used to evaluated the following EF domains: Working Memory, Spatial Working Memory (SWM); Planning – Stocking of Cambridge (SOC), and Cognitive Set-Shifting – Intradimensional/Extradimensional Shift (IED).⁴

Statistical Analysis

All variables were defined dichotomously. The clinical diagnostic was the “gold standard” to differentiate between ADHD and non-ADHD patients. For each cognitive domain, we used one standard deviation (SD) as cutoff for the following measures: TOVA: “ADHD score;” CANTAB SWM – between errors or strategy; CANTAB SOC problems solved in minimum moves or mean initial (and/or) subsequent thinking time; CANTAB IED stage completed or total errors. The association between the clinical evaluation and Cognitive tasks (TOVA and CANTAB) was tested by chi-square test or Fisher's exact test (when the expected count was <5 in at least one cell). Significant association was considered when α level was <0.05. Effect size was computed using the Pearson phi coefficient (r_φ). For comparison of the efficiency of the different tests, we calculated the following measures: sensitivity: CPT/EF identified clinical ADHD from all ADHD (clinical) cases; specificity: CPT/EF identified non-clinical ADHD as not ADHD from all non-ADHD (clinical) cases; positive predictive value; CPT/EF identified clinical ADHD from total CPT/EF identified ADHD and negative predictive value; CPT/EF identified non-clinical ADHD as not ADHD from all non-ADHD CPT/EF-identified cases.

RESULTS

In the group, 27 of the 34 patients recruited were diagnosed clinically as suffering from ADHD. For 7 patients, the diagnosis of ADHD was excluded. These patients were subsequently diagnosed, two with dysthymia, two with conduct disorder and learning disabilities, two with anxiety disorder, and one with pervasive developmental disorder.

There was a significant association between the final clinical diagnosis and the findings of the CPT test (TOVA; p <0.05; Fisher's exact test (FET); r_φ=0.39). Of the 27 patients with a confirmed diagnosis of ADHD, the TOVA verified attention-deficit in 17 and yielded non-pathological findings in 10 patients; thus, the TOVA had a sensitivity of 63%.

Of the seven patients who were diagnosed with another disorder (not ADHD), six had a non-pathological TOVA result; thus, the TOVA had a specificity of 85%. Since 17 of the 18 patients with the pathological result in the TOVA test had a confirmed diagnosis of ADHD, the positive predictive value was 94%. Since, of 16 non-pathological TOVA tests, only 6 were confirmed as not having ADHD, the negative predictive value was 37%.

The tests for executive functions (EFs) were not found to be significantly associated with the clinical diagnosis of ADHD (IED, SOC, SWM; p: NS, FET). Both the sensitivity (IED: 71%; SOC: 71%; SWM: 57%) and the specificity (IED: 7%; SOC: 11%; SWM: 22%) of the tests chosen from the CANTAB were low.

DISCUSSION

This study relates to the common clinical dilemma of diagnosing children with difficulties who are suspected to be suffering from ADHD. The study evaluated the contribution of computerized cognitive tests in the diagnosis of children with ADHD.

The central finding of the study revealed that both the EF tests examined and the CPT did not have a clinically significant contribution to the diagnosis. Although the CPT theoretically had a reasonable specificity of 85%, the fact that approximately 80% of the presented cohort suffered from ADHD diminishes its contribution.

This finding is opposed to the robust findings of malfunctions in these tests in previous research when comparing patients with ADHD to matched-control subjects.^8,15 In our view, the reasonable explanation for this discrepancy lies in the choice of the control group. Although patients with ADHD have measurable deficits in functions related to EF as compared with non- clinical controls, these differences do not typify them in comparison to children who experience other disorders, such as anxiety or depression. Thus, our findings join previous studies that did not find a diagnostic contribution of cognitive batteries in differentiating ADHD patients from children with subclinical difficulties.⁵

The presented study has several limitations: a small sample size and high variability in the patients' ages. Larger studies with more homogenous groups might provide more detailed information about the exact nature of the impairments found by computerized cognitive tests in true clinical samples.