0
Get Alert
Please Wait... Processing your request... Please Wait.
You must sign in to sign-up for alerts.

Please confirm that your email address is correct, so you can successfully receive this alert.

1
Clinical Research Reports   |    
Can Computerized Cognitive Tests Assist in the Clinical Diagnosis of Attention-Deficit Hyperactivity Disorder?
Yuval Bloch, M.D.; Mor Fixman, M.A.; Hagai Maoz, M.D.; Aviva Mimouni Bloch, M.D.; Yechiel Levkovitz, M.D.; Gideon Ratzoni; Shai Aviram, B.A.; Gilad Gal, Ph.D.
The Journal of Neuropsychiatry and Clinical Neurosciences 2012;24:111-114. doi:10.1176/appi.neuropsych.11010014
View Author and Article Information

From The Emotion-Cognition Research Center, Shalvata Mental Health Care Center, Hod-Hasharon, Israel YB, MF, HM, YL, GR, SA, GG); The Pediatric Neurology and Developmental Unit, Loewenstein Rehabilitation Hospital, Ahuza 278, Raanana, Israel (both affiliated with the Sackler Faculty of Medicine, Tel-Aviv University; AMB); Sackler Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel (YB, AMB, YL, GR); and the School of Behavioral Sciences, The Academic College Tel Aviv-Yaffo, Tel Aviv, Israel (GG).

Correspondence: Yuval Bloch, M.D., Shalvata Mental Health Center, Hod-Hasharon, Israel; aviva100@bezeqint.net; yuvalbl@clalit.org.il (e-mail).

Received January 12, 2011; May 30, 2011; Accepted June 9, 2011.

A group of 34 children and adolescents suspected of having attention-deficit hyperactivity disorder were referred for a computerized evaluation that included sustained attention, working memory, planning, and set-shifting. Although only sustained attention had reasonable specificity, all tests had questionable contribution to the diagnostic evaluation.

Abstract Teaser
Figures in this Article

The assessment and diagnosis of attention-deficit hyperactivity disorder (ADHD) is clinical. It is based primarily on behavioral symptoms gathered via clinical examination, patient history, and reports and questionnaires. In the clinical realm, the diagnostic procedure is often a complex one for a variety of reasons: difficulty in obtaining valid information, additional clinical conditions that present with similar symptoms, comorbid conditions, and patients who meet borderline criteria.1,2 Since cognitive difficulties are known to be associated with behavioral symptoms, it is not surprising that professionals in the field are interested in finding objective tools to further validate the diagnosis.

Findings of impaired sustained attention functions were frequently reported1,3 and are therefore considered by some to be central to the understanding of the symptoms of ADHD.4 Thus, continuous performance tests (CPT) that examine these functions are probably the most commonly-used computerized neurocognitive assessments aiding in the clinical diagnosis of ADHD. The Test of Variables of Attention (TOVA) is a commonly-used CPT, with an estimated sensitivity of approximately 85% and specificity of approximately 70%.4 Because of its lower specificity, it carries a risk of overdiagnosis and treatment of children who do not have ADHD.1 One of the important issues related to assessing the sensitivity, specificity, and predictive value of a test is the population selected and the prevalence of the disorder in that population. Most studies used a healthy-control group2,5 or a well-established other diagnosis such as a learning disorder or conduct disorder.2,4 Whereas the study of core deficits in ADHD is based on differentiation from healthy volunteers, referrals differ from healthy volunteers in the sense that they are referred because of some difficulty. Thus, in clinical practice, the validity of the tests should be evaluated in relation to referrals and not to the asymptomatic population or another specific diagnosis. In a previous study,5 the CPT failed to differentiate between children with ADHD and children with subclinical levels of behavioral or cognitive problems, after excluding other diagnoses.

Some consider executive functions (EFs) as the core deficit in ADHD. Among the heterogeneous EFs, working memory, set-shift, response inhibition, and planning were frequently reported to be impaired in patients with ADHD.4,69 Most of the studies that relate to difficulties in these EFs used a normal-control group. The use of various neurocognitive batteries examining a range of EFs is becoming more popular in clinical practice.10

Still, despite the well-replicated EF weakness in ADHD subjects, variability between studies has been noted.8 Although the literature relates to other possible core findings, including delay-aversion, low frustration-tolerance, and emotional lability, neurocognitive batteries assessing CPT and EF have gained popularity in clinical practice.

The aim of the current study is to estimate the potential benefit of commonly used EF and sustained-attention assessment in the establishment of the diagnosis of ADHD, in a clinical setting of ADHD.

A group of 34 children and adolescents, 19 boys and 15 girls, between the ages of 7 and 17, participated in the study (mean age: 11.5 years). Study protocol and consent forms were approved by the Institutional Review Board. All of the participants were children who had been referred by a neurologist or a child psychiatrist for a neurocognitive evaluation in order to substantiate a possible diagnosis of ADHD. Criteria for exclusion were known diagnosis of mental retardation or major psychopathology (namely, major affective disorder, psychotic disorder, pervasive developmental disorder, substance abuse, posttraumatic stress disorder, obsessive-compulsive disorder, panic disorder). After parents signed the informed consent form (and minors signed an assent form), all patients underwent the following:

1. Questionnaires: The Swanson, Nolan, & Pelham Questionnaire (SNAP–IV), a well-validated DSM–IV based questionnaire,11 was completed by one of the parents and, on a separate form, by a teacher; The Strengths and Difficulties Questionnaire (SDQ), a widely-used dimensional measure of child mental health,12,13 was completed by parents and children.

2. Structured interview: MP, a research psychologist trained in ADHD evaluation, performed a clinical interview separately for both child and parents according to the Development and Well-Being Assessment (DAWBA). The DAWBA is constructed to generate ICD–10 and DSM–IV psychiatric diagnosis for minors.14

3. Clinical interview: YB, a certified child and adolescent psychiatrist, blinded to the result of the structured interview, performed a clinical interview of child and parents.

The clinical diagnosis of ADHD was based on consensus between the research team, blinded to results of the computerized test.

4. Neurocognitive tests: For testing sustained attention, we administered the TOVA.1,5 Subtests of the CANTAB were used to evaluated the following EF domains: Working Memory, Spatial Working Memory (SWM); Planning – Stocking of Cambridge (SOC), and Cognitive Set-Shifting – Intradimensional/Extradimensional Shift (IED).4

+

Statistical Analysis

All variables were defined dichotomously. The clinical diagnostic was the “gold standard” to differentiate between ADHD and non-ADHD patients. For each cognitive domain, we used one standard deviation (SD) as cutoff for the following measures: TOVA: “ADHD score;” CANTAB SWM – between errors or strategy; CANTAB SOC problems solved in minimum moves or mean initial (and/or) subsequent thinking time; CANTAB IED stage completed or total errors. The association between the clinical evaluation and Cognitive tasks (TOVA and CANTAB) was tested by chi-square test or Fisher's exact test (when the expected count was <5 in at least one cell). Significant association was considered when α level was <0.05. Effect size was computed using the Pearson phi coefficient (rφ). For comparison of the efficiency of the different tests, we calculated the following measures: sensitivity: CPT/EF identified clinical ADHD from all ADHD (clinical) cases; specificity: CPT/EF identified non-clinical ADHD as not ADHD from all non-ADHD (clinical) cases; positive predictive value; CPT/EF identified clinical ADHD from total CPT/EF identified ADHD and negative predictive value; CPT/EF identified non-clinical ADHD as not ADHD from all non-ADHD CPT/EF-identified cases.

In the group, 27 of the 34 patients recruited were diagnosed clinically as suffering from ADHD. For 7 patients, the diagnosis of ADHD was excluded. These patients were subsequently diagnosed, two with dysthymia, two with conduct disorder and learning disabilities, two with anxiety disorder, and one with pervasive developmental disorder.

There was a significant association between the final clinical diagnosis and the findings of the CPT test (TOVA; p <0.05; Fisher's exact test (FET); rφ=0.39). Of the 27 patients with a confirmed diagnosis of ADHD, the TOVA verified attention-deficit in 17 and yielded non-pathological findings in 10 patients; thus, the TOVA had a sensitivity of 63%.

Of the seven patients who were diagnosed with another disorder (not ADHD), six had a non-pathological TOVA result; thus, the TOVA had a specificity of 85%. Since 17 of the 18 patients with the pathological result in the TOVA test had a confirmed diagnosis of ADHD, the positive predictive value was 94%. Since, of 16 non-pathological TOVA tests, only 6 were confirmed as not having ADHD, the negative predictive value was 37%.

The tests for executive functions (EFs) were not found to be significantly associated with the clinical diagnosis of ADHD (IED, SOC, SWM; p: NS, FET). Both the sensitivity (IED: 71%; SOC: 71%; SWM: 57%) and the specificity (IED: 7%; SOC: 11%; SWM: 22%) of the tests chosen from the CANTAB were low.

This study relates to the common clinical dilemma of diagnosing children with difficulties who are suspected to be suffering from ADHD. The study evaluated the contribution of computerized cognitive tests in the diagnosis of children with ADHD.

The central finding of the study revealed that both the EF tests examined and the CPT did not have a clinically significant contribution to the diagnosis. Although the CPT theoretically had a reasonable specificity of 85%, the fact that approximately 80% of the presented cohort suffered from ADHD diminishes its contribution.

This finding is opposed to the robust findings of malfunctions in these tests in previous research when comparing patients with ADHD to matched-control subjects.8,15 In our view, the reasonable explanation for this discrepancy lies in the choice of the control group. Although patients with ADHD have measurable deficits in functions related to EF as compared with non- clinical controls, these differences do not typify them in comparison to children who experience other disorders, such as anxiety or depression. Thus, our findings join previous studies that did not find a diagnostic contribution of cognitive batteries in differentiating ADHD patients from children with subclinical difficulties.5

The presented study has several limitations: a small sample size and high variability in the patients' ages. Larger studies with more homogenous groups might provide more detailed information about the exact nature of the impairments found by computerized cognitive tests in true clinical samples.

We thank Eve Leibowitz for her editorial assistance during the preparation of the manuscript.

None of the authors has any biomedical financial interests or potential conflicts of interest whatsoever in the publication of this research.

Schatz  AM;  Ballantyne  AO;  Trauner  DA:  Sensitivity and specificity of a computerized test of attention in the diagnosis of attention-deficit/hyperactivity disorder.  Assessment 2001; 8:357–365
[PubMed]
[CrossRef]
 
Sergeant  JA;  Geurts  H;  Oosterlaan  J:  How specific is a deficit of executive functioning for attention-deficit/hyperactivity disorder? Behav Brain Res 2002; 130(1,2):3–28
[PubMed]
[CrossRef]
 
Forbes  GB:  Clinical utility of the test of variables of attention (TOVA) in the diagnosis of attention-deficit/hyperactivity disorder.  J Clin Psychol 1998; 54:461–476
[PubMed]
[CrossRef]
 
Willcutt  EG;  Doyle  AE;  Nigg  JT  et al:  Validity of the executive function theory of attention-deficit/hyperactivity disorder: a meta-analytic review.  Biol Psychiatry 2005; 57:1336–1346
[PubMed]
[CrossRef]
 
Preston  AS;  Fennell  EB;  Bussing  R:  Utility of a CPT in diagnosing ADHD among a representative sample of high-risk children: a cautionary study.  Child Neuropsychol 2005; 11:459–469
[PubMed]
[CrossRef]
 
Barnett  R;  Maruff  P;  Vance  A  et al:  Abnormal executive function in attention-deficit hyperactivity disorder: the effect of stimulant medication and age on spatial working memory.  Psychol Med 2001; 31:1107–1115
[PubMed]
[CrossRef]
 
Goldberg  MC;  Mostofsky  SH;  Cutting  LE  et al:  Subtle executive impairment in children with autism and children with ADHD.  J Autism Dev Disord 2005; 35:279–293
[PubMed]
[CrossRef]
 
Doyle  AE:  Executive functions in attention-deficit/hyperactivity disorder.  J Clin Psychiatry 2006; 67:21–26
[PubMed]
 
Toplak  ME;  Bucciarelli  SM;  Jain  U  et al:  Executive functions: performance-based measures and the Behavior Rating Inventory of Executive Function (BRIEF) in adolescents with attention-deficit/hyperactivity disorder (ADHD).  Child Neuropsychol 2009; 15:53–72
[PubMed]
[CrossRef]
 
Nichols  SL;  Waschbusch  DA:  A review of the validity of laboratory cognitive tasks used to assess symptoms of ADHD.  Child Psychiatry Hum Dev 2004; 34:297–315
[PubMed]
[CrossRef]
 
Swanson  JM;  Kraemer  HC;  Hinshaw  SP  et al:  Clinical relevance of the primary findings of the MTA: success rates based on severity of ADHD and ODD symptoms at the end of treatment.  J Am Acad Child Adolesc Psychiatry 2001; 40:168–179
[PubMed]
[CrossRef]
 
Mansbach-Kleinfeld  I;  Apter  A;  Farbstein  I  et al:  A population-based psychometric validation study of the Strengths and Difficulties Questionnaire, Hebrew version.  Front Psychiatry 2010; 1:151
[PubMed]
[CrossRef]
 
Goodman  R:  Psychometric properties of the Strengths and Difficulties Questionnaire.  J Am Acad Child Adolesc Psychiatry 2001; 40:1337–1345
[PubMed]
[CrossRef]
 
Goodman  R;  Ford  T;  Richards  H  et al:  The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology.  J Child Psychol Psychiatry 2000; 41:645–655
[PubMed]
[CrossRef]
 
Pineda  DA;  Puerta  IC;  Aguirre  DC  et al:  The role of neuropsychologic tests in the diagnosis of attention-deficit hyperactivity disorder.  Pediatr Neurol 2007; 36:373–381
[PubMed]
[CrossRef]
 
References Container
+

References

Schatz  AM;  Ballantyne  AO;  Trauner  DA:  Sensitivity and specificity of a computerized test of attention in the diagnosis of attention-deficit/hyperactivity disorder.  Assessment 2001; 8:357–365
[PubMed]
[CrossRef]
 
Sergeant  JA;  Geurts  H;  Oosterlaan  J:  How specific is a deficit of executive functioning for attention-deficit/hyperactivity disorder? Behav Brain Res 2002; 130(1,2):3–28
[PubMed]
[CrossRef]
 
Forbes  GB:  Clinical utility of the test of variables of attention (TOVA) in the diagnosis of attention-deficit/hyperactivity disorder.  J Clin Psychol 1998; 54:461–476
[PubMed]
[CrossRef]
 
Willcutt  EG;  Doyle  AE;  Nigg  JT  et al:  Validity of the executive function theory of attention-deficit/hyperactivity disorder: a meta-analytic review.  Biol Psychiatry 2005; 57:1336–1346
[PubMed]
[CrossRef]
 
Preston  AS;  Fennell  EB;  Bussing  R:  Utility of a CPT in diagnosing ADHD among a representative sample of high-risk children: a cautionary study.  Child Neuropsychol 2005; 11:459–469
[PubMed]
[CrossRef]
 
Barnett  R;  Maruff  P;  Vance  A  et al:  Abnormal executive function in attention-deficit hyperactivity disorder: the effect of stimulant medication and age on spatial working memory.  Psychol Med 2001; 31:1107–1115
[PubMed]
[CrossRef]
 
Goldberg  MC;  Mostofsky  SH;  Cutting  LE  et al:  Subtle executive impairment in children with autism and children with ADHD.  J Autism Dev Disord 2005; 35:279–293
[PubMed]
[CrossRef]
 
Doyle  AE:  Executive functions in attention-deficit/hyperactivity disorder.  J Clin Psychiatry 2006; 67:21–26
[PubMed]
 
Toplak  ME;  Bucciarelli  SM;  Jain  U  et al:  Executive functions: performance-based measures and the Behavior Rating Inventory of Executive Function (BRIEF) in adolescents with attention-deficit/hyperactivity disorder (ADHD).  Child Neuropsychol 2009; 15:53–72
[PubMed]
[CrossRef]
 
Nichols  SL;  Waschbusch  DA:  A review of the validity of laboratory cognitive tasks used to assess symptoms of ADHD.  Child Psychiatry Hum Dev 2004; 34:297–315
[PubMed]
[CrossRef]
 
Swanson  JM;  Kraemer  HC;  Hinshaw  SP  et al:  Clinical relevance of the primary findings of the MTA: success rates based on severity of ADHD and ODD symptoms at the end of treatment.  J Am Acad Child Adolesc Psychiatry 2001; 40:168–179
[PubMed]
[CrossRef]
 
Mansbach-Kleinfeld  I;  Apter  A;  Farbstein  I  et al:  A population-based psychometric validation study of the Strengths and Difficulties Questionnaire, Hebrew version.  Front Psychiatry 2010; 1:151
[PubMed]
[CrossRef]
 
Goodman  R:  Psychometric properties of the Strengths and Difficulties Questionnaire.  J Am Acad Child Adolesc Psychiatry 2001; 40:1337–1345
[PubMed]
[CrossRef]
 
Goodman  R;  Ford  T;  Richards  H  et al:  The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology.  J Child Psychol Psychiatry 2000; 41:645–655
[PubMed]
[CrossRef]
 
Pineda  DA;  Puerta  IC;  Aguirre  DC  et al:  The role of neuropsychologic tests in the diagnosis of attention-deficit hyperactivity disorder.  Pediatr Neurol 2007; 36:373–381
[PubMed]
[CrossRef]
 
References Container
+
+

CME Activity

There is currently no quiz available for this resource. Please click here to go to the CME page to find another.
Submit a Comments
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discertion of APA editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe



Related Content
Books
Manual of Clinical Psychopharmacology, 7th Edition > Chapter 12.  >
Manual of Clinical Psychopharmacology, 7th Edition > Chapter 12.  >
Manual of Clinical Psychopharmacology, 7th Edition > Chapter 8.  >
The American Psychiatric Publishing Textbook of Psychiatry, 5th Edition > Chapter 21.  >
Gabbard's Treatments of Psychiatric Disorders, 4th Edition > Chapter 4.  >
Topic Collections
Psychiatric News
Read more at Psychiatric News >>
PubMed Articles