Empirical and Authoritative Classification of Neuropsychiatric Syndromes in Neurocognitive Disorders

Significant efforts to define neuropsychiatric syndromes in neurocognitive disorders (NCDs) have surfaced in the past three decades, with surging interest in the past decade (1–13). Neuropsychiatric syndromes are conceptualized with either a bottom-up or a top-down approach, depending on whether a data-driven or an operational definition is the primary orientation. Following the more established literature in general psychopathology, these approaches are called the empirical and authoritative approach, respectively (14). Both approaches have strengths and weaknesses. Although a conceptual framework for these constructs is needed, approaches to classification without critical examination may also lead the field astray (15, 16). Examining the two approaches in a wide context is warranted because the relatively young subdiscipline of neuropsychiatry in NCDs seems to be grappling with the same issues that psychiatry has grappled with (and continues to grapple with). Thus far, the lessons learned from psychiatric classification are not often discussed in unison with the nosological developments in the neuropsychiatry of NCDs. The main objective of this review was to examine the strengths and weaknesses of, and suggest future directions for, the authoritative and empirical approaches to neuropsychiatric syndromes classification.

The Need for the Neuropsychiatric Syndromes Construct

The need for the construct of a neuropsychiatric syndrome arose from the insufficiency of its intellectual predecessor, behavioral and psychological symptoms of dementia (BPSD). BPSD is an umbrella term for behavioral alterations among those with dementia, which received far less attention than cognitive symptoms in 1990s research (17). Although it was useful for drawing attention to the often burdensome noncognitive aspects of dementia, BPSD turned out to be too heterogeneous a clinical target (18). In 2000, the U.S. Food and Drug Administration’s Psychopharmacological Drugs Committee convened to discuss drug development for neuropsychiatric syndromes in Alzheimer’s disease and other dementias (19). The invited experts disagreed on several accounts, but the committee meeting nonetheless led to the formulation of four criteria, influenced by psychiatric indications for treatment, through which neuropsychiatric syndrome indications should be evaluated: universal definition, commonly accepted rating methods, a well-understood pathophysiology, and specific response to drug treatment (18).

Authoritative Approach

Using the authoritative approach, an operational, universal definition of a syndrome is achieved through a top-down clinical threshold for severity and number of symptoms, as is the case for the DSM-5 (20) and ICD-10 (21). The content of these definitions is formed by expert panels, hence the name authoritative approach (14). Neuropsychiatric syndromes have been influenced by their non–dementia-related counterparts in psychiatry (22) and are categorical in nature; the syndrome either is present or it is not (23). Correspondingly, similar approaches to the DSM-5 and ICD-10 are available for neuropsychiatric syndromes in Alzheimer’s disease and other NCDs. Currently, criteria exist for psychosis, apathy, agitation, anxiety, depression, sleep disturbances, and mild behavioral impairment (Table 1). Definitions of syndromes without explicit criteria are not included (e.g., the pioneering apathy definition [24]). Although these syndromes differ in terms of data acquisition strategies, later empirical support, adoption, and applicability, they all have operational definitions. For example, Robert et al. (11) revised transdiagnostic criteria to define apathy as a reduction in goal-directed activity relative to the individual’s previous level of functioning. Additionally, the symptoms must be present for at least 4 weeks, with at least two of three dimensions of apathy present (behavior or cognition, emotion, social interaction); specific symptoms are outlined in more detail in the criteria. Apathy and psychosis have been revised to accommodate advances in research, whereas the construct of syndromal sleep disturbances in Alzheimer’s disease has been less researched and less utilized.

The authoritative approach has both clinical and research benefits. First, having widely agreed-upon criteria for key neuropsychiatric constructs would standardize research efforts and lead to more efficient accumulation of knowledge on the subject. With NCDs, despite research progress, few safe and tolerable treatment options exist for neuropsychiatric symptoms (25, 26). Ideally, operationally defined syndromes would be targets for treatment in clinical trials (10, 18). Heterogeneity in defining the syndrome to be treated is not uncommon, as evidenced in a review of clinical trials for agitation and aggression that found few consistencies in the choice of rating scales and cutoff scores for clinical significance (27). With varying definitions, it is also challenging to establish reliable correlations with clinical and biomarker findings (13, 28) or to observe base rates and temporal changes in the prevalence of neuropsychiatric syndromes.

Diagnostic criteria for neuropsychiatric syndromes could also facilitate routine clinical assessment of these symptoms (29). This practice would fill an important gap, because rating scales for neuropsychiatric symptoms are often underutilized in memory clinics (30). Criteria could also give clinicians and researchers leeway in judgment when culturally appropriate behaviors would produce a false positive for neuropsychiatric syndromes with traditional rating scales (31, 32).

Criteria could also aid in differential diagnosis. Neuropsychiatric syndromes can overlap in some aspects with psychiatric syndromes of primarily nonneurological origins, but their differentiation is crucial for treatment. For example, the nature of symptoms and the cognitive profile differ between very late–onset schizophrenia-like psychosis and psychosis associated with NCD (2, 33, 34) and between late-life depression and depression associated with NCD (4, 35).

The authoritative approach could also benefit communication by offering a shared language for defining syndromes (15, 16, 36). A shared language could offer patients and their caregivers increased understanding of the condition and sources for support and information (37). Additionally, having a common language for neuropsychiatric syndromes could offer administrative, statistical, educational, and legal benefits and provide the conceptual base for scientific inquiry.

Limitations

The authoritative approach has mainly transported the classification framework used in psychiatry into NCDs. However, it is evident that the role of neurobiological underpinnings and the heterogeneity within even a single NCD are not captured by the authoritative criteria. An alternative approach to outlining the neuropsychiatric aspects of an NCD can be found in the literature of cerebellar cognitive affective syndrome (38, 39). With detailed case series on individuals with cerebellar diseases, Schmahmann et al. (39) have outlined the varying neuropsychiatric phenotypes associated with different cerebellar lesions. They outlined five major domains of neuropsychiatric disturbance (i.e., attentional control, emotional control, autism spectrum, psychosis spectrum, social skill set) without resorting to checklists or criteria. Developing criteria for a neuropsychiatric syndrome need not be the goal of systematic studies, but rather one of many possibilities for understanding the clinical phenomenon at hand.

The authoritative approach is useful insofar as the neuropsychiatric syndromes can be detected with adequate reliability and can demonstrate sufficient validity for clinical and research. Limited data are available for assessing these properties in NCDs (29, 40) because the focus on authoritative criteria has mostly been on creating or revising criteria to accommodate research advances.

In the DSM-5, core disorders, such as major depressive disorder, have displayed poor test-retest reliability in field trials (41). Previous iterations of the DSM, conversely, might have overemphasized interrater reliability over clinical utility (16). Furthermore, reliability estimates from field trials may not easily translate to other contexts (15). In DSM-5 field trials, consistently higher test-retest reliability estimates were observed for posttraumatic stress disorder (PTSD) and major NCD than for core disorders, possibly because both have a more tractable etiology than most psychiatric disorders. These estimates are encouraging for neuropsychiatric syndrome research because, for example, subtypes of hallucinations and delusions have discernible pathological bases in dementia (42).

The reliability of higher-order diagnostic constructs may vary on the basis of their constituent symptoms. For example, Seignourel et al. (43) noted that excessive worrying may not be easily appreciable by a caregiver, whereas overt avoidance of anxiety-provoking situations might be easier to observe. Even though both symptoms would fall under the higher-order construct of anxiety, it is plausible that the overt behavioral disturbances would be detected with greater reliability. However, increasing reliability may come at a cost, as the history of the DSM has shown that emphasizing interrater reliability may reduce validity (16) and that emphasizing overt behavioral signs risks missing other important features. For example, research has shown that informant reports of depressive mood in Alzheimer’s disease poorly correlate with the patient’s subjective feelings of hopelessness and worthlessness (44).

In terms of validity, the relationship between criteria and their supporting data tend to be complex, and the criteria could plausibly be formulated in several ways (15). It has been suggested that the symptoms that end up in diagnostic requirements may divert attention from the complexity of the psychopathology associated with the disorder (45, 46). Furthermore, there may be no clinically unique features distinguishing between symptoms that are in the diagnostic criteria and those that failed to make the cut. For example, it has been demonstrated that the symptoms of depression listed in the DSM are not distinguishable from non-DSM symptoms of depression in terms of their network relationships (47).

Suggestions

Although welcome, criteria updated on the basis of the rapidly accumulating evidence base has led to novel criteria being published before establishing, independent of the author panels, the fault with the previous criteria. For example, test-retest and interrater reliabilities are rarely available for existing criteria (29). These basic properties should be explored to assess the utility of the criteria in clinical and research contexts.

Clarifying the gap between published criteria and existing empirical criteria could be facilitated by expertise outside the task force and more transparent literature reviews. The approaches used in literature reviews for novel or revised criteria for neuropsychiatric syndromes could be elaborated in more detail in online supplementary materials or open-access repositories (e.g., osf.io) to make sure important details are not lost due to journal space requirements. This supportive documentation can also be used to discuss the intricacies and challenging decisions posed by equivocal research evidence more thoroughly than would be possible in a single article. Extensive supportive documentation was used in compiling the DSM-5 and has been used in a general neuropsychiatric symptom review (48). Before updated criteria are approved and used, the proposed criteria should be subject to outside commentary, and the expert panel should respond to concerns (15).

Empirical Approach

The empirical approach acknowledges that authoritative criteria may not capture the relevant aspects of psychopathology (14, 47). There is also significant empirical support suggesting that most psychopathological phenomena are more accurately represented as continua rather than categories (49, 50). The empirical approach, thus, aims to capture dimensions of psychopathology in a manner that is consistent with how the symptoms present in data, not as how they are expected to align on the basis of diagnostic schemes. The benefit of this approach is that, at best, the resulting models are free from bias inherent in authoritative approaches. Furthermore, researchers are not limited by the relatively few available diagnostic criteria but are instead free to study, for example, impulsivity. Horizontal, hierarchical, and general structures of psychopathology can be studied, depending on theoretical and statistical commitments.

Horizontal Nosology

NCD researchers have found latent factors underlying correlations between two or more neuropsychiatric symptom domains by using statistical methods such as exploratory factor analysis and principal components analysis (9). These latent factors are often interpreted as psychopathological constructs; a change in the latent factor is reflected as a change in the observed item scores (51). Commonly used approaches, such as exploratory factor analyses of the Neuropsychiatric Inventory (NPI) (52), represent horizontal nosologies, in which structures higher than individual latent variables are not sought. For example, a psychotic syndrome of delusions and hallucinations is at the same level of explanation as an affective syndrome of depression, anxiety, and irritability. The theoretical relevance of these models is often tied to neuroanatomical correlations of the factors (53).

A horizontal nosology based on common rating scales may produce a different view of central symptoms compared with the authoritative criteria. For example, whereas the Robert et al. (11) transdiagnostic criteria for apathy equally emphasize affective changes, changes in social interaction, and behavioral or cognitive changes, a factor analysis of a rating scale covering these domains found a main apathy factor across a diverse neuropsychiatric sample, and changes related to social interaction failed to load on this factor (54). The study’s authors suggested that probing 10 key symptoms of apathy would be sufficient and that many of the features associated with apathy are more feasibly conceived of as consequences rather than determinants of the construct.

In practice, the horizontal nosology approach has yielded relatively few consistent results, at least when using the NPI (9). Exploratory approaches have prevailed, and studies aiming to confirm theoretically meaningful structures or previous empirical findings are scarce (43). Structural solutions using the NPI may also fail to show temporal invariance (55). Heterogeneity in NPI latent-variable models may be attributed, in part, to differences in study design, analytical strategy, and sample size.

Hierarchical Nosology

Hierarchical structures of psychopathology are not yet commonly researched in NCDs (but see Nelson et al.’s [56] recent study on traumatic brain injury). This approach differs from horizontal nosology in that the structure proceeds from individual signs and symptoms to ever-broader dimensions. The Hierarchical Taxonomy of Psychopathology (HiTOP) is the most extensive effort to quantify general psychopathology (49). The HiTOP consortium has reviewed and analyzed the existing psychopathology literature to produce an evidence-based classification of psychopathology from symptoms, through syndromes and subfactors, to higher-order spectra (57). The HiTOP structure yields robust genetic and neural correlates at different levels of the hierarchy and can improve clinical decision making compared with categorical approaches (49). The HiTOP currently represents the gold standard of empirical classification of psychopathology. Whether a similar structure can be found for NCDs is an empirical question that the current data are ill-equipped to answer.

General Factor

The notion that a single factor could explain a substantial portion of the shared variance between psychiatric disorders has gained attention in psychopathology research in the past decade (58). The enthusiasm can be attributed to the finding that the general factor of psychopathology, often called the p factor, can be found in many data sets by using bifactor models. Yet, the p factor approach has not been widely used in NCD research. A 2019 study found structural evidence for the p factor in dementia; however, the limitations of the modeling approach were scarcely acknowledged (59).

Limitations

Small sample sizes have slowed progress with the empirical approach (43). Collecting clinical data of persons with NCD can be resource intensive, particularly with conditions other than Alzheimer’s disease. Correspondingly, the sample sizes used, particularly in the earlier literature, may fall short of common guidelines for structural evaluation (60). Small sample sizes may lead to unstable correlations, which, in turn, can lead to nonrobust structural solutions that often fail to align with previous studies. Comparison of structural solutions across small studies may not be informative for theoretical inferences.

For NCDs, researchers can often find some support from the literature in defense of any given factor model of the NPI. Often, neuroimaging correlations are used when inferring the theoretical meaning of NPI models. However, neuroimaging studies themselves may be limited by low statistical power and high analytical flexibility (61, 62). Additionally, studies in this field have shown preference for exploration over confirmation; at least a part of the explanation may be the general tendency for academic incentive structures to favor novelty over confirmation (63, 64). Extensive exploration has led to the creation of several competing proposals of empirical syndromes, with limited head-to-head comparisons or confirmatory evidence to support one taxonomy over another. An associated challenge for the empirical approach is that several plausible modeling strategies remain for the same data set, but the interpretation of the models may differ drastically (65). To decide on the model that most accurately represents the empirical phenomena, then, requires more than only fit indices (66).

Correspondingly, one of the major hurdles for the empirical approach is the gap between statistical constructs (e.g., a latent variable associated with delusion and psychosis subscales of a measure) and theoretical constructs (e.g., psychosis). The danger in any statistical model is that models can often be overinterpreted as theoretical rather than statistical constructs (67, 68). This issue has been a common source of criticism for p factor research, where a model that can be discovered in many types of data is seen as evidence of an important underlying construct rather than a product of methodological limitations (69). The criticism, however, is not limited to p factor models but rather applies to all situations where inferences are made about the theoretical relevance of a statistical model.

Suggestions

The empirical neuropsychiatric symptom literature can be criticized on several grounds, but the potential of this approach has not yet been fully explored, because the models have been limited by the quality of the available data. Much of the neuropsychiatric symptom data with large sample sizes come from studies where the impressive sample sizes are limited by the crudity of the rating scales assessing these symptoms. This limitation is understandable because data from pioneering research enterprises, such as the Alzheimer’s Disease Neuroimaging Initiative and National Alzheimer’s Coordinating Center have not solely focused on neuropsychiatric symptoms. In these initiatives, brief neuropsychiatric symptom rating scales (e.g., the NPI Questionnaire [70]) are often included to limit the overall burden associated with participating in the study. To address this lack of in-depth data on neuropsychiatric symptoms, a large multinational observational study of neuropsychiatric symptoms was recently proposed (71). Additionally, neuropsychiatric symptom measures that are sensitive to behavioral changes in the preclinical stages should be incorporated into future projects aiming to acquire large amounts of data (72).

Taxometric research has been largely ignored in neuropsychiatric syndrome research, despite its potential to elucidate some unresolved conceptual questions. In this line of research, statistical methods are used to investigate whether data support a categorical or dimensional model for a psychological construct (23). A recent meta-analysis found that the existing psychopathology literature supports dimensional rather than categorical models for psychopathological constructs (50). Similar evidence for dimensionality of neuropsychiatric syndromes would change the way these constructs are conceptualized. Instead of being perceived as syndromes that either are or are not present, neuropsychiatric syndromes would be seen as continuous, similar to cognitive changes, for example. Instead of assuming a general approach toward all neuropsychiatric disturbance in NCD, some of the syndromes could also feasibly be represented as categorical, which could help direct attention to the specificity of different syndromes. With more high-quality data, taxometric research could begin to define key constructs and provide a solid conceptual basis for inquiry.

The empirical approach is, then, not to be confused with dimensionality, even though general psychopathology research has found more support for dimensions than for categories for most psychiatric disorders. Perceiving syndromes as dimensional or categorical often relies on prior implicit or explicit conceptions of the syndromes (23); thus, different disciplines may disagree on the nature of the syndromes in the absence of high-quality data (or even in its presence [57]). Diagnostic guidelines in neurology and psychiatry still endorse categories, whereas psychologists and researchers with an interest in structural psychopathology research may find dimensions more appealing. Dissemination of information and collaboration across different orientations regarding the nature of syndromes may be fruitful in steering the discussion from a priori preferences toward empirically based notions.

A final practical suggestion is to improve the quality of reporting of empirical neuropsychiatric syndrome studies: the data and analytic procedures should be described in detail, the statistical approach should be justified, and the results should be interpreted in light of methodological limitations. Guidelines for reporting the results of commonly used methods in this literature are available elsewhere (73–75), and should be used by researchers, reviewers, and editors.

Discussion

This article has examined the empirical and authoritative approaches to classifying neuropsychiatric syndromes in NCDs. Both frontiers have shown marked progress, but addressing certain limitations could further accelerate advances in this field. The literature on psychiatric classification was used as a reference point because the field of psychiatry has grappled with the same conceptual questions and has influenced the classification of neuropsychiatric syndromes.

This bifurcation of classification approaches is a simple heuristic, and beneficial overlap has occurred between the two. For example, Starkstein et al. (6) modified the ICD-10 criteria for generalized anxiety disorder to better suit Alzheimer’s disease on the basis of empirical examination of sensitivity and specificity. Similarly, some rating scales reflect partially or wholly existing authoritative criteria (e.g., Mild Behavioral Impairment Checklist [76]). Concordance between criteria and rating scales not only aids in systematically assessing symptoms pertinent for the diagnostic construct but also sets conceptual boundaries for empirical syndrome research based on these scales.

After two decades of empirical classification studies, it still seems as if only the surface has been scratched. Two major opportunities remain largely unexplored. First, studies where dimensional and categorical models are compared head-to-head could elucidate whether the existing data support conceptualizing syndromes as dimensional or categorical rather than making a priori assumptions. This type of research could have drastic effects on the conceptualization of neuropsychopathology in NCDs. For example, finding that a key neuropsychiatric construct was better represented as a category would indicate that some level of psychopathological disturbance in a neuropsychiatric domain would be considered normal in the context of that NCD; only after exceeding a clinical threshold would the symptoms indicate the presence of a neuropsychiatric syndrome. In more practical terms, phrases such as “Neuropsychiatric syndrome x affects y% of individuals with Alzheimer’s disease” might start to appear in the literature, which would be more specific and informative than current reporting practices, such as “Neuropsychiatric symptoms affect over 80% of individuals with Alzheimer’s disease.” If, however, a dimensional model was deemed more appropriate, neuropsychiatric syndromes could be perceived as continuous manifestations of the NCD, much like cognitive changes. Dimensionality would contrast with the authoritative approach, which is based on the categorical model (23). In addition to creating a conceptual shift, dimensionality would bring an increase in statistical power (77), which would help advance treatment of these disturbances.

Second, despite a plea from prominent researchers in the field nearly a decade ago (72), refined data on neuropsychiatric symptoms are still scarcely available. This request was recently revived (71). It is evident that brief screening instruments, developed and validated in populations with significantly worse cognitive status compared with those currently enrolling in clinical trials and large research projects, cannot provide the evidence base that is desperately needed. Abundant data on narrow symptom instruments is available, but the full potential of the empirical approach can be realized only when large data sets with extensive data on neuropsychiatric symptom are available.

The caveats of the authoritative approach examined here could be unique to psychiatric classification or more pronounced in psychiatric classification than in neuropsychiatric classification. The two diagnostic frameworks can be distinguished by the weight that neurobiological findings are given; whereas neurobiological findings do not currently play a key role in psychiatric diagnostics, it is evident that the location and extent of neurological insult correlates with the clinical symptoms in neuropsychiatry (16, 78). The reliability of DSM diagnoses seems to be higher for disorders in which an etiology is associated with the clinical presentation, such as NCD or PTSD (41). Neuroanatomical correlates and biomarker signatures of NCD subtypes are increasingly recognized, which in turn could lead to definitions for more granular and reliable phenotypes in terms of neuropsychiatric syndromes. Whereas etiological criteria proved impossible for the DSM-5 (79), the role of biomarker profiles could be far more important for NCDs. It remains to be elucidated whether the neuropsychiatric syndrome criteria could be directly influenced by advances in neurobiology, and if so, whether the criteria would look drastically different from their current DSM-like orientation.

However, the potential for bias using the authoritative approach should be considered within the broader framework of diagnostic formulation and revision. For example, the successive iterations of the DSM have varied in the extent to which autonomy was afforded to construct-specific workgroups in the revision process (79). Additionally, it may be difficult to ascertain how much empirical research contributed to the final criteria (36, 80) and how much expertise outside the working group was utilized. The data for a revision hardly ever “reach off the table, grab you by the throat, and cry out for any one specific change” (15). Therefore, human factors play an important role in the final diagnostic formulation, irrespective of the syndrome in question or its evidence base (15, 16). Maximal transparency should help in judging which factors were considered in forming or refining diagnostic boundaries. In this respect, the most recent criteria for apathy provide ample transparency about why and how previous criteria were modified (13).

Conclusions

Research in both classification traditions accumulates quickly, and it is likely that the criteria and empirical approaches examined here will be refined shortly. Ideally, this revision and exploration would be matched, or exceeded, by testing and confirmation. Much of the conceptual landscape remains uncharted, and the trials and errors of psychiatric classification may help to avoid some pitfalls with neuropsychiatric syndromes.

Some general recommendations might be useful for future studies. First, studying neuropsychiatric syndromes and symptoms as such is valuable and highly needed. Large data sets with broad measures are needed to deepen understanding of these constructs. These studies are a requirement for structural meta-analyses, which could lead to gold-standard taxonomies in quantifying psychopathology and for analyses comparing categorical and dimensional approaches. Much of the current empirical understanding of neuropsychiatry in NCDs is limited by narrow rating scales and small sample sizes.

Second, transparency and appreciation of the expertise outside one’s own niche is required in both approaches. For authoritative approaches, extensive supportive documentation, literature reviews, and engagement with experts outside the working group could maximize the utility of the criteria and reduce the risk of the criteria becoming obsolete in a few years. For empirical approaches, specifying why a modeling approach was used and providing sufficient details for others to assess and replicate the study could encourage confirmatory research and facilitate accumulation, rather than isolation, of scientific knowledge.

Finally, the reliability of authoritative criteria in various settings needs to be examined. As the history of the DSM suggests, problems can arise from both too high and too low reliability. Operating in the dark, however, is even riskier.