Designing Brief Alzheimer’s Screening Tests for Use in General Medical Practice

Presenters at this symposium, held at the American Association for Geriatric Psychiatry’s 2003 Annual Meeting, explored the theoretical, practical, and methodologic issues in designing screening tests for Alzheimer’s disease. In addition, statistical issues related to the design of screening tests and the interpretation of their results were discussed.

The Memory Impairment Screen

Memory impairment usually is the earliest sign of Alzheimer’s disease (AD) and therefore is required and sufficient for diagnosis, according to Herman Buschke, MD, Professor of Neuroscience, Lena and Joseph Gluck Distinguished Scholar in Neurology at the Albert Einstein College of Medicine, New York City. Dr. Buschke described the Memory Impairment Screen (MIS), which consists of controlled learning, a short delay, and free recall, followed by cued recall for items that were not
previously recalled (Buschke, et al. Neurology.1999;52:231). The MIS is recommended by the American Academy of Neurology as a screen for AD.

Subjects are given the names of items in four different categories (animal, city, vegetable, and musical instrument). After a short delay, the subjects are asked to name the item in each category. Such controlled learning assures attention and equal processing of all items, induces deep semantic processing, and provides sufficient time to process each item.

The MIS also demonstrates that individuals can identify the items by their cues and requires all subjects to do the same processing. This ensures that decreased recall is due to impaired memory, not impaired attention or different processing strategies. The MIS induces encoding specificity to improve retrieval by using the same cues for learning and retrieval. This enhances the detection of dementia.

Receiver operating characteristic (ROC) curves, which plot detection of true positives (sensitivity) against detection of false positives (1–specificity), showed high levels of discrimination with the MIS for dementia and AD (Figure 1). The MIS had greater specificity and sensitivity for free recall than the uncontrolled three-word free recall test from the Mini-Mental State Examination. Thus, the MIS improves specificity and discrimination of memory impairment by using controlled learning and optimizes encoding specificity to maximize cued recall.

Dr. Buschke pointed out that most screening tests detect clinical (i.e., existing) dementia, but the real challenge is to detect preclinical dementia. He noted that the accelerated phase of memory decline begins 5–7 years before diagnosis (Hall. Stat Med. 2001;20:3695). These individuals may have normal screening scores but still have declining memory. To address this problem, early screening, using features such as speed of recall or items recalled, is necessary, particularly if drugs are to be used to slow down or stop disease progression.

The Mini-Cog: A Brief Dementia Screening Test for Primary Care

Soo Borson, MD, of the University of Washington, Department of Psychi-atry and Behavioral Sciences in Seattle, described the Mini-Cog test, a 3-minute instrument to screen for cognitive impairment in older adults in the primary care setting. Primary care physicians (PCPs) often do not screen patients because the tests take too long, the PCPs lack confidence in the method or in their own skills, or they are concerned about offending patients. Moreover, most screening tests do not recognize executive dementias, which impair functional abilities out of proportion to memory.

The Mini-Cog uses a three-item recall test for memory and a simply scored clock-drawing test. The latter serves as an “informative distractor,” helping to clarify scores when the memory recall score is intermediate (Figure 1). The Mini-Cog was as effective as or better than established screening tests in both an epidemiologic survey in a mainstream sample and a multi-ethnic, multilingual population comprising many individuals of low socioeconomic status and education level (Borson, et al; Ganguli, et al; ISS, Rome).

Among older adults without dementia, the Mini-Cog misidentified some individuals as impaired (false positive), whereas PCPs misidentified only one patient. However, the percentages of individuals correctly identified by PCPs at different stages of cognitive impairment from the mildest detectable (Clinical Dementia Rating of 0.5) to mild and moderate dementia (stages 1 and 2) were 3%, 40%, and 60%, respectively. In contrast, the Mini-Cog was significantly (P < 0.001) better than PCPs in recognizing the early stages of dementia. Moreover, the Mini-Cog was significantly (P < 0.01) better than PCPs in detecting dementia among minority patients, particularly those who did not speak English or had low levels of education. Furthermore, the Mini-Cog’s performance ranged from 85% to 100% across the spectrum of dementia diagnoses, possibly because the Mini-Cog includes a screen for executive dysfunction as well as memory. In contrast, PCPs recognized dementia in 60% of subjects with a research diagnosis of Alzheimer’s disease, but were considerably less effective in detecting other forms of dementia.

The Mini-Cog may improve early recognition of cognitive impairment in primary care subjects. In comparative tests, the Mini-Cog was at least twice as fast as the Mini-Mental State Examin-ation. The Mini-Cog is less affected by subject ethnicity, language, and education, and can detect a variety of different dementias. Moreover, the Mini-Cog detects many people with mild cognitive impairment (cognitive impairment too mild to meet diagnostic criteria for dementia). Combining the Mini-Cog with informant questions may improve discriminant power, stated Dr. Borson.

The DrD Quick & Easy Dementia Screening Test

Paul Dash, MD, Assistant Clinical Professor of Neurology, Medical School of Louisiana State University of Health Sciences in New Orleans, reported that the DrD Quick & Easy (Q&E) Dementia Screening Test consists of four components: encoding (subject asked to repeat three paired items); temporal orientation (subject must report date, including month, year); verbal fluency (name animals in 1 minute); and uncued recall (subject must name the three paired items that were mentioned earlier).

The three paired items stress the memory system more than do the three single items of the Mini-Mental State Examination (MMSE). Additionally, the time from encoding to recall is longer in the Q&E than in the MMSE. Thus, the Q&E is a more specific test of recall than the MMSE, stated Dr. Dash.

In the Q&E, subjects repeat the items twice to ensure coding and are allowed to hear the items repeated once, to compensate for hearing impairment. The verbal fluency task is allotted a standard time of 1 minute, whereas in
the MMSE, time to recall may change according to the task requested. Furthermore, the distractor items in the MMSE are educationally sensitive, but the “animals-in-a-minute” task is not. Unlike in the MMSE, subjects cannot fail the memory task of the Q&E and still pass the entire test.

Both the Q&E and the MMSE detected dementia in patients with a previous diagnosis of dementia. Among patients referred for evaluation of cognitive impairment, the Q&E detected a few more subjects with cognitive impairment. The Q&E also detected more cases of mild Alzheimer’s disease (AD) than the MMSE among patients referred for evaluation of another problem but who had some noticeable difficulty with memory. In 66 patients referred for another problem but who denied any memory problems (“control” group), the MMSE (using a cut score of  26 as abnormal) misidentified 24 normal subjects, whereas the Q&E misidentified only 3 as abnormal. Thus, the Q&E has a higher specificity than the MMSE. A lower cut score of the MMSE enhanced its specificity, but at the cost of sensitivity. With all subjects, the Q&E was quicker to administer than the MMSE.

Dr. Dash stated that the ideal screening test for dementia should be easily administered and fast, with high specificity and sensitivity, particularly for the early stages of dementia. The test also should not be affected by the patient’s language or education level. Dr. Dash believes that the Q&E test meets many of these criteria, but needs further validation in blinded, head-to-head comparisons with other screening tests.

Further information and copies of the test can be downloaded free of charge at http://www.medschool.lsuhsc.edu/
neurology/Dash_QE.htm.

Designing a Brief Alzheimer Screen

Marta Mendiondo, PhD, of the Sanders-Brown Center on Aging at the University of Kentucky, Lexington, stated that screening tests for cognitive function should be quick, easy to apply, and be able to identify patients with mild dementia and those who need further evaluation. Dr. Mendiondo presented the Brief Alzheimer Screen (BAS), a screening tool for discriminating subjects with mild Alzheimer’s disease (AD) from normal individuals that is shorter and easier to apply than the Mini-Mental State Examination (MMSE). The BAS includes the following tasks from the MMSE: the three memory items (R), date orientation (D), and spelling “world” backwards (S). The BAS also includes the task of naming animals in 30 seconds (A).

Analysis Method
Mendiondo et al. retrospectively designed the BAS using existing entry visit data from the Consortium to Establish a Registry for AD (CERAD). A mild AD group, comprising 342 patients with probable AD (MMSE entry scores > 20) at all visits was compared with a control group, which consisted of 406 normal individuals with a nondementia diag-
nosis (MMSE entry scores > 25) at all visits. Data on the category fluency also were recorded for the AD group.

The AD and control groups were randomly divided into either the derivation or the validation subgroups. A backward logistic regression analysis was performed on the derivation group’s data using all of the MMSE items and the Animal30 task scores. Estimates also were made of the percentages of false negative and false positive findings
according to different cutpoints.

The four most significant (P < 0.001) items were then used in combination to distinguish patients with mild or early AD from control individuals and to derive sensitivity and specificity rates, as well as initial cutpoints. The resulting equation (Figure 1) was applied to the validation subgroup to estimate the percentages of false negative and false positive findings according to different cutpoints.

Mendiondo et al. also used the BAS to compare first visit data for individuals in the database of the University of Kentucky Alzheimer’s Disease Research Center (UK ADRC). The researchers compared data for 504 patients who had probable or possible AD (CDR score 0.5 or 1) at all visits with 70 nondemented patients (CDR score 0.5 or 1) who had memory problems but no mild cognitive impairment, and with a control group of 658 normal individuals. Category fluency data were recorded for the AD and nondemented groups. The receiver operating characteristic (ROC) curves obtained were used to retrospectively compare the performance of the BAS and the MMSE.

Results of the BAS and MMSE in Patients with Probable or Possible AD
The AD patients separated well from the control groups in both the CERAD and UK ADRC datasets. The rates of false positive and false negative findings depended on the BAS cutpoints chosen. With the CERAD dataset validation subgroup, the rates of false positive and false negative findings were 1% and 10%, respectively, at cutpoints > 22 versus > 23. At cutpoints > 26 versus > 27, the rates were 13% and 1% for false positive and false negative findings, respectively. A similar trend was observed with the UK ADRC data.

Although the BAS and MMSE were less effective when comparing AD patients with clinically nondemented individuals, both tests were similar in terms of sensitivity and specificity when comparing subjects with mild AD and control subjects. Thus, these data suggest that the BAS is a short, easy to apply screening tool that can discriminate patients with mild AD from normal individuals.

Statistical Issues in Evaluating Primary Care Screening Tests for Alzheimer’s Disease

Helena C. Kraemer, PhD, Professor of Biostatistics in the Department of Psychiatry and Behavioral Sciences at Stanford University, Palo Alto, California, reviewed important statistical issues regarding the design and interpretation of screening tests for Alzheimer’s disease (AD).

Dr. Kraemer pointed out that statistical significance (P value) indicates only that an observed effect is nonrandom. “The P value is necessary but it doesn’t tell you anything about effect size or whether a test is useful,” stated Dr. Kraemer. In addition, evaluating a test without bias requires a representative sample from the population to which the test will be applied. Sensitivity and specificity may be grossly exaggerated if case-control studies or enriched samples are used.

No screening test is perfect; thus, some persons with AD will not be detected (false negative result) while others will be incorrectly identified as having AD when they do not (false positive result). The false positive results are particularly troubling because they result in unnecessary treatment and potential emotional trauma.

The cost of testing is influenced by several parameters: some objective, such as the proportion of true positive and false positive results, and some subjective, such as the cost of the intervention, iatrogenic damage, and the clinical benefit gained by individuals with true positive results.

The kappa-1, a measure of the quality of a screening test, seeks to optimize sensitivities and to maximize the predictive value of a negative result. In a screening test, a positive result should lead to further testing, not initiation of treatment. Thus, the iatrogenic costs would be almost zero.

Dr. Kraemer recommended that screening tests be as low cost as possible and should be administered only to individuals at reasonably high risk of AD. Screening tests are not diagnostic tests; rather, they select individuals for further (diagnostic) testing.

Statistical methods should take harm and benefit into consideration, and effect size should be emphasized more than statistical significance.