Crosswalk study

In March 2015, the Alzheimer's Disease Research Centers (ADRCs) adopted a new neuropsychological test battery in which non-proprietary tests replaced similar instruments that had been used under licensing restrictions. Each new test is shown below, opposite the older test it replaced:

Old testNew test
Mini Mental State Examination (MMSE)Montreal Cognitive Assessment (MoCA)
Boston Naming TestMultilingual Naming Test (MINT)
Logical Memory, Immediate and DelayedCraft Story 21 Recall, Immediate and Delayed
Digit Span, Forward and BackwardNumber Span, Forward and Backward

Before implementing the new battery, NACC and the ADRCs conducted a crosswalk study, which had two aims: (1) to assess the correlation between the old and new tests; and (2) if the tests are highly correlated, to create a crosswalk between the previous tests and the new tests. The crosswalk study began in April 2014. A data element dictionary is available for reference. The protocol called for both the new and old batteries to be administered to a sample of subjects during the same clinic visit. The "matched" (within-subject) scores were then compared and equated between tests. The new tests listed above are now included in UDS 3; however, some Centers, at their discretion, are choosing to use the old battery for their follow-up visits. The scores of the neuropsychological tests are reported using Form C1 for the old tests and Form C2 for the new tests.

Methods

Crosswalk tables for each test were constructed following the equipercentile equating2 method. The ADRCs administered both the old and new neuropsychological test batteries on a select number of subjects (see paper for details). The order of administration of the two batteries was randomized by Center, such that the subjects from one Center started with the new battery first and subjects from another Center started with the old battery first.

Thresholds for minimum acceptable Spearman Rank Order correlation coefficient between the old and new tests were determined a priori. The minimum acceptable correlation for computing an equipercentile crosswalk table was 0.6. Equating was not advised for any tests with a correlation less than 0.6. Test pairs eligible for equating with a correlation greater than or equal to 0.6 were equated using log linear smoothing (equate package in R), and confidence intervals were calculated using 100 bootstrap samples. Retesting effects were examined; see the full paper1 for findings.

Results

Nine hundred thirty-five subjects from 24 ADRCs contributed data to the Crosswalk Study. Of these subject visits, 102 were initial visits, and the other 833 were follow-up visits. Randomization resulted in 482 subjects receiving the new battery first and 453 receiving the old battery first. Basic demographics of the sample can be found in Table 1 of the full paper1. All pairs of tests had good correlation (r=0.68 to 0.78).

Spearman correlation coefficients between each test pair:

Old testNew testρ
MMSEMoCA (raw)0.77
MMSEMoCA (education-adjusted)0.76
BNTMINT0.76
Logical Memory IA, ImmediateCraft Story 21, Immediate — paraphrase0.73
Logical Memory IIA, DelayedCraft Story 21, Delayed — paraphrase0.77
Digit Span Forward — trials correctNumber Span Forward — trials correct0.75
Digit Span Forward — lengthNumber Span Forward — length0.68
Digit Span Backward — trials correctNumber Span Backward — trials correct0.78
Digit Span Backward — lengthNumber Span Backward — length0.72

Abbreviations: BNT = Boston Naming Test, MINT = Multilingual Naming Test, MMSE = Mini Mental State Examination, MoCA = Montreal Cognitive Assessment, UDS = Uniform Data Set

Equipercentile equatings for each test pair

MoCA and MMSE

MoCA*MMSE equivalent95% CI
064, 9
197, 11
2108, 12
3119, 13
41210, 14
51210, 14
61311, 15
71412, 16
81513, 17
91513, 17
101614, 18
111715, 19
121816, 20
131917, 21
142018, 22
152119, 23
162221, 24
172322, 25
182423, 25
192524, 26
202625, 27
212726, 27
222827, 28
232828, 28
242928, 29
252929, 29
262929, 30
273030, 30
283030, 30
293030, 30
303030, 30

MINT and BNT

MINTBNT equivalent95% CI
010, 2
120, 5
231, 6
341, 7
452, 8
563, 8
663, 9
774, 10
874, 10
985, 11
1085, 11
1196, 12
1296, 12
13107, 13
14118, 13
15119, 14
16129, 15
171310, 16
181411, 16
191513, 17
201614, 18
211715, 19
221817, 20
232018, 21
242120, 22
252221, 24
262423, 25
272524, 26
282626, 27
292727, 28
302828, 29
312929, 29
323030, 30

Craft Story 21, Immediate and Logical Memory, Immediate

Craft 21, ImmediateLogical Memory, Immediate equivalent95% CI
000, 0
100, 1
210, 2
321, 3
421, 4
532, 4
643, 5
754, 6
864, 7
975, 8
1086, 9
1198, 10
12109, 11
131110, 12
141211, 13
151312, 14
161413, 15
171514, 16
181615, 17
191716, 18
201817, 19
211918, 20
222019, 21
232120, 22
242221, 23
252323, 25

Craft Story 21, Delayed and Logical Memory, Delayed

Craft 21, DelayedLogical Memory, Delayed equivalent95% CI
000, 0
111, 2
231, 4
332, 5
442, 6
553, 7
654, 7
764, 8
875, 9
986, 9
1087, 10
1198, 11
12109, 11
131110, 12
141211, 13
151312, 14
161413, 15
171515, 16
181616, 17
191817, 19
201918, 20
212019, 21
222120, 22
232221, 23
242322, 24
252423, 25

Number Span Forward and Digit Span Forward — trials correct

Number Span Forward — trails correctDigit Span Forward — trials correct equivalent 95% CI
000, 1
110, 2
221, 3
333, 4
454, 5
565, 6
676, 7
787, 8
898, 9
999, 10
101010, 11
111111, 11
121111, 12
131212, 12
141212, 12

Number Span Forward, length and Digit Span Forward, length

Number Span Forward — length scoreDigit Span Forward — length score equivalent 95% CI
343, 4
454, 5
555, 5
666, 6
777, 7
888, 8
988, 8

Number Span Backward and Digit Span Backward — trials correct

Number Span Backward — trials correctDigit Span Backward — trials correct equivalent95% CI
000, 1
111, 2
222, 3
333, 3
444, 4
555, 5
666, 6
777, 7
888, 8
999, 9
101010, 10
111110, 11
121211, 12
131212, 12
141212, 13

Number Span Backward and Digit Span Backward — length

Number Span Backward — length scoreDigit Span Backward — length score equivalent95% CI
000, 1
222, 3
333, 3
444, 4
555, 5
666, 6
777, 7
877, 7

References

  1. Monsell SE, Dodge HH, Zhou XH, Bu Y, Besser LM, Mock C, Hawes SE, Kukull WA, Weintraub S, Neuropsychology Work Group Advisory to the Clinical Task Force. Results from the NACC Uniform Data Set neuropsychological battery Crosswalk Study. Alzheimer Dis Assoc Disord, 2:134-139, 2016, PMC4834278
  2. Kolen MJ, Brennan RL. Test Equating. New York, NY: Springer; 1995.