Article Text
PDF
Abstract
Objective To evaluate visual assessment of [11C]PIB and [18F]FDDNP PET images as potential supportive diagnostic markers for Alzheimer's disease (AD).
Methods Twenty-one AD patients and 20 controls were included. Parametric [11C]PIB and [18F]FDDNP global binding potential (BPND) images were visually rated as ‘AD’ or ‘normal.’ Data were compared with ratings of [18F]FDG PET images and MRI-derived medial temporal lobe atrophy (MTA) scores. Inter-rater agreement and agreement with clinical diagnosis were assessed for all imaging modalities. In addition, cut-off values for quantitative global [11C]PIB and [18F]FDDNP BPND were determined. Visual ratings were compared with dichotomised quantitative values.
Results Agreement between readers was excellent for [11C]PIB, [18F]FDDNP and MTA (Cohen kappa κ≥0.85) and moderate for [18F]FDG (κ=0.56). The highest sensitivity was found for [11C]PIB and [18F]FDG (both 1.0). The highest specificity was found for MTA (0.90) and [11C]PIB (0.85). [18F]FDDNP had the lowest sensitivity and specificity (0.67 and 0.53, respectively). The cut-off for quantitative [11C]PIB BPND was 0.54 (sensitivity and specificity both 0.95) and for [18F]FDDNP BPND 0.07 (sensitivity 0.80, specificity 0.73). Agreement between quantitative analyses and visual ratings was excellent for [11C]PIB (κ=0.85) and fair for [18F]FDDNP (κ=0.40).
Conclusion Visual assessment of [11C]PIB images was straightforward and accurate, showing promise as a supportive diagnostic marker for AD. Moreover, [11C]PIB showed the best combination of sensitivity and specificity. Visual assessment of [18F]FDDNP images was insufficient. The quantitative analysis of [18F]FDDNP data showed a considerably higher diagnostic value than the visual analysis.
- Alzheimer's disease
- PET, ligand studies
Statistics from Altmetric.com
Introduction
Recently, several PET tracers have been developed for visualising Alzheimer's disease (AD) pathology directly. Of these ligands, [11C]PIB (Pittsburgh Compound-B)1 and [18F]FDDNP (2-(1-{6-[(2-[F-18]fluoroethyl)(methyl)amino]-2-aphthyl} ethylidene) malononitrile)2 have been used most widely. Both tracers have shown the ability to distinguish AD patients from controls at a group level.1–4 However, little is known about the accuracy and reliability of visual assessment of images acquired using these new biomarkers. The aim of the present study was to compare a visual assessment of [11C]PIB and [18F]FDDNP images as potential supportive diagnostic markers for AD. Results were compared with those of visual assessment of decreased cerebral glucose metabolism [18F]FDG and with medial temporal lobe atrophy (MTA) using MRI.5 In addition, a visual assessment of [11C]PIB and [18F]FDDNP images was compared with quantitative assessment of global binding.
Methods
Subjects
Twenty-one AD patients (mean±SD 63±6 years) and 20 controls (67±6 years) were included in this study. Global and regional [11C]PIB and [18F]FDDNP binding in a largely overlapping sample have been reported elsewhere.4 [11C]PIB and MRI scans were available for all subjects. [18F]FDDNP PET scans were not available for five AD patients and five controls, and [18F]FDG scans were not available for five controls.
Assessment protocol and PET and MRI scanning protocols have been described previously4 and can be found as supplementary material online.
Visual readings
All scans were presented to the readers in a randomised order. All [11C]PIB, [18F]FDDNP and [18F]FDG scans were rated separately by two readers (BvB and PR for [11C]PIB and [18F]FDDNP; BvB and KH for [18F]FDG) and classified as either AD or normal (ie, unlikely to be AD). For rating of [18F]FDG, readers had access to results of the Alzheimer's discrimination tool,6 but the final decision was based on their own assessment. Readers of PET scans were nuclear medicine physicians (BB, PR) and a neurologist (KH) with expertise in PET imaging and they were blinded to clinical information. The results of individual readers were compared, and discrepancies were discussed to establish consensus.
Atrophy rating on T1 weighted MRI was performed separately by two trained neuroradiologists (FB and MW), who were blinded to clinical information but had knowledge of the age of the subject. MTA was rated visually on the oblique/coronal images using a scale ranging from 0 (no atrophy) to 4 (severe atrophy).5 Averaged MTA scores were dichotomised according to age. MTA scores were considered as abnormal depending on age (years), that is, ≥1 for subjects <65, ≥1.5 for subjects ≥65 and <75, and ≥2 for subjects ≥75 years. MTA ratings of the most experienced reader (FB) were used for further analysis, as consensus was not available.
Statistical analysis
Data are presented as mean±SD, unless otherwise stated. Frequency distributions for gender were compared using the χ2 test. Differences between the two groups were assessed using analysis of variance (ANOVA) with age and gender as covariates.
To assess the consistency of visual assessments, the Cohen Kappa (κ)7 for agreement between readers was calculated for all imaging modalities. Sensitivity, specificity, likelihood ratios and accuracy were calculated for visual assessment of [11C]PIB, [18F]FDDNP and [18F]FDG images, and for MTA. Clinical diagnosis was used as reference criterion.
Next, agreement between visual assessment of [11C]PIB and [18F]FDDNP images and corresponding quantitative values of global binding potential (BPND) was assessed. To this end, receiver-operating-characteristic (ROC) curves were generated to determine optimal cut-off values for global [11C]PIB and [18F]FDDNP BPND. Optimal cut-off values were defined as those values that yielded at least 80% sensitivity, with accompanying specificity for detecting AD,8 using clinical diagnosis as reference criterion. Based on these ROC defined cut-off values, quantitative BPND values of all subjects were classified as ‘normal’ or ‘abnormal,’ and agreement between quantitative and visual assessments was assessed using the Cohen κ.
Results
Demographic and clinical characteristics according to diagnostic group, together with examples of normal and AD-like images for all modalities, are provided as supplementary material online (table A and figure A).
Agreement between readers
Agreement between visual readings was excellent for [11C]PIB (κ=0.85), [18F]FDDNP (κ=0.87) and MTA (κ=0.90) and moderate for [18F]FDG (κ=0.56).
Agreement with clinical diagnosis
The sensitivity, specificity, LR+, LR– and accuracy for visual assessment of all imaging modalities are presented in table 1. [11C]PIB and [18F]FDG had the highest sensitivity (all patients were identified correctly). The lowest sensitivity was obtained for [18F]FDDNP and MTA. The highest specificity was found for MTA and [11C]PIB. [18F]FDG and [18F]FDDNP had a low specificity. Sensitivity and specificity were reflected in LR as the highest LR+ for MTA and best LR– for both [11C]PIB and [18F]FDG. The highest accuracy was found for [11C]PIB, while it was lowest for [18F]FDDNP. MTA and [18F]FDG have a similar accuracy.
Sensitivity, specificity, likelihood ratios and accuracy for visual ratings across Alzheimer's disease patients and controls
Agreement with quantitative analysis
ROC curves were generated to determine optimal cut-off values for both ligands (for ROC curves, see supplementary material online, figure B). The [11C]PIB cut-off value was determined at a BPND of 0.54. The area under the curve (AUC) was 0.96 (95% CI 0.90 to 1.03), with both a sensitivity and specificity of 0.95. These values differed only slightly from the sensitivity and specificity of visual ratings. For global [18F]FDDNP binding, the optimal cut-off value was found to be at a BPND of 0.07, resulting in a sensitivity of 0.80, a specificity of 0.73 and an AUC (95% CI) of 0.85 (0.71 to 0.99), which were considerably higher than the corresponding values found with visual ratings.
Agreement of quantitative assessment between both ligands is illustrated in figure 1, where the reference lines indicate cut-off values of both tracers. Agreement between quantitative analysis and visual rating was excellent for [11C]PIB (κ=0.85) and fair for [18F]FDDNP (κ=0.40).
Agreement between the quantitative assessment of global [11C]PIB and [18F]FDDNP global binding potential (BPND) with [18F]FDDNP binding on the x-axis and [11C]PIB binding on the y-axis. Reference lines represent cut-off values. The right upper quadrant contains subjects with relatively high binding for both ligands, while the left lower quadrant displays subjects with relatively low binding for both ligands. Alzheimer's disease (AD) patients are represented by circles, and controls by squares. For [11C]PIB, all AD patients showed a high global BPND, while only one control subject showed binding above the cut-off value. For [18F]FDDNP, three AD patients showed global binding below and four controls global binding above the cut-off value.
Discussion
This study evaluated visual assessment of [11C]PIB and [18F]FDDNP images as potential supportive diagnostic markers for AD. Visual assessment of [11C]PIB images showed a high diagnostic accuracy combined with a high interobserver agreement. Furthermore, agreement between visual and quantitative assessment of [11C]PIB was high. Moreover, in the present cohort, visual rating of [11C]PIB for identification of AD performed equally well as the combination of [18F]FDG (high sensitivity) and MTA (high specificity). The diagnostic accuracy of [11C]PIB was even higher than that of other markers. Visual rating of [18F]FDDNP images for identification of AD had the lowest sensitivity, specificity and accuracy. Additionally, agreement with quantitative assessment was only fair.
The findings are in line with a previously published study9 evaluating the visual assessment of [11C]PIB and [18F]FDG, reporting a similar agreement between readers for both modalities and similar sensitivity and specificity for [11C]PIB.
The low accuracy of visual assessment of [18F]FDDNP may be due to the fact that the difference in [18F]FDDNP BPND between AD patients and controls is subtle,2–4 therefore hampering visual assessment. Quantitative assessment of [18F]FDDNP scans was considerably better in identifying AD and could therefore still be a potential tool in diagnosing dementia. This is especially true, as [18F]FDDNP is the only PET ligand available to date which also has an affinity for tangle pathology.10
Post-mortem studies report presence of AD pathology in around 30% of cognitively healthy elderly subjects11 and other imaging studies report increased [11C]PIB uptake in around 30% of cognitively normal elderly subjects.12–14 A visual assessment of [11C]PIB images in the present study rated three out of 20 cognitively healthy elderly subjects as abnormal. This relatively low percentage, however, is likely to be due to the relatively young age of subjects in the present study.
These control subjects with abnormal [11C]PIB images may be preclinical AD patients, as deposition of pathology is thought to start a decade before cognitive impairment arises.15 Alternatively, deposition of amyloid could have a more benign character, not leading to clinical AD. To verify whether these subjects are indeed preclinical cases, longitudinal follow-up studies will be required. Assuming that these subjects are indeed preclinical rather than false-positive cases, specificity for the other imaging modalities would likely also have been affected, resulting in falsely low specificity estimates.16
A limitation of the study is that readings of imaging modalities were dichotomised. [18F]FDG especially provides a graded answer, and forcing it into just two alternatives does not lead to the use of its full potential. Therefore, the diagnostic accuracy of [18F]FDG in the present study might be lower than the diagnostic value in clinical practice.
The present study contributes to the discussion on new supportive biomarkers for diagnosing AD,17 as it provides a comparison of the relative sensitivity and specificity of [11C]PIB, [18F]FDDNP, [18F]FDG and MTA for visual classification of scans as AD or normal when readers are blind to clinical diagnosis. It should be kept in mind, however, that the values found for sensitivity, specificity, likelihood ratios and accuracy are only an estimation of performance, as they are affected by both size and composition of the study sample. Nevertheless, as all imaging modalities were compared using the same sample, this should still provide a fair assessment of their relative specificity.
References
Footnotes
Funding This work was financially supported by the Internationale Stichting Alzheimer Onderzoek (ISAO, grant 05512), and the American Health Assistance Foundation (AHAF, grant A2005-026).
Competing interests PS serves/has served as Associate Editor of the Journal of Neurology, Neurosurgery & Psychiatry.
Ethics approval Ethics approval was provided by the Medical Ethics Review Committee of the VU University Medical Centre Amsterdam.
Provenance and peer review Not commissioned; externally peer reviewed.