Regional distribution of measurement error in diffusion tensor imaging

doi:10.1016/j.pscychresns.2006.01.008

Psychiatry Research: Neuroimaging

Volume 147, Issue 1, 30 June 2006, Pages 69-78

https://doi.org/10.1016/j.pscychresns.2006.01.008 Get rights and content

Abstract

The characterization of measurement error is critical in assessing the significance of diffusion tensor imaging (DTI) findings in longitudinal and cohort studies of psychiatric disorders. We studied 20 healthy volunteers, each one scanned twice (average interval between scans of 51 ± 46.8 days) with a single shot echo planar DTI technique. Intersession variability for fractional anisotropy (FA) and Trace (D) was represented as absolute variation (standard deviation within subjects: SDw), percent coefficient of variation (CV) and intra-class correlation coefficient (ICC). The values from the two sessions were compared for statistical significance with repeated measures analysis of variance or a non-parametric equivalent of a paired t-test. The results showed good reproducibility for both FA and Trace (CVs below 10% and ICCs at or above 0.70 in most regions of interest) and evidence of systematic global changes in Trace between scans. The regional distribution of reproducibility described here has implications for the interpretation of regional findings and for rigorous pre-processing. The regional distribution of reproducibility measures was different for SDw, CV and ICC. Each one of these measures reveals complementary information that needs to be taken into consideration when performing statistical operations on groups of DT images.

Introduction

Diffusion tensor imaging (DTI) has been evolving rapidly and gaining popularity in psychiatric research. There has been a rapid increase in publications, particularly in the field of schizophrenia where eight original papers were published between 1998 and 2002 and 23 between 2003 and May 2005.

Despite this, the measurement error of this technique has not been fully characterized. The information regarding measurement error is critical in assessing the significance of DTI findings in longitudinal studies and in comparing patient groups. This is particularly true in the case of psychiatric disorders where differences from controls may be quite subtle.

Pfefferbaum et al. (2003) were the first to address reproducibility of FA and Trace images in detail. They studied normal controls three times with a minimum time interval between scans of 1 day. They reported coefficients of variation (CVs) between 1.23% and 2.35% for FA and of 0.84–3.73% for Trace. Their analysis was based either on large collections of voxels (such as all the voxels in the white matter of the supratentorium) or on a single large region of interest (ROI) placed over the entire corpus callosum on a midline slab of tissue with thickness of 5 mm. Thus, the estimates of reproducibility and measurement error derived from this study are likely to be quite liberal as compared with common approaches to DTI data analysis which would include either smaller ROIs or voxel-by-voxel approaches with programs such as statistical parametric mapping (SPM). Moreover, this study did not provide any information on the regional distribution of measurement error, which is important since reproducibility may not be equal across the whole image due to the complex statistical properties of the calculated DTI measures and many other factors. For example, recently claims were made regarding differences in anisotropy between patients with schizophrenia and normal controls in small ROIs of gray matter in the entorhinal cortex (Kalus et al., 2005) and the hippocampus (Kalus et al., 2004). How are we to judge the strength of these findings without knowing if the reproducibility of gray matter ROIs is similar to that found in the corpus callosum?

Also Kubicki et al. (2004) studied the reproducibility of single shot planar imaging (EPI) with ROIs applied to four scans (acquired in separate sessions, with unspecified time interval between sessions) of the same subject. The CV varied between 1.4% and 12% for various regions of white matter and reached 25% for a gray matter ROI. However, the EPI acquisition employed was highly susceptible to artifacts due to the use of a fairly long echo time and the lack of corrections for eddy current distortions. Moreover, the scans were acquired with gaps in between slices, and no effort to register scans acquired on different sessions was described. Hence, the estimates of measurement error presented in this study are likely to be conservative when compared with what would be obtained with more state-of-the-art acquisition and processing schemes.

The gap in knowledge left by these two studies provided the rationale for the current study, where we tested two methods commonly employed in statistical analysis of DT images (ROIs and SPM-based techniques) and described the regional variation in FA and Trace (D) (henceforth referred to as Trace) associated with each strategy.

Section snippets

Methods

We studied 20 healthy volunteers (ages 21–36, mean 26 ± 4.4 SD, three females), each one scanned twice, with an average interval between scans of 51 ± 46.8 days. All scans were performed with the head immobilized by a vacuum cushion. All subjects gave written informed consent according to procedures approved by the NIMH institutional review board.

DTI sessions were conducted on a 1.5 T GE Signa magnet (Waukesha, WI) and consisted of an axial single shot echo planar imaging (EPI) sequence with six

Results

Typical images of FA and Trace obtained during this study are shown in Fig. 2. SNR, SDw, CV and ICC for mean ROI values for FA and Trace are shown in Table 1. A notable regional variation in reproducibility was seen. For FA, the SDw was highest in the Peduncles, PLIC and CSO, and lowest in Put, GCC, FrG, Thal and Ins with values in the Peduncles being twice those of the Put. The CV was highest in the CblCtx and the FrG and lowest in the GCC and the SCC. Nine out of 14 ROIs had CVs below 10%.

Discussion

Characterizing regional variation in measurement error for DTI is important to the interpretation of the results of group comparisons and longitudinal studies. In this article, we present three different ways of looking at measurement error as they apply to ROI measurements and to voxel-by-voxel analysis. We used different measures of reproducibility because they reveal different and complementary information about the regional distribution of measurement error.

Acknowledgments

Some of the material in this article was presented at the 42nd Annual Meeting of the American College of Neuropsychopharmacology (ACNP) in San Juan, Puerto Rico, in 2003 and at the International Society of Magnetic Resonance in Medicine (ISMRM) “Workshop on Methods for Quantitative Diffusion MRI of Human Brain” in Lake Louise, Alberta, Canada, in 2005.

Gustavo K. Rohde is currently a National Research Council Research Associate at the Naval Research Laboratory, Washington, DC.

We thank Andreas

References (18)

P.J. Basser et al.
Microstructural and physiological features of tissues elucidated by quantitative-diffusion-tensor MRI
Journal of Magnetic Resonance, B
(1996)
M.E. Bastin et al.
A theoretical study of the effect of experimental noise on the measurement of anisotropy in diffusion imaging
Magnetic Resonance Imaging
(1998)
O. Ciccarelli et al.
From diffusion tractography to quantitative white matter tract measures: a reproducibility study
Neuroimage
(2003)
P. Kalus et al.
New evidence for involvement of the entorhinal region in schizophrenia: a combined MRI volumetric and DTI study
Neuroimage
(2005)
M. Kubicki et al.
Comparison of single-shot echo-planar and line scan protocols for diffusion tensor imaging
Academy of Radiology
(2004)
A.T. Toosy et al.
Characterizing function-structure relationships in the human visual system with functional MRI and diffusion tensor imaging
Neuroimage
(2004)
J.J. Bartko et al.
On the methods and theory of reliability
Journal of Nervous and Mental Diseases
(1976)
J.M. Bland et al.
Measurement error
British Medical Journal
(1996)
E. Cassol et al.
Diffusion tensor imaging in multiple sclerosis: a tool for monitoring changes in normal-appearing white matter
Multiple Sclerosis
(2004)

There are more references available in the full text version of this article.

Cited by (67)

Diffusion tensor imaging within the healthy cervical spinal cord: Within- participants reliability and measurement error
2024, Magnetic Resonance Imaging
Diffusion tensor imaging (DTI) is a promising technique for the visualization of the cervical spinal cord (CSC) in vivo. It provides information about the tissue structure of axonal white matter, and it is thought to be more sensitive than other MR imaging techniques for the evaluation of damage to tracts in the spinal cord.
The purpose of this study was to determine the within-participants reliability and error magnitude of measurements of DTI metrics in healthy human CSC.
A total of twenty healthy controls (10 male, mean age: 33.9 ± 3.5 years, 10 females, mean age: 47.5 ± 14.4 years), with no family history of any neurological disorders or a contraindication to MRI scanning were recruited over a period of two months. Each participant was scanned twice with an MRI 3 T scanner using standard DTI sequences. Spinal Cord Toolbox (SCT) software was used for image post-processing. Data were first corrected for motion artefact, then segmented, registered to a template, and then the DTI metrics were computed. The within-participants coefficients of variation (CV%), the single and average within-participants intraclass correlation coefficients (ICC) and Bland-Altman plots for WM, VC, DC and LC fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity (AD), and radial diffusivity (RD) were determined for the cervical spinal cord (between the 2nd and 5th cervical vertebrae).
DTI metrics showed poor to excellent within-participants reliability for both single and average ICC and moderate to high reproducibility for CV%, all variation dependent on the location of the ROI. The BA plots showed good within-participants agreement between the scan-rescan values.
Results from this reliability study demonstrate that clinical trials using the DTI technique are feasible and that DTI, in particular regions of the cord is suitable for use for the monitoring of degenerative WM changes.
Normative values of the topological metrics of the structural connectome: A multi-site reproducibility study across the Italian Neuroscience network
2023, Physica Medica
The use of topological metrics to derive quantitative descriptors from structural connectomes is receiving increasing attention but deserves specific studies to investigate their reproducibility and variability in the clinical context. This work exploits the harmonization of diffusion-weighted acquisition for neuroimaging data performed by the Italian Neuroscience and Neurorehabilitation Network initiative to obtain normative values of topological metrics and to investigate their reproducibility and variability across centers.
Different topological metrics, at global and local level, were calculated on multishell diffusion-weighted data acquired at high-field (e.g. 3 T) Magnetic Resonance Imaging scanners in 13 different centers, following the harmonization of the acquisition protocol, on young and healthy adults. A “traveling brains” dataset acquired on a subgroup of subjects at 3 different centers was also analyzed as reference data. All data were processed following a common processing pipeline that includes data pre-processing, tractography, generation of structural connectomes and calculation of graph-based metrics.
The results were evaluated both with statistical analysis of variability and consistency among sites with the traveling brains range. In addition, inter-site reproducibility was assessed in terms of intra-class correlation variability.
The results show an inter-center and inter-subject variability of <10%, except for “clustering coefficient” (variability of 30%). Statistical analysis identifies significant differences among sites, as expected given the wide range of scanners’ hardware.
The results show low variability of connectivity topological metrics across sites running a harmonised protocol.
Scan-rescan repeatability of axonal imaging metrics using high-gradient diffusion MRI and statistical implications for study design
2021, NeuroImage
Citation Excerpt :
Our findings are in line with previous smaller-scale studies of scan-rescan repeatability of axon diameter estimation using ActiveAx in the human and Vervet monkey brains (Alexander et al., 2010; Dyrby et al., 2013), in which broad trends in axon diameter index across the corpus callosum were shown to be quite reproducible, although individual voxel-wise estimates were shown to have greater variability. Of note, many previous studies have assessed the reproducibility of FA measures in the healthy population (Besseling et al., 2012; Boekel et al., 2017; Grech-Sollars et al., 2015; Koller et al., 2021; Lin et al., 2015; Marenco et al., 2006), with reported ICC scores that fall within a range of ~0.7 to 0.98. Our data also yielded ICC scores in the same range for FA, allowing us to compare our results to the existing literature and supporting the consistency of our data and analyses with prior studies.
Axon diameter mapping using diffusion MRI in the living human brain has attracted growing interests with the increasing availability of high gradient strength MRI systems. A systematic assessment of the consistency of axon diameter estimates within and between individuals is needed to gain a comprehensive understanding of how such methods extend to quantifying differences in axon diameter index between groups and facilitate the design of neurobiological studies using such measures. We examined the scan-rescan repeatability of axon diameter index estimation based on the spherical mean technique (SMT) approach using diffusion MRI data acquired with gradient strengths up to 300 mT/m on a 3T Connectom system in 7 healthy volunteers. We performed statistical power analyses using data acquired with the same protocol in a larger cohort consisting of 15 healthy adults to investigate the implications for study design. Results revealed a high degree of repeatability in voxel-wise restricted volume fraction estimates and tract-wise estimates of axon diameter index derived from high-gradient diffusion MRI data. On the region of interest (ROI) level, across white matter tracts in the whole brain, the Pearson's correlation coefficient of the axon diameter index estimated between scan and rescan experiments was r = 0.72 with an absolute deviation of 0.18 μm. For an anticipated 10% effect size in studies of axon diameter index, most white matter regions required a sample size of less than 15 people to observe a measurable difference between groups using an ROI-based approach. To facilitate the use of high-gradient strength diffusion MRI data for neuroscientific studies of axonal microstructure, the comprehensive multi-gradient strength, multi-diffusion time data used in this work will be made publicly available, in support of open science and increasing the accessibility of such data to the greater scientific community.
Longitudinal Reproducibility of Neurite Orientation Dispersion and Density Imaging (NODDI) Derived Metrics in the White Matter
2021, Neuroscience
Diffusion-weighted magnetic resonance imaging (DWI) is undergoing constant evolution with the ambitious goal of developing in-vivo histology of the brain. A recent methodological advancement is Neurite Orientation Dispersion and Density Imaging (NODDI), a histologically validated multi-compartment model to yield microstructural features of brain tissue such as geometric complexity and neurite packing density, which are especially useful in imaging the white matter. Since NODDI is increasingly popular in clinical research and fields such as developmental neuroscience and neuroplasticity, it is of vast importance to characterize its reproducibility (or reliability). We acquired multi-shell DWI data in 29 healthy young subjects twice over a rescan interval of 4 weeks to assess the within-subject coefficient of variation (CV_WS), between-subject coefficient of variation (CV_BS) and the intraclass correlation coefficient (ICC), respectively. Using these metrics, we compared regional and voxel-by-voxel reproducibility of the most common image analysis approaches (tract-based spatial statistics [TBSS], voxel-based analysis with different extents of smoothing [“VBM-style”], ROI-based analysis). We observed high test–retest reproducibility for the orientation dispersion index (ODI) and slightly worse results for the neurite density index (NDI). Our findings also suggest that the choice of analysis approach might have significant consequences for the results of a study. Collectively, the voxel-based approach with Gaussian smoothing kernels of ≥4 mm FWHM and ROI-averaging yielded the highest reproducibility across NDI and ODI maps (CV_WS mostly ≤3%, ICC mostly ≥0.8), respectively, whilst smaller kernels and TBSS performed consistently worse. Furthermore, we demonstrate that image quality (signal-to-noise ratio [SNR]) is an important determinant of NODDI metric reproducibility. We discuss the implications of these results for longitudinal and cross-sectional research designs commonly employed in the neuroimaging field.
Reproducibility, reliability and variability of FA and MD in the older healthy population: A test-retest multiparametric analysis
2020, NeuroImage: Clinical
Citation Excerpt :
Interestingly, only the CVws for MD in overall grey matter is slightly higher (CVws = 4.42%) than previously reported values for this index. When looking at selected regions of interest, the reproducibility values for both FA and MD are consistently better (between 1% and 2.8%) than those reported by a previous study (Marenco et al., 2006) for FA (2.5–10.16%) and MD (2.49–6.20%) with data collected with 8 averages of 6 diffusion directions at 1100 s/mm2 b-values on a 1.5 Tesla system. In our study, the GCC appeared the least reproducible of all the regions for both FA and MD (FA CVws = 1.8%, MD CVws = 2.8%), showing a much higher CVws than the SCC (FA CVws = 1.0%, MD CVws = 2.5%) – the same effect observed by Veenith et al., al. (2013).
Reproducibility of multi-shell diffusion tractography on traveling subjects: A multicenter study prospective
2019, Magnetic Resonance Imaging
Reproducibility of multicenter diffusion magnetic resonance imaging has drawn more attention recently due to rapidly increasing need for large-size brain imaging studies. Advanced multi-shell diffusion models are recommended for their potentials to provide variety of physio-pathological information. While previous studies have investigated the consistency of single-shell diffusion acquisition from various hardware and protocols, a well-controlled study with multi-shell acquisition would be necessary to understand the inherent factors of reproducibility from new complexity of such acquisition protocol. In this study, three traveling subjects were scanned at eight imaging centers equipped with the same type of scanners using the same multi-shell diffusion imaging protocol. Track density imaging and structure connectomes were investigated in local-scale distribution and in distal-scale connectivity, respectively. With evaluations of the coefficient of variation and the intra-class correlation coefficient, our results indicated: 1) similar to single-shell schemes, the intra-center reproducibility of multi-shell is higher than inter-center; 2) multi-shell schemes produce higher reproducibility and precision among centers compared to the single-shell schemes; and 3) in addition to the diffusion schemes, image quality and the presence of complex fiber structure could also associated with multicenter reproducibility.

View all citing articles on Scopus

¹: Currently at the Naval Research Laboratory, Washington, DC 20375.

View full text

Published by Elsevier Ltd.

Regional distribution of measurement error in diffusion tensor imaging

Abstract

Introduction

Section snippets

Methods

Results

Discussion

Acknowledgments

Journal of Magnetic Resonance, B

Magnetic Resonance Imaging

Neuroimage

Neuroimage

Academy of Radiology

Neuroimage

On the methods and theory of reliability

Journal of Nervous and Mental Diseases

Measurement error

British Medical Journal

Diffusion tensor imaging in multiple sclerosis: a tool for monitoring changes in normal-appearing white matter

Multiple Sclerosis