## Abstract

**BACKGROUND AND PURPOSE:** Previous studies quantifying moderate and severe carotid stenosis by direct millimeter measures on CT angiography (CTA) did not consider how prevalence and gender may influence classification cutoff values.

**MATERIALS AND METHODS:** Three hundred nineteen carotid arteries were evaluated in consecutive patients with known or suspected carotid artery disease. Millimeter measures were obtained of the stenotic carotid bulb lumen and distal internal carotid artery (ICA). Interclass correlation coefficients (ICC) defined interobserver and intraobserver agreement. North American Symptomatic Carotid Endarterectomy Trial (NASCET)-style percent stenosis ratios were calculated per carotid artery and used in linear regression and receiver operating characteristic (ROC) curve analysis to define equivalent millimeter quantification and classification values. Likelihood ratios and prevalence-specific positive/negative predictive values (PPV/NPV) were calculated to determine the most appropriate millimeter cutoff values to classify stenosis.

**RESULTS:** Interobserver agreement was excellent for stenosis measures (0.90) and good for distal ICA measures (0.79). Gender-specific regression curves and ROC curves indicated that millimeter stenosis is an excellent tool to quantify and classify carotid stenosis. Assuming a 10% prevalence of severe stenosis, we found that the cutoff value maximizing NPV and PPV was 1.1 mm for both genders (female: PPV = 86.2, NPV = 97.7; male: PPV = 83.2, NPV = 95.9). Assuming a 40% prevalence of moderate stenosis, we found that the cutoff values differed between genders: female = 2.0 mm (PPV = 91.3, NPV = 91.5), male = 2.1 mm (PPV = 91.6, NPV = 92.4). Specific millimeter cutoffs will vary depending upon the clinical scenario, prevalence, and gender.

**CONCLUSIONS:** Direct millimeter stenosis measures are an excellent tool to classify moderate and severe carotid artery stenosis. Millimeter classification cutoff values that best approximate NASCET classifications vary depending on prevalence and gender.

There is a linear relationship between direct millimeter carotid stenosis measures on CT angiography (CTA) and derived percent stenosis as defined by the North American Symptomatic Carotid Endarterectomy Trial (NASCET).^{1–3} This linear relationship allows prediction of NASCET-style percent stenosis from a single direct millimeter measure of stenosis. Quantification of carotid stenosis based on a direct stenosis measure is easy, fast, and reliable.^{1} In addition to eliminating the need for ratio calculations, a direct stenosis measure eliminates the variability of NASCET-style ratios due to differences in distal ICA size within and among patients.

Beyond quantification of stenosis, the NASCET ratio has been used to categorize carotid stenosis as moderate (≥50%–69%) and severe (≥70%). Because millimeter stenosis measures can predict NASCET-style ratios, it could be implied that specific millimeter values may similarly classify carotid artery disease. Prior studies have reported such millimeter classification thresholds, along with their respective sensitivity and specificity values.^{1} However, the sensitivity and specificity values of these millimeter stenosis classifications were <100%. The implication of this difference is that the previously defined millimeter stenosis thresholds are misclassifying some cases of carotid disease.

To decrease the degree of misclassifications from the millimeter stenosis measures, classification threshold values should maximize the positive and negative predictive values (PPV, NPV) of the test. A PPV (the probability that the results in a patient with a positive test result are truly positive) of the test depends not only on the sensitivity and specificity of the test but also on the prevalence of the “condition” within the population being studied. The “condition” in this case is the prevalence of moderate and severe stenosis, as defined by NASCET ratios. The potential impact that gender-specific prevalence could have on the accuracy of millimeter stenosis categorization, in substitution for calculating a NASCET ratio, is an important feature to consider before implementing a change in practice.

## Materials and Methods

### Patients/Subjects

In addition to the 136 carotid arteries with complete data from a prior data base (August 2003 through March 2004),^{1} an additional 200 carotid arteries from the same tertiary medical center were retrospectively reviewed, by using an IMPAX (AGFA Healthcare, Mortsel, Belgium) PACS archived file of patients entered from April 2004 through November 2006. Inclusion criteria were the same as those in the prior data base and included all consecutive patients with a history of known or suspected carotid artery atherosclerotic disease. Exclusion criteria included trauma, dissection, complete carotid occlusion, vascular anomaly/malformation, pre- or postoperative studies unrelated to carotid atherosclerotic disease, cases primarily evaluating posterior circulation, and inadequate coverage and/or technical errors precluding full evaluation of cervical carotid arteries. The study had continuous approval by the research ethics board (project identification number: 411–2004) at our center. Informed consent was not required for inclusion in this study or for the evaluation of records and images.

### Materials/Image Acquisition

All CTA examinations from August 2003 through September 2005 were performed by using a LightSpeed Plus 4-section CT scanner (GE Healthcare, Milwaukee, Wis). Images were obtained from C6 to the vertex by using the helical high-speed mode with 7.5 mm/rotation and 1.25 × 1.25 mm collimation (120 kVp, 350 mA). All subsequent examinations, from October 2005 through November 2006, were performed by using a LightSpeed VCT 64-section CT scanner (GE Healthcare). Images on the 64-section CT scanner were obtained from the aortic arch through the vertex at a thickness of 0.625 mm (140 kVp, auto-mA). Intravenous access was via an antecubital vein by using an 18- or 20-gauge angiocatheter. A total of 100- to 120-mL iohexol (Omnipaque 300; GE Healthcare) or iodixanol (Visipaque 320; GE Healthcare) were injected at a rate of 4.0–5.0 mL/s, with a 17-second delay or the use of SmartPrep software (GE Healthcare) at the pulmonary artery.

Postprocessing multiplanar reformats (MPRs) were created at the CT operator's console. Coronal and sagittal MPR images were created 7.0 mm thick, spaced by 3 mm. Bilateral rotational MPRs were created at the carotid bifurcations with a thickness of 7 mm and spacing by 3 mm. 3D rendered images were created on an Advantage Workstation (GE Healthcare). Images from April 2004 through September 2005 were viewed on IMPAX 4.5 PACS workstations (AGFA Healthcare, Mortsel, Belgium). All subsequent cases were viewed on IMPAX 5.2 PACS workstations (AGFA Healthcare).

### Image Analysis/Interpretation

Within a blinded protocol, 2 neuroradiologists independently reviewed all cases that met the inclusion criteria as in prior original work.^{1} The reviewer's instructions included an initial survey of each carotid artery by evaluating axial source images simultaneously with the MPR images. The following areas were measured with a submillimeter measurement tool: 1) the residual stenotic carotid bulb lumen at its narrowest diameter, and 2) the distal internal carotid artery diameter measured distally where the walls were parallel, well beyond the gradual tapering of the carotid bulb.

The narrowest carotid bulb stenosis and the distal internal carotid artery lumen were measured by manually placing the measurement calipers at the edges of contrast-filled luminograms. As in previous work,^{1,4–6} all measurements were obtained from the axial images. Arteries identified by MPRs as oblique to the axial plane were measured perpendicular to their oblique axis, within the axial plane. These measurements were verified with measures from reformats to ensure accuracy in obtaining the narrowest diameter in a true cross-sectional plane.^{1}

No preset window/level settings were used for image analysis and measurement. Instead, each reviewer modified these settings to best depict the residual stenotic ICA lumen and the distal ICA. Nonetheless, both reviewers agreed that the window/level settings should be relatively wide (approximately W2100:L790), progressing to very wide settings (approximately W3300:L1170) in the cases of dense calcifications, to decrease beam-hardening artifact.

### Statistical Methods

All raw data were analyzed by using the statistical software package Statistical Package for the Social Sciences for Windows (Version 12.0.0; SPSS, Chicago, Ill). All cases of carotid occlusion were removed from the analysis. All cases of carotid near-occlusion stenosis were identified according to published criteria concerning identification of such cases by CTA^{4} and were removed from the analysis because percent stenosis calculations are fallacious in these cases.^{2,3} All missing data were excluded list-wise from calculations.

Interobserver agreement for the 2 reviewers was evaluated by interclass correlation coefficients (ICC) with 95% confidence intervals (CI) for both the narrowest carotid bulb stenosis and the distal ICA measures. Intraobserver agreement was also evaluated by ICC with 95% CIs for the carotid stenosis and distal ICA measures. Mean carotid stenosis values and mean distal ICA values were calculated per carotid artery and were used in the regression analysis and receiver operating characteristic (ROC) curve analysis.

NASCET-style ratios were calculated for each ICA via the mean stenosis and mean distal ICA values and were used as the reference standard in classifying the degree of carotid artery stenosis. Gender-specific linear regression analysis was used to determine how well millimeter stenosis measures predicted NASCET percent stenosis values, with 95% prediction intervals.

Gender-specific ROC curves were created with mean stenosis as the test variable and were based upon the NASCET reference values classifying severe carotid stenosis (≥70%) and moderate carotid stenosis (≥50%–69%).^{2,3} Gender-specific ROC curves were used to determine the most appropriate millimeter stenosis cutoff values to classify severe (≥70% stenosis) and moderate stenosis (≥50% stenosis). More specifically, the ROC curves were used 1) to demonstrate how much “diagnostic classification” accuracy is lost when one chooses to consider only the numerator (millimeter stenosis) rather than the relationship between numerator and denominator (ie, NASCET ratio calculation) when classifying a patient's degree of stenosis, and 2) to determine the numerator millimeter measurement that works best to classify carotid stenosis dichotomously.

## Results

There were a total of 468 carotid arteries (234 CTAs) that met the inclusion criteria (August 2003 to March 2004, 268 carotid arteries [134 CTAs]; April 2004 to November 2006, 200 carotid arteries [100 CTAs]). From the original data base (August 2003 to March 2004), there were only 136 carotid arteries with complete data, minus all near-occlusion and occluded carotid arteries. Many carotid arteries in the original data base had incomplete data because the neuroradiology reviewer's original instructions were to exclude nondiseased carotid bulb measures. These instructions were later revised because all data points could be considered helpful in the regression calculations. From the addition of 200 new carotid arteries, 183 were included in the study after elimination of near-occlusion and occluded carotid arteries.

From the total of 319 carotid arteries eligible for data analysis, there were 157 (49%) female carotid arteries and 162 (51%) male carotid arteries. The ICC between the readers was good for the distal ICA measurements (0.79; 95% CI, 0.75–0.83; *n* = 319), however not nearly as good as for the stenosis measurements (0.90; 95% CI, 0.88–0.92; *n* = 319) (Fig 1). The intraobserver reliability for the distal ICA measurements was also good (ICC = 0.81; 95% CI, 0.70–0.89; *n* = 58) but again fell short of the excellent intraobserver reliability for the stenosis measurements (ICC = 0.92; 95% CI, 0.87–0.95; *n* = 58).

The adjusted *R*^{2} values of the regression curves were excellent (female = 0.93; male = 0.90) (Fig 2). On the basis of the gender-specific linear regression models, estimated percent stenosis values (shown as 95% predictive intervals) were calculated for the direct millimeter stenosis measures at submillimeter intervals from 0.5 mm through 2.4 mm (Fig 3).

Using millimeter stenosis values to classify severe stenosis (≥70% stenosis based on NASCET ratios), the area under the ROC curves was 0.98 for both the female data (95% CI, 0.96–1.0; *n* = 157) and male data (95% CI, 0.96–1.0; *n* = 162) (Fig 4). Positive and negative likelihood ratios (LR+, LR−) and PPV and NPV were calculated (Table 1) for the millimeter stenosis values that corresponded to the estimated percent stenosis 95% predicted intervals that included the 70% stenosis cutoff value, classifying severe stenosis as in NASCET (Fig 3).

In a female population (*n* = 157) with known or suspected carotid artery disease undergoing CTA and a 10% prevalence of severe stenosis (defined by NASCET ratios), the millimeter stenosis cutoff value that minimizes false-negative and false-positive results when classifying severe stenosis is 1.1 mm (Table 1). The PPV for this population is 86.2, and the NPV is 97.7. In other words, in a population of 100 women with a 10% prevalence of ratio-defined severe carotid stenosis, fewer than 14 patients without ratio-defined severe stenosis would be misclassified as having severe stenosis (false-positive) and fewer than 3 patients with ratio-defined severe stenosis would be misclassified as not having severe stenosis (false-negative). With a 30% prevalence of ratio-defined severe stenosis, the 1.1 mm threshold value may also provide a good balance between false-positive and false-negative results when classifying severe carotid stenosis (PPV = 96.0, NPV = 91.6). At a prevalence of 30%, there would be many fewer false-positive cases (4/100) and several more false-negative cases (<9/100) in comparison with a prevalence of 10% at the same millimeter stenosis value.

In a male population (*n* = 162) with known or suspected carotid artery disease undergoing CTA and a 10% prevalence of ratio-defined severe stenosis, the millimeter stenosis cutoff value that minimizes false-negative and false-positive results when classifying severe stenosis is also 1.1 mm (Table 1). The PPV for this population is 83.2, and the NPV is 95.9. In other words, in a population of 100 men with a 10% prevalence of ratio-defined severe carotid stenosis, fewer than 17 patients without ratio-defined severe stenosis would be misclassified as having severe stenosis (false-positive) and fewer than 5 patients with ratio-defined severe stenosis would be misclassified as not having severe stenosis (false-negative). However, in a male population with a 30% prevalence of ratio-defined severe stenosis, a CTA stenosis measure of 1.2 mm may be the best balance between false-positive and false-negative results (PPV = 91.7, NPV = 94.7). With this scenario, there would be many fewer false-positive results (<9/100) and minimally greater numbers of false-negative results (<6/100) in comparison with the 1.1 mm stenosis value and 10% prevalence.

Using millimeter stenosis values to classify moderate stenosis, we found that the area under the curve was 0.98 for the female data (95% CI, 0.97–1.0) and 0.98 for the male data (95% CI, 0.96–1.0) (Fig 5). LR+, LR−, and PPV and NPV were calculated (Table 2) for the millimeter stenosis values that corresponded to the estimated percent stenosis 95% predicted intervals that included the 50% stenosis cutoff value, classifying moderate stenosis as in NASCET (Fig 3).

In a female population with known or suspected carotid artery disease undergoing CTA and a 40% prevalence of ratio-defined moderate stenosis, the millimeter stenosis cutoff value that minimizes false-negative and false-positive results when classifying at least moderate carotid stenosis is 2.0 mm (*n* = 157) (Table 2). The PPV for this population is 91.3, and the NPV is 91.5. Therefore, in a population of 100 women with a 40% prevalence of ratio-defined moderate carotid stenosis, fewer than 9 patients without ratio-defined moderate stenosis would be misclassified as having at least moderate stenosis (false-positive) and fewer than 9 patients with ratio-defined moderate stenosis would be misclassified as having less than moderate stenosis (false-negative). However, in a female population with a 60% prevalence of ratio-defined moderate stenosis, a CTA stenosis measure of 2.3 mm may be the best balance between false-positive and false-negative results (PPV = 90.5, NPV = 94.0). With a stenosis threshold at 2.3 mm and a 60% prevalence of ratio-defined moderate stenosis, there would be slightly more false-positive results (<10/100) and fewer false-negative results (6/100) in comparison with the 2.0 mm stenosis value and 40% prevalence.

In a male population with known or suspected carotid artery disease undergoing CTA and a 40% prevalence of ratio-defined moderate stenosis, the millimeter stenosis cutoff value that minimizes false-negative and false-positive results when classifying at least moderate carotid stenosis is 2.1 mm (*n* = 162) (Table 2). The PPV for this population is 91.6, and the NPV is 92.4. Therefore, in a population of 100 men with a 40% prevalence of ratio-defined moderate carotid stenosis, fewer than 9 patients without ratio-defined moderate stenosis would be misclassified as having at least moderate stenosis (false-positive) and fewer than 8 patients with ratio-defined moderate stenosis would be misclassified as having less than moderate stenosis (false-negative). At 60% prevalence of ratio-defined moderate stenosis in a male population, a 2.3 mm stenosis threshold value may provide optimal balance between false-positive and false-negative cases (PPV = 91.8, NPV = 93.1). At a 60% prevalence, there would be similar false-positive results (<9/100) and fewer false-negative results (<7/100).

## Discussion

The excellent adjusted *R*^{2} values associated with the gender-specific regression curves indicate that millimeter stenosis values are an excellent model to predict NASCET-style percent stenosis (an excellent model to quantify stenosis) (Fig 2). However, the percent stenosis 95% predictive intervals that correspond to the submillimeter stenosis increments greatly overlap one another. In the female population, the 95% predictive intervals that include the ratio-defined 70% stenosis value correspond to submillimeter measures ranging from 1.1 to 1.3 mm (Fig 3). This submillimeter range is even larger for the male population, ranging from 1.1 to 1.4 mm (Fig 3). The 95% predictive intervals that include the ratio-defined 50% stenosis value correspond to even larger submillimeter ranges (female = 1.9–2.3 mm, male = 1.9–2.4 mm) (Fig 3).

The areas under both the severe and moderate stenosis ROC curves are excellent for both genders, signifying that direct CTA measurement of the narrowest carotid body stenosis is an excellent test to classify carotid stenosis (Figs 4, 5). The respective ROC coordinate tables provide the sensitivities and specificities associated with a detailed list of submillimeter stenosis values. However, the interpretation of any diagnostic test depends not only on the balance between sensitivity and specificity but also on the prevalence of the condition that the test is designed to detect. Prevalence can be estimated upwards or downwards for each patient, depending on the clinical information such as age, gender, presence of hypertension, smoking history, and prior ipsilateral stroke/transient ischemic attacks.

NASCET concluded that patients with symptomatic severe carotid stenosis (≥70%) greatly benefit from carotid endarterectomy (CEA), with persistent benefit for at least 5 years following surgery.^{2,3} Therefore, it may be important to select test values that maximize the ability to identify correctly those who have the disease (maximize PPV) when using a test to determine severe stenosis.

For moderate stenosis (≥50%–69%), NASCET concluded that the benefit of CEA is less in men and that no benefit exists for women and for patients with multiple risk factors.^{2,3} Nonetheless, if a patient is not classified as having severe carotid stenosis, it may be important to determine if the patient has at least moderate carotid stenosis.

Likelihood ratios may also be helpful to determine the millimeter stenosis quantification values that may best classify stenosis (Tables 1 and 2). Likelihood ratios represent the odds that a specific millimeter carotid stenosis classification value would be expected in a patient who actually has the disease (according to NASCET-style percent stenosis ratios). Likelihood ratios do not change with prevalence and can be calculated for a range of millimeter stenosis quantification values. The posttest probability can be determined by using a likelihood ratio normogram by plotting the pretest probability and the likelihood ratio corresponding to the millimeter stenosis quantification values on Tables 1 and 2. Likelihood ratios may be most helpful when a patient's carotid stenosis millimeter measure lies within the submillimeter ranges corresponding to the ratio-defined classifications as estimated by the regression analysis and listed as 95% prediction intervals (Fig 3).

Ultimately for the radiologist, it may be difficult to determine prevalence rates that can be translated into a single unique millimeter value to classify carotid stenosis. Knowledge of specific carotid stenosis prevalence rates and individual patient pretest probabilities would be better understood by the clinician. As radiologists, we recommend that the millimeter stenosis quantification value and its corresponding percent stenosis 95% predictive intervals be reported per carotid.

For most patients, the classification of stenosis will be clear-cut. For the patients with millimeter stenosis quantification values that lie within a specific classification range, it is important to recognize that each of these submillimeter classification ranges has a difference of four to five tenths of a millimeter. These submillimeter differences are well within the acceptable limits of measurement error in carotid CTA. Most of the measurements by our trained neuroradiology reviewers differed by at least several tenths of a millimeter for both the narrowest carotid body stenosis and the distal ICA measures (Fig 1). Nonetheless, our ICC values indicate that we have excellent interobserver and intraobserver agreement in our narrowest carotid body stenosis measures and good agreement in our distal ICA measures.

In the future, it is possible that our current understanding of carotid stenosis and how stenosis is best classified will change. Direct submillimeter measures of stenosis allow greater quantification of carotid stenosis than with any other imaging modality. In addition, there have been dramatic improvements in medical and surgical procedures such as antiplatelet treatments and carotid stent placement. Clinical trials using CTA to quantify carotid stenosis have an advantage over the NASCET trials in that a more exact quantification of stenosis is possible. This potentially allows a more consistent relationship to clinical outcome with the ultimate goal of developing more useful clinical algorithms.

### Study Limitations

Although our interobserver and intraobserver agreement was exceptional for the stenosis measures, additional observers may not have similar success. Appropriate window and level settings are critical in the measurement of lumen size with any digital imaging system, including CTA, MR angiography (MRA), and digital subtraction angiography (DSA). If the window and level settings are too narrow, the intraluminal contrast is exaggerated and will contribute to variability between observers and overestimation of the true lumen. Narrow window and level settings may also confound the evaluation of vessels with calcified plaque. Thus, relatively wide window and level settings are needed when classifying carotid artery disease, especially when classifications depend on tenths of a millimeter. Narrow window and level settings can overestimate the true contrast-filled lumen with digital imaging whether on CTA, MRA, or DSA. Wide window and level settings, on the other hand, will never underestimate the true size of a contrast-filled lumen.

Although we are comparing 2 different populations defined by gender, this study does not consider other potentially important factors that may influence carotid artery size, such as chronologic, physiologic, and biometric factors. Additionally, we consider each carotid artery to be statistically independent, even though it could be argued that paired carotid arteries in a single patient are not truly independent because both are subject to the same chronologic, physiologic, and biometric factors. If we were focusing on factors leading to stenosis or consequences of stenosis, such lack of independence could have potentially important ramifications. Instead, this is a study investigating whether, in the era of high-resolution CTA, the dichotomous categorization of a single direct millimeter measurement can reliably replace a similar dichotomous classification based on a ratio measurement (as in NASCET). In this setting, the NPV and PPV do not actually refer to the likelihood of a patient having stenosis; they refer to the likelihood of the patient being classified by NASCET as having severe or moderate stenosis (≥70% or at least 50% stenosis, respectively).

A basic premise of studies examining new diagnostic tests is that the test in question must be independent of the accepted gold standard. This use of the direct millimeter stenosis measurement in this study is not so much the proposition for a new test as the argument that a current gold standard test can be abbreviated without any substantive effect on the diagnostic conclusions. Although probably ideal, we did not use the NASCET calculations of a third observer as an independent gold standard classification of carotid stenosis. Nonetheless, the interobserver consistency of our NASCET ratio measures is excellent (ICC, 0.88; 95% CI, 0.85–0.90; *n* = 319); thus, for our gold standard, we chose to use the mean values of the 2 observers to calculate a NASCET ratio of an individual carotid artery.

The basic proposition of this study is that millimeter carotid stenosis is linearly related to the NASCET ratio calculations (1 − [millimeter carotid bulb stenosis / millimeter distal ICA] × 100) that were used as the reference standard in classifying carotid stenosis. For our proposition to be an exact fit, “millimeter distal ICA” must approximate a constant value. The distal ICA, however, is quite variable between subjects as well as along the course of a single carotid artery, from the distal ICA bulb to the skull base. This intracarotid variability is the reason for the greater interobserver and intraobserver differences of the distal ICA measures, as compared with the measures of a better defined focal maximum bulb stenosis.

The variability of distal ICA measurement is the weakness of ratio measures. In the NASCET study, ratio calculations were necessary to standardize measures between patients because direct millimeter measures could not be obtained from the catheter angiograms. Now that CTA allows direct millimeter measures that are reproducible, there is no longer a need for the ratio calculation. Nonetheless, much of our knowledge of carotid disease and therapeutic decision-making is derived from the NASCET achievements. This study demonstrates that the direct millimeter stenosis measures are an excellent model to predict NASCET ratios and an excellent model to classify stenosis.

## Conclusion

Direct CTA millimeter stenosis values provide an excellent method to classify moderate and severe stenosis in both men and women, in lieu of more cumbersome NASCET ratio calculations. Despite a narrow range of submillimeter values that differ by three to five tenths of a millimeter, a single millimeter threshold value is not practical to classify stenosis. The clinician should consider the pretest probability of moderate or severe stenosis, as defined by NASCET, when interpreting the millimeter stenosis quantification of their patient's carotid CTA.

## References

- Received November 5, 2007.
- Accepted after revision May 16, 2008.

- Copyright © American Society of Neuroradiology