American Journal of Neuroradiology 21:639-642 (4 2000)
© 2000 American Society of Neuroradiology
ARTICLE
Evaluation of Carotid Stenosis by Angiography: Potential Bias toward Overestimated Measurements Introduced by Prior Interpretation of Doppler Sonograms
James Dix
,a and
James Skrockia
a From the David Grant Medical Center, Department of Radiology SGSX, 101 Bodin Circle, Travis Air Force Base, CA 94535.
 |
Abstract
|
|---|
BACKGROUND AND PURPOSE: Doppler sonography of the carotid arteries
is routinely performed before catheter angiography, and its
results may bias the subsequent interpretation of angiograms.
We attempt to establish that Doppler sonography may show an
exagerrated degree of carotid stenosis, introducing bias to
the evaluation of carotid stenosis by subsequent catheter angiography.
METHODS: Angiograms of the carotid arteries obtained to evaluate potential carotid stenosis in patients who had undergone Doppler sonography of the carotid arteries were retrospectively reviewed (from 1993 to 1998). Readers who were blinded to the previous interpretations of the angiography and Doppler sonography results measured carotid stenosis. The results of Doppler sonography of the carotid arteries were not re-evaluated. Based on the original Doppler sonograms, stenoses were categorized as normal/mild (<30%), moderate (30–59%), severe (60–79%), and critical (80–99%). Within these categories, the differences between the original percent stenosis, as determined by angiography, and the blinded measurements were determined.
RESULTS: A total of 106 patients with angiographically measurable stenoses in 128 vessels were identified. The difference between the blinded readers was 3% (±8%), with no category statistically different from the other. The difference between original and remeasured stenoses in carotid arteries in the Doppler categories were as follows: mild stenosis, 2% (±9%); moderate stenosis, 6% (±15%); severe stenosis, 8% (±15%); and critical stenosis, 22% (±12%). A significant overestimation occurred in the severe (P < .05) and critical (P < .0001) stenosis categories. One third of patients with stenoses in the severe or critical Doppler category had significant stenoses on the original angiograms that were less than 60%, according to blinded remeasurement.
CONCLUSION: Doppler sonography of the carotid arteries has the potential to bias the subsequent interpretation of catheter angiography. Care must be taken to measure stenosis accurately, using strict criteria to determine the potential benefit of carotid endarterectomy for the individual patient and to ensure that the criteria for Doppler sonography of the carotid arteries are based on accurate catheter angiography measurements.
|
Introduction
|
|---|
Conventional catheter angiography has been the traditional standard
of reference for evaluation of carotid bifurcation stenosis.
The North American Symptomatic Carotid Endarterectomy Trial
(NASCET) showed that carotid endarterectomy can reduce the risk
of stroke that is associated with atherosclerotic carotid stenosis
if the diameter of the stenosis revealed by angiography is greater
than 50% in symptomatic patients (
1,
2). More controversially,
the Asymptomatic Carotid Atherosclerosis Study (ACAS) showed
that carotid endarterectomy can reduce the risk of stroke that
is associated with atherosclerotic carotid stenosis in asymptomatic
patients with greater than 60% stenosis (
3). In ascertaining
the validity of the NASCET measurement, a tendency to overestimate
percent stenosis was found when the measurements obtained by
neuroradiologists from multiple institutions were compared with
those obtained by the single investigator in the NASCET (
4).
At our facility, we noted that catheter angiography routinely
indicated a higher percent stenosis than our Doppler sonography
studies had suggested. We attempted to determine whether overestimated
percent stenosis by angiography was similar to that experienced
by the NASCET group, indicating a similar mechanism, or whether
the positive results of the Doppler sonography biased the results
of the angiography. Our study was designed to evaluate the potential
bias created in the minds of the angiographers by the "positive"
Doppler results that indicated "severe" or "critical" stenoses.
|
Methods
|
|---|
The angiograms of the carotid arteries of consecutive patients
who were referred to our institution between December 1993 and
May 1998 for evaluation of carotid stenosis and who had undergone
Doppler sonography of the carotid arteries were retrospectively
reviewed. Using the NASCET criteria, two radiologists, who had
not been involved in the original Doppler sonography or angiography
and who were blinded to the previous results, remeasured the
degree of carotid stenosis (
5,
6). Four angiographers interpreted
the original angiograms, and six radiologists interpreted the
Doppler sonograms. Exclusion criteria included an incomplete
examination, carotid occlusion, and partial collapse of the
distal internal carotid artery (approaching near-occlusion,
according to the NASCET criteria) that would preclude accurate
calculation of percent stenosis. Based on Doppler sonography
of the carotid arteries, stenoses were categorized according
to the criteria presented by Bluth et al (
7) into normal/mild
(<30%), moderate (30–59%), severe (60-79%), and critical
(80-99%). The results of the Doppler sonography were not re-evaluated.
Vessels with no stenosis revealed by angiography were removed
from the normal/mild category, leaving only vessels with measurable
stenoses. Within Doppler sonography categories, the differences
between the blinded calculations of the angiography percent
stenosis and the original dictated percent stenosis were determined.
A statistical comparison of the differences between the measurements
obtained by the two blinded readers and of the differences between
the average blinded calculation and the original reported percent
stenosis within the Doppler sonography–derived categories
of mild, moderate, severe, and critical was conducted.
|
Results
|
|---|
A total of 106 patients were identified, and measurable stenoses
were shown in 128 vessels. Of the remaining 84 vessels, 47 carotid
arteries had no measurable stenosis (carotid bulb larger than
distal internal carotid artery), 23 vessels were occluded, five
were "approaching near-occlusion," and the remaining nine had
incomplete Doppler sonography or angiography studies.
The difference between the readings of two blinded angiographers for all Doppler categories was 3% (±8%), with no category statistically different from the others. The differences between the original and remeasured angiography percent stenoses in the carotid arteries within the Doppler categories were as follows: mild stenosis (34 vessels), 2% (±9%); moderate stenosis (11 vessels), 6% (±15%); severe stenosis (38 vessels), 8% (±15%); and critical stenosis (45 vessels), 22% (±12%) (Table). An example of each Doppler category is shown in Figures 1 through 4. There was a statistically significant difference in angiographic stenosis in the severe (P < .05) and critical (P < .0001) stenosis categories when the moderate, severe, and critical categories were compared with the mild category. One third of the patients (27 of 83 patients) in the severe or critical stenosis Doppler category were determined to have greater than 60% stenosis according to the original interpretation but less than 60% according to the blinded measurement.
View larger version (122K):
[in this window]
[in a new window]
|
FIG 1. Results of Doppler sonography of the carotid arteries were interpreted as mild stenosis with a peak systolic velocity of 104 cm/s. Percent stenosis was measured as 25% according to the original interpretation and as 23% by the blinded readers. Arrow indicates point of maximum stenosis, and arrowhead indicates normal distal internal carotid artery.FIG 2. Results of sonography of the carotid arteries were interpreted as moderate stenosis with a peak systolic velocity of 132 cm/s. Percent stenosis was measured as 33% according to the original interpretation and as 28% by the blinded readers. Arrow indicates point of maximum stenosis, and arrowhead indicates normal distal internal carotid artery.FIG 3. Results of Doppler sonography of the carotid arteries were interpreted as severe stenosis with a peak systolic velocity of 170 cm/s. Percent stenosis was measured as 70% according to the original interpretation and as 52% by the blinded readers. Arrow indicates point of maximum stenosis, and arrowhead indicates normal distal internal carotid artery.FIG 4. Results of Doppler sonography of the carotid arteries were interpreted as critical stenosis with a peak systolic velocity of 450 cm/s. Percent stenosis was measured as 83% according to the original interpretation and as 61% by the blinded readers. Arrow indicates point of maximum stenosis, and arrowhead indicates normal distal internal carotid artery
| |
|
Discussion
|
|---|
Radiologists require all available pertinent information when
interpreting a patient's examination results, but they should
be aware that the reported results of previous studies or other
elements of a patient's history might significantly bias their
interpretations. All examinations are potentially biased by
a priori information, but physicians should be cognizant of
the biases to decrease their potential harm. Bias in the workup
of carotid stenosis warrants particular attention because of
the profound societal costs and potential risks of stroke if
carotid endarterectomy is performed inappropriately.
The stroke reduction in the NASCET study was based on conventional catheter angiography, with greater stroke reduction associated with higher grades of stenosis. The correlation of Doppler sonography of the carotid arteries and stroke reduction in the NASCET study was poor, without proven stroke reduction if the endarterectomy had been based on the Doppler examination (4). The ACAS included a combination of catheter angiography and Doppler sonography of the carotid arteries, but the Doppler criteria were variable from site to site, with a requirement that the Doppler criteria produce a 95% positive predictive value (3). The ACAS patient population included a mixture of patients with more severe stenoses, who were entered in the study based on Doppler criteria, and patients with less severe stenoses, who were entered in the study based on catheter angiography criteria.
The published criteria for Doppler sonography of the carotid arteries are markedly variable regarding the velocity threshold chosen to indicate a significant stenosis, but the published reports of the sensitivity and specificity of the examination are more consistent. There are several possible reasons for the variability of the published criteria, including machine and technician variation, spectrum of disease in the patient population, and many definitions of a significant stenosis. Area stenosis and diameter stenosis determined using the NASCET or European Carotid Surgery Trial measurement techniques create marked variation in the measured stenosis on angiograms with which Doppler velocity criteria are correlated. The consistency of the published reports of the sensitivity and specificity with differing velocities is possible by varying the threshold velocity. In most studies, this is done to produce a sensitive screening examination for subsequent confirmatory angiography. Because of this reasoning, the threshold velocities do not equal the stated percent stenosis but merely indicate a high probability of finding most patients with significant stenoses. The criteria published and used by many institutions will therefore consistently lead to overestimated measurements of stenosis when compared with measurements determined by conventional angiography. In our patient population, a peak systolic velocity of 225 cm/s produced a sensitivity of 90% and specificity of 96%, similar to those of recent studies, but the average percent stenosis of patients with a peak systolic velocity of 225 cm/s is only 50% (±18%). If surgery were performed based on the peak systolic velocity of 225 cm/s alone, half of the patients would have angiography percent stenosis of less than 50%. We currently use a low peak systolic velocity threshold of 130 cm/s because of its high sensitivity (identifying patients with potential benefit) but require MR angiography or catheter angiography to confirm surgical disease. Both the radiologist and the referring physician must be aware that the positive Doppler result does not signify a severe stenosis but is only a screening test; severity of stenosis should be based on MR angiography or conventional angiography.
In the original NASCET study, a single radiologist measured the percent stenosis on the original angiograms. To verify the method, several angiograms were sent to multiple institutions and the percent stenosis was remeasured (4). The blinded readers in that study systematically overestimated percent stenosis in 6% of carotid arteries with less than 70% stenosis. Presumably, our blinded readers' interpretations yielded similar overestimation, suggesting that the actual overestimation of stenosis in the original dictation may have been even greater. In the verification study presented by the NASCET group, no remeasurement of severe stenosis vessels is reported. It is possible that an even greater overestimation of stenosis may have occurred in those vessels.
Our results indicate that a bias may be created in the minds of angiographers by the positive results of Doppler sonography. Small sample size and the fact that evaluation was conducted at a single institution limited our study. At our facility, Doppler sonography of the carotid arteries is performed and interpreted in the radiology department. This would increase bias if the radiologist performing the catheter angiography were reluctant to disagree with a colleague. A systematic overestimation of higher grades of stenosis by the original angiographers, regardless of the Doppler result, would create a similar judgment, and is an alternative explanation of our findings. Neither of the two blinded radiologists in our study had a known bias that would lead to systematically underestimated measurements in cases of higher-grade stenosis, but if this bias existed, it would yield underestimated percent stenoses.
Doppler criteria based on the NASCET have been published, but they also are designed to create a particular combination of sensitivity and specificity not to equal a percent stenosis. The result may still constitute a bias on the angiographers' part to agree with the Doppler examiners' impression of a higher-grade stenosis, although the new higher velocity criteria may lessen the degree of overestimation.
|
Conclusion
|
|---|
No significant difference between the original and blinded measurements
of carotid stenosis based on angiography was found if Doppler
sonography of the carotid arteries was predictive of mild or
moderate stenosis. If Doppler sonography of the carotid arteries
were predictive of greater than 60% stenosis, the interpretation
of original angiograms led to overestimated measurements of
stenosis by 16% (
P < .05). Care must be taken to measure
stenosis accurately by using strict criteria to determine the
potential benefit of carotid endarterectomy in the individual
patient and to ensure that the criteria for Doppler sonography
of the carotid arteries are based on accurate catheter angiography
measurements.
|
Footnotes
|
|---|
101 Address correspondence to James Dix, MD.
|
References
|
|---|
-
North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade stenoses. N Engl J Med 1991;325:445-453[Abstract]
-
Barnett HJM,
Taylor DW,
Eliasziw M,
et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenoses. N Engl J Med 1998;339:1415-1425[Abstract/Free Full Text]
-
Executive committee for the Asymptomatic Carotid Atherosclerosis Study. Endarterectomy for asymptomatic carotid artery stenosis. JAMA 1995;273:1421-1428[Abstract]
-
Eliasziw M,
Rankin RN,
Fox AJ,
Haynes RB,
Barnett HJM.
Accuracy and prognostic consequences of ultrasonography in identifying severe carotid artery stenosis: North American Symptomatic Carotid Endarterectomy Trial (NASCET) Group. Stroke 1995;26(10):1747-1752[Abstract/Free Full Text]
-
Eliasziw M,
Fox AJ,
Sharpe BL,
Barnett HJM.
Carotid artery stenosis: external validity of the North American Symptomatic Carotid Endarterectomy Trial measurement method. Radiology 1997;204:229-233[Abstract/Free Full Text]
-
Fox AJ.
How to measure carotid stenosis. Radiology 1993;186:316-318[Free Full Text]
-
Bluth EI,
Wetzner SM,
Stavros AT,
Aufrichtig D,
Marich KW,
Baker JD.
Carotid duplex sonography: a multicenter recommendation for standardized imaging and Doppler criteria. Radiographics 1998;8(3):487-506[Abstract]
Received July 12, 1999;
accepted after revision November 10, 2000.
TOP
Abstract
Introduction