An increasing number of fMRI studies are using the correlation of low-frequency fluctuations between brain regions, believed to reflect synchronized variations in neuronal activity, to infer "functional connectivity". In studies of autism spectrum disorder (ASD), decreases in this measure of connectivity have been found by focusing on the response to task modulation, by using only the rest periods, or by analyzing purely resting-state data. This difference in connectivity, however, could result from a number of different mechanisms--differences in noise, task-related fluctuations, task performance, or spontaneous neuronal activity. In this study, we investigate the difference in functional connectivity between adolescents with high-functioning ASD and typically developing control subjects by examining the residual fluctuations occurring on top of the fMRI response to an overt verbal fluency task. We find decreased correlations of these residuals (a decreased "connectivity") in ASD subjects. Furthermore, we find that this decrease was not due to task-related effects, block-to-block variations in task performance, or increased noise, and the difference was greatest when primarily rest periods are considered. These findings suggest that the estimate of disrupted functional connectivity in ASD is likely driven by differences in task-unrelated neuronal fluctuations.