Fabry disease (FD) is a lysosomal storage disorder that is associated with marked cerebrovascular disease. Conventional MRI shows a progressive load of white matter lesions (WMLs) due to cerebral vasculopathy in the course of FD. To quantify brain structural changes in clinically affected male and female patients with FD we performed a prospective Diffusion-Tensor Imaging (DTI) study in 27 adult Fabry patients (13m, 14f) and 21 age-matched controls (12 m, 9f). Global Mean Diffusivity (MD) was increased in FD (P = 0.003) whereas global Fractional Anisotropy (FA) did not differ significantly between FD and controls. Even FD patients without significant WMLs (9m, 9f) showed increased global MD (P = 0.004). Regions of interest with significant MD elevations were located in the frontal, parietal and temporal white matter. No differences of thalamic and hippocampal DTI measurements could be detected between FD and controls. DTI parameters did not differ between male and female patients. The data provide the first evidence of a pattern of marked structural brain tissue alterations in adult FD male and female patients even without WMLs. DTI seems to be an appropriate diagnostic tool to quantify brain tissue integrity in FD. Moreover, this method could be favorable for longitudinal assessment of brain structure alterations in FD, and for monitoring the cerebral effects of enzyme replacement therapy.