Interictal diffusion imaging studies in patients with medial temporal lobe epilepsy (MTLE) accompanied by hippocampal sclerosis (HS) have shown an increased diffusivity in the epileptogenic hippocampus. In this study, we wanted to explore the whole brain in order to determine if MTLE could have an impact on the organization and the architecture of a large cerebral network and to identify clinical factors that could mediate diffusion abnormalities. Diffusion tensor imaging (DTI) and statistical parametric mapping of the entire brain were performed in 35 well-defined MTLE patients and in 36 healthy volunteers. SPM analyses identified three abnormal areas: an increased diffusivity was detected in the epileptic hippocampus and the ipsilateral temporal structures associated with a decreased anisotropy along the temporal lobe, a decreased diffusivity was found in the contralateral non-sclerotic hippocampus, the amygdala, and the temporal pole, and finally, a decreased anisotropy was noted ipsilaterally in posterior extratemporal regions. Duration of epilepsy, age at onset, and the frequency of generalized tonic-clonic seizures or partial complex seizures did not correlate with the presence of diffusion abnormalities. Region of interest analysis in the hippocampus/parahippocampus demonstrated a correlation between lower ipsilateral diffusivity values and occurrence of epigastric aura and between higher anisotropy values in both hemispheres and history of febrile seizures. In conclusion, this study showed that diffusion abnormalities are not restricted to the pathologic hippocampus and involve a larger network. This pattern may indirectly reflect the epileptogenic network and may be interpreted as a cause or a consequence of epilepsy.