Existing work demonstrates that obesity is independently associated with cognitive dysfunction and macrostructural brain changes; however, little is known about the association between obesity and white matter (WM) integrity. We explore this relationship in a large cohort of otherwise healthy subjects. The present study classified 103 adult participants from the Brain Resource International Database between 21 and 86 years of age without history of neurological, medical, or psychiatric illness according to BMI (normal weight, overweight, obese) and subjected them to diffusion tensor imaging (DTI). Resulting fractional anisotropy (FA) indexes for the corpus callosum and fornix were examined in relation to BMI and age in a multiple regression framework. Results indicated that increasing BMI was independently associated with lower FA in the genu, splenium, and fornix, and a BMI × age interaction emerged for FA in the splenium and body of the corpus callosum. When categorized, obese persons demonstrated lower FA than normal and overweight persons for all WM indexes, but no FA differences emerged between overweight and normal persons. Results indicate both a direct association between obesity and reduced WM tract integrity and an interaction between obesity and aging processes on certain WM tracts in otherwise healthy adults. While such findings suggest a possible role for adiposity in WM dysfunction and associated cognitive deficits, prospective studies are needed to clarify the nature of these relationships and elucidate underlying mechanisms.