Uncovering the functional relationship between temporal lobe amnesia and diencephalic amnesia depends on determining the role of the fornix, the major interlinking fiber tract. In this study relating fornix volume with memory, we made magnetic resonance imaging-based volume estimates of 13 brain structures in 38 individuals with surgically removed colloid cysts. Fornix status was assessed directly by overall volume and indirectly by mammillary body volume (which atrophies after fornix damage). Mammillary body volume significantly correlated with 13 out of 14 tests of episodic memory recall, but correlated poorly with recognition memory. Furthermore, as the volumes of the left fornix and the left mammillary bodies decreased, the difference between recall and recognition scores increased. No other structure was consistently associated with memory. These findings support models of diencephalic memory mechanisms that require hippocampal inputs for recall, but not for key elements of recognition.