Functional magnetic resonance imaging can demonstrate the functional anatomy of cognitive processes. In patients with refractory temporal lobe epilepsy, evaluation of preoperative verbal and visual memory function is important as anterior temporal lobe resections may result in material specific memory impairment, typically verbal memory decline following left and visual memory decline after right anterior temporal lobe resection. This study aimed to investigate reorganization of memory functions in temporal lobe epilepsy and to determine whether preoperative memory functional magnetic resonance imaging may predict memory changes following anterior temporal lobe resection. We studied 72 patients with unilateral medial temporal lobe epilepsy (41 left) and 20 healthy controls. A functional magnetic resonance imaging memory encoding paradigm for pictures, words and faces was used testing verbal and visual memory in a single scanning session on a 3T magnetic resonance imaging scanner. Fifty-four patients subsequently underwent left (29) or right (25) anterior temporal lobe resection. Verbal and design learning were assessed before and 4 months after surgery. Event-related functional magnetic resonance imaging analysis revealed that in left temporal lobe epilepsy, greater left hippocampal activation for word encoding correlated with better verbal memory. In right temporal lobe epilepsy, greater right hippocampal activation for face encoding correlated with better visual memory. In left temporal lobe epilepsy, greater left than right anterior hippocampal activation on word encoding correlated with greater verbal memory decline after left anterior temporal lobe resection, while greater left than right posterior hippocampal activation correlated with better postoperative verbal memory outcome. In right temporal lobe epilepsy, greater right than left anterior hippocampal functional magnetic resonance imaging activation on face encoding predicted greater visual memory decline after right anterior temporal lobe resection, while greater right than left posterior hippocampal activation correlated with better visual memory outcome. Stepwise linear regression identified asymmetry of activation for encoding words and faces in the ipsilateral anterior medial temporal lobe as strongest predictors for postoperative verbal and visual memory decline. Activation asymmetry, language lateralization and performance on preoperative neuropsychological tests predicted clinically significant verbal memory decline in all patients who underwent left anterior temporal lobe resection, but were less able to predict visual memory decline after right anterior temporal lobe resection. Preoperative memory functional magnetic resonance imaging was the strongest predictor of verbal and visual memory decline following anterior temporal lobe resection. Preoperatively, verbal and visual memory function utilized the damaged, ipsilateral hippocampus and also the contralateral hippocampus. Memory function in the ipsilateral posterior hippocampus may contribute to better preservation of memory after surgery.