Retinal lesions caused by eye diseases such as glaucoma and age-related macular degeneration can, over time, eliminate stimulation of parts of the visual cortex. This could lead to degeneration of inactive cortical neuronal tissue, but this has not been established in humans. Here, we used magnetic resonance imaging to assess the effects of prolonged sensory deprivation in human visual cortex. High-resolution anatomical magnetic resonance images were obtained in subjects with foveal (age-related macular degeneration) and peripheral (glaucoma) retinal lesions as well as age-matched controls. Comparison of grey matter between patient and control groups revealed density reductions in the approximate retinal lesion projection zones in visual cortex. This indicates that long-term cortical deprivation, due to retinal lesions acquired later in life, is associated with retinotopic-specific neuronal degeneration of visual cortex. Such degeneration could interfere with therapeutic strategies such as the future application of artificial retinal implants to overcome lesion-induced visual impairment.