Multiple sclerosis is an inflammatory demyelinating disease in which active demyelination and neurodegeneration are associated with lymphocyte infiltrates in the brain. However, so far little is known regarding the phenotype and function of these infiltrating lymphocyte populations. In this study, we performed an in-depth phenotypic characterization of T and B cell infiltrates in a large set of multiple sclerosis cases with different disease and lesion stages and compared the findings with those seen in inflammatory, non-inflammatory and normal human controls. In multiple sclerosis lesions, we found a dominance of CD8+ T cells and a prominent contribution of CD20+ B cells in all disease courses and lesion stages, including acute multiple sclerosis cases with very short disease duration, while CD4+ T cells were sparse. A dominance of CD8+ T cells was also seen in other inflammatory controls, such as Rasmussen's encephalitis and viral encephalitis, but the contribution of B cells in these diseases was modest. Phenotypic analysis of the CD8+ T cells suggested that part of the infiltrating cells in active lesions proliferate, show an activated cytotoxic phenotype and are in part destroyed by apoptosis. Further characterization of the remaining cells suggest that CD8+ T cells acquire features of tissue-resident memory cells, which may be focally reactivated in active lesions of acute, relapsing and progressive multiple sclerosis, while B cells, at least in part, gradually transform into plasma cells. The loss of surface molecules involved in the egress of leucocytes from inflamed tissue, such as S1P1 or CCR7, and the upregulation of CD103 expression may be responsible for the compartmentalization of the inflammatory response in established lesions. Similar phenotypic changes of tissue-infiltrating CD8+ T cells were also seen in Rasmussen's encephalitis. Our data underline the potential importance of CD8+ T lymphocytes and B cells in the inflammatory response in established multiple sclerosis lesions. Tissue-resident T and B cells may represent guardians of previous inflammatory brain disease, which can be reactivated and sustain the inflammatory response, when they are re-exposed to their specific antigen.