Deep Learning in Neuroradiology

Parallels between artificial and biologic neural networks. Hidden layers of artificial neural networks can be thought to be analogous to brain interneurons.

Example of a simple deep network architecture. The goal of this network is to classify MR images into 4 specific diagnoses (normal, tumor, stroke, hemorrhage). Multiple different images form the training set. For each new case, the image is broken down into its constituent voxels, each one of which acts as an input into the network. This example has 3 hidden layers with 7 neurons in each layer, and the final output is the probabilities of the 4 classification states. All layers are fully connected. At the bottom is a zoomed-in view of an individual neuron in the second hidden layer, which receives input from the previous layer, performs a standard matrix multiplication (including a bias term), passes this through a nonlinear function (the rectified linear unit function in this example), and outputs a single value to all the neurons of the next layer.

Example of training and deployment of deep convolutional neural networks. During training, each image is analyzed separately, and at each layer, a small set of weights (convolution kernel) is moved across the image to provide input to the next layer. Each layer can have multiple channels. By pooling adjacent voxels or using larger stride distances between application of the kernel, deeper layers often have smaller spatial dimensions but more channels, each of which can be thought of as representing an abstract feature. In this example, 5 convolutional layers are followed by 3 fully connected layers, which then output a probability of the image belonging to each class. These probabilities are compared with the known class (stroke in the training example) and can be used to measure of how far off the prediction was (cost function), which can then be used to update the weights of the different kernels and fully connected parameters using back-propagation. When the model training is complete and deployed on new images, the process will produce a similar output of probabilities, in which it is hoped that the true diagnosis will have the highest likelihood.

An example of improving the SNR of arterial spin-labeling MR imaging using deep learning. The model is trained using low-SNR ASL images acquired with only a single repetition, while the reference image is a high-SNR ASL image acquired with multiple repetitions (in this case, 6 repetitions). Proton-density-weighted images (acquired routinely as part of the ASL scans for quantitation) and T2-weighted images are also used as inputs to the model to improve performance. The results of passing the low-SNR ASL image through the model are shown on the right, a synthetic image with improved SNR. In this example, the root-mean-squared error (RSME) between the reference image and the synthetic image compared with the original image is reduced nearly 3-fold, from 29.3% to 10.8%.