Generative models for 3D MRI scans

This repo contains experiments on latent space exploration for detection of brain structure differences between genders and ages.

We also implemented several architectures of generative models for 3D brain MRI scans.
This is the first open-souce PyTorch realization of StyleGAN v1 for 3d models.

Presented models:

• Autoencoders:
• • Vanilla Autoencoder
• • Variational Autoencoder (VAE)
• GAN
• StyleGAN v1

Data

The models were trained using Human Connectome Young Adults brain sMRI scans.

This does not contain the data, only code. The data can be downloaded separately only via Human Connectome website. This process is free, but requires registration and confirmation from the Human Connectome organisation.

Requisites

Python3, Pytorch 1.8+, Numpy, Matplotlib

Experiment design

The idea is inspired by [1], where the authors propose to train a logistic regression on StyleGAN's learned latent vectors and disease presence labels. The weights of the classifier can then be used as a direction of disease progression for the latent vector. It allows to predict the brain structure changes caused by e.g. aging, and prevent possible illnesses on early stages.

We train different generative models on 3D MRI scans and then try to explore their latent spaces using logistic regression weights as a direction in which we change the latent vectors in order to observe structural changes.

The models, their code, description and interactive jupyter notebooks can be found in the corresponding folders. The experiment and its results can be found in the experiment folder of the repository.

References

[1] Kathryn Schutte and Olivier Moindrot and Paul Hérent and Jean-Baptiste Schiratti and Simon Jégou. (2021). Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images. arXiv, 2101.07563.