iGAN: Interactive Image Generation via Generative Adversarial Networks

Project | Youtube | Paper

Recent projects:
[pix2pix]: Torch implementation for learning a mapping from input images to output images.
[CycleGAN]: Torch implementation for learning an image-to-image translation (i.e., pix2pix) without input-output pairs.
[pytorch-CycleGAN-and-pix2pix]: PyTorch implementation for both unpaired and paired image-to-image translation.

Overview

iGAN (aka. interactive GAN) is the author's implementation of interactive image generation interface described in:
"Generative Visual Manipulation on the Natural Image Manifold"
Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, Alexei A. Efros
In European Conference on Computer Vision (ECCV) 2016

Given a few user strokes, our system could produce photo-realistic samples that best satisfy the user edits in real-time. Our system is based on deep generative models such as Generative Adversarial Networks (GAN) and DCGAN. The system serves the following two purposes:

• An intelligent drawing interface for automatically generating images inspired by the color and shape of the brush strokes.
• An interactive visual debugging tool for understanding and visualizing deep generative models. By interacting with the generative model, a developer can understand what visual content the model can produce, as well as the limitation of the model.

Please cite our paper if you find this code useful in your research. (Contact: Jun-Yan Zhu, junyanz at mit dot edu)

Getting started

• Install the python libraries. (See Requirements).
• Download the code from GitHub:

git clone https://github.com/junyanz/iGAN
cd iGAN

• Download the model. (See Model Zoo for details):

bash ./models/scripts/download_dcgan_model.sh outdoor_64

• Run the python script:

THEANO_FLAGS='device=gpu0, floatX=float32, nvcc.fastmath=True' python iGAN_main.py --model_name outdoor_64

Requirements

The code is written in Python2 and requires the following 3rd party libraries:

• numpy
• OpenCV

sudo apt-get install python-opencv

• Theano

sudo pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git

• PyQt4: more details on Qt installation can be found here

sudo apt-get install python-qt4

• Qdarkstyle

sudo pip install qdarkstyle

• dominate

sudo pip install dominate

• GPU + CUDA + cuDNN: The code is tested on GTX Titan X + CUDA 7.5 + cuDNN 5. Here are the tutorials on how to install CUDA and cuDNN. A decent GPU is required to run the system in real-time. [Warning] If you run the program on a GPU server, you need to use remote desktop software (e.g., VNC), which may introduce display artifacts and latency problem.

Python3

For Python3 users, you need to replace pip with pip3:

• PyQt4 with Python3:

sudo apt-get install python3-pyqt4

• OpenCV3 with Python3: see the installation instruction.

Interface: