Japanese manga is popular in all ages and all countries. Anime like One Piece, Naruto, Bleach are products of great authors who write manga as their career. However, japanese manga are released as black-and-white and for someone new to reading manga, it may be difficult to imagine or follow the story along the pages. Most of the time, these manga are remained black-and-white by the authors until anime debut with just few pages colored. It is also time-consuming to manually color because every single page contains many panels along with different scenes, acts, and characters. Inspired with this problem, I created an automatic colorization program using artificial intelligence to automate the coloring processes of japanese manga.
There is no readily availble dataset for manga colorization, so I have to scrap manga pages as PNG images from fan-made websites containing colored and black-and-white chapters. I choose three manga titles to gather data, these are One Piece, Black Clover, and My Hero Academia. I used these noteboks to scrap the manga pages each chapter: Notebook-1 for Black Clover and Notebook-2 for One Piece and My Hero Academia. The total file size of the dataset is 5 GB. Note that my dataset is private to prevent copyright issues with the official owners.
Below is the tree of my manga dataset distribution.
Black Clover
│ └───Colored : **1066**
│ └───Black-and-White : **1068**
│
My Hero Academia
│ └───Colored : **1050**
│ └───Black-and-White : **1094**
│
One Piece
│ └───Colored : **1461**
│ └───Black-and-White : **2679**For training and inference, I used MMGeneration codebase. This is a great toolbox because I can easily change the hyperparameters for training. I used model CycleGAN since my dataset is unpaired, meaning a black-and-white image does not necessary have a colored counterpart. Also, one-to-one paired image in manga is difficult to obtain because usually the colored images are just fan-made while the black-and-white are official release.
For the configuration, I set the image size (width, height) = (512, 732), num_iters = 250k, and optimizer = Adam. I actually rented a GPU from Vast.ai, thankfully is was just cheap at $0.260 per hour for RTX A5000. The training lasted for ~2 days but the result was super good.
The inference was done in kaggle notebook since it was just not computationally expensive unlike in training. The FID and IS scores are shown here.
|-----------------------------------------------------+-----------------+------+----------------------------+-------------------------|
| Training configuration | Checkpoint | Eval | FID | IS |
|-----------------------------------------------------+-----------------+------+----------------------------+-------------------------|
| cyclegan_lsgan_resnet_in_summer2winter_b1x1_250k.py | iter_250000.pth | orig | 52.7144(4.72190/47.99250) | mean: 1.746, std: 0.141 |
|-----------------------------------------------------+-----------------+------+----------------------------+-------------------------|
You can access the Training.ipynb and Inference.ipynb here. Both can be edited in kaggle.
To show the results, I present samples below.
This repo is licensed under MIT License