Skip to content

Unofficial re-implementation of a neural supersampling model for real-time rendering

License

Notifications You must be signed in to change notification settings

timmh/neural-supersampling

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Neural Supersampling

This is a work-in-progress unofficial re-implementation of the real-time neural supersampling model proposed in Neural supersampling for real-time rendering [Paper] using PyTorch and PyTorch Lightning. This is in no way endorsed by the original authors.

Differences

This model is implemented as closely to the original paper as possible. However, there are some important differences:

  • the original training data is not freely available. Therefore Blender is used to render images with color, depth and motion data from Blender Open Movies.
  • the original paper seems to use motion data of the target resolution. Here, due to storage constraints, we use motion data of the source resolution
  • the original paper seems to use raw depth values for feature extraction. I found high depth values to negatively impact numerical stability and therefore decided to use inverse depth, i.e. disparity, instead.

Rendering

The training data may be rendered by Blender and the Cycles rendering engine. To achieve this, download any number of Blender Open Movie assets and configure them in render_all.py. Then either run render_all.py directly or use run_blender_headless.sh to run Blender via Docker.

Training

The training, evaluation and visualization are all implemented as separate files in the model directory. Alternatively, take a look at the Jupyter Notebook NeuralSupersampling.ipynb Open in Colab

TODO

  • train to convergence
  • optimize using TensorRT and embed in real-time application

About

Unofficial re-implementation of a neural supersampling model for real-time rendering

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published