List of Curated Papers

Hello friends,

I'm excited to share with you a list of the most influential deep learning papers. I have selected these based on their impact in the community, my own interest in understanding the behaviour and psychology of artificial neural networks and the creative aspects of them. These papers have contributed greatly to the progress of deep learning research and have been invaluable resources for me during the development of our own product.

Please note that there are many other amazing papers out there, and I may have missed some that have been written by my own friends. So, my apologies in advance for any oversight, and please do let me know if there's a paper you think deserves a spot on the list.

But for now, let's focus on the papers that made the cut! Each one is packed with valuable insights, tips, and tricks that have helped me tackle challenging problems and make significant strides in my work.

Whether you're a seasoned researcher or just starting out in the field of deep learning, I believe that these papers will be a valuable resource for you. So, grab a cup of coffee and take some time to explore this list. I look forward to hearing your thoughts and sparking some interesting discussions in the comments.

Thanks for being a part of this amazing community and keep crushing it!

Theory

Year	Month	Title
1998	03	Gradient-Based Learning Applied to Document Recognition
2006	07	Reducing the Dimensionality of Data with Neural Networks
2010		Understanding the difficulty of training deep feedforward neural networks
2012		A Few Useful Things to Know about Machine Learning
2012	06	Improving neural networks by preventing co-adaptation of feature detectors
2013		on the importance of initialization and momentum in deep learning
2014	05	Deep Learning in Neural Networks: An Overview
2014	06	Dropout: A Simple Way to Prevent Neural Networks from Overfitting
2014	12	Neural Turing Machines
2014	12	Adam: A Method for Stochastic Optimization
2014	12	Sequence to Sequence Learning with Neural Networks
2015		Deep learning
2015	02	Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
2015	02	Human-level control through deep reinforcement learning
2015	03	Distilling the Knowledge in a Neural Network
2015	05	U-Net: Convolutional Networks for Biomedical Image Segmentation
2015	06	You Only Look Once: Unified, Real-Time Object Detection
2015	09	Continuous control with deep reinforcement learning
2015	09	Deep Reinforcement Learning with Double Q-learning
2015	12	Deep Residual Learning for Image Recognition
2016	06	XGBoost: A Scalable Tree Boosting System
2016	08	Densely Connected Convolutional Networks
2016	09	Understanding deep learning requires rethinking generalization
2017	03	Mask R-CNN
2017	08	Focal Loss for Dense Object Detection
2017	12	LightGBM: A Highly Efficient Gradient Boosting Decision Tree
2018	03	The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
2018	03	Group Normalization
2018	04	Partial Convolution based Padding
2018	05	Born Again Neural Networks
2018	06	Neural Ordinary Differential Equations
2018	11	Rethinking ImageNet Pre-training
2018	12	Graph neural networks: A review of methods and applications
2019	12	Deep Double Descent: Where Bigger Models and More Data Hurt
2019	12	Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data
2020	03	Autoencoders
2020	06	Hopfield Networks is All You Need

Generative Models (VAE, GAN and Diffusion Models)

GAN

Year	Month	Title
2014	06	Generative Adversarial Nets
2014	10	Neural Turing Machines
2014	11	Conditional Generative Adversarial Nets
2015	11	Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (DCGAN)
2016	06	Improved Techniques for Training GANs
2016	09	Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (SRGAN)
2016	11	Image-to-Image Translation with Conditional Adversarial Networks (Pix2Pix)
2017	01	Wasserstein GAN
2017	03	Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (CycleGAN)
2017	03	BEGAN: Boundary Equilibrium Generative Adversarial Networks
2017	03	Improved Training of Wasserstein GANs
2017	10	Progressive Growing of GANs for Improved Quality, Stability, and Variation (ProGAN)
2017	11	StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
2018	01	Which Training Methods for GANs do actually Converge?
2018	02	Spectral Normalization for Generative Adversarial Networks
2018	04	A Fully Progressive Approach to Single-Image Super-Resolution (ProGanSR)
2018	05	Self-Attention Generative Adversarial Networks
2018	06	The relativistic discriminator: a key element missing from standard GAN
2018	09	Large Scale GAN Training for High Fidelity Natural Image Synthesis (BigGAN)
2018	12	A Style-Based Generator Architecture for Generative Adversarial Networks (StyleGAN)
2018	09	ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
2019	12	Analyzing and Improving the Image Quality of StyleGAN (StyleGAN2)
2021	06	Alias-Free Generative Adversarial Networks (StyleGAN3)
2023	01	StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis
2023	03	GigaGAN: Scaling up GANs for Text-to-Image Synthesis
2023	03	Consistency Models

Diffusion Models

Year	Month	Title	Resources
2008		Extracting and Composing Robust Features with Denoising Autoencoders
2010	12	A Connection Between Score Matching and Denoising Autoencoders
2013	12	Auto-Encoding Variational Bayes
2015	03	Deep Unsupervised Learning using Nonequilibrium Thermodynamics
2016	06	Conditional Image Generation with PixelCNN Decoders (PixelCNN)
2017	06	Attention Is All You Need
2017	11	Neural Discrete Representation Learning (VQVAE)
2017	01	Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications (PixelCNN++)	GitHub
2019	06	Generating Diverse High-Fidelity Images with VQ-VAE-2
2019	07	Generative Modeling by Estimating Gradients of the Data Distribution
2020	06	Denoising Diffusion Probabilistic Models (DDPM)	GitHub
2020	10	Denoising Diffusion Implicit Models (DDIM)	GitHub
2020	10	An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)	On-HF
2020	11	Score-Based Generative Modeling through Stochastic Differential Equations	GitHub
2021	01	How to Train Your Energy-Based Models
2021	02	Learning Transferable Visual Models From Natural Language Supervision (CLIP)
2021	02	Improved Denoising Diffusion Probabilistic Models
2021	05	Diffusion Models Beat GANs on Image Synthesis (Guided Diffusion)	GitHub
2021	06	Low-Rank Adaptation of Large Language Models (LoRA)	HF-Training
2021	12	High-Resolution Image Synthesis with Latent Diffusion Models (Stable Diffusion)	GitHub
2021	12	Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models (GLIDE)	GitHub
2022	02	Pseudo Numerical Methods for Diffusion Models on Manifolds (PNDM)	HF-Pipeline
2022	04	Hierarchical Text-Conditional Image Generation with CLIP Latents (DALL-E 2)	Project Page
2022	05	Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (Imagen)	Project Page
2022	06	Elucidating the Design Space of Diffusion-Based Generative Models (EulerDiscrete)	HF-Schedulers
2022	06	DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
2022	08	Understanding Diffusion Models: A Unified Perspective
2022	11	Null-text Inversion for Editing Real Images using Guided Diffusion Models	YouTube
2022	11	DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models	HF-Scheduler
2022	12	Scalable Diffusion Models with Transformers (DiT)	Project Page
2022	12	Reproducible scaling laws for contrastive language-image learning
2023	02	Adding Conditional Control to Text-to-Image Diffusion Models (ControlNET)	HF-ControlNet
2023	02	UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models	HF-Scheduler
2023	03	MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation	HF-Pipeline
2023	04	Generative Novel View Synthesis with 3D-Aware Diffusion Models	Project Page
2023	07	Improving Latent Diffusion Models for High-Resolution Image Synthesis (SDXL)	HF-Model
2023	11	LCM-LoRA: A Universal Stable-Diffusion Acceleration Module

Others

Year	Month	Title
2012	12	ImageNet Classification with Deep Convolutional Neural Networks
2013	12	Playing Atari with Deep Reinforcement Learning
2014	09	Neural Machine Translation by Jointly Learning to Align and Translate
2015	02	Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
2015	08	A Neural Algorithm of Artistic Style
2016	01	Mastering the game of Go with deep neural networks and tree search
2016	03	Perceptual Losses for Real-Time Style Transfer and Super-Resolution
2016	09	WaveNet: A Generative Model for Raw Audio
2017	01	UNet++: A Nested U-Net Architecture for Medical Image Segmentation
2017	06	Unsupervised Body Part Regression via Spatially Self-ordering Convolutional Neural Networks
2018	04	Image Inpainting for Irregular Holes Using Partial Convolutions
2018	07	IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis
2018	07	Glow: Generative Flow with Invertible 1x1 Convolutions
2018	10	Noise2Noise: Learning Image Restoration without Clean Data
2019	01	Panoptic Feature Pyramid Networks
2019	01	High-Quality Self-Supervised Deep Image Denoising
2019	03	Semantic Image Synthesis with Spatially-Adaptive Normalization
2020	03	PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models
2020	03	NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
2020	12	nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
2022	01	Instant Neural Graphics Primitives with a Multiresolution Hash Encoding