AI Reseach and Artifacts

• This repository contains Artificial Intelligence and Machine Learning related research, artificats, tutorials, articles. My intention is to help you through your journey of becoming Artificial Intelligence and Machine Learning champion!
• If you want to contribute to this list, please add your comments to this gits comments section.

Contents

• Introduction
• Genetic Algorithms
• Datasets
• Cheat Sheets
• Classification
• Linear Regression
• Logistic Regression
• Model Validation using Resampling
  • Cross Validation
  • Bootstraping
• Deep Learning
  • Frameworks
  • Feed Forward Networks
  • Recurrent Neural Nets, LSTM, GRU
  • Restricted Boltzmann Machine, DBNs
  • Autoencoders
  • Convolutional Neural Nets
• Natural Language Processing
  • Topic Modeling, LDA
  • Word2Vec
• Computer Vision
• Support Vector Machine
• Reinforcement Learning
• Decision Trees
• Random Forest / Bagging
• Boosting
• Ensembles
• Stacking Models
• VC Dimension
• Bayesian Machine Learning
• Semi Supervised Learning
• Optimizations
• Tutorials on Python

Imp sites

Here are few Imp sites on AI research. Amazon recenly opened its internal AI traing course, good place to start learning AI. Andrew Ng course on Courera is very good start as well.

• Amazon machine learning course:
• Machine Learning Course by Andrew Ng (Stanford University)
• Fast AI0
• AI training:
• Google AI Research:
• Microsoft AI Research:
• Facebook AI research:
• Uber AI research:
• In-depth introduction to machine learning in 15 hours of expert videos
• Awesome Artificial Intelligence (GitHub Repo)
• Dive into Machine Learning
• A curated list of awesome Machine Learning frameworks, libraries and software
• A curated list of awesome data visualization libraries and resources.
• An awesome Data Science repository to learn and apply for real world problems
• The Open Source Data Science Masters
• Machine Learning FAQs on Cross Validated
• Machine Learning algorithms that you should always have a strong understanding of
• List of Machine Learning Concepts
• MIT Machine Learning Lecture Slides
• Comparison Supervised Learning Algorithms
• Learning Data Science Fundamentals
• Have Fun With Machine Learning
• Twitter's Most Shared #machineLearning Content From The Past 7 Days
• TED talks on AI

Genetic Algorithms

• Genetic Algorithms Wikipedia Page
• Genetic Programming in Python (GitHub)

Useful Blogs

• Fast AI blog Fast AI Blog
• [AWS Machine Learning blog] (https://aws.amazon.com/blogs/machine-learning/) AWS Machine Learning blog, specific to AWS but very helpful
• Data science for beginners! - good beginner level blog
• Andrej Karpathy - A blog about Deep Learning and Data Science in general
• Alex Minnaar's Blog - A blog about Machine Learning and Software Engineering
• fastML - Machine learning made easy
• no free hunch | kaggle - The Kaggle Blog about all things Data Science
• AI Junkie - a blog about Artificial Intellingence

Cheat Sheets

• [Machine learning algorithm cheat sheet from Microsoft] (https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-cheat-sheet) very interesting take
• Machine Learning Cheat Sheet nice for interview prep

Datasets

• https://registry.opendata.aws/
• https://aws.amazon.com/about-aws/whats-new/2018/10/public-datasets/
• https://course.fast.ai/datasets.html
• https://ai.google/tools/datasets/
• https://skymind.ai/wiki/open-datasets
• https://www.kaggle.com/datasets?sortBy=relevance&group=all&search=tag%3A%27artificial%20intelligence%27
• https://en.wikipedia.org/wiki/List_of_datasets_for_machine_learning_research
• https://www.datasciencecentral.com/profiles/blogs/lots-of-free-open-source-datasets-to-make-your-ai-better

Classification

• Does Balancing Classes Improve Classifier Performance?

• What is Deviance?

• When to choose which machine learning classifier?

• What are the advantages of different classification algorithms?

• ROC and AUC Explained (related video)

• An introduction to ROC analysis

• Simple guide to confusion matrix terminology

Linear Regression

• General

  • Assumptions of Linear Regression, Stack Exchange

  • Linear Regression Comprehensive Resource

  • Applying and Interpreting Linear Regression

  • What does having constant variance in a linear regression model mean?

  • Difference between linear regression on y with x and x with y

  • Is linear regression valid when the dependant variable is not normally distributed?

• Multicollinearity and VIF

  • Dummy Variable Trap | Multicollinearity

  • Dealing with multicollinearity using VIFs

• Residual Analysis

  • Interpreting plot.lm() in R

  • How to interpret a QQ plot?

  • Interpreting Residuals vs Fitted Plot

• Outliers

  • How should outliers be dealt with?

• Elastic Net

  • Regularization and Variable Selection via the Elastic Net

Logistic Regression

• Logistic Regression Wiki

• Geometric Intuition of Logistic Regression

• Obtaining predicted categories (choosing threshold)

• Residuals in logistic regression

• Difference between logit and probit models, Logistic Regression Wiki, Probit Model Wiki

• Pseudo R2 for Logistic Regression, How to calculate, Other Details

• Guide to an in-depth understanding of logistic regression

Model Validation using Resampling

• Resampling Explained

• Partioning data set in R

• Implementing hold-out Validaion in R, 2

• Cross Validation

  • How to use cross-validation in predictive modeling

  • Training with Full dataset after CV?

  • Which CV method is best?

  • Variance Estimates in k-fold CV

  • Is CV a subsitute for Validation Set?

  • Choice of k in k-fold CV

  • CV for ensemble learning

  • k-fold CV in R

  • Good Resources

  • Overfitting and Cross Validation

    • Preventing Overfitting the Cross Validation Data | Andrew Ng

    • Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation

    • CV for detecting and preventing Overfitting

    • How does CV overcome the Overfitting Problem

• Bootstrapping

  • Why Bootstrapping Works?

  • Good Animation

  • Example of Bootstapping

  • Understanding Bootstapping for Validation and Model Selection

  • Cross Validation vs Bootstrap to estimate prediction error, Cross-validation vs .632 bootstrapping to evaluate classification performance

Deep Learning

• fast.ai - Practical Deep Learning For Coders

• fast.ai - Cutting Edge Deep Learning For Coders

• A curated list of awesome Deep Learning tutorials, projects and communities

• Lots of Deep Learning Resources

• Interesting Deep Learning and NLP Projects (Stanford), Website

• Core Concepts of Deep Learning

• Understanding Natural Language with Deep Neural Networks Using Torch

• Stanford Deep Learning Tutorial

• Deep Learning FAQs on Quora

• Google+ Deep Learning Page

• Recent Reddit AMAs related to Deep Learning, Another AMA

• Where to Learn Deep Learning?

• Deep Learning nvidia concepts

• Introduction to Deep Learning Using Python (GitHub), Good Introduction Slides

• Video Lectures Oxford 2015, Video Lectures Summer School Montreal

• Deep Learning Software List

• Hacker's guide to Neural Nets

• Top arxiv Deep Learning Papers explained

• Geoff Hinton Youtube Vidoes on Deep Learning

• Awesome Deep Learning Reading List

• Deep Learning Comprehensive Website, Software

• deeplearning Tutorials

• AWESOME! Deep Learning Tutorial

• Deep Learning Basics

• Stanford Tutorials

• Train, Validation & Test in Artificial Neural Networks

• Artificial Neural Networks Tutorials

• Neural Networks FAQs on Stack Overflow

• Deep Learning Tutorials on deeplearning.net

• Neural Networks and Deep Learning Online Book

• Neural Machine Translation

  • Introduction to Neural Machine Translation with GPUs (part 1), Part 2, Part 3

  • Deep Speech: Accurate Speech Recognition with GPU-Accelerated Deep Learning

• Deep Learning Frameworks

  • Torch vs. Theano

  • dl4j vs. torch7 vs. theano

  • Deep Learning Libraries by Language

  • Theano

    • Website

    • Theano Introduction

    • Theano Tutorial

    • Good Theano Tutorial

    • Logistic Regression using Theano for classifying digits

    • MLP using Theano

    • CNN using Theano

    • RNNs using Theano

    • LSTM for Sentiment Analysis in Theano

    • RBM using Theano

    • DBNs using Theano

    • All Codes

    • Deep Learning Implementation Tutorials - Keras and Lasagne

  • Torch

    • Torch ML Tutorial, Code

    • Intro to Torch

    • Learning Torch GitHub Repo

    • Awesome-Torch (Repository on GitHub)

    • Machine Learning using Torch Oxford Univ, Code

    • Torch Internals Overview

    • Torch Cheatsheet

    • Understanding Natural Language with Deep Neural Networks Using Torch

  • Caffe

    • Deep Learning for Computer Vision with Caffe and cuDNN

  • TensorFlow

    • Website

    • TensorFlow Examples for Beginners

    • Stanford Tensorflow for Deep Learning Research Course

      • GitHub Repo

    • Simplified Scikit-learn Style Interface to TensorFlow

    • Learning TensorFlow GitHub Repo

    • Benchmark TensorFlow GitHub

    • Awesome TensorFlow List

    • TensorFlow Book

    • Android TensorFlow Machine Learning Example

      • GitHub Repo

    • Creating Custom Model For Android Using TensorFlow

      • GitHub Repo

• Feed Forward Networks

  • A Quick Introduction to Neural Networks

  • Implementing a Neural Network from scratch, Code

  • Speeding up your Neural Network with Theano and the gpu, Code

  • Basic ANN Theory

  • Role of Bias in Neural Networks

  • Choosing number of hidden layers and nodes,2,3

  • Backpropagation in Matrix Form

  • ANN implemented in C++ | AI Junkie

  • Simple Implementation

  • NN for Beginners

  • Regression and Classification with NNs (Slides)

  • Another Intro

• Recurrent and LSTM Networks

  • awesome-rnn: list of resources (GitHub Repo)

  • Recurrent Neural Net Tutorial Part 1, Part 2, Part 3, Code

  • NLP RNN Representations

  • The Unreasonable effectiveness of RNNs, Torch Code, Python Code

  • Intro to RNN, LSTM

  • An application of RNN

  • Optimizing RNN Performance

  • Simple RNN

  • Auto-Generating Clickbait with RNN

  • Sequence Learning using RNN (Slides)

  • Machine Translation using RNN (Paper)

  • Music generation using RNNs (Keras)

  • Using RNN to create on-the-fly dialogue (Keras)

  • Long Short Term Memory (LSTM)

    • Understanding LSTM Networks

    • LSTM explained

    • Beginner’s Guide to LSTM

    • Implementing LSTM from scratch, Python/Theano code

    • Torch Code for character-level language models using LSTM

    • LSTM for Kaggle EEG Detection competition (Torch Code)

    • LSTM for Sentiment Analysis in Theano

    • Deep Learning for Visual Q&A | LSTM | CNN, Code

    • Computer Responds to email using LSTM | Google

    • LSTM dramatically improves Google Voice Search, Another Article

    • Understanding Natural Language with LSTM Using Torch

    • Torch code for Visual Question Answering using a CNN+LSTM model

    • LSTM for Human Activity Recognition

  • Gated Recurrent Units (GRU)

    • LSTM vs GRU

  • Time series forecasting with Sequence-to-Sequence (seq2seq) rnn models

• Recursive Neural Network (not Recurrent)

  • Recursive Neural Tensor Network (RNTN)

  • word2vec, DBN, RNTN for Sentiment Analysis

• Restricted Boltzmann Machine

  • Beginner's Guide about RBMs

  • Another Good Tutorial

  • Introduction to RBMs

  • Hinton's Guide to Training RBMs

  • RBMs in R

  • Deep Belief Networks Tutorial

  • word2vec, DBN, RNTN for Sentiment Analysis

• Autoencoders: Unsupervised (applies BackProp after setting target = input)

  • Andrew Ng Sparse Autoencoders pdf

  • Deep Autoencoders Tutorial

  • Denoising Autoencoders, Theano Code

  • Stacked Denoising Autoencoders

• Convolutional Neural Networks

  • An Intuitive Explanation of Convolutional Neural Networks

  • Awesome Deep Vision: List of Resources (GitHub)

  • Intro to CNNs

  • Understanding CNN for NLP

  • Stanford Notes, Codes, GitHub

  • JavaScript Library (Browser Based) for CNNs

  • Using CNNs to detect facial keypoints

  • Deep learning to classify business photos at Yelp

  • Interview with Yann LeCun | Kaggle

  • Visualising and Understanding CNNs

Natural Language Processing

• A curated list of speech and natural language processing resources

• Understanding Natural Language with Deep Neural Networks Using Torch

• tf-idf explained

• Interesting Deep Learning NLP Projects Stanford, Website

• NLP from Scratch | Google Paper

• Graph Based Semi Supervised Learning for NLP

• Bag of Words

  • Classification text with Bag of Words

• Topic Modeling

  • LDA, LSA, Probabilistic LSA

  • What is a good explanation of Latent Dirichlet Allocation?

  • Awesome LDA Explanation!. Another good explanation

  • The LDA Buffet- Intuitive Explanation

  • Difference between LSI and LDA

  • Original LDA Paper

  • alpha and beta in LDA

  • Intuitive explanation of the Dirichlet distribution

  • Topic modeling made just simple enough

  • Online LDA, Online LDA with Spark

  • LDA in Scala, Part 2

  • Segmentation of Twitter Timelines via Topic Modeling

  • Topic Modeling of Twitter Followers

• word2vec

  • Google word2vec

  • Bag of Words Model Wiki

  • word2vec Tutorial

  • A closer look at Skip Gram Modeling

  • Skip Gram Model Tutorial, CBoW Model

  • Word Vectors Kaggle Tutorial Python, Part 2

  • Making sense of word2vec

  • word2vec explained on deeplearning4j

  • Quora word2vec

  • Other Quora Resources, 2, 3

  • word2vec, DBN, RNTN for Sentiment Analysis

• Text Clustering

  • How string clustering works

  • Levenshtein distance for measuring the difference between two sequences

  • Text clustering with Levenshtein distances

• Text Classification

  • Classification Text with Bag of Words

• Language learning with NLP and reinforcement learning

• Kaggle Tutorial Bag of Words and Word vectors, Part 2, Part 3

• What would Shakespeare say (NLP Tutorial)

• A closer look at Skip Gram Modeling

Computer Vision

• Awesome computer vision (github)

• Awesome deep vision (github)

Support Vector Machine

• Highest Voted Questions about SVMs on Cross Validated

• Help me Understand SVMs!

• SVM in Layman's terms

• How does SVM Work | Comparisons

• A tutorial on SVMs

• Practical Guide to SVC, Slides

• Introductory Overview of SVMs

• Comparisons

  • SVMs > ANNs, ANNs > SVMs, Another Comparison

  • Trees > SVMs

  • Kernel Logistic Regression vs SVM

  • Logistic Regression vs SVM, 2, 3

• Optimization Algorithms in Support Vector Machines

• Variable Importance from SVM

• Software

  • LIBSVM

  • Intro to SVM in R

• Kernels

  • What are Kernels in ML and SVM?

  • Intuition Behind Gaussian Kernel in SVMs?

• Probabilities post SVM

  • Platt's Probabilistic Outputs for SVM

  • Platt Calibration Wiki

  • Why use Platts Scaling

  • Classifier Classification with Platt's Scaling

Reinforcement Learning

• Awesome Reinforcement Learning (GitHub)

• RL Tutorial Part 1, Part 2

Decision Trees

• Wikipedia Page - Lots of Good Info

• FAQs about Decision Trees

• Brief Tour of Trees and Forests

• Tree Based Models in R

• How Decision Trees work?

• Weak side of Decision Trees

• Thorough Explanation and different algorithms

• What is entropy and information gain in the context of building decision trees?

• Slides Related to Decision Trees

• How do decision tree learning algorithms deal with missing values?

• Using Surrogates to Improve Datasets with Missing Values

• Good Article

• Are decision trees almost always binary trees?

• Pruning Decision Trees, Grafting of Decision Trees

• What is Deviance in context of Decision Trees?

• Discover structure behind data with decision trees - Grow and plot a decision tree to automatically figure out hidden rules in your data

• Comparison of Different Algorithms

  • CART vs CTREE

  • Comparison of complexity or performance

  • CHAID vs CART , CART vs CHAID

  • Good Article on comparison

• CART

  • Recursive Partitioning Wikipedia

  • CART Explained

  • How to measure/rank “variable importance” when using CART?

  • Pruning a Tree in R

  • Does rpart use multivariate splits by default?

  • FAQs about Recursive Partitioning

• CTREE

  • party package in R

  • Show volumne in each node using ctree in R

  • How to extract tree structure from ctree function?

• CHAID

  • Wikipedia Artice on CHAID

  • Basic Introduction to CHAID

  • Good Tutorial on CHAID

• MARS

  • Wikipedia Article on MARS

• Probabilistic Decision Trees

  • Bayesian Learning in Probabilistic Decision Trees

  • Probabilistic Trees Research Paper

Random Forest / Bagging

• Awesome Random Forest (GitHub)**

• How to tune RF parameters in practice?

• Measures of variable importance in random forests

• Compare R-squared from two different Random Forest models

• OOB Estimate Explained | RF vs LDA

• Evaluating Random Forests for Survival Analysis Using Prediction Error Curve

• Why doesn't Random Forest handle missing values in predictors?

• How to build random forests in R with missing (NA) values?

• FAQs about Random Forest, More FAQs

• Obtaining knowledge from a random forest

• Some Questions for R implementation, 2, 3

Boosting

• Boosting for Better Predictions

• Boosting Wikipedia Page

• Introduction to Boosted Trees | Tianqi Chen

• Gradient Boosting Machine

  • Gradiet Boosting Wiki

  • Guidelines for GBM parameters in R, Strategy to set parameters

  • Meaning of Interaction Depth, 2

  • Role of n.minobsinnode parameter of GBM in R

  • GBM in R

  • FAQs about GBM

  • GBM vs xgboost

• xgboost

  • xgboost tuning kaggle

  • xgboost vs gbm

  • xgboost survey

  • Practical XGBoost in Python online course (free)

• AdaBoost

  • AdaBoost Wiki, Python Code

  • AdaBoost Sparse Input Support

  • adaBag R package

  • Tutorial

Ensembles

• Wikipedia Article on Ensemble Learning

• Kaggle Ensembling Guide

• The Power of Simple Ensembles

• Ensemble Learning Intro

• Ensemble Learning Paper

• Ensembling models with R, Ensembling Regression Models in R, Intro to Ensembles in R

• Ensembling Models with caret

• Bagging vs Boosting vs Stacking

• Good Resources | Kaggle Africa Soil Property Prediction

• Boosting vs Bagging

• Resources for learning how to implement ensemble methods

• How are classifications merged in an ensemble classifier?

Stacking Models

• Stacking, Blending and Stacked Generalization

• Stacked Generalization (Stacking)

• Stacked Generalization: when does it work?

• Stacked Generalization Paper

Vapnik–Chervonenkis Dimension

• Wikipedia article on VC Dimension

• Intuitive Explanantion of VC Dimension

• Video explaining VC Dimension

• Introduction to VC Dimension

• FAQs about VC Dimension

• Do ensemble techniques increase VC-dimension?

Bayesian Machine Learning

• Bayesian Methods for Hackers (using pyMC)

• Should all Machine Learning be Bayesian?

• Tutorial on Bayesian Optimisation for Machine Learning

• Bayesian Reasoning and Deep Learning, Slides

• Bayesian Statistics Made Simple

• Kalman & Bayesian Filters in Python

• Markov Chain Wikipedia Page

Semi Supervised Learning

• Wikipedia article on Semi Supervised Learning

• Tutorial on Semi Supervised Learning

• Graph Based Semi Supervised Learning for NLP

• Taxonomy

• Video Tutorial Weka

• Unsupervised, Supervised and Semi Supervised learning

• Research Papers 1, 2, 3

Optimization

• Mean Variance Portfolio Optimization with R and Quadratic Programming

• Algorithms for Sparse Optimization and Machine Learning

• Optimization Algorithms in Machine Learning, Video Lecture

• Optimization Algorithms for Data Analysis

• Video Lectures on Optimization

• Optimization Algorithms in Support Vector Machines

• The Interplay of Optimization and Machine Learning Research

• Hyperopt tutorial for Optimizing Neural Networks’ Hyperparameters

Tutorials on Python

• For a collection of Data Science Tutorials using Python, please refer to this list.