SLRProject.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple Linear Regression Project\n",
    "\n",
    "\n",
    "## Modelling the linear relationship between Sales and Advertising dataset\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Project overview\n",
    "\n",
    "\n",
    "In this project, I build a Simple Linear Regression model to study the linear relationship between Sales and Advertising dataset for a dietary weight control product.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Linear Regression\n",
    "\n",
    "\n",
    "Linear Regression is a statistical technique which is used to find the linear relationship between dependent and one or more independent variables. This technique is applicable for Supervised learning Regression problems where we try to predict a continuous variable.\n",
    "\n",
    "\n",
    "Linear Regression can be further classified into two types – Simple and Multiple Linear Regression. In this project, I employ Simple Linear Regression technique where I have one independent and one dependent variable. It is the simplest form of Linear Regression where we fit a straight line to the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Simple Linear Regression (SLR)\n",
    "\n",
    "Simple Linear Regression (or SLR) is the simplest model in machine learning. It models the linear relationship between the independent and dependent variables. \n",
    "\n",
    "In this project, there is one independent or input variable which represents the Sales data and is denoted by X. Similarly, there is one dependent or output variable which represents the Advertising data and is denoted by y. We want to build a linear relationship between these variables. This linear relationship can be modelled by mathematical equation of the form:-\n",
    "\t\t\t\t \n",
    "                 \n",
    "                 Y = β0   + β1*X    -------------   (1)\n",
    "                 \n",
    "\n",
    "In this equation, X and Y are called independent and dependent variables respectively,\n",
    "\n",
    "β1 is the coefficient for independent variable and\n",
    "\n",
    "β0 is the constant term.\n",
    "\n",
    "β0 and β1 are called parameters of the model.\n",
    " \n",
    "\n",
    "\n",
    "For simplicity, we can compare the above equation with the basic line equation of the form:-\n",
    " \n",
    "                   y = ax + b       ----------------- (2)\n",
    "\n",
    "We can see that \n",
    "\n",
    "slope of the line is given by, a =  β1,  and\n",
    "\n",
    "intercept of the line by b =  β0. \n",
    "\n",
    "\n",
    "In this Simple Linear Regression model, we want to fit a line which estimates the linear relationship between X and Y. So, the question of fitting reduces to estimating the parameters of the model β0 and β1. \n",
    "\n",
    " \n",
    "\n",
    "## Ordinary Least Square Method\n",
    "\n",
    "As I have described earlier, the Sales and Advertising data are given by X and y respectively. We can draw a scatter plot between X and y which shows the relationship between them.\n",
    "\n",
    " \n",
    "\n",
    "Now, our task is to find a line which best fits this scatter plot. This line will help us to predict the value of any Target variable for any given Feature variable. This line is called **Regression line**. \n",
    "\n",
    "\n",
    "We can define an error function for any line. Then, the regression line is the one which minimizes the error function. Such an error function is also called a **Cost function**. \n",
    "\n",
    "\n",
    "## Cost Function\n",
    "\n",
    "We want the Regression line to resemble the dataset as closely as possible. In other words, we want the line to be as close to actual data points as possible. It can be achieved by minimizing the vertical distance between the actual data point and fitted line. I calculate the vertical distance between each data point and the line. This distance is called the **residual**. \n",
    "\n",
    "\n",
    "So, in a regression model, we try to minimize the residuals by finding the line of best fit. The residuals are represented by the vertical dotted lines from actual data points to the line.\n",
    "\n",
    " \n",
    "We can try to minimize the sum of the residuals, but then a large positive residual would cancel out a large negative residual. For this reason, we minimize the sum of the squares of the residuals. \n",
    "\n",
    "\n",
    "Mathematically, we denote actual data points by yi and predicted data points by ŷi. So, the residual for a data point i would be given as \n",
    "\t\t\t\t\n",
    "                di = yi -  ŷi\n",
    "\n",
    "Sum of the squares of the residuals is given as:\n",
    "\n",
    "\t\t\t\tD = Ʃ di**2       for all data points\n",
    "                \n",
    "\n",
    "This is the **Cost function**. It denotes the total error present in the model which is the sum of the total errors of each individual data point. \n",
    "\n",
    "We can estimate the parameters of the model β0 and β1 by minimize the error in the model by minimizing D. Thus, we can find the regression line given by equation (1).\n",
    "\n",
    "\n",
    "This method of finding the parameters of the model and thus regression line is called **Ordinary Least Square Method**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The problem statement\n",
    "\n",
    "The aim of building a machine learning model is to solve a problem and to define a metric to measure model performance. \n",
    "\n",
    "The problem is to model and investigate the linear relationship between Sales and Advertising dataset for a dietary weight control product. \n",
    "\n",
    "I have used two performance metrics RMSE (Root Mean Square Value) and R2 Score value to compute our model performance.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Software information\n",
    "\n",
    "I did this project using Jupyter notebook (Jupyter notebook server 5.5.0).\n",
    "\n",
    "The server is running on Python (Python 3.6.5), Anaconda distribution.\n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python libraries\n",
    "\n",
    "I have Anaconda Python distribution installed on my system. It comes with most of the standard Python libraries I need for this project. The basic Python libraries used in this project are:-\n",
    "\n",
    " •\tNumpy – It provides a fast numerical array structure and operating functions.\n",
    " \n",
    " •\tpandas – It provides tools for data storage, manipulation and analysis tasks.\n",
    " \n",
    " •\tScikit-Learn – The required machine learning library in Python.\n",
    " \n",
    " •\tMatplotlib – It is the basic plotting library in Python. It provides tools for making plots. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import necessary libraries\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "% matplotlib inline\n",
    "\n",
    "# The above command sets the backend of matplotlib to the 'inline' backend. \n",
    "# It means the output of plotting commands is displayed inline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## About the dataset\n",
    "\n",
    "The data set has been imported from the econometrics website with the following url:-\n",
    "\n",
    "http://www.econometrics.com/intro/sales.htm\n",
    "\n",
    "This data set contains Sales and Advertising expenditures for a dietary weight control product. It contains monthly data for 36 months. The variables in this data set are Sales and Advertising.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the data\n",
    "\n",
    "url = \"C:/project_datasets/SALES.txt\"\n",
    "df = pd.read_csv(url, sep='\\t', header=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploratory data analysis\n",
    "\n",
    "\n",
    "First, I import the dataset into the dataframe with the standard read_csv () function of pandas library and assign it to the df variable. Then, I conducted exploratory data analysis to get a feel for the data.\n",
    "\n",
    "\n",
    "### pandas shape attribute\n",
    "\n",
    "The shape attribute of the pandas dataframe gives the dimensions of the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(36, 2)\n"
     ]
    }
   ],
   "source": [
    "# Exploratory data analysis\n",
    "\n",
    "# View the dimensions of df\n",
    "\n",
    "print(df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### pandas head() method\n",
    "\n",
    "I viewed the top 5 rows of the pandas dataframe with the pandas head() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0     1\n",
      "0  12.0  15.0\n",
      "1  20.5  16.0\n",
      "2  21.0  18.0\n",
      "3  15.5  27.0\n",
      "4  15.3  21.0\n"
     ]
    }
   ],
   "source": [
    "# View the top 5 rows of df\n",
    "\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### pandas columns attribute\n",
    "\n",
    "I renamed the column labels of the dataframe with the columns attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Rename columns of df dataframe\n",
    "\n",
    "df.columns = ['Sales', 'Advertising']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### column names renamed\n",
    "\n",
    "I viewed the renamed column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Sales  Advertising\n",
      "0   12.0         15.0\n",
      "1   20.5         16.0\n",
      "2   21.0         18.0\n",
      "3   15.5         27.0\n",
      "4   15.3         21.0\n"
     ]
    }
   ],
   "source": [
    "# View the top 5 rows of df with column names renamed\n",
    "\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### pandas info() method\n",
    "\n",
    "I viewed the summary of the dataframe with the pandas info() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 36 entries, 0 to 35\n",
      "Data columns (total 2 columns):\n",
      "Sales          36 non-null float64\n",
      "Advertising    36 non-null float64\n",
      "dtypes: float64(2)\n",
      "memory usage: 656.0 bytes\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# View dataframe summary\n",
    "\n",
    "print(df.info())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### pandas describe() method\n",
    "\n",
    "I look at the descriptive statistics of the dataframe with the pandas describe() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "           Sales  Advertising\n",
      "count  36.000000    36.000000\n",
      "mean   24.255556    28.527778\n",
      "std     6.185118    18.777625\n",
      "min    12.000000     1.000000\n",
      "25%    20.300000    15.750000\n",
      "50%    24.250000    23.000000\n",
      "75%    28.600000    41.000000\n",
      "max    36.500000    65.000000\n"
     ]
    }
   ],
   "source": [
    "# View descriptive statistics\n",
    "\n",
    "print(df.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Independent and Dependent Variables\n",
    "\n",
    "\n",
    "In this project, I refer Independent variable as Feature variable and Dependent variable as Target variable. These variables are also recognized by different names as follows: -\n",
    "\n",
    "\n",
    "### Independent variable\n",
    "\n",
    "Independent variable is also called Input variable and is denoted by X. In practical applications, independent variable is also called Feature variable or Predictor variable. We can denote it as:-\n",
    "\n",
    "Independent or Input variable (X) = Feature variable = Predictor variable \n",
    "\n",
    "\n",
    "### Dependent variable\n",
    "\n",
    "Dependent variable is also called Output variable and is denoted by y. \n",
    "\n",
    "Dependent variable is also called Target variable or Response variable. It can be denoted it as follows:-\n",
    "\n",
    "Dependent or Output variable (y) = Target variable = Response variable\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Declare feature variable and target variable\n",
    "\n",
    "X = df['Sales'].values\n",
    "y = df['Advertising'].values\n",
    "\n",
    "# Sales and Advertising data values are given by X and y respectively.\n",
    "\n",
    "# Values attribute of pandas dataframe returns the numpy arrays."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visual exploratory data analysis\n",
    "\n",
    "I visualize the relationship between X and y by plotting a scatterplot between X and y."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm8HGWd7/HPNwkhBKIhEGIgGzrInoR4QFzAgICoIMvAIEYnIFdcGWfmjgIyIM7IvcGrRq8zwkQRIgYIAyiIXAWUsCjbCQSERBYhgZgQToBgEggk4Xf/qOcknUOfc/os1dXL9/169au7q6uqf7V0/bqep56nFBGYmVnzGlB0AGZmViwnAjOzJudEYGbW5JwIzMyanBOBmVmTcyIwM2tyTgRVJmmqpKV9mP5iSef2Z0xlviMk/U0nn02TdHMv53u+pJ/1LbrGJOkySd8sOo7O9HW/LTO/qu4Lkh6VNLWbccZJWiNpYJXCqhlOBL0gabGkV9NO81z6EW+Xw/ecIumu0mER8bmI+Pf+/q5KRcSciDii2t9bD0lE0hhJ10paKellSX+UdErRcVWLMk9JWlhwHG9KqhGxd0TM62q6iHgmIraLiI25BliDnAh67+iI2A6YDOwHnF1wPFa8y4FngfHADsDfAysKjai6DgZ2At4uaf8iAmjGf/P9wYmgjyLiOeA3ZAkBAElbS/q2pGckrUjFOduUm17SWZL+LGm1pIWSjkvD9wQuBt6TzjxWpeFb/NuR9BlJT0p6UdINknYu+SwkfU7SE5JekvSfkpQ++xtJt6d/rislze0Q2mGdTLfFWUr6jn9I/wRXSvo/krrar4ZImpuW9wFJk0rmtXP6R90m6WlJ/5CGHwl8DTgprYuHJB0i6Y8l094q6b6S93dJOrar+abPBpRsgxckXS1pRPpsQlq+6WlbrpR0ThfLtj9wWUSsjYgNEfFgRPy/ku/673QG+bKkOyTt3dmMJB0laYGkVZL+IGliyWdnSvpLWoePSfpgJ/P4qKQHJf1V0rOSzi/5rMtlk7RN2tdeSv/wKzmwTweuB25Kr0tj2TXtb6sl3QLsWPLZryV9qcP4D0k6Pr3eQ9ItaR9/TNLflYx3maSLJN0kaS1wGjAN+GraV36Zxlss6bD0+gBJrWm9rJD03Q7rZFB6P0/Sv0v6fYr7Zkmlcf+9pCVpvzm39DvqTkT40cMHsBg4LL0eA/wR+H7J598DbgBGAMOAXwL/O302FVhaMu6JwM5kSfkkYC0wOn12CnBXh+++DPhmen0osBKYAmwN/AC4o2TcAG4EhgPjgDbgyPTZlcA56XuHAO+vcLotYkrj3paWdRzwOPA/Ollv5wPrgROArYB/AZ5OrwcA84HzgMHA24GngA+VTPuzknkNAV4lO6AMAp4DlqX1vU36bIcK5vuPwD1pO24N/BdwZfpsQlq+H6V5TgJeA/bsZPluBX4PfBwYV+bzT6f4tk77yIJOtusU4Hng3cBAsoPq4jTd7mRnHTuXxPiOTuKZCuyb1sFEsrOTYytZNmAGcGfarmOBRyjZb8t811Dgr8BHgL8l2y8Hl3x+N/DdtAwHA6vbtyfZmdPvS8bdC1iVxt02Le+paTtPSfPeu2S9vQy8j8378qZ12clv9m7gU+n1dsCBHdbJoPR+HvBn4J1pHc0DZpTEuAZ4P9l+9W2yffuwoo9PvTqmFR1APT7STrUm7cwB/BYYnj4T2cH8HSXjvwd4Or2e2s0PagFwTHp9Cl0ngkuAb5V8tl3aGSek98GWB/irgbPS658Cs4AxZWLoarotYkrjHlny/gvAbztZtvOBe0reDwCWAweRHfSe6TD+2cClJdP+rMPndwLHAwcCN6c4jwQOAR5O43Q330XAB0s+G53W4aCSA8OYks/vAz7eyfJtT3YAfRTYmLbl/p2MOzzN+61ltutFwL93GP8x4APA35AlicOArXq4334PmJled7lsZMmydLueTtf77SfJ/jAMIjuArwKOS5+NAzYA25aMfwWbE8Ewst/M+PT+AuAn6fVJwJ0dvuu/gK+XrLefdvYb6fCbbU8EdwDfAHbsME77OilNBP/aYd/+dXp9HukPQ3o/FHidOk0ELhrqvWMjYhjZgX0PNp/qjiTbKean0/pVwK/T8DdJp5cLSsbdp2Re3dkZWNL+JiLWAC8Au5SM81zJ61fIkgXAV8mS1n3Krqj4dId5dzZdOc+WvF6S4up23Ih4A1iaxh8P7Ny+HtK6+Bowqot53U62/g9Or+eRHSw/kN5TwXzHAz8v+WwR2UG89HsrWhcR8VJEnBURe6fpFwC/UGagpBmpCOqvZAcmKL+txwP/s0PMY8nOAp4kO4s5H3he0lUqKQ4sJendkm5LRWIvA58r832dLdvOvHm7dmU6cHVkRWKvAdexuXhoZ+CliFhbbn4RsRr4FdmZFOl5Tno9Hnh3h3UxDXhbybxK46zEaWT/8v8k6X5JR3UxbkXrJyJeIfvt1SUngj6KiNvJ/oF8Ow1aSVYssXdEDE+Pt0ZWsbwFSePJTs2/BOwQEcPJTsHVPvtuvn4Z2Q+lfX7bkhWH/KWCuJ+LiM9ExM7AZ4EfqpNLRiswtuT1uBRXt+Mqq0sYk8Z/luysaXjJY1hEfKQ95DLz6pgIbufNiaC7+T4LfLjD50Miott12JWIWEm2T+xMVrzyCeAYsn/ybyX79wmbt3WpZ4ELOsQ0NCKuTPO+IiLeT7btA7iwkzCuICuiHBsRbyWrcyr3feUs583btSxJY8iKKT+Z6kCeIyv++0gqU18ObJ/2z87mdyVwsqT3kBXD3JaGPwvc3mFdbBcRny+ZtuO+0eXvJiKeiIiTySq2LwSu6RBbJZaT7btAVqdC9turS04E/eN7wOGSJqd/uT8CZkraCUDSLpI+VGa6bcl22rY03qlkZwTtVgBjJA3u5HuvAE6VNFnS1sD/Au6NiMXdBSzpxPQDBngpxdHby+a+Iml7SWOBLwMdK55LvUvS8alC7h/JyqXvISuW+KuyitBt0j/ofbT56pMVwARtWRH9B7Iy8wOA+yLiUdI/SLLTfyqY78XABSkpI2mkpGN6sxIkXZjmPUjSMODzwJMR8QJZ8cdrZP8ah5Jtq878CPhc+kcvSdsqq/gdJml3SYem7b2O7E9HZ9ttGPBiRKyTdABZMqrU1cDZabuOAc7oYtxPkdUN7U520cRksn/cS4GTI2IJ0Ap8Q9JgSe8Hju4wj5vItt2/AXPT7wiyuqp3SvqUpK3SY39lF1N0ZgVZXVBZkj4paWT6jlVpcE/3/WuAoyW9N/0+v0HlSbbmOBH0g4hoIytzb2/odSbwJHBPKga4lexH0nG6hcB3yCqvVpBV7P2+ZJTfkZU3PydpZZnpf5u+81qyfyjvYPPpdXf2B+6VtIbsX+OXI+LpCqft6HqyCtkFZKf4l3Qz7klkyedTwPERsT6ya7ePJjuIPE12ZvVjsn/PAP+dnl+Q9ABAKmp4AHg0Il5Pn98NLImI59M43c33+2n5b5a0miwpvbt3q4GhwM/JDi5PkR3YPpY++ylZcchfgIXpe8qKiFbgM8B/kK2nJ8nqZiArf5+RluM5sn+1X+tkVl8A/i0t13lkB/dKfSPF+zRZ/cvlXYw7HfhhOsvc9CBLsu3FQ58gW68vAl8nWx+blBQnHUb2B6d9+GrgCLL9ehnZMl9Ith46cwmwVypK+kWZz48EHk37/vfJ6kXWdTG/N0l/Os4AriL77a0mq7t5rSfzqRVKFR1mvSIpgN1S2bVZU1LWoHQV2W+ht3+oCuMzAjOzXpB0tKShqX7h22SXkS8uNqrecSIwM+udY8iKq5YBu5EVMdVlEYuLhszMmpzPCMzMmtygogOoxI477hgTJkwoOgwzs7oyf/78lRFRtjFrqbpIBBMmTKC1tbXoMMzM6oqk7lqEAy4aMjNrek4EZmZNzonAzKzJORGYmTU5JwIzsybnRGBmNWfOHJgwAQYMyJ7nzOluisZS7eWvi8tHzax5zJkDp58Or7ySvV+yJHsPMG1acXFVSxHLXxddTLS0tITbEZg1hwkTsoNfR+PHw+LF1Y6m+vpz+SXNj4iW7sZz0ZCZ1ZRnnunZ8EZTxPI7EZhZTRnXyU0xOxveaIpYficCM6spF1wAQ4duOWzo0Gx4Myhi+Z0IzKymTJsGs2ZlZeJS9jxrVnNUFEMxy+/KYjOzBuXKYjMzq4gTgZlZk3MiMMtRLbaQrcWYrFhuWWyWk1psIVuLMVnxXFlslpNabCFbizFZflxZbFawWmwhW4sxWfGcCMxyUostZGsxJiueE4FZTmqxhWwtxmTFyzURSBou6RpJf5K0SNJ7JI2QdIukJ9Lz9nnGYFaUWmwhW4sxWfFyrSyWNBu4MyJ+LGkwMBT4GvBiRMyQdBawfUSc2dV8XFlsZtZzhVcWS3oLcDBwCUBEvB4Rq4BjgNlptNnAsXnFYGZm3cuzaOjtQBtwqaQHJf1Y0rbAqIhYDpCedyo3saTTJbVKam1ra8sxTDOz5pZnIhgETAEuioj9gLXAWZVOHBGzIqIlIlpGjhyZV4xmZk0vz0SwFFgaEfem99eQJYYVkkYDpOfnc4zBzMy6kVsiiIjngGcl7Z4GfRBYCNwATE/DpgPX5xWDmZl1L+++hs4A5qQrhp4CTiVLPldLOg14Bjgx5xjMzKwLuSaCiFgAlLt06YN5fq+ZmVXOLYvNzJqcE4GZWZNzIjAza3JOBGZmTc6JwMysyTkRmJk1OScCM7Mm50RgZtbknAjMzJqcE4GZWZNzIjAza3JOBGZmTc6JwMysyTkRmJk1OScCM7Mm50Rg1uTmzIEJE2DAgOx5zpyiI7Jqy/sOZWZWw+bMgdNPh1deyd4vWZK9B5g2rbi4rLp8RmDWxM45Z3MSaPfKK9lwax5OBGZN7JlnejbcGpMTgVkTGzeuZ8OtMTkRmDWxCy6AoUO3HDZ0aDbcmkeuiUDSYkl/lLRAUmsaNkLSLZKeSM/b5xmDmXVu2jSYNQvGjwcpe541yxXFzUYRkd/MpcVAS0SsLBn2LeDFiJgh6Sxg+4g4s6v5tLS0RGtra25xmpk1IknzI6Klu/GKKBo6BpidXs8Gji0gBjMzS/JOBAHcLGm+pHR1MqMiYjlAet4p5xjMzKwLeSeC90XEFODDwBclHVzphJJOl9QqqbWtrS2/CM06cEvb+uNt1je5JoKIWJaenwd+DhwArJA0GiA9P9/JtLMioiUiWkaOHJlnmGabtLe0XbIEIja3tPWBpXZ5m/VdbolA0raShrW/Bo4AHgFuAKan0aYD1+cVg1lPuaVt/fE267s8+xoaBfxcUvv3XBERv5Z0P3C1pNOAZ4ATc4zBrEfc0rb+eJv1XW6JICKeAiaVGf4C8MG8vtesL8aNy4oWyg232uRt1nduWWxWwi1t64+3Wd85EZiVcEvb+uNt1ne5tizuL25ZbGbWc7XcstjMzGqIE4GZWY2pdgM536rSzKyGFHH7UJ8RmJnVkCIayDkRmJnVkCIayDkRmJnVkCJuH+pEYGZWQ4poIOdEYGZ94i6g+1cRDeR81ZCZ9VoRV7g0g2nTqrv+fEZgZr3mLqAbgxOBmfWau4BuDE4EZtZrRVzhYv3PicDMes1dQDcGJwIz6zV3Ad0YfNWQmfVJta9wsf7nMwIzsybnRGBm1uScCMzMmly3dQSSRpQZvDoi1ucQj5mZVVklZwQPAG3A48AT6fXTkh6Q9K7uJpY0UNKDkm5M73eVdK+kJyTNlTS4LwtgVi3uU6d/eD3WnkoSwa+Bj0TEjhGxA/Bh4GrgC8APK5j+y8CikvcXAjMjYjfgJeC0noVsVn3tfeosWQIRm/vU8UGsZ7wea1MliaAlIn7T/iYibgYOjoh7gK27mlDSGOCjwI/TewGHAtekUWYDx/YibrOqcp86/cPrsTZVkghelHSmpPHp8VXgJUkDgTe6mfZ7wFdLxtsBWBURG9L7pcAu5SaUdLqkVkmtbW1tFYRplh/3qdM/vB5rUyWJ4BPAGOAXwPXAuDRsIPB3nU0k6Sjg+YiYXzq4zKhRbvqImBURLRHRMnLkyArCNMuP+9TpH16PtanbRBARKyPijIjYLyImR8SXIqItIl6PiCe7mPR9wMckLQauIisS+h4wXFL71UpjgGV9XAaz3LlPnf7h9Vibuk0Ekt4paZakmyX9rv3R3XQRcXZEjImICcDHgd9FxDTgNuCENNp0srMMs5rmPnX6h9djbVJE2ZKZzSNIDwEXA/OBje3DOxT5dDePqcC/RMRRkt5OdoYwAngQ+GREvNbV9C0tLdHa2lrp15mZGSBpfkS0dDdeJZ3ObYiIi/oSTETMA+al108BB/RlfmZm1n8qqSz+paQvSBotaUT7I/fIzMysKio5I5ienr9SMiyAt/d/OGZmVm3dJoKI2LUagZiZWTE6TQSSDo2I30k6vtznEXFdfmGZmVm1dFVH8IH0fHSZx1E5x2VmJfqzozZ3+mYddXpGEBFfT8+nVi8cM+uovaO29j562jtqg55ff9+f87LGUUmDsi9LeosyP07dTx9RjeDMrH87anOnb1ZOJZePfjoi/gocAewEnArMyDUqM9ukPztqc6dvVk4liaC9o7iPAJdGxEOU7zzOzHLQnx21udM3K6eSRDBf0s1kieA3kobRfffTZtZP+rOjNnf6ZuVUkghOA84C9o+IV4DBZMVDZlYF/dlRmzt9s3I67XRO0h4R8SdJU8p9HhEP5BpZCXc6Z2bWc/3R6dw/A6cD3ynzWZDdX8DMzOpcp0VDEZGuLubDEXFI6YOsvsDMrEfcmK02VVJH8IcKh5mZdaq9MduSJRCxuTGbk0Hxuupr6G1kN5bfRtJ+bL5k9C3A0M6mMzMrp6vGbK6sLlZXdQQfAk4hu6/wd9icCP4KfC3fsMys0bgxW+3qqq+h2ZIuB06OCJ+8mVmfjBuXFQeVG27F6rKOICLeAD5bpVjMrIG5MVvtqqSy+BZJ/yJprG9VaWa95cZstavTBmWbRpCeLjM4IqJqt6p0gzIzs57rjwZlgG9VaWbW6Cq5H8FQSf8qaVZ6v5ukbu9QJmmIpPskPSTpUUnfSMN3lXSvpCckzZU0uO+LYWZmvVVJHcGlwOvAe9P7pcA3K5juNeDQiJgETAaOlHQgcCEwMyJ2A14i69TOzMwKUkkieEdEfAtYDxARr1LB/Qgisya93So92vsouiYNnw0c29Ogzcys/1SSCF6XtA3ZQRxJ7yD7t98tSQMlLQCeB24B/gysiogNaZSlZK2Xy017uqRWSa1tbW2VfJ1ZTXM/O1arKkkE5wO/BsZKmgP8FvhqJTOPiI0RMZmsdfIBwJ7lRutk2lkR0RIRLSNHjqzk68xqlvvZsVrWbSKIiJuB48m6m7gSaImIeT35kohYBcwDDgSGS2q/WmkMsKwn8zKrR75pvNWySq4auoHsxvXzIuLGiFhZyYwljZQ0PL3eBjgMWATcBpyQRpsOXN+bwM3qifvZsVpWSdHQd4CDgIWS/lvSCZKGVDDdaOA2SQ8D9wO3RMSNwJnAP0t6EtgBuKSXsZvlIo+yfN803mpZJQ3KbgdulzSQ7IqfzwA/IeuOuqvpHgb2KzP8KbL6ArOa016W316M016WD33rCuGCC7acL7ifHasdlZwRtBft/C3wOWB/sss+zRpOXmX57mfHalkldQRzycr2DwX+k6xdwRl5B9ZsfGlhbcizLH/aNFi8GN54I3t2ErBa0W3REFnL4k9ExMa8g2lWeRVHWM+5z3xrRp32Pirp+K4mjIjrcomojEbvfXTChPIHn/Hjs3+OVj0dkzJkZfkuxrF61B+9jx6dnnci62fod+n9IWRtAqqWCBqdLy2sHe0H+3POydb/uHFZha6TgDWyrm5VeSqApBuBvSJieXo/mqyuwPqJiyNqy7RpPvBbc6nkqqEJ7UkgWQHsnlM8Tcm38DOzIlVSWTxP0m/IupcI4ONk/Q1ZP3FxhJkVqZIGZV+SdBxwcBp0NzAq16iakIsjzKwoFTUoA54mux/BcWSVxYtyi8jMzKqq00Qg6Z2SzpO0CPgP4Fmyy00PiYj/qFqEVpfcQM6sfnRVNPQn4E7g6Ih4EkDSP1UlKqtrbiBnVl+6Khr6W+A5sh5EfyTpg1Rwi0oz971vVl86TQQR8fOIOAnYg6wB2T8BoyRdJOmIKsVndcgN5MzqSyV3KFsbEXMi4iiyO4otAM7KPTKrW+5736y+VHrVEAAR8WJE/FdEHJpXQFb/3EDOrL70KBGYVSKPvveb5SqkZllOqy2d9j5aSxq991HrWrP0CNosy2nVU2nvo04EVvOapZvuZllOq55KE4GLhgpUjWKARihqqPWrkPprHdf6clrjqqTTOctBNRpdNUrDrlruprs/13EtL6c1Np8RFKQaja4apWFXLV+F1J/ruJaX0xpbbolA0lhJt0laJOlRSV9Ow0dIukXSE+l5+7xiqGXVKAZolKKGPK5C6i/9uY5reTmtseVWWZzuZDY6Ih6QNAyYDxwLnAK8GBEzJJ0FbB8RZ3Y1r0asLK5GxaArH/PndWy1rPDK4ohYHhEPpNerybqu3gU4BpidRptNlhyaTjWKAVzUkD+vY2sEVakjkDQB2A+4FxjVfuvL9LxTJ9OcLqlVUmtbW1s1wqyqahQDuKghf17H1ghyb0cgaTvgduCCiLhO0qqIGF7y+UsR0WU9QSMWDZmZ5a3woqEUxFbAtcCciLguDV6R6g/a6xGezzMGMzPrWp5XDQm4BFgUEd8t+egGYHp6PR24Pq8YrH41QkM4s3qRZ4Oy9wGfAv4oaUEa9jVgBnC1pNOAZ4ATc4zB6lCjNIQzqxfua8hqji/JNOsfNVFHYNYbRTaEc5FUffJ26xv3NWQ1p6g+d1wkVZ+83frOZwRWc4pqpNUofTM1G2+3vnMisJpTVCOtRumbqdl4u/WdE4EVqrOy3WnTsorhN97Inqtxit9Z0ZO7ga5t3m5950RghWkv212yBCI2l+0WVdHnfoPqk7db3zkRWGFqrWzX/QbVJ2+3vmvYdgRz5mQHlGeeyU4RL7jAO0atGTAgOxPoSMqKhMysb5q6HUGtFTlYeS7bNasNDZkIaq3Iwcpz2a5ZbWjIRODLyepDadkuwMCBmxO2z97MqqchE4GLHOrHtGmbzww2bsyGuSjPrLoaMhG4yKG+uCjPrFgNmQh8OVl9cVGeWbEattO5adN84K8XRXUyZ2aZhjwjsPriojyzYjkRWOF6UpTnfufN+l/DFg1ZfamkKM/9zpvlw2cEVjd8dZFZPpwIrG746iKzfDgRWN1wQ0GzfDgRWN3w1UVm+cgtEUj6iaTnJT1SMmyEpFskPZGet8/r+63xuKGgWT7yPCO4DDiyw7CzgN9GxG7Ab9N7s4oVcQtLs0aXWyKIiDuAFzsMPgaYnV7PBo7N6/vNzKwy1a4jGBURywHS806djSjpdEmtklrb2tqqFqCZWbOp2criiJgVES0R0TJy5MiiwzEza1jVTgQrJI0GSM/PV/n7zcysg2onghuA6en1dOD6Kn+/mZl1kOflo1cCdwO7S1oq6TRgBnC4pCeAw9N7MzMrUG6dzkXEyZ189MG8vtPMzHquZiuLzay2uUvwxuFuqM2sx9wleGPxGYGZ9Zi7BG8sTgRm1mPuEryxOBGYWY+5S/DG4kRgZj3mLsEbixOBmfWYuwRvLE4ENc6X6Fmt7gPuErxx+PLRGuZL9Mz7gFWDIqLoGLrV0tISra2tRYdRdRMmZD/8jsaPz/6BWeOrx31g/fr1LF26lHXr1hUdStMYMmQIY8aMYautttpiuKT5EdHS3fQ+I6hhvkTP6nEfWLp0KcOGDWPChAlIKjqchhcRvPDCCyxdupRdd921V/NwHUEN8yV6Vo/7wLp169hhhx2cBKpEEjvssEOfzsCcCGqYL9Gzet0HnASqq6/r24mgRpS7MqRWL9Gr1atYGlGt7gPWYCKi5h/vete7opH97GcRQ4dGwObH0KHZ8FpTT7FaMRYuXFh0CPHNb34z9tprr9h3331j0qRJcc899/R4Hk8//XTMmTNn0/sHH3wwfvWrX/U5tgEDBsSkSZNi7733jhNOOCHWrl0bERHbbrttj+LpqNx6B1qjgmOszwhqQD114FVPsVp96O8zzLvvvpsbb7yRBx54gIcffphbb72VsWPH9ng+ixcv5oorrtj0fsGCBdx00009mseGDRveNGybbbZhwYIFPPLIIwwePJiLL764V/H0JyeCGlBPV4bUU6xW+9rbSSxZkp1ftreT6EsyWL58OTvuuCNbb701ADvuuCM777wzAPfffz/vfe97mTRpEgcccACrV69m8eLFHHTQQUyZMoUpU6bwhz/8AYCzzjqLO++8k8mTJ3PhhRdy3nnnMXfuXCZPnszcuXNZu3Ytn/70p9l///3Zb7/9uP767M67l112GSeeeCJHH300RxxxRJexHnTQQTz55JNbDIsIvvKVr7DPPvuw7777Mnfu3DfFM3PmzN6voHIqOW0o+tHoRUPjx29Z1NL+GD++6MjerJ5itWL0pGgoj/1p9erVMWnSpNhtt93i85//fMybNy8iIl577bXYdddd47777ouIiJdffjnWr18fa9eujVdffTUiIh5//PFoP97cdttt8dGPfnTTfC+99NL44he/uOn92WefHZdffnlERLz00kux2267xZo1a+LSSy+NXXbZJV544YWy8bUXAa1fvz4+9rGPxQ9/+MMthl9zzTVx2GGHxYYNG+K5556LsWPHxrJly94UT0cuGqpz9XRlSD3FarUvjzPM7bbbjvnz5zNr1ixGjhzJSSedxGWXXcZjjz3G6NGj2X///QF4y1vewqBBg1i/fj2f+cxn2HfffTnxxBNZuHBhRd9z8803M2PGDCZPnszUqVNZt24dz6TADz/8cEaMGFF2uldffZXJkyfT0tLCuHHjOO2007b4/K677uLkk09m4MCBjBo1ig984APcf//9vV8hFXCDshrQfgXIOedkP4Bx47IDa2dXhsyZU/m4Rcdq1pVx48q3nO5rO4mBAwcydepUpk6dyr777svs2bOZMmVK2cssZ86cyahRo3jooYd44403GDJkSEXfERFce+217L777ltRHKnVAAAHQElEQVQMv/fee9l22207na69jqCr+VabzwhqRKUdeOVRpppXrGbdyeMM87HHHuOJJ57Y9H7BggWMHz+ePfbYg2XLlm36d7169Wo2bNjAyy+/zOjRoxkwYACXX345GzduBGDYsGGsXr1603w6vv/Qhz7ED37wg00H7gcffLD3QZc4+OCDmTt3Lhs3bqStrY077riDAw444E3f35+cCOqMr9qxRpJHO4k1a9Ywffp09tprLyZOnMjChQs5//zzGTx4MHPnzuWMM85g0qRJHH744axbt44vfOELzJ49mwMPPJDHH39807/5iRMnMmjQICZNmsTMmTM55JBDWLhw4abK4nPPPZf169czceJE9tlnH84999x+WSfHHXccEydOZNKkSRx66KF861vf4m1ve9ub4ulPhXQ6J+lI4PvAQODHETGjq/GbtdO5cgYMyM4EOpKyf+hmRVu0aBF77rln0WE0nXLrvdJO56p+RiBpIPCfwIeBvYCTJe1V7TjqVT32PWNmta2IoqEDgCcj4qmIeB24CjimgDjqkq/aMbP+VkQi2AV4tuT90jRsC5JOl9QqqbWtra1qwdU69z1j9aCIIudm1tf1XUQiKNdN3puWIiJmRURLRLSMHDmyCmHVD1+1Y7VsyJAhvPDCC04GVRKR3Y+g0steyymiHcFSoLTjjzHAsgLiMLMcjBkzhqVLl+Iz+eppv0NZbxWRCO4HdpO0K/AX4OPAJwqIw8xysNVWW/X6TllWjKongojYIOlLwG/ILh/9SUQ8Wu04zMwsU0gXExFxE9Cz/lzNzCwXbllsZtbkCmlZ3FOS2oAyXVPVrR2BlUUHUYBmXW5o3mX3chdrfER0e9llXSSCRiOptZJm342mWZcbmnfZvdz1wUVDZmZNzonAzKzJOREUY1bRARSkWZcbmnfZvdx1wHUEZmZNzmcEZmZNzonAzKzJORHkTNJPJD0v6ZGSYedL+oukBenxkSJjzIOksZJuk7RI0qOSvpyGj5B0i6Qn0vP2Rcfan7pY7obe5pKGSLpP0kNpub+Rhu8q6d60vedKGlx0rP2pi+W+TNLTJdt7ctGxdsV1BDmTdDCwBvhpROyThp0PrImIbxcZW54kjQZGR8QDkoYB84FjgVOAFyNihqSzgO0j4swCQ+1XXSz339HA21ySgG0jYo2krYC7gC8D/wxcFxFXSboYeCgiLioy1v7UxXJ/DrgxIq4pNMAK+YwgZxFxB/Bi0XFUW0Qsj4gH0uvVwCKyGxAdA8xOo80mO0g2jC6Wu6FFZk16u1V6BHAo0H4wbMTt3dly1xUnguJ8SdLDqeiooYpHOpI0AdgPuBcYFRHLITtoAjsVF1m+Oiw3NPg2lzRQ0gLgeeAW4M/AqojYkEYpezfCetdxuSOifXtfkLb3TElbFxhit5wIinER8A5gMrAc+E6x4eRH0nbAtcA/RsRfi46nWsosd8Nv84jYGBGTyW42dQCwZ7nRqhtV/jout6R9gLOBPYD9gRFATRd/OhEUICJWpJ3nDeBHZD+ahpPKTK8F5kTEdWnwilSO3l6e/nxR8eWl3HI3yzYHiIhVwDzgQGC4pPbu7hv6boQly31kKiKMiHgNuJQa395OBAVoPxAmxwGPdDZuvUqVaJcAiyLiuyUf3QBMT6+nA9dXO7Y8dbbcjb7NJY2UNDy93gY4jKx+5DbghDRaI27vcsv9p5I/OyKrF6np7e2rhnIm6UpgKlm3tCuAr6f3k8lOkxcDn20vN28Ukt4P3An8EXgjDf4aWXn51cA44BngxIhomMr0Lpb7ZBp4m0uaSFYZPJDsD+bVEfFvkt4OXEVWPPIg8Mn0L7khdLHcvwNGAgIWAJ8rqVSuOU4EZmZNzkVDZmZNzonAzKzJORGYmTU5JwIzsybnRGBm1uScCMw6kHRO6kny4dRz5Lu7GPcySSd09rlZPRjU/ShmzUPSe4CjgCkR8ZqkHYGG6jrZrCOfEZhtaTSwsr3RU0SsjIhlks6TdL+kRyTNSi1GtyDpXZJulzRf0m9KWpf+g6SF6Qzjqiovj1m33KDMrETqLO4uYChwKzA3Im6XNKK9BbSky8lakP5S0mXAjWRdJ9wOHBMRbZJOAj4UEZ+WtAzYNZ1hDE990pjVDBcNmZVINxh5F3AQcAgwN91AZ7Wkr5IliBHAo8AvSybdHdgHuCWdLAwk62UU4GFgjqRfAL+oyoKY9YATgVkHEbGRrBfJeZL+CHwWmAi0RMSz6Q5zQzpMJuDRiHhPmVl+FDgY+BhwrqS9S/roNyuc6wjMSkjaXdJuJYMmA4+l1ytT0VG5q4QeA0amymYkbSVpb0kDgLERcRvwVWA4sF1+S2DWcz4jMNvSdsAPUtfCG4AngdOBVWQ9ii4G7u84UUS8ni4j/b+S3kr22/oe8DjwszRMwEzXEVitcWWxmVmTc9GQmVmTcyIwM2tyTgRmZk3OicDMrMk5EZiZNTknAjOzJudEYGbW5P4/kkA+VxUYcxYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot scatter plot between X and y\n",
    "\n",
    "plt.scatter(X, y, color = 'blue', label='Scatter Plot')\n",
    "plt.title('Relationship between Sales and Advertising')\n",
    "plt.xlabel('Sales')\n",
    "plt.ylabel('Advertising')\n",
    "plt.legend(loc=4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking dimensions of X and y\n",
    "\n",
    "We need to check the dimensions of X and y to make sure they are in right format for Scikit-Learn API. \n",
    "\n",
    "It is an important precursor to model building. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(36,)\n",
      "(36,)\n"
     ]
    }
   ],
   "source": [
    "# Print the dimensions of X and y\n",
    "\n",
    "print(X.shape)\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reshaping X and y\n",
    "\n",
    "Since we are working with only one feature variable, so we need to reshape using Numpy reshape() method.\n",
    "\n",
    "It specifies first dimension to be -1, which means \"unspecified\".\n",
    "\n",
    "Its value is inferred from the length of the array and the remaining dimensions.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reshape X and y\n",
    "\n",
    "X = X.reshape(-1,1)\n",
    "y = y.reshape(-1,1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(36, 1)\n",
      "(36, 1)\n"
     ]
    }
   ],
   "source": [
    "# Print the dimensions of X and y after reshaping\n",
    "\n",
    "print(X.shape)\n",
    "print(y.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Difference in dimensions of X and y after reshaping\n",
    "\n",
    "\n",
    "We can see the difference in diminsions of X and y before and after reshaping.\n",
    "\n",
    "It is essential in this case because getting the feature and target variable right is an important precursor to model building."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train test split\n",
    "\n",
    "\n",
    "I split the dataset into two sets namely - train set and test set.\n",
    "\n",
    "The model learn the relationships from the training data and predict on test data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split X and y into training and test data sets\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X, y, test_size=0.33, random_state=42)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(24, 1)\n",
      "(24, 1)\n",
      "(12, 1)\n",
      "(12, 1)\n"
     ]
    }
   ],
   "source": [
    "# Print the dimensions of X_train,X_test,y_train,y_test\n",
    "\n",
    "print(X_train.shape)\n",
    "print(y_train.shape)\n",
    "print(X_test.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mechanics of the model\n",
    "\n",
    "\n",
    "I split the dataset into two sets – the training set and the test set. Then, I instantiate the regressor lm and fit it on the training set with the fit method. \n",
    "\n",
    "In this step, the model learned the relationships between the training data (X_train, y_train). \n",
    "\n",
    "Now the model is ready to make predictions on the test data (X_test). Hence, I predict on the test data using the predict method. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fit the linear model\n",
    "\n",
    "# Instantiate the linear regression object lm\n",
    "from sklearn.linear_model import LinearRegression\n",
    "lm = LinearRegression()\n",
    "\n",
    "\n",
    "# Train the model using training data sets\n",
    "lm.fit(X_train,y_train)\n",
    "\n",
    "\n",
    "# Predict on the test data\n",
    "y_pred=lm.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model slope and intercept term\n",
    "\n",
    "The model slope is given by lm.coef_ and model intercept term is given by lm.intercept_. \n",
    "\n",
    "The estimated model slope and intercept values are 1.60509347 and  -11.16003616.\n",
    "\n",
    "So, the equation of the fitted regression line is\n",
    "\n",
    "y = 1.60509347 * x - 11.16003616  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated model slope, a: [[1.60509347]]\n",
      "Estimated model intercept, b: (array([-11.16003616]),)\n"
     ]
    }
   ],
   "source": [
    "# Compute model slope and intercept\n",
    "\n",
    "a = lm.coef_\n",
    "b = lm.intercept_,\n",
    "print(\"Estimated model slope, a:\" , a)\n",
    "print(\"Estimated model intercept, b:\" , b) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# So, our fitted regression line is \n",
    "\n",
    "# y = 1.60509347 * x - 11.16003616 \n",
    "\n",
    "# That is our linear model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making predictions\n",
    "\n",
    "\n",
    "I have predicted the Advertising values on first five 5 Sales datasets by writing code\n",
    "\n",
    "\n",
    "\t\tlm.predict(X) [0:5]  \n",
    "        \n",
    "\n",
    "If I remove [0:5], then I will get predicted Advertising values for the whole Sales dataset.\n",
    "\n",
    "\n",
    "To make prediction, on an individual Sales value, I write\n",
    "\n",
    "\n",
    "\t\tlm.predict(Xi)\n",
    "        \n",
    "\n",
    "where Xi is the Sales data value of the ith observation.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 8.10108551],\n",
       "       [21.74438002],\n",
       "       [22.54692675],\n",
       "       [13.71891266],\n",
       "       [13.39789396]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predicting Advertising values\n",
    "\n",
    "lm.predict(X)[0:5]\n",
    "\n",
    "# Predicting Advertising values on first five Sales values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[27.36220717]]\n"
     ]
    }
   ],
   "source": [
    "# To make an individual prediction using the linear regression model.\n",
    "\n",
    "print(str(lm.predict(24)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regression metrics for model performance\n",
    "\n",
    "\n",
    "Now, it is the time to evaluate model performance. \n",
    "\n",
    "For regression problems, there are two ways to compute the model performance. They are RMSE (Root Mean Square Error) and R-Squared Value. These are explained below:-  \n",
    "\n",
    "\n",
    "### RMSE\n",
    "\n",
    "RMSE is the standard deviation of the residuals. So, RMSE gives us the standard deviation of the unexplained variance by the model. It can be calculated by taking square root of Mean Squared Error.\n",
    "RMSE is an absolute measure of fit. It gives us how spread the residuals are, given by the standard deviation of the residuals. The more concentrated the data is around the regression line, the lower the residuals and hence lower the standard deviation of residuals. It results in lower values of RMSE. So, lower values of RMSE indicate better fit of data. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE value: 11.2273\n"
     ]
    }
   ],
   "source": [
    "# Calculate and print Root Mean Square Error(RMSE)\n",
    "\n",
    "from sklearn.metrics import mean_squared_error\n",
    "mse = mean_squared_error(y_test, y_pred)\n",
    "rmse = np.sqrt(mse)\n",
    "print(\"RMSE value: {:.4f}\".format(rmse))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  R2 Score\n",
    "\n",
    "\n",
    "R2 Score is another metric to evaluate performance of a regression model. It is also called coefficient of determination. It gives us an idea of goodness of fit for the linear regression models. It indicates the percentage of variance that is explained by the model. \n",
    "\n",
    "\n",
    "Mathematically, \n",
    "\n",
    "\n",
    "R2 Score = Explained Variation/Total Variation\n",
    "\n",
    "\n",
    "In general, the higher the R2 Score value, the better the model fits the data. Usually, its value ranges from 0 to 1. So, we want its value to be as close to 1. Its value can become negative if our model is wrong.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2 Score value: 0.5789\n"
     ]
    }
   ],
   "source": [
    "# Calculate and print r2_score\n",
    "\n",
    "from sklearn.metrics import r2_score\n",
    "print (\"R2 Score value: {:.4f}\".format(r2_score(y_test, y_pred)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interpretation and Conclusion\n",
    "\n",
    "\n",
    "The RMSE value has been found to be 11.2273. It means the standard deviation for our prediction is 11.2273. So, sometimes we expect the predictions to be off by more than 11.2273 and other times we expect less than 11.2273. So, the model is not good fit to the data. \n",
    "\n",
    "\n",
    "In business decisions, the benchmark for the R2 score value is 0.7. It means if R2 score value >= 0.7, then the model is good enough to deploy on unseen data whereas if R2 score value < 0.7, then the model is not good enough to deploy. Our R2 score value has been found to be .5789. It means that this model explains 57.89 % of the variance in our dependent variable. So, the R2 score value confirms that the model is not good enough to deploy because it does not provide good fit to the data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmYFOW59/HvbwaQVVlVZBnQoIgIAwxEjSIxblFx93gIGlQOJNGYmOS4x2gSfY9GDRpjNBMjYsCAu8Z9A8XEGEFwAVxQB0VAhk1mWJSZud8/qmZohp7unqX3+3NdfXV39VPVd1V11131VD1PycxwzjmXvwrSHYBzzrn08kTgnHN5zhOBc87lOU8EzjmX5zwROOdcnvNE4Jxzec4TQYpJGiNpeTPGv1PSVS0ZU5TvMEnfaOCz8ZKea+J0r5E0vXnR5SZJ90i6Nt1xNKS5v9so00vpb0HSIklj4pTpK6lSUmGKwsoYngiaQFKZpC3hj2ZV+CfumITvOUfSq5HDzOyHZvbblv6uRJnZDDM7OtXfmw1JRFJvSQ9JWiPpS0nvSDon3XGligIfS1qc5jh2SqpmdoCZzYk1npl9amYdzaw6qQFmIE8ETTfWzDoCxcAw4PI0x+PS72/AZ0AR0A34PvBFWiNKrdHA7sDekkamI4B83JtvCZ4ImsnMVgHPEiQEACTtIukmSZ9K+iKszmkXbXxJl0n6SFKFpMWSTgmH7w/cCRwcHnlsCIfvsLcjaZKkpZLWSXpc0l4Rn5mkH0r6UNJ6SbdLUvjZNyS9HO65rpE0q15oRzYw3g5HKeF3/CTcE1wj6UZJsX5XbSXNCuf3TUlDI6a1V7hHXS7pE0k/CYcfC1wBnBkui7ckfVvSOxHjviDpPxHvX5V0cqzphp8VRKyDtZLul9Q1/KxfOH8TwnW5RtKVMeZtJHCPmW0ysyozW2BmT0d81wPhEeSXkl6RdEBDE5J0gqSFkjZI+pekIRGfXSrp83AZvi/pOw1M43hJCyRtlPSZpGsiPos5b5Lahb+19eEefiIb9gnAY8BT4evIWPqHv7cKSc8D3SM+e0bSj+uVf0vSqeHrgZKeD3/j70v6r4hy90i6Q9JTkjYBE4HxwCXhb+UfYbkySUeGr0dJmhculy8k/b7eMmkVvp8j6beS/hnG/ZykyLi/L2lZ+Lu5KvI7so6Z+aORD6AMODJ83Rt4B7g14vNbgMeBrkAn4B/A/4WfjQGWR5Q9A9iLICmfCWwCeoafnQO8Wu+77wGuDV8fAawBhgO7ALcBr0SUNeAJoDPQFygHjg0/+ztwZfi9bYFDExxvh5jCsrPDee0LfAD8TwPL7RpgG3A60Br4X+CT8HUBMB/4FdAG2Bv4GDgmYtzpEdNqC2wh2KC0AlYBK8Ll3S78rFsC070I+He4HncB/gz8PfysXzh/fwmnORT4Cti/gfl7Afgn8N9A3yifnxfGt0v4G1nYwHodDqwGvgkUEmxUy8Lx9iM46tgrIsZ9GohnDHBguAyGEBydnJzIvAHXA3PD9doHeJeI322U72oPbASOA04j+F22ifj8NeD34TyMBipq1yfBkdM/I8oOAjaEZTuE83tuuJ6Hh9M+IGK5fQl8i+2/5bpl2cB/9jXg7PB1R+CgesukVfh+DvARsG+4jOYA10fEWAkcSvC7uongt31kurdPTdqmpTuAbHyEP6rK8MdswItA5/AzEWzM94kofzDwSfh6TJw/1ELgpPD1OcROBH8FfhfxWcfwx9gvfG/suIG/H7gsfH0vUAr0jhJDrPF2iCkse2zE+/OBFxuYt2uAf0e8LwBWAocRbPQ+rVf+cmBqxLjT630+FzgVOAh4LozzWODbwNthmXjTXQJ8J+KznuEybBWxYegd8fl/gP9uYP66EGxAFwHV4boc2UDZzuG0d4uyXu8Afluv/PvA4cA3CJLEkUDrRv5ubwGmhK9jzhtBsoxcr5OJ/bs9i2CHoRXBBnwDcEr4WV+gCugQUf4+tieCTgT/maLw/XXA3eHrM4G59b7rz8DVEcvt3ob+I/X+s7WJ4BXg10D3emVql0lkIvhlvd/2M+HrXxHuMITv2wNfk6WJwKuGmu5kM+tEsGEfyPZD3R4EP4r54WH9BuCZcPhOwsPLhRFlB0dMK569gGW1b8ysElgL9Ioosyri9WaCZAFwCUHS+o+CKyrOqzfthsaL5rOI18vCuOKWNbMaYHlYvgjYq3Y5hMviCmCPGNN6mWD5jw5fzyHYWB4evieB6RYBj0R8toRgIx75vQktCzNbb2aXmdkB4fgLgUcVKJR0fVgFtZFgwwTR13UR8It6MfchOApYSnAUcw2wWtJMRVQHRpL0TUmzwyqxL4EfRvm+huZtL3Zer7FMAO63oErsK+BhtlcP7QWsN7NN0aZnZhXAkwRHUoTPM8LXRcA36y2L8cCeEdOKjDMREwn28t+T9IakE2KUTWj5mNlmgv9eVvJE0Exm9jLBHshN4aA1BNUSB5hZ5/CxmwUnlncgqYjg0PzHQDcz60xwCK7aycf5+hUEf5Ta6XUgqA75PIG4V5nZJDPbC/gB8Cc1cMloAvpEvO4bxhW3rIJzCb3D8p8RHDV1jnh0MrPjakOOMq36ieBldk4E8ab7GfDdep+3NbO4yzAWM1tD8JvYi6B65XvASQR78rsR7H3C9nUd6TPgunoxtTezv4fTvs/MDiVY9wbc0EAY9xFUUfYxs90IzjlF+75oVrLzeo1KUm+CasqzwnMgqwiq/44L69RXAl3C32dD0/s7ME7SwQTVMLPD4Z8BL9dbFh3N7EcR49b/bcT835jZh2Y2juDE9g3Ag/ViS8RKgt8uEJxTIfjvZSVPBC3jFuAoScXhXu5fgCmSdgeQ1EvSMVHG60Dwoy0Py51LcERQ6wugt6Q2DXzvfcC5kool7QL8P+B1MyuLF7CkM8I/MMD6MI6mXjZ3saQukvoAPwXqn3iONELSqeEJuYsI6qX/TVAtsVHBidB24R70YG2/+uQLoJ92PBH9L4I681HAf8xsEeEeJMHhPwlM907gujApI6mHpJOashAk3RBOu5WkTsCPgKVmtpag+uMrgr3G9gTrqiF/AX4Y7tFLUgcFJ347SdpP0hHh+t5KsNPR0HrrBKwzs62SRhEko0TdD1wertfewIUxyp5NcG5oP4KLJooJ9riXA+PMbBkwD/i1pDaSDgXG1pvGUwTr7jfArPB/BMG5qn0lnS2pdfgYqeBiioZ8QXAuKCpJZ0nqEX7HhnBwY3/7DwJjJR0S/j9/TeJJNuN4ImgBZlZOUOde29DrUmAp8O+wGuAFgj9J/fEWAzcTnLz6guDE3j8jirxEUN+8StKaKOO/GH7nQwR7KPuw/fA6npHA65IqCfYaf2pmnyQ4bn2PEZyQXUhwiP/XOGXPJEg+ZwOnmtk2C67dHkuwEfmE4MjqLoK9Z4AHwue1kt4ECKsa3gQWmdnX4eevAcvMbHVYJt50bw3n/zlJFQRJ6ZtNWwy0Bx4h2Lh8TLBhOzH87F6C6pDPgcXh90RlZvOAScAfCZbTUoJzMxDUv18fzscqgr3aKxqY1PnAb8L5+hXBxj1Rvw7j/YTg/MvfYpSdAPwpPMqsexAk2drqoe8RLNd1wNUEy6NORHXSkQQ7OLXDK4CjCX7XKwjm+QaC5dCQvwKDwqqkR6N8fiywKPzt30pwXmRrjOntJNzpuBCYSfDfqyA4d/NVY6aTKRSe6HCuSSQZMCCsu3YuLyloULqB4L/Q1B2qtPEjAuecawJJYyW1D88v3ERwGXlZeqNqGk8EzjnXNCcRVFetAAYQVDFlZRWLVw0551ye8yMC55zLc63SHUAiunfvbv369Ut3GM45l1Xmz5+/xsyiNmaNlBWJoF+/fsybNy/dYTjnXFaRFK9FOOBVQ845l/c8ETjnXJ7zROCcc3nOE4FzzuU5TwTOOZfnPBE45zLOjBnQrx8UFATPM2bEGyO3pHr+s+LyUedc/pgxAyZPhs2bg/fLlgXvAcaPT19cqZKO+c+KLiZKSkrM2xE4lx/69Qs2fvUVFUFZWaqjSb2WnH9J882sJF45rxpyzmWUTz9t3PBck47590TgnMsofRu4KWZDw3NNOubfE4FzLqNcdx20b7/jsPbtg+H5IB3z74nAOZdRxo+H0tKgTlwKnktL8+NEMaRn/v1ksXPO5Sg/Weyccy4hngiccy7PeSJwLokysYVsJsbk0stbFjuXJJnYQjYTY3Lp5yeLnUuSTGwhm4kxueTxk8XOpVkmtpDNxJhc+nkicC5JMrGFbCbG5NLPE4FzSZKJLWQzMSaXfklNBJI6S3pQ0nuSlkg6WFJXSc9L+jB87pLMGJxLl0xsIZuJMbn0S+rJYknTgLlmdpekNkB74ApgnZldL+kyoIuZXRprOn6y2DnnGi/tJ4sl7QqMBv4KYGZfm9kG4CRgWlhsGnBysmJwzjkXXzKrhvYGyoGpkhZIuktSB2APM1sJED7vHm1kSZMlzZM0r7y8PIlhOudcfktmImgFDAfuMLNhwCbgskRHNrNSMysxs5IePXokK0bnnMt7yUwEy4HlZvZ6+P5BgsTwhaSeAOHz6iTG4JxzLo6kJQIzWwV8Jmm/cNB3gMXA48CEcNgE4LFkxeCccy6+ZPc1dCEwI7xi6GPgXILkc7+kicCnwBlJjsE551wMSU0EZrYQiHbp0neS+b3OOecS5y2LnXMuz3kicM65POeJwDnn8pwnAuecy3OeCJxzLs95InDOuTznicA55/KcJwLnnMtzngiccy7PeSJwzrk854nAOefynCcC55zLc54InHMuz3kicM65POeJwDnn8pwnAufy3IwZ0K8fFBQEzzNmpDsil2rJvkOZcy6DzZgBkyfD5s3B+2XLgvcA48enLy6XWn5E4Fweu/LK7Umg1ubNwXCXPzwROJfHPv20ccNdbvJE4Fwe69u3ccNdbvJE4Fweu+46aN9+x2Ht2wfDXf5IaiKQVCbpHUkLJc0Lh3WV9LykD8PnLsmMwTnXsPHjobQUiopACp5LS/1Ecb6RmSVv4lIZUGJmayKG/Q5YZ2bXS7oM6GJml8aaTklJic2bNy9pcTrnXC6SNN/MSuKVS0fV0EnAtPD1NODkNMTgnHMulOxEYMBzkuZLCq9OZg8zWwkQPu+e5Bicc87FkOxE8C0zGw58F7hA0uhER5Q0WdI8SfPKy8uTF6Fz9XhL2+zj66x5kpoIzGxF+LwaeAQYBXwhqSdA+Ly6gXFLzazEzEp69OiRzDCdq1Pb0nbZMjDb3tLWNyyZy9dZ8yUtEUjqIKlT7WvgaOBd4HFgQlhsAvBYsmJwrrG8pW328XXWfMnsa2gP4BFJtd9zn5k9I+kN4H5JE4FPgTOSGINzjeItbbOPr7PmS1oiMLOPgaFRhq8FvpOs73WuOfr2DaoWog13mcnXWfN5y2LnInhL2+zj66z5PBE4F8Fb2mafXFtn69ev59hjj0US++23H7Nnz076dya1ZXFL8ZbFzrlc9+CDD3LGGTufMj3llFN4+OGHmzTNTG5Z7JxzDli3bh1HH300kqImAYCzzz476XH4Hcqccy7F7r//fs4888yYZaROmL3Mz342jM2bk1vV5UcEzjmXAuvWrePII49EUswkcNxxv6Bdu22YbQSGpaSBnCcC55xLopkzZyKJbt268eKLL0Yts9tuu7FgwQLMjEWLbmLLlh0ra5LdQM4TgXPOtbC1a9dyxBFHIIlx48Y1WO6SSy6hqqqKDRs2UFxcDKSngZyfI3DOuRZy3333MT5OZX6XLl2YM2cOQ4YMifp5OhrI+RGBc841w5o1axgzZgySYiaBSy+9lKqqKtatW9dgEoD0NJDzROCca5Z87QJ6+vTpSKJHjx68/PLLUct069aNt99+GzPj+uuvp7CwMO5009FAzquGnHNNVtsFdG3vn7VXuED2tuyNpby8nNNOO425c+fGLHfFFVfwm9/8JqENfzTjx6d2+XnLYudck/XrF70+u6gIyspSHU3y3HvvvUyYMCFmmR49evDSSy8xePDgFEUVn7csds4lXS53Ab169WoOPfRQJMVMAr/85S+pqqpi9erVGZUEGsOrhpxzTZaLXUDfc889nHvuuTHL7LHHHrz00ksMGjQoRVEllx8ROOeaLFe6gP7iiy845JBDkBQzCfzqV7+iurqaVatW5UwSAD8icM41Q+0JzSuvDKqD+vYNkkC2nCi+++67mThxYswyPXv25MUXX2T//fdPUVSp54nAOdcsqb7CpblWrVrFySefzOuvvx6z3DXXXMNVV11FQUHuV5x4InDO5YW77rqLSZMmxSzTq1cvXnjhBQYOHJiiqDJD7qc651zeWrlyJaNGjUJSzCTw29/+lurqapYvX553SQD8iMA5l4NKS0v5wQ9+ELNMnz59eOGFF9h3331TFFXmintEIKlrlEfrVATnnHOJWrFiBSUlJUiKmQSuu+46qqur+fTTTz0JhBKpGnoTKAc+AD4MX38i6U1JI+KNLKlQ0gJJT4Tv+0t6XdKHkmZJatOcGXAuVfK1T52W1tLL8c9//jOS6NWrF/Pnz49apqioiA8++AAz44orrsiLE8CNkcjSeAY4zsy6m1k34LvA/cD5wJ8SGP+nwJKI9zcAU8xsALAeiH3tlnMZoLZPnWXLwIyU3DUqF7XUcvz8888ZPnw4kvjhD3/YYLn/+7//o6amhrKyMgYMGNDM6HOYmcV8APMaGgYsjDNub+BF4AjgCUDAGqBV+PnBwLPxYhgxYoQ5l05FRWbBpmvHR1FRuiPLLs1ZjjU1NXb77bcbEPPRv39/W7p0abJnJStE235HeyRyRLBO0qWSisLHJcB6SYVATZxxbwEuiSjXDdhgZlXh++VAr2gjSposaZ6keeXl5QmE6Vzy5HKfOqnUlOW4fPlyiouLKSgo4IILLmiw3A033EBNTQ0ff/wx++yzTzMjzS+JJILvEezZPwo8BvQNhxUC/9XQSJJOAFabWWSlnaIUjdr9qZmVmlmJmZX06NEjgTCdS56G+s7J5j510iHR5Whm3H777UiiT58+vPXWW1HH22effVi6dClmxiWXXIIUbRPj4ombCMxsjZldaGbDzKzYzH5sZuVm9rWZLY0x6reAEyWVATMJqoduATpLqr1stTewopnz4FzS5UqfOukWbzl+9tlnDBkyhIKCAn784x83OJ0bb7yRmpoali5d6nv/LSBuOwJJ+wL/C/SLLG9mR8Qaz8wuBy4PpzEG+F8zGy/pAeB0guQwgeAow7mMlu196mSKaMvx2muNtWtvQ/ppzHH33XdfnnnmGfr375+CSPNL3BvTSHoLuBOYD1TXDq9X5RNvGmMIEsEJkvYmSAJdgQXAWWb2Vazx/cY0zuWeTz/9lOOOO45FixbFLHfzzTfzs5/9zKt9miDRG9Mk0rK4yszuaE4wZjYHmBO+/hgY1ZzpOeeyk5nxhz/8gYsuuihmuYEDB/L000/Tr1+/1ASW5xI5WfwPSedL6hnZujjpkTnncsayZcsYNGgQBQUFMZPAlClTqKmpYcmSJZ4EUiiRI4Lae7RdHDHMgL1bPhznXK4wM6ZMmcIvfvGLmOUGDRrE008/TV+/BCtt4iYCM/MzM865hJWVlXHsscfy/vvvxyx36623cuGFF3rdfwZoMBFIOsLMXpJ0arTPzezh5IXlnMsmZsbNN9/MxRdfHLPc4MGDefLJJ33vP8PEOkdwePg8NsrjhCTH5ZyL0JIdtbXktF566SUkUVBQEDMJ3HbbbdTU1PDOO+94EshADR4RmNnV4XPDd3J2ziVdbUdtmzcH72s7aoPGt2NoiWnV1NSw++67s3bt2pjlhgwZwpNPPknv3r0bF6RLuUTuR/BTSbsqcFfY/fTRqQjOORc0vqrdcNfavDkYnsppPf/880iisLAwZhK4/fbbqamp4a233vIkkCUSuWroPDO7VdIxwO7AucBU4LmkRuacA1q2w7vGTqumpoZu3bqxYcOGuNN+++23OfDAAxsflEu7RNoR1J7SPw6YamZvEb3zOOdcErRkh3eJTuvZZ5+t2/uPlQSGDh1KTU0NZuZJIIslkgjmS3qOIBE8K6kT8bufds61kJbs8C7WtGpqaujYsSOSOPbYY2NO54033sDMWLhwoV/+mQMSSQQTgcuAkWa2GWhDUD3knEuB8eOhtBSKikAKnktLm9bhXbRpnX/+U5x1VrD3v2nTpgbHHTFiRN2NTEpK4nZf47JIg53OSRpoZu9JGh7tczN7M6mRRfBO55xrWdXV1XTo0IGvvorZ3yMA8+fPZ/jwqJsBl+FaotO5nwOTgZujfGYE9xdwzmWRJ554grFjx8YtN2rUKF5//fUUROQyQax2BOHVxXzXzLZGfiapbVKjcs61mOrqanbZZReqq6vjll2wYAHFxcVJi2XGDL+nQyZK5BzBvxIc5pzLII8++iiSaNWqVcwk8K1vfauu7j/ZSWDy5KARm9n2xmzNadnsWkasvob2JLixfDtJw9h+yeiuQPuGxnPOpU9VVRWtW7dOqOxbb73FkCFDkhzRdrEas/lRQXrFOkdwDHAOwX2Fb2Z7ItgIXJHcsJxzjfHwww9z2mmnxS13+OGHM2fOnOQHFEVLNoxzLSvWOYJpkv4GjDMzP3hzLsM0Zu//nXfeYfDgwUmOKLa+fYPqoGjDXXrFPEdgZjXAD1IUi3MuAQ888ACS4iaBI488sq7uP91JAFq2YZxrWYn0NfS8pP8FZgF1rU3MbF3SonLO7WDbtm20adMmobKLFi1i0KBBSY6o8WrPA/hVQ5mnwQZldQWkT6IMNjNL2a0qvUGZy1czZ85k3Lhxccsde+yxPP300ymIyGWTlmhQBvitKp1Ltcbs/S9ZsoSBAwcmOSKX6xK5H0F7Sb+UVBq+HyAp7h3KJLWV9B9Jb0laJOnX4fD+kl6X9KGkWZIS+8U7l+Puu+8+JMVNAmPHjq2r+/ck4FpCIucIpgLzgUPC98uBB4An4oz3FXCEmVVKag28Kulpgq4rppjZTEl3EnRqd0eToncuy3399dfssssuCZV9//332XfffZMckctHibQs3sfMfgdsAzCzLSRwPwILVIZvW4eP2j6KHgyHTwNObmzQzmW76dOnIyluEjj55JPr9v49CbhkSeSI4GtJ7Qg24kjah2BvPy5JhQRHE98Abgc+AjaYWVVYZDlB6+Vo404m6PTOb3btcsI993zFuecm1k3XBx98wIABA5IckXOBRI4IrgGeAfpImgG8CFySyMTNrNrMiglaJ48C9o9WrIFxS82sxMxKevTokcjXOZeRpk2bhqS4SeD000+v2/v3JOBSKZGrhp6TNB84iKBK6KdmtqYxX2JmGyTNCafRWVKr8KigN7Ci8WE7l9m2bt1Ku3btEir70UcfsffeKbsa27mdJHLV0OPA0cAcM3si0SQgqYekzuHrdsCRwBJgNnB6WGwC8FhTAncuE919991ISiAJnAkYknkScGmXSNXQzcBhwGJJD0g6PcH7EfQEZkt6G3gDeN7MngAuBX4uaSnQDfhrE2N3LilmzIB+/aCgIHiO103y1q1bkYQkJk6cGGfqHxPUhs4EvJ8dlxniJgIze9nMzgf2BkqB/wJWJzDe22Y2zMyGmNlgM/tNOPxjMxtlZt8wszPMLKETz86lQmP6zP/LX/6S0N7/WWedxfTpRvv2Bmxvn+n97LhMkchVQ7VVO2MJjmeHE1z26VzOiddn/pYtW2hfv+e0BpSVlVFUVLTT9L2fHZdpEjlHMIugbv8IgktA9zGzC5MdWL5pbHWES46G+sZftuxOJMVNAhMmTKi78qd+Ehg/HsrKoKYmePYk4DJFoi2Lv2dm8W946pqktjqidk+0tjoCfGORajv2mb8Z6JDQeMuWLfP2Li5rNXhEIOlUSacS3JbypNr3EcNdC4lVHeFS67rroHXr2wmulI6dBCZOnFi39+9JwGWzWEcEY8Pn3Qn6GXopfP9tYA7wcPLCyi9+C7/027p1K507d+arr+Jfu/DZZ5/Ru3fvFETlXGo0eERgZuea2bkE17oNMrPTzOw04ICURZcnGtqZ9J3M5HvooYfqrvyJlQQmT55ct/fvScDlmkTOEfQzs5UR778A9ktSPHnpuut2PEcAfmlhMm3ZsoV9992X5cuXxy37+eefs9dee6UgKufSJ5EGZXMkPSvpHEkTgCcJ+htyLWT8eCgthaIikILn0lI/UdzSau/12759+5hJ4Pzzz6/b+/ck4PJBIn0N/VjSKcDocNBrwB5JjSoPjR/vG/5k2Lx5M/vssw+rVq2KW3bNmjV069YtBVE5l1kSOSIA+ITgfgSnEJwsXpK0iJxrAbNmzUISHTp0iJkEbr311rq9f08CLl81eEQgaV/gv4FxwFpgFsHN7r+dothcFpsxI/WtaDdt2kT//v0pLy+PW3bt2rV07do1uQE5lyViHRG8B3wHGGtmh5rZbYA3KnNxNaa/npbw97//HUl07NgxZhL44x//WLf370nAue1inSM4jeCIYLakZwi6S4x7i0rn4vXX0xIqKyspKipi3bp1McsVFBSwZs0aunTp0jJf7FwOitWO4BEzOxMYSNCA7GfAHpLukHR0iuJzWSiZDeRq7/XbqVOnmEngjjvuwMyorq72JOBcHIlcNbQJmAHMkNQVOAO4DHguybG5LLVjfz07Dm+KyspKevfuzZdffhmzXOvWrVm9ejWdO3du2hc5l6cSvWoIADNbZ2Z/NrMjkhWQy37XXRc0iIvUlAZy9957b93ef6wkUFpaipnx9ddfexJwrgkalQicS0RzGshVVFTQqVMnJDFhwoQGy7Vu3ZYNGzZgZkyaNKkFo08v747cpYMnApcUje17f+rUqUhi1113pbKyMkbJuwCjdestPPHEbi0XcAZI9dVWztWSmaU7hrhKSkps3rx56Q7DtbCNGzey5557smXLlpjlpA6YrQB23WF4UVGQZHJFv37Rz63k2ny61JE038xK4pXzI4I0SkU1QCZWNdx1111IYrfddouZBKZOnUqwo1JJ/SQAmdNNd0stY++O3KVNbQObTH6MGDHCcs306Wbt25sFlQDBo337YHg2fUeiNmzYYG3atDGCbs0bfOy66662ceMC9SrfAAAXQElEQVTGHcYtKtpxHmofRUWpn4/6WnIZZ/J8uuwEzLMEtrFp38gn8sjFRJCKP30mbFj+/Oc/x934AzZt2rQGp5FJCa2+llzGmTyfLjulPREAfYDZBB3ULQJ+Gg7vCjwPfBg+d4k3rVxMBFL0DYiUXd8Rzfr1662wsDDuxr9r165WUVGR0DSnTw82rlLwnCkbx5Zexpk6ny47JZoIknmOoAr4hZntDxwEXCBpEEFjtBfNbADBfQ0uS2IMGSsVdyVL9Z3P7rjjDiTRpUsXqqsb7pZq+vTpmBlr166lY8eOCU27sVchpUpLL+NMnU+X25KWCMxspZm9Gb6uIDgy6AWcBEwLi00DTk5WDJmspRpdpfs71q9fjyQkcf755zdYrkePHlRWVmJmjM+hrVsqlrFzSZfIYUNzH0A/4FOCSz821PtsfQPjTAbmAfP69u3b4odMmSAV1QDJ+o7bbrstobr/++67r2W+MIN5dY7LVCRYNZT0dgSSOgIvA9eZ2cOSNphZ54jP15tZzF7BvB1BZli3bl1CN2/p2bMnS5cupX39XWXnXEplRDsCSa2Bh4AZZvZwOPgLST3Dz3sCq5MZg2u+W2+9FUlxk8DMmTMxM1asWOFJwLkskrREIEnAX4ElZvb7iI8eB2o7kZkAPJasGFzTrV27tq7u/6KLLmqwXK9evdi0aRNmxplnntli35+JDeGcy1XJPCL4FnA2cISkheHjOOB64ChJHwJHhe9dhpgyZQqS6N69e8xyDzzwAGbG8uXLW3zv3/vccS61vK8hR3l5Obvvvnvccn379uW9996jXbt2SY3H+9xxrmVkxDkCl9luuukmJMVNAg899BBmxrJly5KeBCC9fe54lVR28vXWPHHvUOZyy+rVq9ljjz3iltt7771ZtGgRbdu2TUFUO2rpO5wlqrZKqvZ+y7VVUuANuzKZr7fm8yOCPHHDDTcgKW4SeOSRRzAzPvroo7QkAUhfI60rr9y+Mam1eXMw3GUuX2/N50cEOWzr1q1MnTo1ZotfgAEDBvDOO++wyy67pCiy2Gr34q68MqgO6ts3SALJ3rvzbqCzk6+35vMjghy0ZMkSfv7zn9OrV6+YSeDxxx/HzPjggw/SlgQaqttNR587qe6bybUMX2/N54kgR2zdupXp06czevRoBg0axJQpU1i3bt1O5QYOHMjWrVsxM8aOHZuGSLfLtMtEvd+g7OTrrQUk0g9Fuh+52A11S1m0aJFddNFF1qVLl6h9/RQVFdlll11mS5YsSXeoO8mE+yXU5/0GZSdfb9GRKX0NtYSmtCOYMSP1dcypsmXLFh588EFKS0t59dVXd/q8sLCQk046icmTJ3PUUUdRUJCZB34FBcGmvz4pqBJyzjVPou0IcvJkcS5fTlZZWUn//v1Zs2bNTp/169ePSZMmce6559KzZ880RNc46bpM1Dm3o8zcVWymXL6crGPHjhx00EF171u1asVpp53Gs88+y0cffcQVV1yRFUkAvG7XuUyRk0cEuX452eTJk1m0aFHd3v+ee+6Z7pCaJPIy0WXLoLBwx4Sd7UdvzmWLnEwEuV7lcPzxx3P88cdnbN1/Y9Ru7HO1Ks+5bJD9W5Iocr3KoaCgICeSQK1crspzLhvkztYkwvjxUFoa9FYpBc+lpb53malyvSrPuUyXk1VDEGz0fcOfHXK9Ks+5TJeTRwQuu+R6VZ5zmc4TgUu7xlTleb/zzrW8nK0actklkaq8XG4o6Fw6+RGByxp+dZFzyeGJwGUNv7rIueTwROCyhvc771xyeCJwWcOvLnIuOZKWCCTdLWm1pHcjhnWV9LykD8PnLsn6fpd7vKGgc8mRzCOCe4Bj6w27DHjRzAYAL4bvnUtYOm5h6VyuS1oiMLNXgPr3SjwJmBa+ngacnKzvd845l5hUtyPYw8xWApjZSkm7N1RQ0mRgMkBfPxvoXIvbtm0by5cvZ+vWrekOxTVT27Zt6d27N61bt27S+BnboMzMSoFSCG5VmeZwnMs5y5cvp1OnTvTr1w9J6Q7HNZGZsXbtWpYvX07//v2bNI1UXzX0haSeAOHz6hR/v3MutHXrVrp16+ZJIMtJolu3bs06skt1IngcmBC+ngA8luLvd85F8CSQG5q7HpN5+ejfgdeA/SQtlzQRuB44StKHwFHhe+ecc2mUzKuGxplZTzNrbWa9zeyvZrbWzL5jZgPC5/pXFTnn8khhYSHFxcUMHjyYsWPHsmHDhnSHtIPjjjuuRWK65ppruOmmm3YafsghhzR72i3BWxY755pk7Vp4+22YNy94Xru28dNo164dCxcu5N1336Vr167cfvvtLRJbVVVVi0znqaeeonPnzi0yrWj+9a9/JW3ajeGJwDmHpEY/uncXQ4eKkSOD5+7do5dL1MEHH8znn39e9/7GG29k5MiRDBkyhKuvvrpu+G9/+1sGDhzIUUcdxbhx4+r2tMeMGcMVV1zB4Ycfzq233kp5eTmnnXYaI0eOZOTIkfzzn/8E4OWXX6a4uJji4mKGDRtGRUUFK1euZPTo0XVHJ3PnzgWgX79+rFmzBoDf//73DB48mMGDB3PLLbcAUFZWxv7778+kSZM44IADOProo9myZUvC89yxY0cA5syZw5gxYzj99NMZOHAg48ePxyy4WHL+/PkcfvjhjBgxgmOOOYaVK1cmPP2EmVnGP0aMGGHOuZa1ePHiutdA0h6xdOjQwczMqqqq7PTTT7enn37azMyeffZZmzRpktXU1Fh1dbUdf/zx9vLLL9sbb7xhQ4cOtc2bN9vGjRvtG9/4ht14441mZnb44Yfbj370o7ppjxs3zubOnWtmZsuWLbOBAweamdkJJ5xgr776qpmZVVRU2LZt2+ymm26ya6+9ti6WjRs3mplZUVGRlZeX27x582zw4MFWWVlpFRUVNmjQIHvzzTftk08+scLCQluwYIGZmZ1xxhn2t7/9baf5vPrqq+vijDb/s2fPtl133dU+++wzq66utoMOOsjmzp1rX3/9tR188MG2evVqMzObOXOmnXvuuXHXZy1gniWwjc3YdgTOudy3ZcsWiouLKSsrY8SIERx11FEAPPfcczz33HMMGzYMgMrKSj788EMqKio46aSTaNeuHQBjx47dYXpnnnlm3esXXniBxYsX173fuHEjFRUVfOtb3+LnP/8548eP59RTT6V3796MHDmS8847j23btnHyySdTXFy8w3RfffVVTjnlFDp06ADAqaeeyty5cznxxBPp379/XfkRI0ZQVlbWpGUxatQoevfuDVC3TDp37sy7775bt1yqq6vp2bNnk6YfiycC51xdNUSi3n4bvv565+Ft2sCQIYlPp/YcwZdffskJJ5zA7bffzk9+8hPMjMsvv5wf/OAHO5SfMmVKzOnVbqgBampqeO211+qSRq3LLruM448/nqeeeoqDDjqIF154gdGjR/PKK6/w5JNPcvbZZ3PxxRfz/e9/v26cWMtnl112qXtdWFjYqKqhWNOpqqrCzDjggAN47bXXmjTNRPk5Audco/XqFdw3OlJBQTC8KXbbbTf+8Ic/cNNNN7Ft2zaOOeYY7r77biorKwH4/PPPWb16NYceeij/+Mc/2Lp1K5WVlTz55JMNTvPoo4/mj3/8Y937hQsXAvDRRx9x4IEHcumll1JSUsJ7773HsmXL2H333Zk0aRITJ07kzTff3GFao0eP5tFHH2Xz5s1s2rSJRx55hMMOO6xpM9sI++23H+Xl5XWJYNu2bSxatKjFv8ePCJxzjdatW/D8+efBkUGbNkESqB3eFMOGDWPo0KHMnDmTs88+myVLlnDwwQcDwUnV6dOnM3LkSE488USGDh1KUVERJSUl7LbbblGn94c//IELLriAIUOGUFVVxejRo7nzzju55ZZbmD17NoWFhQwaNIjvfve7zJw5kxtvvJHWrVvTsWNH7r333h2mNXz4cM455xxGjRoFwP/8z/8wbNiwRlUDXXvttXUnmSHo4iOeNm3a8OCDD/KTn/yEL7/8kqqqKi666CIOOOCAhL83IYmcSEj3I59PFk+fblZUZCYFz9Onpzsil2rJ+g1EO7mYDSoqKszMbNOmTTZixAibP39+miPKDH6yOEfNmAGTJ2+/YfuyZcF78H7484X/BnY2efJkFi9ezNatW5kwYQLDhw9Pd0hZT9bIk0TpUFJSYvPmzUt3GCnXr1/wx6+vqCi4KYvLfcn8DSxZsoT999+/eRNxGSPa+pQ038xK4o3rJ4sz2KefNm64yz3+G3Cp4IkggzV0Px6/T0/+8N+ASwVPBBnsuuugffsdh7VvHwx3+cF/Ay4VPBFkiBkzgvrggoLgecaM4GRgaWlQHywFz6Wl6T9JGC1WlxyZ+htwOSaRS4vS/cj1y0enTzdr394Mtj/at8/MS0WzKVYXWyZcPnrttdfaoEGD7MADD7ShQ4fav//970ZP45NPPrEZM2bUvV+wYIE9+eSTzY6toKDAhg4dagcccICdfvrptmnTJjPb3j9QovGkSnMuH/Ujggxw5ZXbLw+stXlzMDzTZFOsrmW19JHga6+9xhNPPMGbb77J22+/zQsvvECfPn0aPZ2ysjLuu+++uvcLFy7kqaeeatQ0onVbHdlFdps2bbjzzjubFE828ESQAbLpypBsitW1nNr2DMuWBceBte0ZmpMMVq5cSffu3ev62OnevTt77bUXAG+88QaHHHIIQ4cOZdSoUVRUVFBWVsZhhx3G8OHDGT58eF1f/pdddhlz586luLiYG264gV/96lfMmjWL4uJiZs2axaZNmzjvvPMYOXIkw4YN47HHgjvk3nPPPZxxxhmMHTuWo48+Omashx12GEuXLt1hmJlx8cUXM3jwYA488EBmzZq1Uzzx+kbKGIkcNqT7ketVQ0VFO1a11D6KitId2c6yKVYXW2OqhpKx3isqKmzo0KE2YMAA+9GPfmRz5swxM7OvvvrK+vfvb//5z3/MzOzLL7+0bdu22aZNm2zLli1mZvbBBx9Y7XZh9uzZdvzxx9dNd+rUqXbBBRfUvb/88svruoZev369DRgwwCorK23q1KnWq1cvW7t2bdT4aquAtm3bZieeeKL96U9/2mH4gw8+aEceeaRVVVXZqlWrrE+fPrZixYqd4kkVrxrKctl0ZUg2xepaTjKOBDt27Mj8+fMpLS2lR48enHnmmdxzzz28//779OzZk5EjRwKw66670qpVK7Zt28akSZM48MADOeOMM3boYjqW5557juuvv57i4mLGjBnD1q1b+TQM/KijjqJr165Rx6vtIrukpIS+ffsyceLEHT5/9dVXGTduHIWFheyxxx4cfvjhvPHGG01fIGnkXUxkgNorQK68Mvhj9e0bbFgbujJkxozEy6Y7Vpcb+vaN3sK5ue0ZCgsLGTNmDGPGjOHAAw9k2rRpDB8+POqdzaZMmcIee+zBW2+9RU1NDW3btk3oO8yMhx56iP3222+H4a+//voO3VbXV3uOINZ0c4UfEWSI8eODLgNqaoLnWEmgpetqkxWryx3JOBJ8//33+fDDD+veL1y4kKKiIgYOHMiKFSvq9q4rKiqoqqriyy+/pGfPnhQUFPC3v/2N6upqADp16kRFRUXddOq/P+aYY7jtttvqNtwLFixoetARRo8ezaxZs6iurqa8vJxXXnmFUaNG7fT92cATQZbxq3ZcOiSjPUNlZSUTJkxg0KBBDBkyhMWLF3PNNdfQpk0bZs2axYUXXsjQoUM56qij2Lp1K+effz7Tpk3joIMO4oMPPqjbmx8yZAitWrVi6NChTJkyhW9/+9ssXry47mTxVVddxbZt2xgyZAiDBw/mqquuapFlcsoppzBkyBCGDh3KEUccwe9+9zv23HPPneLJBmnpdE7SscCtQCFwl5ldH6t8vnY6F01BQXAkUJ8U7KE7lyjvdC63ZFWnc5IKgduB7wKDgHGSBqU6jmzlfc8451paOqqGRgFLzexjM/samAmclIY4spJfteOca2npSAS9gM8i3i8Ph+1A0mRJ8yTNKy8vT1lwmc77nnEtKZeufMlnzV2P6UgEO18XBjvNhZmVmlmJmZX06NEjBWFlD79qx7WEtm3bsnbtWk8GWc7MWLt2bcKX00aTjnYEy4HIDkV6AyvSEIdzea13794sX74cP+LOfm3btqV3795NHj8dieANYICk/sDnwH8D30tDHM7ltdatW9O/f/90h+EyQMoTgZlVSfox8CzB5aN3m9miVMfhnHMukJYuJszsKaBx/cQ655xLCm9Z7JxzeS4tLYsbS1I5EKXLq6zVHViT7iDSIF/nG/J33n2+06vIzOJedpkViSDXSJqXSLPvXJOv8w35O+8+39nBq4accy7PeSJwzrk854kgPUrTHUCa5Ot8Q/7Ou893FvBzBM45l+f8iMA55/KcJwLnnMtzngiSTNLdklZLejdi2DWSPpe0MHwcl84Yk0FSH0mzJS2RtEjST8PhXSU9L+nD8LlLumNtSTHmO6fXuaS2kv4j6a1wvn8dDu8v6fVwfc+S1CbdsbakGPN9j6RPItZ3cbpjjcXPESSZpNFAJXCvmQ0Oh10DVJrZTemMLZkk9QR6mtmbkjoB84GTgXOAdWZ2vaTLgC5mdmkaQ21RMeb7v8jhdS5JQAczq5TUGngV+Cnwc+BhM5sp6U7gLTO7I52xtqQY8/1D4AkzezCtASbIjwiSzMxeAdalO45UM7OVZvZm+LoCWEJwA6KTgGlhsWkEG8mcEWO+c5oFKsO3rcOHAUcAtRvDXFzfDc13VvFEkD4/lvR2WHWUU9Uj9UnqBwwDXgf2MLOVEGw0gd3TF1ly1ZtvyPF1LqlQ0kJgNfA88BGwwcyqwiJR70aY7erPt5nVru/rwvU9RdIuaQwxLk8E6XEHsA9QDKwEbk5vOMkjqSPwEHCRmW1MdzypEmW+c36dm1m1mRUT3GxqFLB/tGKpjSr56s+3pMHA5cBAYCTQFcjo6k9PBGlgZl+EP54a4C8Ef5qcE9aZPgTMMLOHw8FfhPXotfXpq9MVX7JEm+98WecAZrYBmAMcBHSWVNvdfU7fjTBivo8NqwjNzL4CppLh69sTQRrUbghDpwDvNlQ2W4Un0f4KLDGz30d89DgwIXw9AXgs1bElU0PznevrXFIPSZ3D1+2AIwnOj8wGTg+L5eL6jjbf70Xs7IjgvEhGr2+/aijJJP0dGEPQLe0XwNXh+2KCw+Qy4Ae19ea5QtKhwFzgHaAmHHwFQX35/UBf4FPgDDPLmZPpMeZ7HDm8ziUNITgZXEiwg3m/mf1G0t7ATILqkQXAWeFeck6IMd8vAT0AAQuBH0acVM44ngiccy7PedWQc87lOU8EzjmX5zwROOdcnvNE4Jxzec4TgXPO5TlPBM7VI+nKsCfJt8OeI78Zo+w9kk5v6HPnskGr+EWcyx+SDgZOAIab2VeSugM51XWyc/X5EYFzO+oJrKlt9GRma8xshaRfSXpD0ruSSsMWozuQNELSy5LmS3o2onXpTyQtDo8wZqZ4fpyLyxuUORch7CzuVaA98AIwy8xeltS1tgW0pL8RtCD9h6R7gCcIuk54GTjJzMolnQkcY2bnSVoB9A+PMDqHfdI4lzG8asi5COENRkYAhwHfBmaFN9CpkHQJQYLoCiwC/hEx6n7AYOD58GChkKCXUYC3gRmSHgUeTcmMONcIngicq8fMqgl6kZwj6R3gB8AQoMTMPgvvMNe23mgCFpnZwVEmeTwwGjgRuErSARF99DuXdn6OwLkIkvaTNCBiUDHwfvh6TVh1FO0qofeBHuHJZiS1lnSApAKgj5nNBi4BOgMdkzcHzjWeHxE4t6OOwG1h18JVwFJgMrCBoEfRMuCN+iOZ2dfhZaR/kLQbwX/rFuADYHo4TMAUP0fgMo2fLHbOuTznVUPOOZfnPBE451ye80TgnHN5zhOBc87lOU8EzjmX5zwROOdcnvNE4Jxzee7/Aw9qUPVSLSEmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the Regression Line\n",
    "\n",
    "\n",
    "plt.scatter(X, y, color = 'blue', label='Scatter Plot')\n",
    "plt.plot(X_test, y_pred, color = 'black', linewidth=3, label = 'Regression Line')\n",
    "plt.title('Relationship between Sales and Advertising')\n",
    "plt.xlabel('Sales')\n",
    "plt.ylabel('Advertising')\n",
    "plt.legend(loc=4)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Residual analysis\n",
    "\n",
    "\n",
    "\n",
    "A linear regression model may not represent the data appropriately. The model may be a poor fit to the data. So, we should validate our model by defining and examining residual plots.\n",
    "\n",
    "The difference between the observed value of the dependent variable (y) and the predicted value (ŷi) is called the residual and is denoted by e. The scatter-plot of these residuals is called residual plot.\n",
    "\n",
    "If the data points in a residual plot are randomly dispersed around horizontal axis and an approximate zero residual mean, a linear regression model may be appropriate for the data. Otherwise a non-linear model may be more appropriate.\n",
    "\n",
    "If we take a look at the generated ‘Residual errors’ plot, we can clearly see that the train data plot pattern is non-random. Same is the case with the test data plot pattern.\n",
    "So, it suggests a better-fit for a non-linear model. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHoVJREFUeJzt3X+UVOWd5/H3BwUBxRCQ+IMG2jHGFQ200PFH9DgZg4oxhuxu3Jgl0SgR9cQddbJng+FkUGdJnONkTNyYnWGNhtl0dDyTiRonGQNG1+RsDLQJapBhQcOPVpQGJWJQfsh3/7i3oWga+kfdqrpV9/M6p07Xfe6te59bFN966rnP872KCMzMrPENqnUFzMysOhzwzcwKwgHfzKwgHPDNzArCAd/MrCAc8M3MCsIB3xqKpOWSPnKAdR+R1JHRcZ6U9IUs9mVWLYfWugJWTJLWAEcD7wJvAf8KXB8Rb5Wz34g4pfzamTUmt/Ctli6JiCOAFuA04OYa16fqJO3X6OqprL/7MOuJA77VXES8CjxGEvgBkHSYpL+RtE7Sa5L+TtKwdN1Rkh6VtEXS65J+IWlQum6NpGnp82GSvifpDUkvAB8qPa6kkPT+kuXvSfrv6fP3psfoTF//qKSmvpyPpEGS5kh6UdJmSQ9KGpWua06PO0vSOuDnPZWl234i7aLaknYhnVxyjDWSvizpOeCPkg5Nl1+WtFXSSkkf7f+/hjUyB3yruTSQXgSsLin+a+ADJF8C7wfGAn+ZrvsS0AGMIekW+grQU46QecAJ6eNC4Ip+VGsQcB8wARgPvA18u4+v/XPgk8CfAscBbwB3d9vmT4GT03rtVybpA8D9wI0k5/kT4MeShpRs/xngYmAkyTleD3woIkak+13Tx/paQTjgWy09JGkrsB7YSBKgkSTgauCmiHg9IrYCXwMuS1+3EzgWmBAROyPiF9FzUqj/BMxP97EeuKuvFYuIzRHxw4jYlh5/PklA7otrgLkR0RER24FbgE9163q5JSL+GBFvH6Ds08C/RMSiiNgJ/A0wDPhwyfZ3RcT6dPt3gcOAiZIGR8SaiHixr+drxeCAb7X0ybQ1+hHg3wFHpeVjgOHAM2l3xhaSi7pj0vV3kPwa+JmklyTNOcD+jyP5Mumytq8VkzRc0t9LWivpTeApYKSkQ/rw8gnAj0rqvoIkIB9dss36Hl5XWnZcaX0jYne6fmxP20fEapJfA7cAGyU9IOm4PtTVCsQB32ouIv4P8D2SVizAJpIulFMiYmT6eE96gZeI2BoRX4qIPwEuAf7iAP3VG4BxJcvju63fRvLF0uWYkudfAk4CzoiII4Fz03L14ZTWAxeV1H1kRAyNiJdLT7uH15WWvULyxZEcNPnVMw444D4i4gcRcU76uiDpFjPbwwHf8uKbwPmSWtLW7P8C7pT0PgBJYyVdmD7/uKT3p0HwTZLW87s97PNB4Ob0AmwT8F+6rV8G/GdJh0iazr5dNiNIvnS2pBdc5/XjXP4OmC9pQlrfMZJm9OP1XXW/WNJHJQ0m+QLaDvzfnjaWdJKk8yQdBryT1r2n98QKzAHfciEiOoF/AL6aFn2ZpNvm6bRLZTFJixvgxHT5LeBXwHci4skednsrSbfI74GfAf+72/obSH4hbAFmAg+VrPsmSZ/5JuBpki6lvvoW8AhJl9PW9PVn9OP1RMRK4LPA/0jrcAnJMNYdB3jJYcDt6bavAu8juZhttod8AxQzs2JwC9/MrCAc8M3MCsIB38ysIBzwzcwKIldJl4466qhobm6udTXMzOrKM888sykixvS2Xa4CfnNzM+3t7bWuhplZXZHUp1nk7tIxMysIB3wzs4JwwDczK4iyA76koZKWSHo2vVnDrWn58ZJ+LWmVpH/slsfbzMyqLIsW/nbgvIiYTHKziumSziTJ1HdnRJxIcgOIWRkcy8zMBqjsgB+JrhtPD04fAZwH/FNavpDkDkBmVu/a2qC5GQYNSv62tdW6RtZHmfThp+lll5HctWgR8CKwJSJ2pZt0sO+NG0pfO1tSu6T2zs7OLKpjZpXS1gazZ8PatRCR/J0920G/TmQS8CPi3YhoAZqA00nuy7nfZgd47YKIaI2I1jFjep03YGa1NHcubNu2b9m2bUm55V6mo3QiYgvwJHAmye3guiZ2NZHcwcfM6tm6df0rt1zJYpTOGEkj0+fDgGkk9/B8AvhUutkVwMPlHsvMamx897tE9lJuuZJFC/9Y4AlJzwFLgUUR8SjJHYv+QtJqYDTw3QyOZWa1NH8+DB++b9nw4Um55V7ZuXQi4jngtB7KXyLpzzezRjFzZvJ37tykG2f8+CTYd5VbruUqeZqZ1YGZMx3g65RTK5iZFYQDvplZQTjgm9Ujz3a1AXAfvlm96Zrt2jUBqmu2K7hv3Q7KLXyzeuPZrjZADvhm9cazXW2AHPDN6o1nu9oAOeCb1RvPdrUBcsA3qzczZ8KCBTBhAkjJ3wULfMHWeuVROmb1yLNdbQDcwjez/PJ8g0y5hW9m+eT5BplzC9/M8snzDTLngG9m+eT5BplzwDezfPJ8g8w54JtZPnm+QeYc8M0snzzfIHMepWNm+eX5BplyC9/MrCAc8M3MCsIB38ysIMoO+JLGSXpC0gpJyyXdkJaPkrRI0qr073vLr66ZmQ1UFi38XcCXIuJk4Ezgi5ImAnOAxyPiRODxdNnMzGqk7IAfERsi4jfp863ACmAsMANYmG62EPhkuccyM7OBy7QPX1IzcBrwa+DoiNgAyZcC8L4DvGa2pHZJ7Z2dnVlWx8zMSmQW8CUdAfwQuDEi3uzr6yJiQUS0RkTrmDFjsqqOmZl1k0nAlzSYJNi3RcQ/p8WvSTo2XX8ssDGLY5mZ2cBkMUpHwHeBFRHxtyWrHgGuSJ9fATxc7rHMzGzgskitcDbwOeB5ScvSsq8AtwMPSpoFrAMuzeBYZmY2QGUH/Ij4JaADrP5oufs3M7NseKatmVlBOOCbmRWEA76ZWUE44Fs+tbVBczMMGpT8bWurdY3M6p5vgGL509YGs2fDtm3J8tq1yTL4ZhhmZXAL3/Jn7ty9wb7Ltm1JuZkNmAO+5c+6df0rN7M+ccC3/Bk/vn/lZtYnDviWP/Pnw/Dh+5YNH56Um9mAOeBb/sycCQsWwIQJICV/FyzwBVuzMjngW79VZcTkzJmwZg3s3p38dbA3K5uHZVq/eMSkWf1yC9/6xSMmzeqXA771i0dMmtUvB3zrF4+YNKtfDvjWLx4xaVa/HPCtXzxi0qx+eZSO9dvMmQ7wZvXILXwzs4JwwLfCcsp9Kxp36VgheQKZFZFb+LavgjR7PYHMisgtfNurQM1eTyCzIsqkhS/pXkkbJf2upGyUpEWSVqV/35vFsayCCtTs9QQyK6KsunS+B0zvVjYHeDwiTgQeT5ctzwrU7PUEMiuiTAJ+RDwFvN6teAawMH2+EPhkFseyCjpQ83bUqIbr189yAllBLntYA1BEZLMjqRl4NCJOTZe3RMTIkvVvRMR+3TqSZgOzAcaPHz917dq1mdTHBqB7Hz7A4MFJRNyxY2/Z8OGeXpvq6S3z22PVJumZiGjtbbuaj9KJiAUR0RoRrWPGjKl1dYqtp2bvkUfuG+yhYfv1B6JAlz2sAVQy4L8m6ViA9O/GCh7LstL9TlOvd++pSzVgv/5AVPWyh/uOrEyVDPiPAFekz68AHq7gsaxS6nk4SxUCZNXenq6+o7VrIWLvkFkHfeuHrIZl3g/8CjhJUoekWcDtwPmSVgHnp8tWb+p1OEuVAmTV3h73HVkGMrtom4XW1tZob2+vdTWsu7a2JLCsW5c0XefPz/8VyebmJMh3N2FC0lWVoaq8PYMGJV9c3UlJ95sVWl8v2jrgW2PKa4Ac6LdDFb/ArP7UzSgds4rI47WHcrqZ6rVrzXLFAd8aUx4DZDn98L7VmGXAXTrWuPJ27SGv3UxW9/rapeNsmda48nYvxvHje+6Hr4chrtYQ3KVjjSevE5Ty2M1kheKAb40lzxOU3A9vNeY+fGssHr5oBeRhmVZMBcrpn2t57VYrOAd8aywFyumfW3nuVis4B3xrLD1dGB08GLZudQCqFuf9yS0HfGsszulfe+5Wyy0HfGs8zulfW3lMa2GAA74VgQNQdXm+QW454FvjcwCqLs83yC2nVrDG1xVo8pRXp9HlLa2FAQ74VhQOQGbu0jEzKwoHfDOzMtTTpGJ36ZiZDVDXpOKueWZdc/ognz2IbuFb8dRTk8xyrd4mFbuFb8VSb00yy7V6m1TsFr4VS86bZP7xUV/qbU5fxQO+pOmSVkpaLWlOpY9ndlA5bpI5yWT9qbc5fRUN+JIOAe4GLgImAp+RNLGSxzQ7qBw3yXL+48N6UHeTiiOiYg/gLOCxkuWbgZsPtP3UqVNjoObNmxeAH3744UddPubNmzfg+Ae09yUmV7pLZyywvmS5Iy3bQ9JsSe2S2js7OytcHTOz4qp0wFcPZbHPQsSCiGiNiNYxY8ZUuDpmZsVV6YDfAYwrWW4CXqnEgW655ZaKdk/54cfBHt//fjBhQiAlf7///drux4/6e9xyyy2VCI37qHTAXwqcKOl4SUOAy4BHKnxMs6rKcnRN93u35PbiX194jGnuVDTgR8Qu4HrgMWAF8GBELK/kMc2qrR5H11Q8FnuMaS4pInrfqkpaW1ujvb291tUw65dBg5KY1p2UtNTzpvtkY0jGjmc6nLC5OQny3U2YkPx0sUxJeiYiWnvbzjNtzcqU46H9ParKL5IcT3ArMgd8szLV22zLqsTievsWLAgHfKs/ObsYWG+zLasSi+vtW7AgHPCtvuT0YmA9ja6pSiyut2/BgvBFW6svvhiYibY239O9kfiirdVUxXpdfDEwE/X0i8Sy44Bvmator4svBu6Rs0sZVgcc8C1zFR3254uBQG4vZVjOOeBb5ira6+KLgUB9zu612vM9bS1z48f3fF01s16XmTMLF+C786UMGwi38C1z7nWpPF/KsIFwwLfMudel8vylagPhLh2rCPe6VFbXe+ux9NYfbuHnhcfYWT95LL31l1v4edA9X23XGDvw/2Izy4xb+HmQhzF2/oVh/eCPS31yCz8Paj3Gzr8wrB/8calfbuHnQa3H2OXhF4bVjap+XPxTIlMO+HlQ6zF2tf6FYXWlah8X54/InAN+HtR64Hqtf2FYXanax8W/PDPngJ8XtRxjV+tfGFZXqvZx8S/PzDngW+1/YVhdqdrHxb88M+c7XplZPnUfDgTJTwk3RvZTlTteSbpU0nJJuyW1dlt3s6TVklZKurCc45hZAfmXZ+bKHYf/O+A/AH9fWihpInAZcApwHLBY0gci4t0yj2dmReKkTJkqq4UfESsiYmUPq2YAD0TE9oj4PbAaOL2cY5mZWXkqddF2LLC+ZLkjLduPpNmS2iW1d3Z2Vqg6ZmbWa5eOpMXAMT2smhsRDx/oZT2U9Xh1OCIWAAsguWjbW33MzGxgeg34ETFtAPvtAMaVLDcBrwxgP2ZmlpFKdek8Alwm6TBJxwMnAksqdCwzM+uDcodl/ntJHcBZwL9IegwgIpYDDwIvAP8KfNEjdMzMaqvcUTo/ioimiDgsIo6OiAtL1s2PiBMi4qSI+Gn5VTWrM870aDnjfPhmleCk8ZZDzqVj+3KrNBvO9Gg55Ba+7eVWaXac6dFyyC1828ut0uw406PlkAO+7eVWaXZ8jwHLIQd828ut0uw406PlkAO+7eVWabZqeRczsx444NtebpWaNTSP0rF9Of+4WcNyC9/MrCAc8M3MCsIB38ysIBzwzcwKwgHfzKwgHPDNzArCAd+s4JwgtUx19AZ6HL5ZgTlBapnq7A1URNS6Dnu0trZGe3t7rathVhjNzUmM6m7ChCQbhPUiJ2+gpGciorW37dylY1ZgTpBapjp7Ax3wzQrMCVLLVGdvoAO+WYE5QWqZ6uwNdMA3KzAnSC1Tnb2BZV20lXQHcAmwA3gRuDIitqTrbgZmAe8Cfx4Rj/W2P1+0NTPrv2pdtF0EnBoRk4D/B9ycHnwicBlwCjAd+I6kQ8o8lpmZlaGsgB8RP4uIXeni00BT+nwG8EBEbI+I3wOrgdPLOZaZmZUnyz78q4Cfps/HAutL1nWkZfuRNFtSu6T2zs7ODKtjZmalep1pK2kxcEwPq+ZGxMPpNnOBXUDXnGL1sH2PFwsiYgGwAJI+/D7U2czMBqDXgB8R0w62XtIVwMeBj8beK8AdwLiSzZqAVwZaSTMzK19ZXTqSpgNfBj4REdtKVj0CXCbpMEnHAycCS8o5lpmZlafc5GnfBg4DFkkCeDoiro2I5ZIeBF4g6er5YkS8W+axzMysDOWO0nl/RIyLiJb0cW3JuvkRcUJEnBQRPz3YfszqXh2lyLXicnpks3LVWYpcKy6nVjAr19y5e4N9l23bknKzHHHANytXnaXIteJywDcrV52lyLXicsA3K1edpci14nLANytXnaXIteLyKB2zLMyc6QBvuecWvplZQbiFb2aZ27lzJx0dHbzzzju1rkpDGTp0KE1NTQwePHhAr3fAN7PMdXR0MGLECJqbm0nTrliZIoLNmzfT0dHB8ccfP6B9OOCbWebeeeedgwb7zZvh5Zdhxw4YMgTGjoXRo6tcyTojidGjR1POfUMc8M2sIg4W7Neuhd27k+UdO5JlcNDvTbm/lnzR1syq6uWX9wb7Lrt3J+VWWQ74ZlZVO3b0r3wgNm/eTEtLCy0tLRxzzDGMHTt2z/KOPh7oyiuvZOXKlQOuQ1NTE1u2bDng+t27d3P77bcPeP8D4YBvZlU1ZMj+ZaN+2sakTzRnll569OjRLFu2jGXLlnHttddy00037VkeklYgItjd/adGifvuu4+TTjqprHocjAO+mTW8sWOTuN5l1E/bmPC12QzZsBYi9qaXrsA9BVavXs2pp57Ktddey5QpU9iwYQOzZ8+mtbWVU045hdtuu23Ptueccw7Lli1j165djBw5kjlz5jB58mTOOussNm7cuN++Ozs7Of/885kyZQrXXXcde+/4CpdccglTp07llFNO4Z577gFgzpw5bN26lZaWFi6//PIDbpepiMjNY+rUqWFm9e+FF1446PpNmyKefTZi6dKI7cdOiEhC/b6PCRMyqcu8efPijjvuiIiIVatWhaRYsmTJnvWbN2+OiIidO3fGOeecE8uXL4+IiLPPPjt++9vfxs6dOwOIn/zkJxERcdNNN8XXv/71/Y5z3XXXxfz58yMi4qGHHgog3njjjX2O8cc//jFOPvnkeP3112Pnzp3xnve8Z5999LRddz29t0B79CHGuoVvZlU3ejRMmgStrTDk1eqmlz7hhBP40Ic+tGf5/vvvZ8qUKUyZMoUVK1bwwgsv7PeaYcOGcdFFFwEwdepU1qxZs982Tz31FJ/97GcBmDFjBiNGjNiz7s4779zz66Cjo4MXX3yxx7r1dbuBcsA3K6I83ZKxyumlDz/88D3PV61axbe+9S1+/vOf89xzzzF9+vQeZwcPKbnwcMghh7Br164e993TsMnFixfz1FNP8fTTT/Pss88yadKkHo/R1+3K4YBvVjRdt2RcW/k+8z6pYXrpN998kxEjRnDkkUeyYcMGHnvssQHv69xzz6UtfQ9//OMfs3XrVgD+8Ic/MGrUKIYNG8by5ctZunQpAIcemkyD6vryONB2WXLANyuavN2SsYbppadMmcLEiRM59dRTufrqqzn77LMHvK9bb72VxYsXM2XKFJ588knGjh0LwMUXX8y2bduYPHkyt912G2ecccae18yaNYtJkyZx+eWXH3S7rChKriTXWmtra7S3t9e6GmaNbdCgpGXfnbT/jKgBWrFiBSeffHIm+7J99fTeSnomIlp7e61b+GZF41syFlZZAV/SX0l6TtIyST+TdFxaLkl3SVqdrp+STXXNrGy+JWNhldvCvyMiJkVEC/Ao8Jdp+UXAieljNvA/yzyOmWXFt2QsrLKyZUbEmyWLhwNdHYMzgH9IJwQ8LWmkpGMjYkM5xzOzjPiWjIVUdnpkSfOBy4E/AH+WFo8F1pds1pGW7RfwJc0m+RXAePchmplVTK9dOpIWS/pdD48ZABExNyLGAW3A9V0v62FXPQ4HiogFEdEaEa1jxowZ6HmYmVkveg34ETEtIk7t4fFwt01/APzH9HkHMK5kXRPwSjZVNjM7uCzSIwPce++9vPrqq71ut3r1alpaWg66zUsvvcQDDzzQ52NXQrmjdE4sWfwE8G/p80eAy9PROmcCf3D/vZkdSNaZHvqSHrkv+hrw+yIPAb/cPvzbJZ0E7AbWAtem5T8BPgasBrYBV5Z5HDNrUF2ZHrom/3ZleoDKXFdeuHAhd999Nzt27ODDH/4w3/72t9m9ezdXXnkly5YtIyKYPXs2Rx99NMuWLePTn/40w4YNY8mSJft8WSxdupRZs2Zx+OGH7zND98UXX+Tzn/88b731FoMGDeI73/kOZ5xxBnPmzGHVqlW0tLRw1VVXcfHFF/e4XUX1JaVmtR5Oj2zWGHpLj1xqwoSKZkfeJz3y888/HzNmzIidO3dGRMTVV18dbW1t8fTTT8f06dP3vKYrrXFXiuSeTJw4MX75y19GRMSNN94YkydPjogktfHbb78dERErVqyI008/PSIiFi1aFDNmzNjz+gNt15ty0iP7JuZmVlMHyoJciezIixcvZunSpbS2JlkI3n77bcaNG8eFF17IypUrueGGG/jYxz7GBRdccND9bNq0ibfffntPy/5zn/scTzzxBADbt2/n+uuv59lnn+XQQw89YIrjvm6XJadWMKu1PKUqroFqZnqICK666qo9/fkrV67kq1/9KqNHj+a5557jnHPO4a677uKaa67pdV89pUIG+MY3vsG4ceN4/vnnWbJkCdu3by9ruyw54JvVUt5SFddANTM9TJs2jQcffJBNmzYByWiedevW0dnZSURw6aWXcuutt/Kb3/wGgBEjRuxJc1zqqKOOYujQofzqV78C2JMWGZI0x8ceeyySWLhw4Z5bHXbf14G2qyQHfLNayluq4hqoZqaHD37wg8ybN49p06YxadIkLrjgAl577TXWr1/PueeeS0tLC1dffTVf+9rXALjyyiv5whe+0ONwzvvuu49rrrmGs846iyOOOGJP+fXXX88999zDmWeeydq1aznssMMAOO2003j33XeZPHkyd9111wG3qySnRzarpSqkKq4Fp0euHKdHNqtXTlVsVeSAb1ZLTlVsVeSAb1ZLDZyqOE/dxY2i3PfU4/DNaq0BUxUPHTqUzZs3M3r06AMOX7T+iQg2b97M0KFDB7wPB3wzy1xTUxMdHR10dnbWuioNZejQoTQ1NQ349Q74Zpa5wYMHc/zxx9e6GtaN+/DNzArCAd/MrCAc8M3MCiJXM20ldZLk1R+Io4BNGVanHvici8HnXAzlnPOEiOj1HrG5CvjlkNTel6nFjcTnXAw+52Koxjm7S8fMrCAc8M3MCqKRAv6CWlegBnzOxeBzLoaKn3PD9OGbmdnBNVIL38zMDsIB38ysIBoi4EuaLmmlpNWS5tS6PpUg6V5JGyX9rqRslKRFklalf99byzpmTdI4SU9IWiFpuaQb0vKGPW9JQyUtkfRses63puXHS/p1es7/KGlIreuaJUmHSPqtpEfT5UY/3zWSnpe0TFJ7Wlbxz3XdB3xJhwB3AxcBE4HPSJpY21pVxPeA6d3K5gCPR8SJwOPpciPZBXwpIk4GzgS+mP7bNvJ5bwfOi4jJQAswXdKZwF8Dd6bn/AYwq4Z1rIQbgBUly41+vgB/FhEtJWPvK/65rvuAD5wOrI6IlyJiB/AAMKPGdcpcRDwFvN6teAawMH2+EPhkVStVYRGxISJ+kz7fShIQxtLA5x2Jt9LFwekjgPOAf0rLG+qcJTUBFwP3pMuigc/3ICr+uW6EgD8WWF+y3JGWFcHREbEBkuAIvK/G9akYSc3AacCvafDzTrs3lgEbgUXAi8CWiNiVbtJon/FvAv8N6Lpr+2ga+3wh+RL/maRnJM1Oyyr+uW6EfPg93U7HY00biKQjgB8CN0bEm41+B6WIeBdokTQS+BFwck+bVbdWlSHp48DGiHhG0ke6invYtCHOt8TZEfGKpPcBiyT9WzUO2ggt/A5gXMlyE/BKjepSba9JOhYg/buxxvXJnKTBJMG+LSL+OS1u+PMGiIgtwJMk1y9GSupqoDXSZ/xs4BOS1pB0x55H0uJv1PMFICJeSf9uJPlSP50qfK4bIeAvBU5Mr+oPAS4DHqlxnarlEeCK9PkVwMM1rEvm0r7c7wIrIuJvS1Y17HlLGpO27JE0DJhGcu3iCeBT6WYNc84RcXNENEVEM8n/3Z9HxEwa9HwBJB0uaUTXc+AC4HdU4XPdEDNtJX2MpFVwCHBvRMyvcZUyJ+l+4CMkKVRfA+YBDwEPAuOBdcClEdH9wm7dknQO8Avgefb2736FpB+/Ic9b0iSSC3aHkDTIHoyI2yT9CUkLeBTwW+CzEbG9djXNXtql818j4uONfL7puf0oXTwU+EFEzJc0mgp/rhsi4JuZWe8aoUvHzMz6wAHfzKwgHPDNzArCAd/MrCAc8M3MCsIB38ysIBzwzcwK4v8Dyf/lH6xKsGYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting residual errors\n",
    "\n",
    "plt.scatter(lm.predict(X_train), lm.predict(X_train) - y_train, color = 'red', label = 'Train data')\n",
    "plt.scatter(lm.predict(X_test), lm.predict(X_test) - y_test, color = 'blue', label = 'Test data')\n",
    "plt.hlines(xmin = 0, xmax = 50, y = 0, linewidth = 3)\n",
    "plt.title('Residual errors')\n",
    "plt.legend(loc = 4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking for Overfitting and Underfitting\n",
    "\n",
    "\n",
    "I calculate training set score as 0.2861. Similarly, I calculate test set score as 0.5789. \n",
    "The training set score is very poor. So, the model does not learn the relationships appropriately from the training data. Thus, the model performs poorly on the training data. It is a clear sign of Underfitting. Hence, I validated my finding that the linear regression model does not provide good fit to the data. \n",
    "\n",
    "\n",
    "Underfitting means our model performs poorly on the training data. It means the model does not capture the relationships between the training data. This problem can be improved by increasing model complexity. We should use more powerful models like Polynomial regression to increase model complexity. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.2861\n",
      "Test set score: 0.5789\n"
     ]
    }
   ],
   "source": [
    "# Checking for Overfitting or Underfitting the data\n",
    "\n",
    "print(\"Training set score: {:.4f}\".format(lm.score(X_train,y_train)))\n",
    "\n",
    "print(\"Test set score: {:.4f}\".format(lm.score(X_test,y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['lm_regressor.pkl']"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save model for future use\n",
    "\n",
    "from sklearn.externals import joblib\n",
    "joblib.dump(lm, 'lm_regressor.pkl')\n",
    "\n",
    "# To load the model\n",
    "\n",
    "# lm2=joblib.load('lm_regressor.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple Linear Regression - Model Assumptions\n",
    "\n",
    "\n",
    "\n",
    "The Linear Regression Model is based on several assumptions which are listed below:-\n",
    "\n",
    "i.\tLinear relationship\n",
    
    "ii.\tMultivariate normality\n",
    
    "iii.\tNo or little multicollinearity\n",
    
    "iv.\tNo auto-correlation\n",
    
    "v.\tHomoscedasticity\n",
    "\n",
    "\n",
    "### i.\tLinear relationship\n",
    "\n",
    "\n",
    "The relationship between response and feature variables should be linear. This linear relationship assumption can be tested by plotting a scatter-plot between response and feature variables.\n",
    "\n",
    "\n",
    "### ii.\tMultivariate normality\n",
    "\n",
    "The linear regression model requires all variables to be multivariate normal. A multivariate normal distribution means a vector in multiple normally distributed variables, where any linear combination of the variables is also normally distributed.\n",
    "\n",
    "\n",
    "### iii.\tNo or little multicollinearity\n",
    "\n",
    "It is assumed that there is little or no multicollinearity in the data. Multicollinearity occurs when the features (or independent variables) are highly correlated.\n",
    "\n",
    "\n",
    "### iv.\tNo auto-correlation\n",
    "\n",
    "Also, it is assumed that there is little or no auto-correlation in the data. Autocorrelation occurs when the residual errors are not independent from each other.\n",
    "\n",
    "\n",
    "### v.\tHomoscedasticity\n",
    "\n",
    "Homoscedasticity describes a situation in which the error term (that is, the noise in the model) is the same across all values of the independent variables. It means the residuals are same across the regression line. It can be checked by looking at scatter plot.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "\n",
    "The concepts and ideas in this project have been taken from the following websites and books:-\n",
    "\n",
    "i.\tMachine learning notes by Andrew Ng\n",
    "\n",
    "ii.\thttps://en.wikipedia.org/wiki/Linear_regression\n",
    "\n",
    "iii.https://en.wikipedia.org/wiki/Simple_linear_regression\n",
    "\n",
    "iv.\thttps://en.wikipedia.org/wiki/Ordinary_least_squares\n",
    "\n",
    "v.\thttps://en.wikipedia.org/wiki/Root-mean-square_deviation\n",
    "\n",
    "vi.\thttps://en.wikipedia.org/wiki/Coefficient_of_determination\n",
    "\n",
    "vii.https://www.statisticssolutions.com/assumptions-of-linear-regression/\n",
    "\n",
    "viii.Python Data Science Handbook by Jake VanderPlas\n",
    "\n",
    "ix.\tHands-On Machine Learning with Scikit Learn and Tensorflow by Aurilien Geron\n",
    "\n",
    "x.\tIntroduction to Machine Learning with Python by Andreas C Muller and Sarah Guido\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}