XOR-Net-Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XOR Prediction Neural Network\n",
    "#### A simple neural network which will learn the XOR logic gate.\n",
    "\n",
    "I will provide you with any links necessary so that you can read about the different aspects of this NN(Neural Network)."
   ]
  },
  {
   "attachments": {
    "nn_pic.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAADnCAYAAAAtkT6aAAAABmJLR0QA/wD/AP+gvaeTAABL5UlEQVR42u2dB5zP9R/HKaWh6F+hMrNHiTJCFKUlo0hlZCR7ZGVmlU2EkHEoe+89ssdZ53DGccPt+9nOuHOv/+P14adzDsf9fne/8Xo+Ht8Hobv7re/r816vdyoIIYQQIkVJpadACCGEkBgLIYQQEmMhhBBCSIyFEEIIibEQQgghJMZCCCGExFgIIYQQEmMhhBBCYiyEEEIIibEQQgghMRZCCCGExFgIIYSQGAshhBBCYiyEEEJIjIUQQgghMRZCCCEkxkI4ChcuXMCMGTPs+j3OnTuH4OBgeHt74+DBg3dchw4dgr+/P86cOYMbN27oBRFCSIyF+3Ht2jUcOXLEpl/zypUrOHz4MKZMmYIGDRqgVatWGDJkCCZPnox//vnnjmvq1KkYPnw4OnfujMqVK2PkyJHYtGkTIiMj9eIIISTGQiSW69evIyAgAGvWrMHQoUPRuHFjDBs2DJs3b74d9SbmioiIgKenp4nUq1SpggEDBmDFihXmsBAVFaUnWgghMRYiPjExMSb1PH78eIwYMcJcO3fuNCnpxArwvS5Gxnv37oWHh4f5un/++Se2bduGq1ev6okXQkiMhSAWi8WIcN++fbFw4UITvVJAkyrC8S9+H0bXy5cvR58+fdClSxeEhobqBRBCSIyF+3L58mUsWrQIXbt2xdy5cxESEmIE09YinJAoh4eHm8i7WbNmmD17NsLCwhAbG6sXRQghMRbuAUXv9OnTpimrffv2pgva3gJ8r8vPzw8TJ040Ubmtm8+EEBJjIRyWkydPmo7oOXPmGFFOKSG2XoyK161bh19++QVbt27VWJQQQmIsXBeKHMeUWrZsaUSPHc8pLcRxL/5sAwcOxKpVq0xXtxBCSIyFS8Fu6R07dqBTp044cOCAQ4lw3Ispc45BcbRKEbIQQmIsXArWY3v06IEtW7YkS5NWUi7+rP3795cgCyEkxsJ1uHjxoumYZvrXHiNL9hJk1rUZxavLWgghMRZODceXKGoLFixwChGOO/7Eunb37t0RGBioF1IIibEQzglTvDTx+PXXX50mIo57cRZ57Nix5hJCSIyFcEpOnTplmqFoRelsQmy96NDVu3dvYxAihJAYC6fjGk557cQ2n2BcjUmo5hiDi0E+WLPDBxdjXO/RR0dHG4tLblNy9IatB11eXl4mXc3HIYSQGAun4QYunVyNH6s2w6gNJ3A5OhY3oi/h4Oqp6NtrMg5FcmtQNM4cXY0Wdepj+LqTuOpiPUJc8MDuaUbHzizEVlMQLpjgBih1VwshMRZ2Es5d03ugXLEvMWSBNxIKYm+E70K7Sh/hyyZDsT80gU0/0eewZVw3fFKiIoZtDkbstVAs610fDQavxdnoWCDmLP6dPgj1GtREwcL1sOzk2Zv/X+xVHF8xDN9UGYB94a61QWjChAlYunSp0wux9eIKxz/++MP8XgghMRZ2wH/LOHyUKRMqdJ2DqLvU+DoCVg3AK/97Gk/meR8jN/kjflY52nIMfeuWxZPpq2JD6BVYDsxD5bKNsezkxZtiHuKJ2eu9cWjjOHzw7g//iTG/+qXj6FmvGvosOoJrLhIds87atm1b0wDlKmLMx8L6965du/SBEUJiLOzBeb9NaPxBDjxdrA2OXIy+8y9jLFj3W228VqgUihQrizbjtuHSHYIdi8gjy1G7xKt45pO+CIuJgvesnijbYAACr8T9WtE49e/dYhwbfQmz+7ZA/R4LEHHVNVKgTOfSd9pVhNh60QSEu5CFEBJjYQdiLvhjbLNPkD5NRSz0vXTH38WeOYQe1Uqg8A8D0Pn7Cvig1QSE3CHYN3Bize8o90ImfNpnMaKvn8HK3s3wWZsJuHj9xgPFGDeuYcufnVGzwe84eeaa0z+XFy5cMKNMjCBdTYy54rFx48ZmdloIITEWNicKB2f3whuZMuK78XvwnyTGIHzfXHxatBR+mrYdywc3wOtlmmBLwPn/xDrmMub0+BxPZ/wQf27yQ+y1SCzp2gLfdZ6JqOjEiPF17PHojdp1B+G4xXnEmIsUoqKi7rro7fzXX3/B39//0YUvIhB71q7Fxi2HERzpWILs4eFh6sdCCImxsH1sjIg9M/BhwUwoWGsSAqJvpaFvXIanR1e8mbkyZhwIwrGVI1AudwWM2HDqv//zmj9++Sg7Xni/GdafOgfEnMP6/s3xceuxuHAtEWIccw0bR/2MWo1Hw+9cym8K8vHxMbt973f16dMHnTt3Rps2be64mJ6myQf3AyfJ5MNvI5q+nhPvfDQEe04/xNexRMDPaxPGDOuHvv1m43CI7Ueqdu/ejcGDB+sjI4TEWNiDG+cPY0DlIshe5EesC7hy88+iIvBP5+p47b222B9xCWd9VqJGmXdQa9QmWKvG1wJW4sPnM+PDJuPhd57p66s4tmQQStfuixOXH1wzvnH9PKZ0b4xGv62A5XrKd3DRoKNw4cJIlSrVQ11VqlSBp6cn5s2bh0WLFiVN9B5JjEOxa8kYfF82PzI8mxb/y9QQa0/ZfkVjQEAAOnbsqFS1EBJjYR+i4OXRHOleK4CO84+YVPWlEE+0qfA2qnabg7NXbiD24kmMqVcR2T/sj4CYmwJ7clE3PP5yYbSfcQBRRktjcenkBvz4QW1M2BWGWEQjaMt4VP64AsqWKY2SJUuhTNn3UKlBO6w4dhZR4TvRtnpNjN12Go7g/cGab5MmTfDYY48lSoSzZMliNhwxEqZA/f3339i/f3+yirElwAvjf66F7HnfwleN2qPeu4Xwqp3EmDPHo0aNMml4IYTEWNg+NsbZ3X8hR/psqPHLMoRfi0Xo3un4OHdJdJq86+bIU+wFbB/RGHnzfYe1QVeBmDDMbVUB/yvwGTw8w3A7KR1zDjvHtsVXrf5BwOX7dEjHnMO28R3xdcOJOHEh2mGeCaabX3rppQcKcaFChTB79mxcu3az1n3+/HkMHDgQvr6+thPjwGAc378Zkwf/ahrDhkxZAG/f4Dv+feSxrejdviOGzlyHk7578Xu1sshqJzGOiIgwzw+zAM5ObMxVHN+2HivXeePitYTfp7FXwrBxyXzsCTgP2Z0IibFIHi4fQss3sqHYV33hFRYFz6ltkKlIZUzdE3LrRhQDyy4PlMhbGO0WnsD1M3vRsXhevPHZEHifiyumsbgWeQA96zdEp+m7cC7BAeJrCNr+D6p92w6Lj0Qi2oGeBjZh5c2b954inDZtWpOW3rhxI2Ji/ovnGTU2aNDA/GoTMa7QF7OmD0f1CsXw0hM3v/eTL2fHF/WGYZvvf9/DEnoaPidOISzSgjNB++0qxswAcAPVunXrnP7tfsn/X3SoVQ9/Lj2EqOvXEXxgNfp2aoJq1aqhSfdp8Ao+j5hrZ7BkSGs0ajsexyxXdY8QEmORHERhRqfyeP6trzBrnw8mNSmPYtX6YH/YfzehGxE70aRYAXzYZSkObZ2AorlexcfDtuHi3TEFrl6+iAtR13EjwVJwLGKuReHcxagEXb+SE/pI03eZzUnTpk1D69atUbp06QSFOEOGDOjUqZMx9oi/5zcoKAhff/110kXPiHE2vJL1PXz4YweMXbwNvr4n4LNvPX5v+iGeTJ8XjYavRWBCndZ2FmM+T3PnzjVNasmXtLmCg3P74f2cr+Pr0ZsSLGfciDqCDqVzIssHP2C5TyTufktdhtffnZA335toMsETF2POYtXQpqjaZylCr8bgwtEF+LL0h+i/YDeCwnwxuVN9VGs4BEcir+JS8F50bd4CA5d447rCYyExFsmQuIPXnK7Ilf4d9PhrMhrlz41q3VcgIu7dL/YMFnb4DEUqdsWUoa2RN9Pb+GNXBJzNPItCSnHhPDBHkSZNmoR27dph06ZNJvrbsmXLXUKcK1cujBkzBleuXEnwax47dsx0WttGjF/F09lKo/ec3Qi1WP/OglDPSciXOhO++HEqfBLqlrazGPOaNWuWqZMnoxoj8uBCfFH8NbxeazRCo+9+t0Ud+gd5+TqlLY4+8w/dfQC8GoAp9d/Hq69WxOQDZ3A9aCMalamOv/ZaeCxEiOc89B26HIGXbqrtyXW/o0q1hthw6jxw/RzWDfsZ33SajMgrMbpNCImxsD+Wo8tR+63C+OKrb1Ek2zvotvg47hw4ioHP3M7Ik78capUvgVfeaAnvS84VLpw4ccL4LLMreOrUqVi8eLFpSLLWfnGrkStPnjy3hZiR8vLly3H16r1Tld7e3kbYbSPGWfF6qR+x1Cvozr/zXYqKqf+Hit+PhFdgpJuIMRAdfgCdK7+Dl7N8hWV+UfGVFoentUXWTKXwzhulUaXvEly/cadL3OVTG1G/XF68Wqg99p6/htDNo1Cq3C/wSvC9G4NNY9qjap1+OBh+xTQq+v87DhWqdcVeyxXdJITEWNif62dOoG/9sngm7VN4Jfd3WOV3983not8KVM2YFmmfeAq5Go/HJQd+PNwyxC5nbk9atmwZfv75ZwwaNAjbt283aWVGufHTzeZ2HBNjxPqZZ54xqWduYXrQxiLbinFOvP3hQOyOL7hGjJ/He98Nxj6/iBQT4+bNmyfqObHdC3kWGwc3RJZX3kDv1fH80aMD4FHzbbz1eV/0aVIVr3zUE6fjub/5bxqP9/O+jHe7LsSl2MvYN7kt3vxuFIJj78wMxVw9D5+1f6F2xaaYsT0AN/u6buCM90JUKv4D5h89p5uEkBiL5FDjC9g9ezRa1quHn/osRnACs7/XL/tjavdmqFevLSasP+6QD4Miy0YsRrPcKcx9vOwCPn36dILiGx/+GzpN0eDi3LnE3YCPHDli/r2txDjB0SYHEeMKFSqYx7pixQrzZ4l5TpOoxrDsHI+8r72Gim3mIvh2U2AsrpxajVpv5EP1sZuxzaMTCmT9HEv8/puDjr12FssG1kfWJ97EoHV+bN3Cnomt8Wbj8YiIG1+fC8CyP/uiyhfNsXB/eJymwlhc8FmOT0o0xtwjEmMhMRbigXD0ZsmSJejVqxfGjx+P+fPn48CBA49kUsHGrvulpeMTGBiIunXrurQYs4GLSzCss8bMBAwbNsw0v9lbkGPCt6J+gax486Me8Ay1lhSu4cSygSicrThGe0bizP7Z+KhgPjRfeOJ2H8P1c34Y+ENpPJ7lW6zyO2/S2kfnd0eRKsPgdyuAjo25gGWju6Bmw9+waO/peD0QNxCxbxYqvtcSqwMv6kMmJMZC3BWlX79uBJhiQFFgapk14KNHj5q537ijR/aGixRq1qxpBMtVxZjNbbT8XLly5e0MhJeXl+kw5w5ndpnb7TmPPYv5nSrh+QKV8JfVJOZ6BJb2+g7ZX68Pz/MxuBG5F53eK4R32ixA+K0f46zvGtR7IxsKN/gDQRev3zSm8Z6Nj4s3x7qga+a/LfumoUqJmhi94gBOh4SaxxEafgaXr8Vw2Bje8/qj0g8DceriNX3ohMRYCKsAM928atUqE6V169YNv/32m4mAUxKms7nz9+TJky4rxjz4zJw5E9u2bbsri8DXgnucly5dav6tHdQY/qv649Un86Dp7xtxPjoWV0P3o8OXJZGv6USYHEbsOWzqXwu5irTC5qCbWY0Ty/uj4OP50Wr0Fly+NUcXe+kYhterglYzvHHlxnX4rh2FGjVq3Hl93xsrvMNw/UIAxnVsiXYj1yMqOlYfQCExFu7NpUuXsGPHDtOENXr0aHOxrstINNkaiR7w8zE6PHjwYDKKcRgOrv4HPVs1MTaeTRp9h3Kvv4p0zxRE9fqNzZ81bdEaEzYfh8VGdpjjxo0zB4748DXw8/PD9OnTTZaCY2NxO9RtcuA5vgxVXn0N5VuMx+kL1xDmtQA1ipZE/T8330otxyBs/QBky/0+Rm0Nxg1EYWmfKkiXowJGrvPFf1oajROrh6NqnZ7YHnT5vgcAv01TUO/b9ljgFSYXLiExFu4Hb+6s2dJecvLkySbqmjJlihFgpoRtfaO3RcTORRGrV69Omuj5b0LrwoXw3hcjsPcuMV6Bz9O9gkoNh2O/fwTOWILw75h2eCdjOqRLl/D1/EuZ0Xj6XkTaQIzZQc1muPs1tfF1YZaia9euJmNBAbfVYen6xQAMrPcOni7bHJsDLDgw/Re8lflTjN8W+N/75sw21MzxOur8tgaWa4EY9FlhFPz0Z+w4fTne1wrE313b4uf+SxB+OWEPuNjzRzG4VV0MmLMPF+T4ISTGwp0EmJEu65Cs/fbu3dvc1BkR38tsw1FgAxNTtHSnSnLd2EEvLsH45ZdfEn04Wbt2rTlEcawsPDw86U1e0Zew+vfGeOW59zBy3UFM71oFuT/vgN1BcRqrYkMxuEpxlK87Cj77FqDsC9nwRev5CLmmFLMQyS7G/PBz56wjXgml+Nwd3rjpYMUO6OHDh5t0L4WNHbusRzoDUVFRxsGLj4E1bVcUY3anr1mz5qGeFz4X3GY1ZMgQbN26NWkNXrHR8F03Gh9kz4Nvh05G90/exuetpyLoUtyvGYN1w+sj9weNMG1iN2TNXhQ/zzsCOUsLkQJi3L59e3NT5xykI13ffvst/v33X72ytyLJs2fPmoNThw4dTJ2RKd7jx4+b+quzPAa6dbG7mLVs1kxHjBiBw4cPu6QY07ebEe7DwnIDa+l0PuvXr59JXT9alByLqMAdaPFJEeSpXhOfFngTrSfuQnwjLd81Q/HuG8VQvWopZC/QEKv9o/SBEyKlxJgneEe7mfHnclcx5s3X6obFgwlnVatWrWrH7lv7wcfBQ8Pvv/+OIkWKmNostxkR+lfzgOFqQswOaq6ITAosQzB7wAwRG73Ynf3Qonw1GLNbVcNLjz+GF978FFP33e2NfuH0DrSolMNYmeavOhp+SlELITGWGN9MVXIkhqlK1lQZPTJSYmrXmWBzEhvIevTogXfffRdp0qQxN3yOwlgPFHTiYpMThcZVhJiPhd3rTDPbAs7xcjkHD2T8nD5cP8BVBOxYjMF9e+P3v5bg1Lnrd/2L6EthWDVrtHmdJvzri2jdS4WQGLurGDM1uXPnTrPJiF21XL3H6MpRRpEeVoQ9PT3NvuLcuXPjqaeeur1EIkeOHCYStj4m1rj79u1rmp1cRYzpu01Pbx6qbAX7BHggGzlypIm4mboWQkiMJcZJhCLEkSOKFv2gOeP6559/mi1JFDP7+xfbFooFl0nwEPHFF1/giSeeQOrUqe9Yq8jImI+TdeO4cByrZ8+eJgJ0haiYESzT7/Zw1+LXZPd8mzZtzGhYQECA071XhJAYS4xTHD4uaxcxU9Bjx47F+vXrjR2lsx4q2NnNBsBPPvkEzz777F27ja1X4cKFzUKKhIScAsZ6uLOPOfn4+Bivb4qkvd9H7Lqmixkb+R7FT1wIITF2KzGmYLERy8PDA126dDGmHLyB0hSCQuTM8BDB14lNWfcSYevF1G1C0SIju71795oULDusnVmMechixJocpQVmUHgQYtf1r7/+aprkFCULITGWGN+CN2KKFKMk1kebNWtmRnlYE7548aIRYFe5afJx8KBRunTp+wpx0aJF71tD5YGFmYKpU6c6ZXTMpRA8YLGem9yz3mzosrp4zZ4929STk3PphxBCYuxQMFXImVmuyqOosEuYUVKyLpZPIbZs2YL8+fMnKMQvvPCCyQY8SCBoG8kIb/v27U7ptsXXmzX/lILz5ny/cTaZjmzxa/NCCImxS4sxjR1482MTEtOUFGKuJaQ4u0vakClTLkV48cUX7xLjzz77LNGdxRQzjtgw0nMWIWbjGpvvaGaS0g5oPPCwLs9RqKZNm5o0tqsfBIWQGLuhGFNcmRZk1yxnaDmGRKeldevWITAw0EQn7nbz43NCAWAzEVPNjIStQpwxY0Zj8JHYQwn/HZ9LNkFxBtnRhZgd8ax1syPekfzAOSq3Z88ek2ngwhB3yM4IITF2AzFmnZcdsnTDYuQ7dOhQEwUzLe1oG5GSE97guf6PWQH+SjFlVJY1a1Yjxt99950RhoeB0aV18QWbkhxViNlsxoiYYuyonuDMzixcuBCDBw82Y2e0VFWTlxASY6cTY97M6KTEmxmdsCjE/G82Gbk7PKCwaYk1SjZxWSMvOoVxZIvWl5yJfdSvvXz5clOHZSOco703mQXhY+R7wtHfC3xd+PrQjpSZHG71EkK4pRhbEBEWhKO7FqNt9dLI9UFtjNt45IE7Y1NCjBnhsPGFNUumXFu2bGkiCrofsWPW2UeRbCnEixYtMunkhDZrsWuc3eNJeb5Y/1y1apVZQ8gDkCNYZlJ4+Xh5AOHBzJmapJhG5+GIGQc21LGOr65rIdxGjC04eWADJvRsjOJFiiBvtpeQKlcFDFp5EBEOIsaMHNiExY7gOXPmmNQj98rSjpKiIu4WWh5UWCO2d1TIlCoFhLXPadOmmTWRKSXErLuyTMHDAY1anGVdZXxYVpk1a5b5fHHNo7MtHhFCYvxIVyhm9vwaRQt/hDYj/safP3/hMGLMqI2NR6z9MoXHbmBGe4oY7i/ErAkzMkzOmzgbpfj6cHkGD03MUiSnELN2ze/N1ZUcY3L2Zij+/DzYMLrn42K9/2Fr+0KIFBXjCPhsno5GlT9Gg8GLEWy58+/8ds5Bg88q4NtfF+F4OM0bgrHqn7FYuOEgToeFYtnQ71JMjBllMSrgKAqtF1u1amWiYN7c2ZzFWqeaW+4N54DposVaaUrYd/L12b17Nzp16mQEhONjzGjY08iDTVrMlnCH9MaNG12uAcq6gIK1ZFqashSgrmshnCQytoQcxazu3yBXviqYtO3EbVENObEd/euUR7ZcNbD4UHCCQp4SYsy6HsdkODLD6IppRt5gebMViTvEMDJlsxLHlFJ6hIfp4Q0bNhiBtK4V5DytLWrK1powx9bo/UyR4nuFI2uuDA+pfE75mWN2iEs7dDAVwsHFmHXgkCNr0LJSUbxbbwi2nwiBJTIEq8a2RYE8xdF41EqE3yOqTi4xZpqZNT7aUbITmhEwF7QzmlI67uGEmELHSHTJkiUOk77nz8VIndEq0+Z8fenRzJ+RpQceHh4mAub7Yu3atebAwcfKcSXWUxl5uxPsEudccv/+/U3TnMo1Qji0GPMKx/5FA1Eyx1toM2o1Tviswff5cuDdr/th27Hge6a47SnGvEEzgmEqky5O3A3MeVVGxYx4dGN5eJim5Y2ZoueoneQ8XFFEWPdkVMd69ldffYXOnTsboaZA86K48L1Bpyz+N4062BjGbVNc4MFFHnycdAFjNsUWkSG/hrOlffl88jPDrAMzA+yhUJQshMOK8RlEhvpheq9vkKvoB6hXsxxeK/U9pm0/cZ+RJduLMW8cjICZgma3bd26dTF69GiTapT4Pjp87rhrmaNd3KjkjD8/U6379u0zAsuLEd+ECRPM/DL/m4+Phw17dkVzppfvS2eEzwvXfXLJCbuvuYBC9WQhHFCMz5yJhO+O2fi6RDakeeJ/qDlkDQIi7t/8ZSsxZiMN/45RELuh+Stvrkxd6hSfNFg/5OGG86hM3wr3hiLMDAM7yTn37Ui2n0JIjHlZwuE5dwDKvfosHns8HSo0nQTvoEi7ijHrgawNMq3IxhqKBtNozjrz6YhwhtbqfKWDjbBGyey6ZgMkU/tslhNCOIgYBx3djFaflUCx73ph3uQeKJjzTbQetxYB4RabiTFrvt7e3qYRi//dt29fY4DPMRelzGyLdQVft27dTOZBiPjwM0cTljZt2piGSI4E6sB2NyyTMIPAMpruUxJj+4pxxCks/K0BCuSvipErvRAcfAgDvy6DfMW/x8xtx2wmxl9++aVZ2M65YM4H64NvH2jmMWPGDOOqJSEWD4INkWPGjDGfTaauHX30i+9te8CmRnbu84DC0TCWzDi9wbQ+6+wcj+P8Nv+Mv6enAQ1kOKeve5nE2AZiHI6j68bhvUKv47OOM+ATEokzlgh4rRyLSm/mQ/G2ExFiIzFmA1G9evWMEAvY7YbCmwWbm2SNKBILU9e+vr6mbGTdsOWoUARtGfXycbMxr3bt2mYUjs2BzCrNnDnT/DltU7dv324a4FhOY+c+DwSzZ882//ann34ypTaWhGS5KzF+RDG2IPTYZnSvXBp53m2FdcdDYbEaJoSdwvLfWyB35rIYtsYH4ZYziNjzD954+UVkyJABGTKkR7qnn0Sqx9Lg6XTPI735s9z4pus0nAyJTFCMWRNm3ZipU75xNSNsO3gyZ3MOG3NYBmDqX4iHxbqAok6dOmY2mwdnV5tkYFMjDxvM0PG+xIZRjsjxzzjNwc8R6+jMGNzPUIb/hl3+HMXjMhrakdIHn59Bjubx32kKRGKc+A7qPSvR/+duGLfywG0htgp18OF/MajLT+g5aSP8IiyIPLYBHX9sjEaNGt3jaoPh0zfidAJ1ZmsDF0WDo0qcd+W2Gb7xRdKFmGYenCPliV4IW4gyU7PMsnB8LCUsU+0R/XPunPcdOvcxNc8mUlvarlKcGblT4Gk6w/3TapCTGDv0PmOOLbFphKJ8+PBh1VySAE/zzDZwH7FGVYStYFTHLnymaps0aWJ+76wNTDR+Yb2XKXgKJFPT9lxQwiCDxjSsOzOFTYMaRckSY4cUY9yqb3JrDpcFME2k1OrD3yw5olKzZk1j5qEPu7AHTOty6qFr165mHIrRn7OIMn92OpDx/sNDBUX4fulne4gyJ0h69uxptsnx4Owu3LAcQr9atdF3WXL1H8QgfPc0fFy2A3acsd290C3E2Ao7ftn5+9dff5kUtqLkxKXc2PHJkz5vNkLYGx6WGVWyJsrUtaNvvWL2jXVvHvbp3paS9z+KMJu9eJ9jTd71R6RuwOI1Hx8VfRtt5ybT/enGBewY2xp5XqyFTZES40e2w+Q4BT2G2XFN20Fxf/hc0rDBFfbxCueC0aV1D/bOnTsd8mdkuYYLQhjJM73uCPdApsXZlc2SEpssHcYfPvoi/I9swYi2P6Bhw4Zo2LAjpqzahQBL1B1RZ+TBpejYoR1GrPPFna23sbjoswIN23bHDM8gROM6dsz6FbU+L4NM6dOjUIWv0LDFr1jsGYSrV89i5eRf0WLoDPidvYCAfWswtG9HNGz0A7oMmov9ARG4duO/7xm2Zzaa/dQZk7YF4Ho8sQ/cORMNO/XD0kPhuHr5JCa1rY1Kb72OZ9PkwOe1G6DVkMk4Gpn0sp3bibE12mOUx8UQHCGwlcG/q0UnjEp4I+QWIgmxSCmxY/MT06/MaNE5z1HKJLxvsHmKI0jsBne0eyEjYy6+YTd3Sj9n1y8EYG6f+iiWLyeyZs6EzJkzI3PmV5EtR16892U3rD8agRvmFhwNv5X9kD3rK6g8cgcuxRPGiA0DkSpTQbSZewhXcQXzelfGi+mfRZrUj+HZDC8jc57PMGzZUURdDsfI1h/gsVKNMH3WcFQuWAA5s2dF5syZkPHlV5Gv4veY6RmIm3f9azg+rwOefSU3Gk7djyvxvqfXjJ+QKkcZ/LbGF5fO78VPb7yM555+EqlTPYX/ZcqM/LU6YHtg0kfN3FKMrXBWj40PNCFQCvbO54XzjbzRaH+zcIigKjra7MXm4ZAp7JSebecBlRuqOGbkaPfAuBe7umlTy4N1ikXIsdHYPbs73syRA+Xb/g6vwPM3RfCaBes9eqHYa1nxdoNRCLh4/SHFmF/7MjwntEPhZzPfkaaOuRyOUW0qINUrBfD5N00xZ8dJXIqOReyNy9g+uSdKvZoRuWoMQ8T12IcSY/6EsedOoH+99/Fsmi+VprblCkVGgNxJyw85Z5IddeVfcnH58mVTq5s4caJpQBHCUWD2ipExR4bYscwyExunkhv6FjBKpzsWa7SOLMa8mO7niBVHoVIiwxUbE4lh376Bl8o0wvIjkYi9I3MdiHEtP8HzL1bGvMMW24txqlxo6bELcSXz+pljGFivHJ556nOsDbkmMXYUMcYt71z65TJCHjt2rFs6SvFGx8fN52Du3LkmBSeEI0IBZucw592HDh2arGUUfh963/PA6gxCbL04BcGfOSUcz25cOYTmOV/C+w1HwfdcvOU8Ny5j14T2KPxsYQxZ5297MX6xCmZ5xfOYiDmL9f0bIMdThTB2z1mJsSOJsRXWp1j/4Yec9RZ32urEHb0dOnQwUYe7ZweEc8A6KGds27VrZ1LYHIWyZ+8Hvza/H+8xPAA4ixBbnbzoD8CpiOT2kY8JWYNKqTKheutZCImKf2i6Bp+lg1HqtZxoNu2A7cW4VBNsOh3fSOYy9v3TBflfzIk+60Mkxo4oxtaTLwfne/XqZUYEmLZ1dfz9/U2jB+tKMvMQzvj+tR6iKZYPSl1zFOlRYJMWR4bY2e1MQhx3Fpkb7OiFnfxinBlftpmDkCt3i/HR5UPxbpbkFOMoHJjeDQVeel1i7MhibOXUqVMmBca0rSsLMscx2ODB50z7nYUzp67psMemKn5m6eOcUOqaWR+O/DzsqBQ/GxQxLkZxtqg47sVyHE1VWEdONjG+tBf1X86A8g1GwfdsvKxbbBT2/9MNRdPnx68rTz5AjK/h+Oy2DyfGWWph8bF4vS+xF7B1eFPkfvYNTNx//pYYd7yHGF/Fpj++lRinpBhb02DcoNK6dWvz4XWlhRO8UfGx8aTPm5gQrgBFk41dPGDS85qrCq2izF9pzpExY0YUKlQI27ZtS/TIDw/nzB7RLMhZhdh6sVGVEXJyBRmxMWEYUrMwsr7XBKuO3DmdEXM5BFPbV0PGDFWxyOeMmSU+t2cKXs+ZDZ/024Bzcc5TN674YXL9sg/ZwFUAnWcdQNxjWcz5k/ij0Yd47rkqWGcauG7AsvUPpMuUC3XH7MDFOJWOG1HHMODTAhLjlBZjKzS8GDRokEmFucLqMqsQ06+bj00z1sLV4NIEji1SdNhwxakJRsuffvopUqVKZa5KlSqZzFBiGD16tImMnV2IeTGtz6wfA4xk+ezHRmPH1I4okDEnPu3mgYCzV28FxRZsXzwKFQvkRfGGI2+NNsXiyrHFKJ/tVRT88GesOx6J6NhYXL8cjp3TRuGH8m/FE+ObkXWRFzKj0dgduBobT4xffB2fNuiK9YeDcSWGdf8r8Fr8Bz7K9xperTkMFjPcHIvL3tOR77mXUaLGb9gecA43YmNx9UIQNo0fjK/eznunGF84hRENPsTzj1fCfN9LsNUzmCQxLlq0KMqXL+9QV7Zs2WwuxnzDMjXFhRM8cTuz7yvTdHQgo6sWH4eEWLgqfK9z5SBT14xq+Z5/7rnnbotx2rRp8eWXXz4wQmRjGLNjjmjs8ajNXJzV5ox0co2GXQ71xpi2lZElew68U+Z9VKxYERXLl0b+HNmRv3hTzNkVgGjrrehaKKa3/xxPPvMi8hd7Fx9UqIj3y32AL77phlnjuuD5uGKMGITs/AfvF3oNGXO9hfIVv8avf2/DhfNhN8X47a8xaEBrVHyjBMpXqIiKFcrjzZyZ8L9c72PgikP/CenV0xhWuyTSPpcJhYuXRYWKFVG+bAVU+/5XTBnWFOniiLHpxh7yI7KlexEFSpRDxWqdsMgrNMmi/MhizPQOX0hHvOwlMIwouT+0Y8eOJivgbAsn+PMyumeULzMP4S7wc7tixQqkT5/+thBbr8cee+x2d/S9/l/WiWkr6QpCbL2YdmftmHPbyUMsbly14NDOVej5/ScoWbIkSpasg9ELNsA3PArxb9nXo4Kw4s9f8PX7JVGqTFm0GLMMJ8Iu4vqplaj86TcYtv7kf7aV189h/6LhqFC+LEqW/wZDF3rh0sX/Grg2+ofDd+sctPq+KkqWKo0azUZj6/FQXL1x5zeNOnsCCwZ3QNVyJVG6QiV09FiLAMtlhHtOw6fVGmPyrtOIvvVYrlmOYXqfRuZxfFC3Kzb4nkk5MXZXKPRsfvjjjz9Mo4iz7BBlBylFmK5ayT3aIERKR8jNmze/S4it1/PPP29GfhKaradwceSPDn2uJMa8mMbnFjtX5P7d1I6JxPgR4RJ0zuTyQ+zoe5JZ5+aCcx4gZOYh3I2NGzfi5ZdfvqcY82JTF13n4ndgs67KZRVsBHM1MWYKnzPaEmOJsdPDzk2ag3Tu3NmYsXMjlKPB9BvrZbTu0x5n4W6wzlu5cmWkTp36vmLM66mnnjLNXtYRP5bi6MhntYZ1NTHmY+K9izVxibHE2CXgG5vpX/rVshbjCFEyfwaaxNMCj847ctUS7gi7pRn9ValSBe+8845JSd9PkLNnz272d1OQebhmj4Wzmnwk5uKoF/tfJMYSY5eBHZmsvzAK5S7RlBZi1rg41rFq1SqtPxRuDV3lmGZmFotdxDw4N2nSBCVKlDDRcFwxTpMmDd577z3s27fPOFbxgM0eEVcVY85aDx482OVe89iYqzi2dz1mrtmN8KhHC0QYwCSnEZLE2JansZgYExl36tTp9pq35I6SKbysBXELFc3hHWX3qxAOcZOOjTU3WB6emabl4oRFixaZudsyZcoga9asePXVV1GzZk1zuG7atKlp0nRVMeahnV3VssG9G66R5Vw6R8D4PLEJ1p7iLDG200mcLyAjU5pqJBd8o3DGmnZ/Bw8e1AshxEPCUR+mpT08PMyBdsiQIXYRwchgX6xdMB1/zdkI/4iE6tFh2LFsGjzmrcWx0/az3+SCGDZ3OstUSHLCXgFrxoQNgHXr1jWrOxnkcCLF1hlHibEdBZmLF+hytW7dOrtHqEyp0CGoS5cuxrJPCJE0Nm/ebA7VdhHjwP0YWq88MuWsjNkHEjATCd2DFkWyoXiN37DTN9RuYszDBxfiUJTFvcU47lz6W2+9herVq5tyB/tybGWcIjG2Izw5sVORNRmOFbEGZS8hpikBF4hTiOWqJUTSoVMdD9J2EUJLCPbO6o18WbKizp+bEWKJ+/eRCPj3D+R+JT/a/PUvTkfaL03NbvNp06alyJ5jZxTjuNezzz6LLFmyoGrVqsYHnUs4OLHyqBGzxDiZouQ5c+aYFW+s59qys5npEtY2+LUdcbRKCGeFfR8s+9httMhvM9qULIAin/fDDv84qehIXyzsXBk5ijXBssMhdl+rOG7cOGMVSkHR9d/FfoEHjcPFvV588UW0aNHC3OsfZYeBxDiZYD2XtQY2ijCdbAvhpBAzxcQTnFy1hLAtCxYssKsYnzkTgEW/VMf/8n2M0WsOI+LWn4cfXYumpXPj3ZYzcCLCvg1c9CFgxzgtQenhreu/q3bt2g8lxrTGpIc5dxjQFEpi7OBwOQNT1mwO4QfhUVPKPHkxGqYYy1VLCGcUYwtO7f4HFTNmw9edZ+BESKRp3PJaNBDFchZBr2VHYDljfzFmJJfS45iOyP3S1I8//jiefvpp031Pq1XOa3MbFu/LSlM7EWzmYnMIu57Xrl37UM5YfKE5PkWfadazknMOTgh3wt5pal4RIT7o/93byFqxJdYeCoQlxAceP32J3B92wP4g+49Ucf6aNWNm7cSDxZhjb2XLlkWbNm2M6yLHnWyFxDgFYf24Z8+exuUnsdEt591Y3+EWGs0QC2E/6Fxn9Quw2xUZitVjWiNP9goYuHgfTnuvQdNKH6BG3wWIsNh/ztjawEVjE5GwGD/xxBNm9zUXhvD9wOymPYIgiXEKwhQ15/tYQ6ZRCMcM7pW25p9z/KBt27YmpSR7SyHsy44dO0wZyL6CaMHJ3Qvwdeli+KjHLOxdPBjl3voEw1Z62T1FzYsdwBQd/iruhFlLjooy8AkMDDQZTHtOqkiMHQRa9Vn3JMfvxOMpjLZ1NWrUMHNtQgj7Q8MeHpLtviQi+DDGN6uMjIXroVPLqihZsye2H0+eLVGMiGlsor6TlEdi7EBRMmeEOWbABi+rIw5rxKxNMDVN03shRPLAzyOtIpmWtK8ohuLA/IEo9lpOvJLvLTQZuQaBEZZkW6PI6E+9JxJjEQ+eULnGjZtmaGnJVAm7pjmUr4UPQiQfjIi5y5h9GvYWxUi/zehYJhcyFayOqdtPJUuKmhczcePHj9eLLTEWCUHR9fT0xFdffYVPPvlEvrFCpABXr141dphcGGFvUQz33Yj2ZQugTL2JOBSWfLuTaenI6FhIjEUCsGbMxhF+UFiz4lC+bC6FSF74eaOBAz9/dt3cFB6ADWPbIVemd9Bvhc9t8w97X0y/02WKG6yExFgkIMTc9kQxposLrTS5k7h79+7YtGmTniAhkhGmqCdMmGBGgOxiiRkZhj1LR+HLd/KhYOVe8A6OTLaomDPUbN4SEmMR7xTO0aXevXubtFjchgqmrdlaT5MQuuWkxJ5kIdwRpqoZPXKtom3FMAhb/voJ+bO8iAwvZES2MvXgselIsgkxI332ouzbt08vssRYxIVpaKakufThXiu5GCWzoWTYsGFm7EKCLIR9obEOI0h6FdtWEMPgvWYKOrRqhg7dRmHN3hMIt5xJNjGm4xbXu8rTXmIs4sDZYY4X8EPPk/j9oFBv2bIFP//8s0lfy4VLCNvDqQbazXKZPGf8uQAgObqqk+OiHzWNPmbPnq2RJomxsJ662TXNPcQ8qSZ2dIn/jmmmkSNHmhP7/Zy7hBAP/hxyixobmmh3yINuwYIFzUq8Vq1amd4NWmOyvmrXRq5kug4dOmQW1bAsJiTGbg8FlZEtP+A083gUMbWmrSnmFHXNIQuROPh5o8hSmGiyw8j3rbfeQurUqW8vBfjwww/NJh7Cfzt48GDTROnsYsx7Bv3wdb+QGEuIbwkxdxsz9ZWUqJYe1ZwT5I2CHzBb7EkWwlVhmefw4cPm88I5fu6gTWhFXqZMmcxmtbifWWvampkoZxViZuB69OjxSPt2hcTYpaBYcjEEbwZMedkivcyvwToQrTQZaXMtmk69QvwHV92NHj0aFSpUMGvwnnnmmXvuqk2bNi0GDhx4VyMlFwXwczt37lz7+1Xb4WKTaJ06dbQuUWIsOEPMxgmmxfjhsDWMkmlvx5GF9evXmzS2EAKma/j777+/pwDHFWKOMt3L9S4yMtKs0mMTpTMJMuvhvPewJq6mLYmxW8MPBOeE6eZj7xTRsWPHMGDAAPO97CH6QjgbzB4x7VyoUKH7ijEXx7Psc7+Mlbe3t/EDoHe8s4jxkiVLMGbMGHOAEBJjt4WnaVpb0lUrOazneCPh96RBCJu7eCpW2lq4O8wc8TP40ksvJSjETz75pBGtB40L8u+5wIUHXjZfOrII8z7A0SzuQWf5SlMXEmO3hTZ6PXv2NDN9vBkkNzzlc1SDrl5q7hLuDEWU3dOfffbZXUKcLl06E+0+zHghG7qsq00dUYjDwsKwfPlytG/fXstmJMbunRbjB5+nZ9aXUqpOw5/j1KlT8PDwwO+//25OykK4G/z8Mert1auXORiXKlXqDjGuXLny7TGmxEJBZm8GezTYFOVINWT+LGwU5c+mPegSY7cWYtaVmCKmq1ZKp4b4/dk8xuYu7kl+1LlmIZwRNjLyMMpRQo41UZi5MzxbtmxGiPPkyYMNGzY8UimH0TZnj2misXHjRocQZI5e0QyIPxMzcypRSYzd9gTOSJibl44ePepwP9/x48dNtD5t2jTNGgqXhmUhRqy0mmVUHDc7xbElRo2cJ+aaxKRCAaxXr575XPFzzwxUSqSl2aTGIICeA0Ji7NYf/hUrVjisEFth/Yhr4Th3SXFWlCxcDc4EU2RZmtmxY0eCTVkUL0bMD/KDTyxWDwEedjmLbK+1iwmlpBnxW+1xuWFK40sSY7eOiNmpyRnEh609pQS8AbEBhTU0nqaVyhKuAkWQs8Kcqw0NDb3vYdPWTZX8euwV4WG3evXqpqZMQx57pqRp9sPon81aFGYdriXGbgtPxLNmzTI1Gmea4+OHlh9mdnszvWYrRzAhUgLWhrnQgZmprVu3puhGM36OeCinYxc/X/PmzTP+8XTBetRlE7y3cDyJUTDtdHnYsAo+e0L02ZUYuzWsDVk7lZNjhtge8OfmHmU+Bi2cEM4GRYjRMKNR2sFyG5GjvIf5czA6Zwc3nfd4r+A4FP+bvSUsEwUGBt5z1SHFm7vLmcWiaQf9Cvgrd5ozo8VVj0Ji7PbQ73b8+PEmquTvnRmmrVlbY7qLXdeqOQlngU5Y1ll+R21K5IGBGTQeFBjVLlq0CFOnTjUH4DZt2qBSpUp3XVxkwWwbnfRYg166dCl27dplomNbptcZWbPXRUiMnfIkzg8ETQLochXfUN6ZHxdP5Dx5s7nLkSIMIeK/V3kAXrZsGWrVqmUsYFMyLf0oETMFlYdgijQPEfEvRr1MvfP+wsOxvdLQ/Dlc5R4mMXYj+CHi2kPWgjhD7Ew3gMTCDyYN5Tn6QSs9V3yMwrmF2No9zMyUUrVCYuyG0DCje/fuZsDflVO5PC17eXmZ2hRHRGSlKRwBHgxZK+3UqZNJrzprn4YQEuNHhMLL+T06WPFU7g7RojUVyOiD+11pq6m0tUip9yKbtFg+4RwvO5L1XhQSYzeDUSJP4ZxddGQzD3vBtDUzAexU5egIDRWESM6D8J49e0yTFjum9f4TEmM3hU1aHEdwZ7cqRiGMjAcNGmQiZe1FFckByyO0dhw+fLgZ8UmJzWdCSIxTGHYy0lCe4wfaBXqTs2fPYv78+ejTpw98fX01AiXsdvhjNz9T0mPHjpUhjZAY34t9+/YZowhHvDgsn1QY+TElxgYmW3nXuhKsm3NPMg34tXBC2Doa5hxujx49ZNUqJMYPggurW7dubUZ8HOmqWLGiGTlKCuzQ5Gn877//Vjr2PjBy4YGFKURbHICE4PuINo8sC3F2WAiJcSLEmE5NjrJM23rx53pUMWYajOmwbt26GTcfNYokLoqhVR+7zDkKpbS1eBQ4ncCeDL6PVq5cabr4lZYWEmM3FGN+8Llthakxiot4OBjFsLmLzTZ8HYRILPR3p91j165dFQ0LibG7izEN2bn+kM5TsoZ7tKwCU4xcI8mZZD6fqvWJB8EOfTZp8RCnUoeQGLuxGFNEaG/Ztm1b05Sm0YmkwWa3DRs2mDV2mzZt0vMpEoQHXn5OOTtMy1VOLgghMXZTMaZQUDB69eplbC6F7eAoGL2t2YwTFham+p+4ffjlSkE6adFEhilqIYQbizEbRjg+MXToULc287AnXHg+a9YsM6dNByUtnHBvePjdunWr2cnLGrF8pYVwczGmKFjNPDieIyG27w2Ynt6dO3c26+6Ee8LPHPd+Mwt14MAB9RMI4e5ibF18wNM5IzeRPPB5//PPP016Us1d7gNH3diTwblhzu3rMyeExPi2IFCMaesokhc2dy1dutQ0d23ZskVpaxeHTVkLFixA7969k2zEI4TE2EXEmE5abdq0waRJk7SbN4UjJaYpWaundanqhq4JD7ssA9HJ7sSJE8qECCExvmnbSEMK1ok1apPysEbPqIkzyUxfspP9fjdrRtD3+3t+PTbh2Rp+T6ZV2Q1OL27Wvrdv327WabL5j1EfL0b7/PMdO3Zg7969ppOcWRh3fK9xZGn37t3o0qULVqxYIUc2ISTGN2F9kjd8CbFjRskcLaNJyNq1axO0H6Vor1q1yjgz3avRjqL5xx9/2Ozn4rgNhXf69OnGjIL+2zSEYWaFTUi0SuV+Zy4x4LV8+XLz50y///DDD/jrr78wdepUeHh4mP3PgYGBbiFKjIb5uJn1OHjwoKJhISTG/5obN/efsl7l6emp+qSDYl2VN3LkSIwZM+auWj6j0Dx58hgxtGfzD38ORrQU1Y4dO2LcuHFmjzVFmVE3Z2MT8/6jkNNVihEyRXry5MlmnpYz1xQoVxRlftZ4WPrll1/M4YX+7kIIFxHjsMCj2LBoErrU/ghZXsmEjBlz4YOarTBq5nqcDAqH5T5iTAco3kQ5RsGbok7ojs+FCxdMyrdfv34mJcwshr+/P15//XWkSpUKTz31lOnGtfVryaYy7mVmYx/fO5x/ZTaFwsw+g6S+j2nxyMiYaWyKMt+TTOPy8brCSB0zF1zsQCetXbt2KfskhEuJccQpTOtZB3myvYb8pT/Gt7XroM63NfFpubfw8gsF8EP/ufAJirinGHPrUv/+/U3jiHAuOAbTvXt3E2E1bdoUTz75pBFjXoUKFTLZDltFc4xgGQlzBppR7OnTp+2atWHUzMMh35ujRo0youysPujWjAYjf/ZjBAcH680rhGuJcSRO/vsnCr2SCW9W6YR5O3wQwQglIgQn9q9Eq4oF8FLR6piy9USC0TF/rpw5c6J69erGZMLb29s00wjngZFqo0aNkC5duttCzCtNmjSoW7euaahKKnSCogizo5tp6OQspTDlzc1gXJDA7v7z58873WtEP2mWgJYsWaLZYSGcR4wj4LNpGup9VA41e81BYGScv7OE4/imqaj5XinU6LMIx8MD8e8fTZC7WFn0WrAP4ZY7v86W8S2QPn1RdJq0FSGWhMWYtT5GIExXs6GkZcuWaNCggYmCeBNhxMy6FsdqmFaTA5djwZRu3rx57xBi68V0NUXgUUeieDDje4LRKfcvM42cEr0NTIFbI3P+LIwyHb2cws8Jnz82sLGuzkOMuqWFcLbIOPQ45vWuizz5PsO4f48h3Py5BcHHtqDPN+8he/5vsfxIyAOj5h1T2iLDy8XRY9pOhCZytIk3EUYfbKCZN28efv75Z1PjYiqUdUreXFhjDggIMDcb1ZhTDh6SPv744wSF2Hply5btkaw1mYZm3Zazr0yrOkrTId+vjNJ5CHFUceNniPPhTEkzNa01o0I4qxifsSD02Aa0++xtlPiuP7YcD4ElMhjLR7ZA3twl0XzcGkQ86GuEH8XoRuXxWonamL3b755p6ge5/fDGQuMPNuowXcmbCwWZUQrtGtu1a2du9tbISSQPjHZ5UGI6+n5izKtEiRIPVadkapgHMGZNgoKCHGoCgFEyMzY9evQwh0JHg8LLzwNH0Jht0rpDIZxajG+mmb1X/oFyuYrgxyGLcfjAEtTKlRPvfz8Mu33vHxVHhp/CujHtkP/13Kg9YAlOhllsss/YKs684XCshjdqRgC0a2TkzHEN1p9pUMGZUqYWGT1zJpYpbkXRtoPP6cyZM1GpUiU8//zzePrpp5E6deoExZiCzWgyMW5qfE0pdDTnYOTtaON4VkFmaaV169YOM6PLzwW7yocNG2a83Hkw1ftdCJcQ4zOIDPPHrF/rIm/RcqhVtTSylv0Bs3aeROT9Gl78vTF3RDsUzp8fFdtOwtEwi032GT9MxMYuXzarUJwbN25sImgKB9PcFGk2HfGGqtGOpMNULQ89fH6rVKmCt99+GxkyZLhLkF966SXjM36/lCkFhDXZ+fPnm25mRxTiuNfRo0fNe5gRckoKH0e+mDZnNoHvcaWlhXAxMTbd0jvn4puS2ZHmyRfx7fD1dzZ0xZ/T9PPE8ObVUKBQYVRuNxK7TgTbZJ9xUqIFRnCcgeXNihEzDSL4K00rKNb8PWcuWYOWOCcNRr48CLGUwEa80qVL39FlXbBgQfM6JASNXijWfH0cLTV9vwiZWRnOW/P9kxJYm9yGDx9unntFw0K4ohhbwrBjRh+UyZwOadM+h/cajsG+wIgERTvw0Hr0rVURubKURLuRC3HQP+SeZh/JJcYJiTOjBn5v3jz37Nljvj9HZmiUz8iOKT4aI/DGxnobIz/e4NTF/XDPM0do2HVMe0zOIufOndvMIX/99dd31TH57ynSdL5yFiG2XkylM5LnLHJyvkf4vWhQ0qdPH2MFykyCEMJFxfj04fVo8uE7KFG/P5bN/A1v5CiEpn+sgF/YnSlEvwOr8PPnpfB6zgoYOM8TYTbaZ5wSsFGMto6MnGvUqGFEetasWUZU2EDGFDdvfEwNisTDQxDr++yQ5lKCuKLC8TWO3zDd7UxCHFeQ2blMb+vkEGROG/B78Tnj4g4hhCuLccRJzO1ZFwUK1sCYNd4ICT6CobXLIW/R2vhns0+crunjGNOyMgrl/xh9p29FaKTFJvuMHQFGxYzU6CTFMSs2x7BRjJ3cXHbAtCoFmhFgQosTRMLEnTvm7ydOnGiWgzhDnfheF98DXbt2NSNZ9oQlF74POfLF96YQwqXFOBw+q0ehZP4cqNxlDo6FRuKMJRLeayfis7fyokiLcQi51XF9fPVQZH8tJ6p0mwmf4Aib7DN2ZHFmjY433H379hlnpoULF5oUJd2mmI6lVSNT3IqcE8ehQ4fMBiWKmbMKsfXiyB0PFvaaP+ahsG3btiZLQ79sIYRLi7EFIUfWo8NHxVGgfCds84vjehQRYEaW8r1cHL8uOYjQiMPw+OYdpEmdCqlTP4bHHkvoyoyqHabANzjS6cU4IZiW5MW6Mmdp+Xi4lq9atWpmHSRX+rEGzfWDTMfSHpJRtGrQNz2T2T3N58zZhZgX0+wsb9g6dcz3Fa04WR/mDLYQwi3EOBKn9q/DiL79MGXdwXhNWBaEHN2Kkb/2wMBpW+Af7o8tkwajXdu25sSe8NUN4+ZuRVC4xSXF+H5w7pPRDGdmOXbCbUPsNGb0xDQ39wLzxs0oxx27YClezCbwgOIKYszuapYxGCHb4vXk12DmgAYeLI04oy+2EBLjRxbj5N9n7C5RIFPc7IBlipsCzTo0Vw3SEKNZs2bm9zSToKGJO0TO7Fx3lajYenFXMF3hkjrry/+fKyI5fufMG6OEkBhLjB0aii1naznbzLQ1048UYkZVNCrhSkJG0HzdduzYYaJI68IMV4iimTXgIYSP25XEmBcbrGiZ+SjwPcHnZujQoeawwtdcJQ0hJMYS4xSCqWumsNkkxiX3TGszBcpoiY5i/PPDhw+b59EZxZnvR6bsnbmD+l4XD1V8bz8snL/m88K0NJu0ZEIjhMRYYuxg0N2KjTwUYFoeLlq0yMw/cw76xx9/hIeHh4nGKG6MrhwZigyjftbMXU2IjRNdeLh5TVhDTizsvudzMnjwYLPXWwghMZYYOwEcn+ENnBG0tQ5NgeYmpTp16hgTCnZxM8XNXbYUaaa4bT12wwajh90MxJ+FUTGXLDz6+ycSJ1cPRsb8pdBu+q4EV3Qm2GQVGYaAk8fgtdfT2KDu2XcIvgHBiHiI+fjEXBRWOrwl5mDCDAg/C3Ty4uuptLQQEmOJsQtA0bU6idE3uUOHDsbzmfVomkWweYxCyHptUuah+X1y5sxpbCwfZnsRm5w4AkRL0qSIse/S3kj9cj786LHt1tz7A5yygk9gw8zf8X3VD5AtQ1qkSpUaz79cCJUbdMGkpbsRbrHde5vPMZ/r+0Hh5Zw6R5YccR2jEEJiLGwII1emuBnFsoPbOn7D2VWOFjHVTeGm8cbDiDNT5s8++yyeeuops7OYtWx2jD8IptM56pW0FYkPKcaRgdg8pTfK5M2MrOW/Q/eBw82o2cCuTVCmQC5kyV8Ns/cE2Oy9zfc1O6HvlYmgexZnrLnkgYcSRcNCSIwlxm4GBYId3FY3MS8vL9MQxs0/jRo1MiJCe0rOSFujV+vSjLiiweiPQmzdysQNTeXLlzf/L7/2vQSG70XWuJP2/oknxpHhCDx1AocOHsTBg0dwMiAIEZH/zccH7FuKuhXyIn2+bzBv3ymERdxMS0eEnsL6CZ1RIFMGfDZgDYJt9N5mpoB2qVyYERcedngY4XPM51xObUJIjCXGIkGhZvqas61MbXfp0sVEzqxn8lq2bNntFDej6jRp0ty1u5jXd999Z1zH4osRBZqWoRRy24hxXvwwbgU2zB6J2pWK4mnz/f+H975piynrDiHUcuvfei5FzxZ10HrS2rv2cYd5LULVItlRsPIYeEfa5r3NOj2jXo4pWWFjF2vlvXv3Nm5sioaFkBhLjEWixZmvFf22OW5DJ6g5c+YYUSlUqFCCQsyLVqh58+Y1m4WYzo4rxuwC54INm4jxC6/h/bo/4ofvG6Lbb4MwdOgg9GzzLfKky4A3a/6CjT7BD7R/Ddw5HR8UfBWl287FKRvVjVmz52pDZh7Y3c4sAzul+WeJSeULISTGEmNxT9ikxRQ3x2+KFy9+TzG2Xkxjc5cx/bpZJ6W4z5w50+wvtokYP/40cr7/A6Zv9Lq1McyC8KBDGFbnbaTJ+yGGrfZGxP26q8NOYvnwlsiVqRD6LPd54A7uxF5M71N4eYhh2r5bt26m412zw0JIjCXGwmZwbKdw4cIPFOO4F5u8pkyZYmqpthPjTKjcfAp8QuKOJkVg1fB6SPVaGfRdsBfh9/oalmBsmNobJQvkQqmmo3AywnbjTWyYY0MbMwMjR440FqdCCIkxatSogebNmzvU9fbbb0uMnRSKavr06R8owK+88grKlSuHmjVrGiOMAQMGGJtHm4lx+lxoMGI9Tt+RXr4lxumKobPHtlt143gRcYgv1s8YgNJv5kPRb3titZdtVzgyC8BZb3asq0lLCImxgSkzGhw44qW9rM4HU60cTXryySeRNm1ak4rOlSsX6tWrZxq+KLasK9Nhi+lsf39/07zErUOcTZ4xY4btxDjB0ab7ibEFYQEHMe2X71E0Rya812wE/j3khwiLbbM+HBPj4g/+KoSQGAthc9iQxG5gHqbo/hW/Y/p+sIGLXdkbN25MATG2IOjYDvz27fvIkfFtNB84HwEW+5Rg4jZwCSEkxkI4FBTj1atXG/OR5BbjiMADmPBTVeTNUh7tRy7GsdPhduuHOHr0qBltovWnEEJiLITDQTHu27dv8opxhD/W/tkeRbIURvPhq3AiJMKuzYkHDhwwlp8PkzUQQkiMhUg2OOLDmduk7TJ+GDGOxIkt0/FFsVfwTLEG+HvZcmM8Ev9at3EP/EJt01HNejldtoQQEmMhHBI2FHKrEevOySPGYdjxT3cUf+7+nd9ZCn6PJd6hNhFjLoCgy5gQQmIshENCByqagNBy89EFz4Kwk/swe+EybD8SGM/i0oJTB7di9ryV8DwSgEi6bB3dg1XzZ5sFGfe6Fi/fCt+QSJuIMZdQxHUeE0JIjIVwKNjERb9rCqDFYnE4M5qkXky/f/7558apTAghMRbCYWHdmIsokrZG0TEv1ovp3S2EkBgL4dDQ7OWnn34yphiuJMQ8XNSvXx++vr56kYWQGAvh+HCBApdGuFKqeufOnRg6dKgWQgghMRbCOeAMbosWLczuX1cQ4rCwMDNbvGHDBu0qFkJiLITzQP9mbnGiU5WzizHr4EOGDEFISIheWCEkxkI4D5w5HjZsmHGscmYh5mGC40wrV640O5+FEBJjIZwGChc9nBlRMs3rrGK8bds29OjRQ+NMQkiMhXBOaAJCe8xNmzY5pRAfO3bMrI7kpiYhhMRYCKeFQtayZUscOnTIqYSYKxJ5kKD9pZq2hJAYC+H0bN68Gd26dYOPj4/TdE9PmTLFNKBdunRJL6AQEmMhnJ+rV6+aCJNWmf7+/g4vxlwE0b9/f2N/KYSQGAvhMrABau7cuRg0aJAZEXJEEQ4PD8eaNWtMFB8cHKwXTQiJsRCuyaJFi9CvXz8cPHjQoRy6WCP+559/zM+meWIhJMZCuDRMWc+fPx9t2rQxjlaOIMT0m6bVJevESk0LITEWwi2gv/P27dvRvn1742FNAUwJEWZkvmvXLnTt2hVz5swxNp5CCImxEG4FxY/rFmmqwSg5KCgo2TYweXl5YeLEiWaOmPPEGl8SQkiMhVtHyfv37zfWmXTrYgOVPUXZ09PTHAAGDBhg6teKhoUQEmMhbkG3Ls4jcztSq1atsGDBAlPLtVWXNNPijMAHDhyIJUuWIDAwUF7TQgiJsRDxYao4Ojra1HK59YlrGDnvyxllWmpy6QRTypxV5ugRu57jXlxOQcevw4cPG0/p5cuXm1R0u3btMGLECNPBzQYypaSFEBJjIRIJU9h+fn4mncyIdvTo0fDw8DBCzYujSHEvLqbgn1OAOaLE33P94YULF2z6c+3ZswcnTpzQCySExFgI9yImJgaXL182VpWnTp2Ct7e3acKKex0/ftzMCzPlzQjbXjA6Z11bCCExFkIIIYTEWAghhJAYCyGEEEJiLIQQQkiMhRBCCCExFkIIISTGQgghhJAYCyGEEBJjIYQQQkiMhRBCCImxEEIIISTGQgghhMRYCCGEEI/A/wEnKokR702y2AAAAABJRU5ErkJggg=="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural Network Info\n",
    "\n",
    "#### All information regarding the neural network:\n",
    "\n",
    "- Input Layer Units = 2 (Can be modified)\n",
    "- Hidden Layer Units = 2 (Can be modified)\n",
    "- Output Layer Units = 1 (Since this is problem specific, it can't be modified)\n",
    "\n",
    "- No. of hidden layers = 1\n",
    "- Learning Algorithm = Backpropagation\n",
    "\n",
    "![nn_pic.png](attachment:nn_pic.png)\n",
    "\n",
    "Feel free to mess around with it and try out different things."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np # For matrix math\n",
    "import matplotlib.pyplot as plt # For plotting\n",
    "\n",
    "import sys # For printing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Neural Network Implementation\n",
    "Initially, I was going to approach this in an Object Oriented manner but I think that it would be much easier to read and implement, functionally. So, let's get started."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training Data\n",
    "\n",
    "The XOR logic gate returns true when the number of inputs given is odd and false when they're even. Here is the simple training dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The training data.\n",
    "X = np.array([\n",
    "    [0, 1],\n",
    "    [1, 0],\n",
    "    [1, 1],\n",
    "    [0, 0]\n",
    "])\n",
    "\n",
    "# The labels for the training data.\n",
    "y = np.array([\n",
    "    [1],\n",
    "    [1],\n",
    "    [0],\n",
    "    [0]\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1],\n",
       "       [1, 0],\n",
       "       [1, 1],\n",
       "       [0, 0]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1],\n",
       "       [1],\n",
       "       [0],\n",
       "       [0]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Additional Parameters\n",
    "These are just additional parameters which are required by the weights for their dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_i_units = 2 # Number of Input units\n",
    "num_h_units = 2 # Number of Hidden units\n",
    "num_o_units = 1 # Number of Output units"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Neural Network Parameters\n",
    "These are the parameters required directly by the NN. Comments should describe the variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The learning rate for Gradient Descent.\n",
    "learning_rate = 0.01\n",
    "\n",
    "# The parameter to help with overfitting.\n",
    "reg_param = 0\n",
    "\n",
    "# Maximum iterations for Gradient Descent.\n",
    "max_iter = 5000\n",
    "\n",
    "# Number of training examples\n",
    "m = 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Weights and Biases\n",
    "These are the numbers the NN needs to learn to make accurate predictions.\n",
    "\n",
    "For the connections being made from the input layer to the hidden layer, the weights and biases are arranged in the following order: **each row contains the weights for each hidden unit**. Then, the shape of these set of weights is: *(number of hidden units X number of input units)* and the shape of the biases for this connection will be: *(number of hidden units X 1)*.\n",
    "\n",
    "So, the overall shape of the weights and biases are:\n",
    "\n",
    "**Weights1(Connection from input to hidden layers)**: num_h_units X num_i_units\n",
    "**Biases1(Connection from input to hidden layers)**: num_h_units X 1\n",
    "\n",
    "**Weights2(Connection from hidden to output layers)**: num_o_units X num_h_units\n",
    "**Biases2(Connection from hidden to output layers)**: num_o_units X 1\n",
    "\n",
    "### Generating the Weights\n",
    "\n",
    "The weights here are going to be generated using a [Normal Distribution(Gaussian Distribution)](http://mathworld.wolfram.com/NormalDistribution.html). They will also be seeded so that the outcome always comes out the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(1)\n",
    "W1 = np.random.normal(0, 1, (num_h_units, num_i_units)) # 2x2\n",
    "W2 = np.random.normal(0, 1, (num_o_units, num_h_units)) # 1x2\n",
    "\n",
    "B1 = np.random.random((num_h_units, 1)) # 2x1\n",
    "B2 = np.random.random((num_o_units, 1)) # 1x1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.62434536, -0.61175641],\n",
       "       [-0.52817175, -1.07296862]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "W1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.86540763, -2.3015387 ]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "W2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.41919451],\n",
       "       [0.6852195 ]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.20445225]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B2"
   ]
  },
  {
   "attachments": {
    "sigmoid-curve.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAADVCAYAAAAmTCnuAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAABmJLR0QA/wD/AP+gvaeTAAAAB3RJTUUH4QgIFhkb7ca+nwAAFhdJREFUeNrt3Xt0VOW5BvDn3XtmQkK4aQKCrdXWChYvqFgFITMTLhaRXLBTa3tc1NZST3VhUVs41Sp6BBOt2uKyxXqtrWKJkgtaKkJmJkSUikgXokWPlypSICi3XJjL/t7zR8Z1KEesuhCy53t+a80iCZNhr4edJ++3Z88egIiIiIiIiIiIiIiIiIiIiIjI94QR0EGi3K+IiEVI5BMOIyAiFiAREQuQiMgOgYP9gDU1NSeLyEhGa6dZs2ahtrb2kn2/lkgkhq1du7bso+5/1VVX/Y6p5T/Pc5yudEEhAHTtDRSKuJJOS8ggEDQGSGeCRQCQNU7IqBsEgGwmUAQACjeo6plTv/o/t86ePXt9jy5AETkHQNxv/0EicpyqFgNY77dtV9VKEWnsQZu0at9P2tvbt4lIav87ZbPZMQsXLjzloosuWsDMD5mTRaRdVd/6pN+wa1dx4bZdfUvbU0VHeJmCI1KeFqvnFhvj9DYifdU4xapOb6PSG0ChZ9APil6AU6hqihXSGxD3X3/gdK+opkWcrEI7AEAc7YQiC6jnOtKeu2eXwGQUZneuW3p2AQI4dtasWRv9tlfU1NQcKSIhP257bW3tET1puz9iWzYCeGr/+w0ZMuSxbdu2DWbmh3S7hxpjds+ePXvjuIrlg4zrHAXBF9STwRDnaIgZDJUhUJRCMFiBUgGKAKQAtHXf5AOI7lZgj6jsFuh2VdkFYCfEdIg4nWrMboHZ48HtDAWcDtdxd3Z0etr61Ngdn3Xbx59ZW9Pjl8BkLf2Ij3k+4GE2NtZS6ma9oVBnKMR8NbE2HU5lAkdFqhKDPaAAiu1QbIHgXUC3isq7KrJeVTeLaJvjaZtb4G1ZXjdhVz7mwwKkg3YUgREcPmdMXxPsv6XzRCM6QgWnAToCwKnImP6AvAPoa6ryep/ee9f3d1N1W9uObCxIpTYtXXpeyubcWIBEPlRWmfyiC0TV0bFQOQ3b2k/yHHQB8jcAL0HkIaj5WypYsPG5utFd+yyBK1R1++x7v/UGU2QBEvnChIrWIRnXiwIagSIK6JcM8FdHkYRgnuu465Y/MfZNJsUCJMoL5VNbTjeexgCtykj2eKi8CNEEBFcggNZkXbSdKbEAifJGpCI+Uh2JAfimMeYIddDkwMxOe6HmZ5vG7GFCLECi/Jr0qlZ8xYM7XYBvAegn0CZxdEaJO+iZurrhaSbEAiTKv+KrTpZ70CuNwUSBPiFwLi8JlSxn6bEAifLSpEl/Ltgb6n2hcfRqT/UoEX0QLi5PLI5uYjosQKK8dG5s1RGpdOqqLpEfAfquqN5ZmOpaaPt5eCxAojw2Zcqaoj2BPTNSmfQsiLQ6Ri5obgq3MBkWIFEeU4lWt1yyW9v/W4zzFhwzJVEfbWUuLECivFZWkTzNkeRvVFHiiPlxvKG8kamwAIny2jkVrX2CkrkFotNEcJv201vjD5XvZTIsQKK8FqmMj4dk7wPkFePJKS1Lwm8xFRYgUf5PfW7mNqhcqIqrko2RB5kKC5Ao742raDnFc7KPw8gbcPXkJM/lYwES2eD1d78wPOuYlQK9M3Fa5CbMEcNUWIBEeS0W2xDalmm76+1/elFHzOR4fXkSDczFr/i2mESfUGRKvKQt07ZcVIafffKGh+L15UmmwgIkyntlFyS/CldaoXgvFQpO6FPYxWvxsQCJ8l+4OjHB8XQ1RB9KNEYu2vcS8+RvPAZI9DGi1cmpRvVBEbkkXh9ZzERYgER2lF9l8juq+hsXEmuuDy9jIlwCE1khUp283Ije5aic19zA8uMESGSJcFXiP6F6gxoZ39wUfomJsACJ7Jj8quL/AWCewIxvaSpn+bEAiawpvyqF3O0amdLcVP4iE8l/PAZIBKC8KjlRIb8XVV6xmQVIZI9odfOpBvongU5LNEaXMxEWIJEVJlS0DlF1lojghkRDtIGJ2IXHAMla51S09sk42acAeSJeH57PRDgBElkhFlvkBiW7CMDbiRFlVzMRToBE1mjLDqpR0YF9veKxvJYfC5DIGtHq5FRVvUQ9OXPJkpGdTIRLYCIrlFUnT1TV+0XNd/nGRcQCJGuMmbxygKPaBMjN8cbyp5kIsQDJEipuyHsIijWJhvDtzIMAHgMkS0SqWmZA8bXCdK8zmAaxAMkaZZXJkwG90XGc8qVLz97NRIhLYLLCxIlP93agi0RwffPisrVMhDgBkjUyRQW/VcFrifrwXUyDWIBkjWhV8wUGGBdA9hRAlIkQl8BkhXHVy49UOHcL9PIV9ePfZyLEAiRrGA3cJcAyXuGFuAQmq0Sq4+erYrwXdIYzDeIESNYYH3umHyALROSylXVlbUyEOAGSNTLZ4HwxWBlvDPNNzIkFSPaIVibPU9XzQk6QS1/iEpjsMWnS831VdIEKfrys/pxtTIRYgGSNroKuO1TwUrI+Usc0iEtgskZ5dbLcqFYGvOxJTIM4AZI1Jk58urdRfQDQmSuaxm9lIsQJkKyRKSq4FpC/Jxoif2QaxAIka0Qq4scrcIUY70ymQVwCk1XUkV9D5e54U/lGpkGcAMme6a8qXgXgFITMhUyDOAGSNUbFVhUCcgdUrknURduZCLEAyRoFmdQsAJsSjWWLmAZxCUzWGD+l9ZgMsjNVZQwvckqcAMkqWdebD+C+lsbweqZBnADJGuVVyYkGenYwmJnGNIgTIFkjFtsQMtD5Cvx0ed2EXUyEWIBkje3ptmsUeD/ZEOYrPohLYLJo6Xv+iqM9wSyBhvnEB3ECJKuYgDPXUfwp0RBdxzSIEyBZI1rdfKpRqfZcZxjTIE6AZBVV5zZHcPvKxWX/ZBrECZDsmf6mNk82BsNDHalqpkGcAMkasdgiV9WpheK6ZcvO7WAixAIka7RlS3+ogDcwtO1hpkFcApM1IrF4MdJygysyra7uWx4TIU6AZI80ZqtgbXNDeBnDIE6AZI3y81ccbURmiOhopkEsQLKKCThzFViYrI++zDSIBUjW+PCkZ+PwpGc6NHgMkHoMVfd2AL/kSc/ECZCsEq5KTILosIKOVCXTIE6AdLgJgFoAOwB8AOCW3NcOOMB9xO2TmaOOAPMEcj1PeiZOgNQTTAcwHsDpuc8fB/AGgPv+TWl++unvpeQ0CAp0h+FJz8QJkHqEaQCuA/BW7nYdgO8d9DHTKXAhmCOO+WkiEc0ydmIBUk8wHMAL+3y+Jve1j9MGoAPAegAzALj/7h8p/eK3T4Dg7fji8qcYOXEJTD1FMYB933tjJ4A+n2D5WwDgZAC/AnAcgJkH+gY3OCDUp6RsGIAI46bDQQ72A9bU1PxBRF7zYRbHiEiRqv7dh9s+AcAzB/MBr7322mtnzpx5Z0lJSScAbN26tff8+fOvnDt37rxP8v3vvPNOv3vuuefyuXPnzkskEkNXr159zv73GXTstKMHlHyls3Jcx+3M/BD9wIsMU9VOAO/4bdtV9YTZs2df3KMnQBH5p+M4d/hwhz7L87wS13V9txQzxvQ/2Jl7njd5wYIFf73++uuXAcC999470fO8dZ/031m6dOnRnudd7DjOHa+88sox6XT6lH3/vu+RI0sHfmHSrza9etPzjlN5BzM/NDzPm+y67nYAq32Y+Q09fiNra2tr/DgK19TUjK6tra3w47Z/TplfBuDF3DL2uNzHl+77C3m/+y8EcBKAEIATc9PR/AM9eLgq8cgZZbe+NmTIkMeY+SHd7oqamprRzPxzmgApb9wD4MsA1u7z+f0fc//FAB4BMBTAJgCPAbj5o+4YrWo+Q4Hztm96JMGY6XBiAdKBKICf5W4febRjv8/rcrdP8MBurajeltnbdgpjpsOJp8HQIRWd2jwZosOKTfGvmAaxAMkasdgiV41Towa/WLJkZCcTIRYgWaMtXXoJAOX7fFBPwWOAdEiMiq0qRCZ9A0R/yPf5IE6AZJVQJn0NBK8n6qN/YRrECZCsMTbWUioZc7WoGcc0iBMg2bWTZcwcAZriDeUvMg3iBEjWGFe98gRPvYvF8U5mGsQJkKziwatR4LfNi8f9g2kQJ0CyRmRq/Gw1CPcKhi5lGsQJkCyiAk/uhOLmp+tGf8A8iAVI1ohWtlwMQUlRuvM3TIO4BCZ7lr6xeLHJ6C2OY6YvXXpeiokQJ0CyZ/Gblf8SYAPf54M4AZJVyqYkjxPVGeKZs5kGcQIku3aogN6uwO/iS8o3MA3iBEjWKK9OlhvVMQL9PtMgToBkjVhsketB71TBtYmG6E4mQixAskZbdtBlojADA9seYBrEJTBZY8zklQOg3hwR801e6484AZJVgkHvRkBXxOvLk0yDOAGSNcZVtnzNg/keAL7DG3ECJJuoeKLzVXFnoiH6NvMgToBkjUhVYhogXy7oSlUyDWIBkj3lNyVeAsitUP3OsmXndjAR4hKYrCGuzBfBk4nG6HKmQZwAyZ7przr+DVVMQFZPZBrECZCsMWXKmiJVuVuBGYkl0e1MhFiAZI09bsdcUbyWbIgsZBrEJTBZIzw1cSaM/iAQcEcwDWIBkjUikXgABvcAev3yJ8a+yUSIS2CyxwBcAyBbGmy7i2EQJ0CyRtmU5HFQnW2MRHmxA+IESPaYo47j6gNQ+W1LU/glBkIsQLJG5KXEVQoMTIWCNzEN4hKYrBGd0jxcRa4TaOS5utFdTIQ4AZIVJk36c4EJOI8KMC/REF3HRIgFSNboLCiaJ4rdJcFttzMN4hKYrBGpjI8H8H1AT+OzvsQJkKwxrmL5IIg8DJEreJFTYgGSPeao4zmBPwrQkKgPP8JAiEtgskZ0XXKOEQxMBUIVTINYgGSNcEUiqsAMxzNnPVfPU16IBUiWiFTFjwXwJ4FMjzeVb2QiZAseA7TcqNiqQsB5HCr3xxvCi5gIsQDJEiqhTPohANtLQ1uvYx7EJTBZI1qd/IUqTgXM2Tzfj1iAZI1wdSJmFFcGxB21on7sTiZCLECyQnlFssyo3g/VqSsaxr7GRIgFSFYYV9nytayYegCXJ/mevmQ5PglikQkVrUM8MX8W1VuSDZE/MBFiAZIVxsZaSjNO9mkBmhKN0V8yESIWoDXl52bMChG8EB8R/gkTIWIB2lZ+a+Knhi/FHDFMhYgFmPcmVj870Mma5QBeZPkRsQCtUT51xZfSmmkR1RcSI8I/YPkR/X88DSYPRS6IDzOeLFNFfbIx8hM0iDIVIk6A+T/5VbV8HZ6shMrdycbIlQDLj4gFaMPkVx3/toFZroKfJxrDtUyEiEtgK36PRaric6ByBVSnJhv4Cg8iFqAFxkxeOaD1b3tiKtKuWTmzZUnkLaZCxCVw3iuvTI52g9461zWpgo7U6JYlYZYfEQsw36lEqpJXGtG/COSOUSdtaFq27NwO5kLEAsxr4y9Y+eVwVXK5QqcblXMSDeFfMxWiz4bHAH0iFlvktmUG/STreTcI9IE+Xp8pS5aM7GQyRCzAvBaemjizLYO7AS2EoxMTi6PPMxUiFmBem1DROiTjZObBYCpEaksDJbfV1Q1PMxkiFmDeGjN55YBA0MzIIHu1itS7Ge/E5ifHvcdkiFiAeWvs1JbBAc/MVPEuU+A5OBiXXBx5gckQsQDzVqQqfqyqzBRjfqCCuKiZkGgsX81kiFiA+Vx8IxTyUwDVjoPH4Jkz4k3lG5kMEQswL02sfnZgWrMXAToNwPFQPGggQ1sawu8yHSIWYN6JxTaEtqXbznUEF6c1UwGVNSKyQIPm0WRdtJ0JEeVJAa5fv34wABeA56cgXn755RLHcQYerMebNOn5vp2hvePE0XO3ZdpiImiD4mEDubql8aBOe04ucwHgq2v/ZTKZIp/+3EgucweAr660vWHDhoHG+PLi4G4u84P7H/k5bKgCOBLAB35Kd8iQIfcBOGrz5s3nf6YHmKNO+bqVIw28iYBMBDAKgo2qeBqCx5P1kec+p03vDaAdQC8AKT9lXlhY+CoAdHV1neizH8YCAHsBFAPo8Nl+/iSALZs3b77UZ5kfAeD9g91ZXAJ/RlOmrCna7bSfISJfh+jZWJeM5n6vLheRhySTvYjn7hFZtgTOR5HvxXthJ4aJOKcb1bMEOGsP2ocLsEWhf3UUqwXObc0jxq6x+M2HBEANgOm5VcA9AH7ut2U5sQDtDSN0ZChS/cgYGAyDYJhCTgQwTHfiWAF2K/QlUVmt0BtDGlj9TNOYzf/yAA1WxzcdwHgAp+c+fxzAGwDu455FLMDDaY4649auKPVct1RUB6m4X4DiWMB8EZBjABwDzR4PCbiqGCrdx+5eFUeXqCe/lAJ9NVEX3cLd5WNNA3AdgA8vynodgGtZgMQCPEgmTXq+b8rt6JMNBvoGVPoA2l+B/gD6q2IAch8D2l8FAwUYCKBU1yVLPSfgQLFTVbYodLMA7wD4h4isUmPe2fT3m3/U1fF27/c2vX0ed4vPZDiAfV+2tyb3NSI7CnBUbFXh660/wxGDzrou4Bal3GBhbwBw3IKQSCAEAG6wVzEAiASCIoGC7r8P9nbgiEiwCI7jOOL2UpGgiBMScQsdJ1AoEijuwl4ALsTLdmQ122lMtlNNtt2YdIcxmXbjpTqMl2r3sl3t2fTu91LpHbtSnVt3dux6c0f7rtd2pVI7sgfY9GC/fv36iMgAAJN89n/YK/fnuQAyh3E7+gAYBSC7z77V9+PyNMb0/vB3m88yD+b+nIjuZ4N9o7Ozs1RVAz7MvM/n8aAH9SnlCRWtQ7qymzdk0nuKPG+vAoB6XarGKKDIZju6v2bSakxaAcB4XQYAvGynAkY9b6+qZmC8tKpJq/G61Mt0Gs/r6P442/G5PMlgjAkAEMdxMj7bMeB5XqHrul2Hcxt27NgxuH///ltFup8EUlVn586dAwcMGLAlnU4XdnZ2ftQOLH379n3fcZwsMz80jDFBAOrHzPv377/6zTffnMq5lXqiVftNFZMAtOY+LgEw8gA3IiLfuwzAiwCOy91eBHApYyEiGwiAWwHsyN1q8Pm80oiIiIiIiIiIiIiIiIiIDrdTATyN7mvWvQfg+z7b/iT8dUWTbwBYAaALwGZ0vxb3yB62jQKgFt3PFn8A4Bb45xljP+Sbb/u0bzvkBHS/5va7AAYAOAbAoz4KfhqAZ322syxD90nIxeh+LfQCAE/2sG38Efx7zqAf8s2nfdrXHfIogMt9OrkOAPAPAEPh72va9QGwq4dt08e9aoT5cp/Olw7BFgA35ZYK7wP4Pbqv2OIHCwBck/vYzwVYCaClh23TLnS/PO5DpbnlMPPlPp1PHYIsgD/klgoDASwE8KAPtvus3DLB8XkBngbgXQAjeth2efi/K6og93GW+XKf9kOH6L+57f+bvnSfzwcCaPPBtq8CMGy/7/NL5h+K5H44y3rgTp0PE2BPzvdAhx160j79afaVntQhn8rK3AZ/aJBPNv7TFE1PdCG6ny37eg/+YfTzMcCenm8+7dN+7RAA3U9X/3G/8fUBHy51/FR+V6H7QHdPfqtJP185xg/55tM+7fsOuTHX2B8AeBg+OoDp0wI80G/64h60jX6+cowf8s23fTofOoSIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiOiz+F+uAYT6iQDobQAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAxNy0wOC0wOFQyMjoyNToyNyswMDowMN3n2dUAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMTctMDgtMDhUMjI6MjU6MjcrMDA6MDCsumFpAAAAAElFTkSuQmCC"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid Function\n",
    "[This](http://mathworld.wolfram.com/SigmoidFunction.html) function maps any input to a value between 0 and 1.\n",
    "\n",
    "![sigmoid-curve.png](attachment:sigmoid-curve.png)\n",
    "\n",
    "In my implementation, I have added a boolean which if set to true, will return [Sigmoid Prime(the derivative of the sigmoid function)](http://www.ai.mit.edu/courses/6.892/lecture8-html/sld015.htm) of the input value. This will be used in backpropagation later on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(z, derv=False):\n",
    "    if derv: return z * (1 - z)\n",
    "    return 1 / (1 + np.exp(-z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forward Propagation\n",
    "[This](https://en.wikipedia.org/wiki/Feedforward_neural_network) is how predictions are made. Propagating the input through the NN to get the output.\n",
    "\n",
    "In my implementation, the forward function only accepts a feature vector as row vector which is then converted to a column vector. Also, the predict boolean, if set to true, only returns the output. Otherwise, it returns a tuple of the outputs of all the layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def forward(x, predict=False):\n",
    "    a1 = x.reshape(x.shape[0], 1) # Getting the training example as a column vector.\n",
    "\n",
    "    z2 = W1.dot(a1) + B1 # 2x2 * 2x1 + 2x1 = 2x1\n",
    "    a2 = sigmoid(z2) # 2x1\n",
    "\n",
    "    z3 = W2.dot(a2) + B2 # 1x2 * 2x1 + 1x1 = 1x1\n",
    "    a3 = sigmoid(z3)\n",
    "\n",
    "    if predict: return a3\n",
    "    return (a1, a2, a3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradients for the Weights and Biases\n",
    "These variables will contain the gradients for the weights and biases which will be used by gradient descent to update the weights and biases.\n",
    "\n",
    "Also, creating the vector which will be storing the cost values for each gradient descent iteration to help visualize the cost as the weights and biases are updated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "dW1 = 0 # Gradient for W1\n",
    "dW2 = 0 # Gradient for W2\n",
    "\n",
    "dB1 = 0 # Gradient for B1\n",
    "dB2 = 0 # Gradient for B2\n",
    "\n",
    "cost = np.zeros((max_iter, 1)) # Column vector to record the cost of the NN after each Gradient Descent iteration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training\n",
    "This is the training function which contains the meat of NN. This contains forward propagation and [Backpropagation](http://neuralnetworksanddeeplearning.com/chap2.html).\n",
    "\n",
    "### Backpropagation\n",
    "The process of propagating the error in the output layer, backwards through the NN to calculate the error in each layer. Intuition: It's like forward propagation, but backwards.\n",
    "\n",
    "Steps(for this NN):\n",
    "1. Calculate the error in the output layer(dz2).\n",
    "2. Calculate the error in the weights connecting the hidden layer to the output layer using dz2 (dW2).\n",
    "3. Calculate the error in the hidden layer(dz1).\n",
    "4. Calculate the error in the weights connecting the input layer to the hidden layer using dz1 (dW1).\n",
    "5. The errors in the biases are just the errors in the respective layers.\n",
    "\n",
    "Afterwards, the gradients(errors) of the weights and biases are used to update the corresponding weights and biases by multiplying them with the negative of the learning rate and scaling it by divinding it by the number of training examples.\n",
    "\n",
    "While iterating over all the training examples, the cost is also being calculated simultaneously for each example. Then, a regurlization parameter is added, although for such a small dataset, regularization is unnecessary since to perform well, the NN will have to over fit to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(_W1, _W2, _B1, _B2): # The arguments are to bypass UnboundLocalError error\n",
    "    for i in range(max_iter):\n",
    "        c = 0\n",
    "        \n",
    "        dW1 = 0\n",
    "        dW2 = 0\n",
    "\n",
    "        dB1 = 0\n",
    "        dB2 = 0\n",
    "        \n",
    "        for j in range(m):\n",
    "            sys.stdout.write(\"\\rIteration: {} and {}\".format(i + 1, j + 1))\n",
    "\n",
    "            # Forward Prop.\n",
    "            a0 = X[j].reshape(X[j].shape[0], 1) # 2x1\n",
    "\n",
    "            z1 = _W1.dot(a0) + _B1 # 2x2 * 2x1 + 2x1 = 2x1\n",
    "            a1 = sigmoid(z1) # 2x1\n",
    "\n",
    "            z2 = _W2.dot(a1) + _B2 # 1x2 * 2x1 + 1x1 = 1x1\n",
    "            a2 = sigmoid(z2) # 1x1\n",
    "\n",
    "            # Back prop.\n",
    "            dz2 = a2 - y[j] # 1x1\n",
    "            dW2 += dz2 * a1.T # 1x1 .* 1x2 = 1x2\n",
    "\n",
    "            dz1 = np.multiply((_W2.T * dz2), sigmoid(a1, derv=True)) # (2x1 * 1x1) .* 2x1 = 2x1\n",
    "            dW1 += dz1.dot(a0.T) # 2x1 * 1x2 = 2x2\n",
    "\n",
    "            dB1 += dz1 # 2x1\n",
    "            dB2 += dz2 # 1x1\n",
    "\n",
    "            c = c + (-(y[j] * np.log(a2)) - ((1 - y[j]) * np.log(1 - a2)))\n",
    "            sys.stdout.flush() # Updating the text.\n",
    "        \n",
    "        _W1 = _W1 - learning_rate * (dW1 / m) + ( (reg_param / m) * _W1)\n",
    "        _W2 = _W2 - learning_rate * (dW2 / m) + ( (reg_param / m) * _W2)\n",
    "\n",
    "        _B1 = _B1 - learning_rate * (dB1 / m)\n",
    "        _B2 = _B2 - learning_rate * (dB2 / m)\n",
    "        cost[i] = (c / m) + ( \n",
    "            (reg_param / (2 * m)) * \n",
    "            (\n",
    "                np.sum(np.power(_W1, 2)) + \n",
    "                np.sum(np.power(_W2, 2))\n",
    "            )\n",
    "        )\n",
    "    return (_W1, _W2, _B1, _B2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running\n",
    "Now, let's try out the NN. Here, I have called the train() function. You can make any changes you like and then run all the kernels again. I have also plotted the cost function to visual how the NN performed.\n",
    "\n",
    "The console printing might be off.\n",
    "\n",
    "The weights and biases are then shown."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 5000 and 4"
     ]
    }
   ],
   "source": [
    "W1, W2, B1, B2 = train(W1, W2, B1, B2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.32260643, -0.42332921],\n",
       "       [-1.4336158 , -1.67239068]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "W1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.25299514, -2.21317287]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "W2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.37348207],\n",
       "       [-0.22080277]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.24523225]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting\n",
    "Now, let's plot a simple plot showing the cost function with respect to the number of iterations of gradient descent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4lFX6//H3nUboNShNCF2KIIaitIANbLjqIuiqa0FdK7YVt6/72+82RdS1sahrQ2SxgA0ElF4D0hEITUIRpEsxEO7fH/PgzsbABMhkUj6v68rFPGfOzNwnV8gnTznnMXdHRETkeOJiXYCIiBR9CgsREYlIYSEiIhEpLEREJCKFhYiIRKSwEBGRiBQWIiISkcJCREQiUliIiEhECbEuoKDUqFHDGzRoEOsyRESKlXnz5n3r7imR+pWYsGjQoAEZGRmxLkNEpFgxs/X56afDUCIiEpHCQkREIlJYiIhIRAoLERGJSGEhIiIRKSxERCQihYWIiERU6sNi1/5snp6wisVZu2NdiohIkVViJuWdrLg446kJK4mPg9Z1K8e6HBGRIqnU71lUSk4ktUZ5Fm/UnoWIyLGU+rAAaFWnMks27ol1GSIiRZbCAmhVuxIbdx1gx77sWJciIlIkKSyA1nVC5yqW6FCUiEieFBZAyyAsdN5CRCRvCgugctlE6lcvp8tnRUSOQWERaHdGVTLW78DdY12KiEiRo7AIdEitxrffZbP2232xLkVEpMhRWATaN6gGwNx1O2JciYhI0aOwCDRKKU/18knMXquwEBHJTWERMDPaN6jGHIWFiMiPKCzCnNuoOlk7D+i8hYhILgqLMD2a1QRg0oqtMa5ERKRoUViEOaN6ORqmlOfzrxQWIiLhohoWZtbLzFaYWaaZDcrj+afMbEHwtdLMdgXtbc1sppktNbNFZnZtNOsM17NZTWav2cH+7MOF9ZEiIkVe1MLCzOKB54DeQAugv5m1CO/j7g+4e1t3bws8C7wXPLUfuNHdWwK9gCFmViVatYbr2bwm2TlHmLJyW2F8nIhIsRDNPYsOQKa7r3H3bGAE0Oc4/fsDbwO4+0p3XxU83gRsBVKiWOsPOqRWo0aFJMYs3FQYHyciUixEMyzqABvCtrOCth8xs/pAKvB5Hs91AJKA1Xk8d7uZZZhZxrZtBbMnkBAfx2Vn1WbC8q3sPXioQN5TRKS4i2ZYWB5tx1p4qR8wyt1z/ucNzGoBbwA3u/uRH72Z+1B3T3P3tJSUgtvxuLxNbbIPH2Hc0m8K7D1FRIqzaIZFFlAvbLsucKxjO/0IDkEdZWaVgI+B37j7rKhUeAztzqhC/erlGJmxIXJnEZFSIJphMRdoYmapZpZEKBDG5O5kZs2AqsDMsLYk4H3gdXf/TxRrzJOZcX3HM5izdgdfbdHtVkVEohYW7n4YuAcYBywHRrr7UjN73MyuCOvaHxjh/7s2eF+gG/DzsEtr20ar1rz0TatHmYQ4Xp+5vjA/VkSkSLKScv+GtLQ0z8jIKND3fHTUIkYv3MjUX/YkpWKZAn1vEZGiwMzmuXtapH6awX0cd6Y34lCO89LkH12IJSJSqigsjiO1RnmubFuHN2atZ+ueg7EuR0QkZhQWEdx3fmOOuPO3sStiXYqISMwoLCKoX708t3VtyLvzs3QXPREptRQW+XBvz8bUrpzMY+8t5kB2TuQXiIiUMAqLfCiXlMBfrz6LzK3f8edPlsW6HBGRQqewyKduTVMY0DWVN2d9zYdaZFBEShmFxQl45OLmpNWvykP/Wci89TtjXY6ISKFRWJyApIQ4ht6YRq3KyQx4PYOV3+yNdUkiIoVCYXGCqpVP4t83dyAhzug3dBbLNmntKBEp+RQWJyG1RnneueNcyiTE0W/oTKat+jbWJYmIRJXC4iSl1ijPyDvOpVblstz06hxem7GOkrLOlohIbgqLU1CvWjlG/eJc0pum8PsxS7nrrfns2p8d67JERAqcwuIUVUxOZOiNaQzq3ZwJy7/h4iFT+GLF1liXJSJSoBQWBSA+zrizeyPev6szFcokcPOrc7nrrXls2a3FB0WkZFBYFKBWdSrzyf1defiipkxcvpXzn5zE85MyOXhIS4SISPGmsChgZRLiuadnE8Y/0J1zG9Xg72NXkP6PSbwz92sO5xyJdXkiIidFYRElZ1Qvx7Cb0hh5x7nUrpLMo+8uptfTU/lw4SZyjuiqKREpXhQWUdYhtRrv/uI8XrrhHAy49+0vufCpybw7L0t7GiJSbOge3IXoyBFn7NItPPt5Jss37+GMauW4K70RP2lXhzIJ8bEuT0RKoSJxD24z62VmK8ws08wG5fH8U2a2IPhaaWa7wp4ba2a7zOyjaNZYmOLijEta1+KT+7ow7MY0qpZLZNB7i+n6ty94flImu/cfinWJIiJ5itqehZnFAyuBC4EsYC7Q393zvCGEmd0LnO3utwTb5wPlgDvc/bJIn1cc9ixyc3emZX7L0ClrmLrqW8olxXNt+3rc0jmVetXKxbo8ESkF8rtnkRDFGjoAme6+JihoBNAHONbdg/oDvz+64e4TzSw9ivXFnJnRtUkKXZuksGzTHoZNW8MbM9fz2ox1XNK6Frd3a8hZdavEukwRkagehqoDbAjbzgrafsTM6gOpwOcn8gFmdruZZZhZxrZt20660KKgRe1KDO7blqmP9mBAt4ZMXrGNK/45nb4vzeSzpVt0BZWIxFQ0w8LyaDvWb7x+wCh3P6HZa+4+1N3T3D0tJSXlhAssimpVLstjvc9kxmM9+c2lZ7Jx5wFuf2MePZ6YxMvT1rL3oM5riEjhi2ZYZAH1wrbrAse6H2k/4O0o1lLsVExO5LauDZn8SDovXN+O0yqV4U8fLaPT/03kD2OWsu7bfbEuUURKkWies5gLNDGzVGAjoUC4LncnM2sGVAVmRrGWYishPo7erWvRu3UtFmft5tXpa3lr9npem7mO85vX5ObOqZzXqDpmee3IiYgUjKjOszCzS4AhQDzwirv/2cweBzLcfUzQ5w9AsrsPyvXaqUBzoAKwHbjV3ccd67OK49VQJ2vrnoO8Oftr3pq1nu37sml2WkVu7tyAK8+uQ3Ki5muISP7l92ooTcorxg4eyuHDhZt4Zfo6lm/eQ9VyiVzX8Qxu6NSA0ysnx7o8ESkGFBaliLsze+0OXpm2lvHLvyHeQpP/bumSStt6uvRWRI6tKMyzkEJiZnRqWJ1ODavz9fb9vDZzHSPnbmDMwk2cfUYVbumcSq9Wp5MYr6XAROTkaM+ihPru+8OMytjAv2esY932/ZxeKZkbz6vPdR3OoEq5pFiXJyJFhA5DCRBavPCLFVt5Zfpapmdup2xiPFefU4ebO6fSKKVCrMsTkRhTWMiPLN+8h1emrWX0gk1k5xyhZ/Oa3NpFl96KlGYKCzmmbXu/581Z63kzuPS2+ekVuaVLKle0qa1Lb0VKGYWFRHTwUA5jFmzilelr+WrLXmpUSOL6jvX5Waf6pFQsE+vyRKQQKCwk39ydGau38/K0tXz+1VaS4uPo07Y2t3ZNpfnplWJdnohEkS6dlXwzMzo3rkHnxjVYve07Xp2+llHzsvjPvCw6N67OrV1SSW9ak7g4ndcQKa20ZyF52rU/m+Fzvub1GevZsucgDVPKc3PnVK5uV4dySfobQ6Sk0GEoKRCHco7wyeLNvDxtLYuydlO5bGhJkZvO1ZIiIiWBwkIKlLuTsX4nL09dy2fLthAfZ1zRpg63d2tIs9Mrxro8ETlJOmchBcrMaN+gGu0bVGPDjv28PG0t78zdwLvzs+jRLIU7ujeiY2o1zdcQKaG0ZyEnbee+bN6YFbpn+PZ92bSpW5k7ujfi4panE6+T4SLFgg5DSaE5eCiHUfOyGDZ1Deu276d+9XLc1rUhPz2nrib5iRRxCgspdDlHnM+WbuHFKWtYuGEX1concdO5Dbjh3PpUK6/FC0WKIoWFxIy7M2ftDoZOWcPEr7aSnBjHtWn1uK1rQ+pVKxfr8kQkjE5wS8yYGR0bVqdjw+qs/GYvQ6esYficr3lj1noub1Obu9Ib6woqkWJGexZSKLbsPsgr09fy5qz17M/O4aIWp3F3j8a00Z38RGIqv3sWUb11mpn1MrMVZpZpZoPyeP4pM1sQfK00s11hz91kZquCr5uiWadE3+mVk/nVJWcy/dGe3Hd+E2at2U6f56Zzw8uzmbVmOyXljxaRkipqexZmFg+sBC4EsoC5QH93X3aM/vcCZ7v7LWZWDcgA0gAH5gHnuPvOY32e9iyKl70HD/HW7K8ZNnUN336XTVr9qtzdozHpzVI0V0OkEBWFPYsOQKa7r3H3bGAE0Oc4/fsDbwePLwbGu/uOICDGA72iWKsUsorJidzZvRHTHu3JH69oyaZdB7j533O57NlpfLJ4M0eOaE9DpCiJZljUATaEbWcFbT9iZvWBVODzE32tFG/JifHcdF4DJj3Sg79fcxb7s3O46635XPjUZN6dl8XhnCOxLlFEiG5Y5HUs4Vh/LvYDRrl7zom81sxuN7MMM8vYtm3bSZYpRUFSQhx90+ox4cHuPNv/bBLj43joPwu5YLBCQ6QoiGZYZAH1wrbrApuO0bcf/z0Ele/XuvtQd09z97SUlJRTLFeKgvg44/I2tfn0/q4MveEcyiUl8NB/FnLhU1N4b75CQyRWonmCO4HQCe7zgY2ETnBf5+5Lc/VrBowDUj0oJjjBPQ9oF3SbT+gE945jfZ5OcJdM7s5ny75hyIRVLN+8h4Y1ynPv+Y25/KzaJMRH9WI+kVIh5ie43f0wcA+hIFgOjHT3pWb2uJldEda1PzDCw1IrCIU/EQqYucDjxwsKKbnMjItbns7H93bhxZ+dQ1JCHA+8s5CLnprC+19mkaMT4SKFQpPypFg5cuTonsZKvtqyl4Yp5bmvZxMub1NbK92KnISY71mIRENcnNGr1el8cl9XXvxZO5Li4xj4zgIuemoyHy/SJbci0aKwkGIpFBq1+OS+rrxwfTvi44y7h8/n8n9O44sVWzUjXKSAKSykWIuLM3q3rsWn93fjqWvbsOfgIW5+dS7XvjSLuet0mkukoOichZQo2YeP8E7GBp6duIqte78nvVkKD1/UjFZ1Kse6NJEiSfezkFLtQHYOr81cxwuTVrP7wCEuPasWD17YlEYpFWJdmkiRorAQAfYcPMSwKWsYNm0t3x8+wjXt6nLfBU2oU6VsrEsTKRIUFiJhvv3ue57/YjVvzloPBjef14C70htTuVxirEsTiSmFhUgeNu46wODPVvLel1lUSk7k3p6NueHc+pRJiI91aSIxoXkWInmoU6UsT/Ztw8f3dqVNvSr8v4+Xc/6Tkxm9YKPmaIgcR77CwszeyE+bSHHRonYlXr+lA2/c2oGKyYncP2IBVz4/nZmrt8e6NJEiKb97Fi3DN4K74J1T8OWIFK6uTVL4+N4uDO7bhm/3fk//f83iln/PZeU3e2NdmkiRctywMLPHzGwvcJaZ7Qm+9gJbgdGFUqFIlMXFGVe1q8vnD6czqHdz5q7bQa8hUxj07iK+2XMw1uWJFAn5OsFtZn9x98cKoZ6TphPcUlB27svm2c8zeWPWOhLi4rgrvREDujUkOVEnwaXkKegT3B+ZWfngjX9mZoODW6GKlDhVyyfxu8tbMOHB7nRvmsKT41dy/pOT+XDhJq05JaVWfsPiBWC/mbUBfgmsB16PWlUiRUD96uV58YZzeHtAJyqVTeTet7/kmhdnsnDDrliXJlLo8hsWh4ObE/UBnnb3p4GK0StLpOg4t1F1Prq3C3+9qjXrt++jz3PTeXDkAp3PkFIlv2Gx18weA24APg6uhtLUVyk14uOMfh3O4IuH07mzeyM+WriZ9H9M4tmJqzh4KCfW5YlEXX7D4lrge+AWd98C1AH+EbWqRIqoismJDOrdnAkPdie9Weh8Rs8nJjFG5zOkhMv3ch9mdhrQPtic4+5bo1bVSdDVUBILM1dv508fLWPZ5j2k1a/KH/u0pGVtLYcuxUeBXg1lZn2BOcBPgb7AbDO75tRKFCn+zm1UnQ+D8xlrvt3H5c9O47cfLGHX/uxYlyZSoPJ7GOrXQHt3v8ndbwQ6AL+N9CIz62VmK8ws08wGHaNPXzNbZmZLzWx4WPvfzGxJ8HVtPusUKXQ/nM94KJ0bz23AW7PX0+OJSbw952tytN6UlBD5DYu4XIedtkd6bXAS/DmgN9AC6G9mLXL1aQI8BnR295bAwKD9UqAd0BboCDxiZpXyWatITFQul8gfrmjJR/d2pUnNijz23mJ+8vx0vvx6Z6xLEzll+Q2LsWY2zsx+bmY/Bz4GPonwmg5ApruvcfdsYAShS2/DDQCec/edAGGB1AKY7O6H3X0fsBDolc9aRWKqRe1KvHNHJ57u15Ytuw/yk+dn8MtRC/n2u+9jXZrISYu0d9DYzDq7+yPAS8BZQBtgJjA0wnvXATaEbWcFbeGaAk3NbLqZzTKzo4GwEOhtZuXMrAbQA6iXrxGJFAFmRp+2dfj84XTu6NaQ9+ZvpMcTk/j39LUczjkS6/JETlikPYshwF4Ad3/P3R909wcI7VUMifBay6Mt9wHcBKAJkA70B4aZWRV3/yz4jBnA24TC6fCPPsDsdjPLMLOMbdu2RShHpPBVKJPAY5ecydiB3WhTtwp/+HAZlz07jdlrtBS6FC+RwqKBuy/K3ejuGUCDCK/N4n/3BuoCm/LoM9rdD7n7WmAFofDA3f/s7m3d/UJCwbMqjzqGunuau6elpKREKEckdhrXrMAbt3bgxZ+1Y+/Bw1w7dBYPvLOAbXt1aEqKh0hhkXyc5yLd8X4u0MTMUs0sCegHjMnV5wNCh5gIDjc1BdaYWbyZVQ/azyJ0+OuzCJ8nUqSZGb1a1WLCg925p0djPlq0ifOfnMQbs9brqikp8iKFxVwzG5C70cxuBeYd74Xufhi4BxgHLAdGuvtSM3vczK4Iuo0DtpvZMuAL4BF3305oKZGpQftQ4GfB+4kUe2WT4nn44mZ8en83WtauzG8/WMJVL8xgycbdsS5N5JiOO4M7mLX9PpDNf8MhDUgCfhIs/VEkaAa3FEfuzugFm/h/Hy9jx75sbjy3AQ9d1JSKyVp6TQpHfmdw5/fmRz2AVsHmUnf//BTrK3AKCynOdh84xBPjVvDm7PWkVCjDby9rwWVn1cIsr+tERApOgYZFcaCwkJJg4YZd/OaDJSzeuJuuTWrweJ9WpNYoH+uypAQr6DvliUghaFOvCh/c3Zk/XtGSBV/v4uIhUxgyYaWWQZeYU1iIFDHxccZN5zVg4kPd6dXydIZMWEWvIVOYukpziSR2FBYiRVTNSsk80/9s3ry1I2bGDS/P4b63v9TcDIkJhYVIEdelSQ0+vb8rAy9owtglW7hg8GRGzt2gmy1JoVJYiBQDyYnxDLygKZ/c35Vmp1fkl+8uov+/ZrF623exLk1KCYWFSDHSuGYFRgzoxN+ubs2yTXvoPWQqz0xcRfZhLU4o0aWwEClm4uKMa9ufwYSHunNxq9MZPH4llzwzlbnrdsS6NCnBFBYixVTNisk82/9sXr25PQeyc/jpizP51fuL2X3gUKxLkxJIYSFSzPVoVpPxD3ZjQNdURsz5mgsGT+bjRZt1AlwKlMJCpAQol5TAry9twZh7unBapTLcPXw+t72WwcZdB2JdmpQQCguREqRVncp8cFdnfnPpmcxYvZ0LB0/m5WlrtQS6nDKFhUgJkxAfx21dGzL+wW50TK3Gnz5axpXPTdcS6HJKFBYiJVTdquV45eft+ed1Z7N590H6PDedv3y6nAPZWmdKTpzCQqQEMzMuO6s2Ex/szk/PqctLk9fQ6+kpzFj9baxLk2JGYSFSClQul8hfrz6L4bd1BOC6f81m0LuL2L1fl9lK/igsREqR8xrXYNzAbtzRvSH/mZfFBU9N5tPFm2NdlhQDCguRUiY5MZ7Hep/J6Ls7k1KhDL94az53vJHBN3sOxro0KcIUFiKlVKs6lRl9T2ce7dWcSSu2ccHgybw952tN5pM8RTUszKyXma0ws0wzG3SMPn3NbJmZLTWz4WHtfw/alpvZM6abEYsUuMT4OH6R3oixA7vRsnYlHntvMf3/NYt13+6LdWlSxEQtLMwsHngO6A20APqbWYtcfZoAjwGd3b0lMDBoPw/oDJwFtALaA92jVatIaZdaozzDb+vEX65qzdJNe7h4yBRemLSawzlazVZCorln0QHIdPc17p4NjAD65OozAHjO3XcCuPvWoN2BZCAJKAMkAt9EsVaRUi8uzujf4QwmPNid9GYp/G3sV/TRZD4JRDMs6gAbwrazgrZwTYGmZjbdzGaZWS8Ad58JfAFsDr7GufvyKNYqIoHTKiXz0g1pvHB9O7bu/V6T+QSIbljkdY4h95mzBKAJkA70B4aZWRUzawycCdQlFDA9zazbjz7A7HYzyzCzjG3bdDN7kYLUu3UtJjzQnWvaaTKfRDcssoB6Ydt1gU159Bnt7ofcfS2wglB4/ASY5e7fuft3wKdAp9wf4O5D3T3N3dNSUlKiMgiR0qxyuUT+dk0ek/l0z4xSJ5phMRdoYmapZpYE9APG5OrzAdADwMxqEDostQb4GuhuZglmlkjo5LYOQ4nEyHmNazD2/rDJfIMnM3aJJvOVJlELC3c/DNwDjCP0i36kuy81s8fN7Iqg2zhgu5ktI3SO4hF33w6MAlYDi4GFwEJ3/zBatYpIZGWT/ncy351vzufON+axVZP5SgUrKRNw0tLSPCMjI9ZliJQKh3KOMGzqWoZMWElSQhy/vuRMrm1fD02HKn7MbJ67p0XqpxncInLCwifztahViUGazFfiKSxE5KSl1ijP2wOCyXwbNZmvJFNYiMgp+WEy30Pd6d5Uk/lKKoWFiBSI0GS+c3jh+nZ8s0eT+UoahYWIFBgzo3frWkx8sDtXt6vDS5PXcPGQKUxbpcl8xZ3CQkQKXOVyifz9mjYMH9CROIOfvTybh0YuZOe+7FiXJidJYSEiUXNeoxqMHdiNu9IbMXrBRi4YPJnRCzbqnhnFkMJCRKIqOTGeX/Zqzof3dqFu1bLcP2IBN/97Llk798e6NDkBCgsRKRRn1qrEe3d15neXtWDO2h1c9NQUXp62lpwj2ssoDhQWIlJo4uOMW7qk8tkD3eiYWo0/fbSMq56fzvLNe2JdmkSgsBCRQle3ajle+Xl7nu7XlqydB7j82Wn8fexXHDyky2yLKoWFiMSEmdGnbR0mPNidK8+uw/OTVtNriO6ZUVQpLEQkpqqWT+KJn7bhrds64oTumfHoqEXs3q97ZhQlCgsRKRI6B/fMuLN7I0bNz+L8wZP5aNEmXWZbRCgsRKTIKJsUz6DezRlzT2dqV0nmnuFfcttrGWzadSDWpZV6CgsRKXJa1q7Me784j99ceiYzVm/nwsGTeW3GOl1mG0MKCxEpkhLi47ita0M+e6Ab5zSoxu/HLOWaF2ewYsveWJdWKiksRKRIq1etHK/d3J4h17Zl/fb9XPbsVAZ/tkKX2RYyhYWIFHlmxpVnhy6zvbxNbZ75PJNLnpnKzNXbY11aqaGwEJFio1r5JAb3bcsbt3Yg54jT/1+zeGjkQnZoNduoi2pYmFkvM1thZplmNugYffqa2TIzW2pmw4O2Hma2IOzroJldGc1aRaT46NokhXEDu3FPj8aMWbiRnk9OYuTcDbrMNoosWt9cM4sHVgIXAlnAXKC/uy8L69MEGAn0dPedZlbT3bfmep9qQCZQ192PuUxlWlqaZ2RkRGEkIlKUrfpmL79+fwlz1u2gQ2o1/u8nrWhcs2Ksyyo2zGyeu6dF6hfNPYsOQKa7r3H3bGAE0CdXnwHAc+6+EyB3UASuAT49XlCISOnV5LSKjLi9E3+/+ixWfrOX3k9P5YlxOgFe0KIZFnWADWHbWUFbuKZAUzObbmazzKxXHu/TD3g7SjWKSAkQF2f0bV+PicEJ8H9+kcnFQ6YwZeW2WJdWYkQzLCyPttzHvBKAJkA60B8YZmZVfngDs1pAa2Bcnh9gdruZZZhZxrZt+qEQKe2qVyjD4L5tGX5bR+LNuPGVOdz39pds3Xsw1qUVe9EMiyygXth2XWBTHn1Gu/shd18LrCAUHkf1Bd539zxXFHP3oe6e5u5pKSkpBVi6iBRn5zWuwSf3d2XgBU0Yu2QL5z85mbdmr+eIZoCftGiGxVygiZmlmlkSocNJY3L1+QDoAWBmNQgdlloT9nx/dAhKRE5CcmI8Ay9oyqcDu9KqdmV+/f4SrnlxBl9t0Y2WTkbUwsLdDwP3EDqEtBwY6e5LzexxM7si6DYO2G5my4AvgEfcfTuAmTUgtGcyOVo1ikjJ1yilAsMHdOTJn7Zh3fb9XPrMNP7yyXL2Zx+OdWnFStQunS1sunRWRCLZuS+bv3y6nJEZWdSpUpY/XdmSns1Pi3VZMVUULp0VESlSqpZP4u/XtOGd2ztRNimeW/6dwZ1vzNMS6PmgsBCRUqdjw+p8cl9XHrm4GZNWbuX8JyfzwqTVZB8+EuvSiiyFhYiUSkkJcdzdozHjH+hO58Y1+NvYr7Q44XEoLESkVKtXrRzDbkpj2I1pHDyUQ/9/zeL+EV+ydY/mZoRTWIiIABe0OI3xD3Tn3p6N+XRxaG7Gq9PXcjhHh6ZAYSEi8oOySfE8dFEzxg7sStszqvDHD5dxxT+nM2/9zliXFnMKCxGRXBqmVOD1Wzrw3HXt2LEvm6tfmMGjoxaV6vtmKCxERPJgZlx6Vi0mPNSd27s1ZNT8LHo+OYnhs78ulcuGKCxERI6jQpkEfnXJmXxyX1ea1qzIr95fzFUvzGDJxt2xLq1QKSxERPKh2ekVeeeOTgzu24asnfu54p/T+N3oJew+kOc6pyWOwkJEJJ/MjKva1WXiQ+nc0Kk+b85az/lPTuI/GRtK/KEphYWIyAmqXDaRP/ZpxZh7ulCvWjkeGbWIq1+cwaKsXbEuLWoUFiIiJ6lVncq8e+d5PPHTNmzYcYA+z03nsfdK5lVTCgsRkVMQF2dcc05dPn+4O7d0TmVkRhbp//iC12euK1ET+hQWIiIFoFJyIr+9rAVj7+9K67qV+d3opVz27DTmrN0R69IKhMJCRKQANTmtIm+J1hA3AAAKd0lEQVTe2pHnr2/HngOH6PvSTO4f8SVbdhfvtaYUFiIiBczMuKR1LSY+lM59PRvz6ZIt9HxyEi9OLr7LoCssRESipGxSPA9e1IwJD3TnvEY1+OunX9FryBQmrdga69JOmMJCRCTKzqgeWgb91Zvb48DPX53LgNcz2LBjf6xLyzeFhYhIIenRrCZjB3bl0V7NmZ75LecPnszg8Ss5kJ0T69IiimpYmFkvM1thZplmNugYffqa2TIzW2pmw8PazzCzz8xsefB8g2jWKiJSGMokxPOL9EZ8/lA6vVqezjMTV3HB4Ml8tGgT7kV3FrhFqzgziwdWAhcCWcBcoL+7Lwvr0wQYCfR0951mVtPdtwbPTQL+7O7jzawCcMTdj7nPlpaW5hkZGVEZi4hItMxes50/friMZZv30KFBNX53eQta1alcaJ9vZvPcPS1Sv2juWXQAMt19jbtnAyOAPrn6DACec/edAGFB0QJIcPfxQft3xwsKEZHiqmPD6nx4bxf+clVrMrd9x+X/nMagdxexbe/3sS7tf0QzLOoAG8K2s4K2cE2BpmY23cxmmVmvsPZdZvaemX1pZv8I9lREREqc+Dijf4cz+OLhdG7tnMqoeVn0fGISQ6cUnUttoxkWlkdb7mNeCUATIB3oDwwzsypBe1fgYaA90BD4+Y8+wOx2M8sws4xt27YVXOUiIjFQuWwiv7msBeMe6Eb71Gr83ydfcfGQKUxc/k3Mz2dEMyyygHph23WBTXn0Ge3uh9x9LbCCUHhkAV8Gh7AOAx8A7XJ/gLsPdfc0d09LSUmJyiBERApbo5QKvPLz9rx6c3vM4NbXMrjp1blkbt0bs5qiGRZzgSZmlmpmSUA/YEyuPh8APQDMrAahw09rgtdWNbOjCdATWIaISCnSo1lNxg3sxm8va8GXX+/k4iFT+cOYpezeX/g3XIpaWAR7BPcA44DlwEh3X2pmj5vZFUG3ccB2M1sGfAE84u7b3T2H0CGoiWa2mNAhrX9Fq1YRkaIqMT6OW7ukMunhdPq1r8frM9eR/sQXvDGzcFe1jdqls4VNl86KSGmwbNMeHv9oKbPW7KDZaRX5/eUtOK9xjZN+v6Jw6ayIiBSwFrUr8faATrxwfTv2ZR/mumGzufut+VE/AZ4Q1XcXEZECZ2b0bl2LHs1r8vK0tRzIzsEsrwtQC47CQkSkmEpOjOfuHo0L5bN0GEpERCJSWIiISEQKCxERiUhhISIiESksREQkIoWFiIhEpLAQEZGIFBYiIhJRiVkbysy2AetP4S1qAN8WUDnFRWkbc2kbL2jMpcWpjLm+u0e8x0OJCYtTZWYZ+VlMqyQpbWMubeMFjbm0KIwx6zCUiIhEpLAQEZGIFBb/NTTWBcRAaRtzaRsvaMylRdTHrHMWIiISkfYsREQkolIfFmbWy8xWmFmmmQ2KdT2nwsxeMbOtZrYkrK2amY03s1XBv1WDdjOzZ4JxLzKzdmGvuSnov8rMborFWPLLzOqZ2RdmttzMlprZ/UF7iR23mSWb2RwzWxiM+Y9Be6qZzQ7qf8fMkoL2MsF2ZvB8g7D3eixoX2FmF8dmRPljZvFm9qWZfRRsl/TxrjOzxWa2wMwygrbY/Vy7e6n9AuKB1UBDIAlYCLSIdV2nMJ5uQDtgSVjb34FBweNBwN+Cx5cAnwIGdAJmB+3VgDXBv1WDx1VjPbbjjLkW0C54XBFYCbQoyeMOaq8QPE4EZgdjGQn0C9pfBH4RPL4LeDF43A94J3jcIviZLwOkBv8X4mM9vuOM+0FgOPBRsF3Sx7sOqJGrLWY/16V9z6IDkOnua9w9GxgB9IlxTSfN3acAO3I19wFeCx6/BlwZ1v66h8wCqphZLeBiYLy773D3ncB4oFf0qz857r7Z3ecHj/cCy4E6lOBxB7V/F2wmBl8O9ARGBe25x3z0ezEKON9C9+DsA4xw9+/dfS2QSej/RJFjZnWBS4FhwbZRgsd7HDH7uS7tYVEH2BC2nRW0lSSnuftmCP1iBWoG7ccae7H9ngSHG84m9Jd2iR53cEhmAbCV0C+A1cAudz8cdAmv/4exBc/vBqpTvMY8BPglcCTYrk7JHi+E/gD4zMzmmdntQVvMfq5L+z2487rDeWm5POxYYy+W3xMzqwC8Cwx09z127JvXl4hxu3sO0NbMqgDvA2fm1S34t1iP2cwuA7a6+zwzSz/anEfXEjHeMJ3dfZOZ1QTGm9lXx+kb9TGX9j2LLKBe2HZdYFOMaomWb4LdUYJ/twbtxxp7sfuemFkioaB4y93fC5pL/LgB3H0XMInQceoqZnb0D8Dw+n8YW/B8ZUKHK4vLmDsDV5jZOkKHinsS2tMoqeMFwN03Bf9uJfQHQQdi+HNd2sNiLtAkuKoiidDJsDExrqmgjQGOXgFxEzA6rP3G4CqKTsDuYLd2HHCRmVUNrrS4KGgrkoJj0S8Dy919cNhTJXbcZpYS7FFgZmWBCwidq/kCuCbolnvMR78X1wCfe+js5xigX3D1UCrQBJhTOKPIP3d/zN3runsDQv9HP3f36ymh4wUws/JmVvHoY0I/j0uI5c91rM/4x/qL0FUEKwkd8/11rOs5xbG8DWwGDhH6i+JWQsdqJwKrgn+rBX0NeC4Y92IgLex9biF08i8TuDnW44ow5i6EdqsXAQuCr0tK8riBs4AvgzEvAX4XtDck9MsvE/gPUCZoTw62M4PnG4a916+D78UKoHesx5aPsafz36uhSux4g7EtDL6WHv3dFMufa83gFhGRiEr7YSgREckHhYWIiESksBARkYgUFiIiEpHCQkREIlJYiATM7Lvg3wZmdl0Bv/evcm3PKMj3F4k2hYXIjzUATigszCw+Qpf/CQt3P+8EaxKJKYWFyI/9Fega3EfggWDRvn+Y2dzgXgF3AJhZuoXupTGc0EQozOyDYOG3pUcXfzOzvwJlg/d7K2g7uhdjwXsvCe5dcG3Ye08ys1Fm9pWZvRXMVsfM/mpmy4Janij0746USqV9IUGRvAwCHnb3ywCCX/q73b29mZUBppvZZ0HfDkArDy15DXCLu+8IluGYa2bvuvsgM7vH3dvm8VlXAW2BNkCN4DVTgufOBloSWstnOtDZzJYBPwGau7sfXfZDJNq0ZyES2UWE1t1ZQGj58+qE1hUCmBMWFAD3mdlCYBahBdyacHxdgLfdPcfdvwEmA+3D3jvL3Y8QWsakAbAHOAgMM7OrgP2nPDqRfFBYiERmwL3u3jb4SnX3o3sW+37oFFo++wLgXHdvQ2j9puR8vPexfB/2OAdI8ND9GToQWmX3SmDsCY1E5CQpLER+bC+hW7QeNQ74RbAUOmbWNFgJNLfKwE53329mzQktG37UoaOvz2UKcG1wXiSF0K1xj7kSanDfjsru/gkwkNAhLJGo0zkLkR9bBBwODif9G3ia0CGg+cFJ5m3893aW4cYCd5rZIkKrms4Ke24osMjM5ntoee2j3gfOJbS6qAO/dPctQdjkpSIw2sySCe2VPHByQxQ5MVp1VkREItJhKBERiUhhISIiESksREQkIoWFiIhEpLAQEZGIFBYiIhKRwkJERCJSWIiISET/Hxo17bwdPSrgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x6e74710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Assigning the axes to the different elements.\n",
    "plt.plot(range(max_iter), cost)\n",
    "\n",
    "# Labelling the x axis as the iterations axis.\n",
    "plt.xlabel(\"Iterations\")\n",
    "\n",
    "# Labelling the y axis as the cost axis.\n",
    "plt.ylabel(\"Cost\")\n",
    "\n",
    "# Showing the plot.\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Observation\n",
    "With the initial parameters, the cost function doesn't look that good. It is decreasing which is a good sign but it isn't flattening out. I have tried, multiple different values but this some seems like the best fit.\n",
    "\n",
    "Try out your own values, run the notebook again and see what you get."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}