connectx-with-q-learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "# About Q-Learning algorithm\n",
    "> \"Q-learning is a model-free reinforcement learning algorithm. The goal of Q-learning is to learn a policy, which tells an agent what action to take under what circumstances. It does not require a model (hence the connotation \"model-free\") of the environment, and it can handle problems with stochastic transitions and rewards, without requiring adaptations.\"\n",
    "[*wiki*](https://en.wikipedia.org/wiki/Q-learning)"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"ToC\"></a>\n",
    "# Table of Contents\n",
    "1. [Install libraries](#install_libraries)\n",
    "1. [Import libraries](#import_libraries)\n",
    "1. [Define useful classes](#define_useful_classes)\n",
    "1. [Create ConnectX environment](#create_connectx_environment)\n",
    "1. [Configure hyper-parameters](#configure_hyper_parameters)\n",
    "1. [Train the agent](#train_the_agent)\n",
    "1. [Create an agent](#create_an_agent)\n",
    "1. [Evaluate the agent](#evaluate_the_agent)"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"install_libraries\"></a>\n",
    "# Install libraries\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install 'kaggle-environments==0.1.6' > /dev/null 2>&1"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"import_libraries\"></a>\n",
    "# Import libraries\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import gym\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "from random import choice\n",
    "from tqdm.notebook import tqdm\n",
    "from kaggle_environments import evaluate, make"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"define_useful_classes\"></a>\n",
    "# Define useful classes\n",
    "NOTE: It's not easy to generate a Q-Table with all possible states; and even if I can do so, the huge number of states will cost much of memory. So, I use the approach that dynamically adding newly discovered states into an object of QTable class created below.\n",
    "\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
   },
   "outputs": [],
   "source": [
    "class ConnectX(gym.Env):\n",
    "    def __init__(self, switch_prob=0.5):\n",
    "        self.env = make('connectx', debug=True)\n",
    "        self.pair = [None, 'negamax']\n",
    "        self.trainer = self.env.train(self.pair)\n",
    "        self.switch_prob = switch_prob\n",
    "        \n",
    "        # Define required gym fields (examples):\n",
    "        config = self.env.configuration\n",
    "        self.action_space = gym.spaces.Discrete(config.columns)\n",
    "        self.observation_space = gym.spaces.Discrete(config.columns * config.rows)\n",
    "\n",
    "    def switch_trainer(self):\n",
    "        self.pair = self.pair[::-1]\n",
    "        self.trainer = self.env.train(self.pair)\n",
    "\n",
    "    def step(self, action):\n",
    "        return self.trainer.step(action)\n",
    "    \n",
    "    def reset(self):\n",
    "        if random.uniform(0, 1) < self.switch_prob:\n",
    "            self.switch_trainer()\n",
    "        return self.trainer.reset()\n",
    "    \n",
    "    def render(self, **kwargs):\n",
    "        return self.env.render(**kwargs)\n",
    "\n",
    "\n",
    "class QTable:\n",
    "    def __init__(self, action_space):\n",
    "        self.table = dict()\n",
    "        self.action_space = action_space\n",
    "        \n",
    "    def add_item(self, state_key):\n",
    "        self.table[state_key] = list(np.zeros(self.action_space.n))\n",
    "        \n",
    "    def __call__(self, state):\n",
    "        board = state.board[:] # Get a copy\n",
    "        board.append(state.mark)\n",
    "        state_key = np.array(board).astype(str)\n",
    "        state_key = hex(int(''.join(state_key), 3))[2:]\n",
    "        if state_key not in self.table.keys():\n",
    "            self.add_item(state_key)\n",
    "        \n",
    "        return self.table[state_key]"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"create_connectx_environment\"></a>\n",
    "# Create ConnectX environment\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "env = ConnectX()"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"configure_hyper_parameters\"></a>\n",
    "# Configure hyper-parameters\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha = 0.1\n",
    "gamma = 0.6\n",
    "epsilon = 0.99\n",
    "min_epsilon = 0.1\n",
    "\n",
    "episodes = 10000\n",
    "\n",
    "alpha_decay_step = 1000\n",
    "alpha_decay_rate = 0.9\n",
    "epsilon_decay_rate = 0.9999"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"train_the_agent\"></a>\n",
    "# Train the agent\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0f0f043967bb40b3a5d848c028977045",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=10000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "q_table = QTable(env.action_space)\n",
    "\n",
    "all_epochs = []\n",
    "all_total_rewards = []\n",
    "all_avg_rewards = [] # Last 100 steps\n",
    "all_qtable_rows = []\n",
    "all_epsilons = []\n",
    "\n",
    "for i in tqdm(range(episodes)):\n",
    "    state = env.reset()\n",
    "\n",
    "    epsilon = max(min_epsilon, epsilon * epsilon_decay_rate)\n",
    "    epochs, total_rewards = 0, 0\n",
    "    done = False\n",
    "    \n",
    "    while not done:\n",
    "        if random.uniform(0, 1) < epsilon:\n",
    "            action = choice([c for c in range(env.action_space.n) if state.board[c] == 0])\n",
    "        else:\n",
    "            row = q_table(state)[:]\n",
    "            selected_items = []\n",
    "            for j in range(env.action_space.n):\n",
    "                if state.board[j] == 0:\n",
    "                    selected_items.append(row[j])\n",
    "                else:\n",
    "                    selected_items.append(-1e7)\n",
    "            action = int(np.argmax(selected_items))\n",
    "\n",
    "        next_state, reward, done, info = env.step(action)\n",
    "\n",
    "        # Apply new rules\n",
    "        if done:\n",
    "            if reward == 1: # Won\n",
    "                reward = 20\n",
    "            elif reward == 0: # Lost\n",
    "                reward = -20\n",
    "            else: # Draw\n",
    "                reward = 10\n",
    "        else:\n",
    "            reward = -0.05 # Try to prevent the agent from taking a long move\n",
    "\n",
    "        old_value = q_table(state)[action]\n",
    "        next_max = np.max(q_table(next_state))\n",
    "        \n",
    "        # Update Q-value\n",
    "        new_value = (1 - alpha) * old_value + alpha * (reward + gamma * next_max)\n",
    "        q_table(state)[action] = new_value\n",
    "\n",
    "        state = next_state\n",
    "        epochs += 1\n",
    "        total_rewards += reward\n",
    "\n",
    "\n",
    "    all_epochs.append(epochs)\n",
    "    all_total_rewards.append(total_rewards)\n",
    "    avg_rewards = np.mean(all_total_rewards[max(0, i-100):(i+1)])\n",
    "    all_avg_rewards.append(avg_rewards)\n",
    "    all_qtable_rows.append(len(q_table.table))\n",
    "    all_epsilons.append(epsilon)\n",
    "\n",
    "    if (i+1) % alpha_decay_step == 0:\n",
    "        alpha *= alpha_decay_rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "40789"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(q_table.table)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_kg_hide-input": true
   },
   "outputs": [],
   "source": [
    "# for k in q_table.table.keys():\n",
    "#     print('State:', k)\n",
    "#     print('Action-Value:', list(q_table.table[k]), '\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_kg_hide-input": true
   },
   "outputs": [],
   "source": [
    "# plt.plot(all_epochs)\n",
    "# plt.xlabel('Episode')\n",
    "# plt.ylabel('Num of steps')\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_kg_hide-input": true
   },
   "outputs": [],
   "source": [
    "# plt.plot(all_total_rewards)\n",
    "# plt.xlabel('Episode')\n",
    "# plt.ylabel('Total rewards')\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEKCAYAAAA1qaOTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsfXe8HUXZ//c59+am904SEkglJBBI6NJC6CpdsCCiyE/FhjUoNhRFygsWpIgKKIIKAr4EiCTUF0IJEFJII4X05KaXm+SW8/z+2J3d2d2Z3dlz9pSbzPd+7uecs2Xm2d3ZeebpxMywsLCwsLAoFrlKE2BhYWFhsW/AMhQLCwsLi0xgGYqFhYWFRSawDMXCwsLCIhNYhmJhYWFhkQksQ7GwsLCwyASWoVhYWFhYZALLUCwsLCwsMoFlKBYWFhYWmaC20gSUE7169eIhQ4ZUmgwLCwuLVoW33357IzP3Tjpuv2IoQ4YMwcyZMytNhoWFhUWrAhF9aHKcVXlZWFhYWGQCy1AsLCwsLDKBZSgWFhYWFpnAMhQLCwsLi0xgGYqFhYWFRSawDMXCwsLCIhNYhmJhYWFhkQksQ7GwsNjvsHrrbrywcEOlydjnYBmKhYXFfoez7ngZV/7lrUqTsc/BMhQLC4v9Djv2NFeahH0SFWEoRHQJEc0jojwRTQjtO4yIZrj75xBRO8X5PyWi1UQ0y/0/p3zUW1hYWFioUKlcXnMBXAjgHnkjEdUC+BuAy5n5PSLqCaBJ08btzHxracm0sLCwsDBFRRgKM88HACIK7zoDwGxmfs89blOZSbOwsLCwKBDVZkMZAYCJaCoRvUNE34s59qtENJuI/kxE3ctFoIWFhYWFGiVjKEQ0jYjmKv7PizmtFsBHAHza/byAiE5THHcXgKEAxgFYC+C2GDquJqKZRDSzvr6+8AuysLCwyBBPvLsaQyZPwdL6nZUmJTOUjKEw8yRmHqP4fzLmtFUAXmLmjczcAOBpAEcq2l7PzC3MnAfwRwBHx9BxLzNPYOYJvXsn1oexsLCwKAt+9/xiAMDE216qMCXZodpUXlMBHEZEHVwD/ckA3g8fRET9pZ8XwDHyW1hYWLQaTDqkLwDgpBH7zkK3Um7DFxDRKgDHAZhCRFMBgJm3APgfAG8BmAXgHWae4p5zn+RifLPrUjwbwKkAri37RVhYWFgUA9cnqVenusrSkSEq5eX1OIDHNfv+Bsd1OLz9Kun75aWjzsLCwqIM4NDnPoBqU3lZWFhY7BfYB/mJZSgWFhYWlQCzw0ryvO+wFMtQLCwsLCoAwUf2IX5iGYqFhYVFJWBVXhYWFhYWmcCXUPYdlmIZioWFhUUFwK5ssu+wE8tQLCwsLCoCK6FYWFhYWGQCwUj2IX5iGYqFhcX+i227deWWSg/PKG8ZioWFhUXrx6adeyvWt6fy2oesKJahWFhY7LdQFPkrGwQjye87/MQyFAsLi/0XlTSI28BGCwsLi30IlZzLWfGttcMyFAsLi/0W+4qEcucLH2DI5Cn4cNOu4hsrApahWFhY7LeorLopu+SQd0xbBABYsG5H0W0VA8tQLCws9ltUVOXldv7Cwvqi2xo/uDsAoHPbipS48lCpio2XENE8IspLVRhBRJ8molnSf56IxinO70FEzxHRYveze3mvwMJi/8ai9TuwtaGx0mQUjUpKKHLfjc35otoit/xjpa0xlZJQ5gK4EMDL8kZmfoiZxzHzOACXA1jOzLMU508GMJ2ZhwOY7v62sLAoAxas244zbn8Z4254rtKkFI1K1iKR40+aWopjKF6bFeYoFWEozDyfmRcmHPZJAA9r9p0H4AH3+wMAzs+KNgsLi3hs391caRIyQ7VIKJm1WWEZpZptKJdCz1D6MvNaAHA/++gaIaKriWgmEc2sry9eV2lhsb+jJle5YMCsUckJWA5ozIqKSksoJbPgENE0AP0Uu37IzE8mnHsMgAZmnlssHcx8L4B7AWDChAmVVjFaWOwD2Hdeo4pKKCW4j5V+MiVjKMw8qYjTL4NeOgGA9UTUn5nXElF/ABuK6MvCwmI/RUVX9LKEkhEhlU6FX3UqLyLKAbgEwCMxh/0HwBXu9ysAxEo8FhYW2aHSapUsUUmVV1Y9b9/ThBlLN2XaZqGolNvwBUS0CsBxAKYQ0VRp90kAVjHz0tA590kuxjcBOJ2IFgM43f1tYWFhkQqVNcpnk3xlab0UHb+v2lDiwMyPA3hcs+9FAMcqtl8lfd8E4LRS0WdhYbF/oEo0XkWhXRtfLrBeXhYWFq0KlVarZImKxqFk1DWz+nslUNk4fQsLC4sygplxw1PvS78rSIv8vQg6qomhWAnFwsJiv8FDb6zAX15d7v1etaWhYrRk5tlVRYnwLUOxsLBIhUqvgotBm5pgUGb9jgqWANb+SNmOwv142vvr8c1H3i27G7FlKBYWFvsNcqGSvwO6ta8QJVAykc/++U0MmTwFu/YWlt5GNHnVgzPxxKw1mLVya+H0FQDLUCwsLPYbhGvIV9bLK9r7y4uc9FA/eHyOeTvWhmJhYdFaUelo7GIQzkJWLckhw8xlxWZz207wXOd7+zY1xZBWMCxDsbCw2G9R0Uj5mK7zKciKk1DKfXWWoVhYWKRC65VPgJDGq6ISihwDE2EEKQiLyif+dVqjvIWFRVWjJc3yucoQYSiVISPS9xE/DxYrSxNwyQrGFP4sFyxDsbCwSIWbp/q18Vobc6GQFaWS9qBYlVeKAo5BCYW1+8oBy1AsLCxSYd223d73La2srnxYQqksGL061Sn3mEoozIwL//Ca9Du8v2DiCoJlKBYWFqlw6YRB3veqmp8LQCVtKNPmb8DGnWqGbErXw2+uDJ4XacfaUCwsLKoYKiNwa0WlvLy2Jkh2phJKn85tA7/DDMSqvCz2KyzfuAvX/P0d7GlqqTQpFoaI806qdoQDG5OwYlNDSexES9waJtedPUq5v9gsyOIyy51NuVIFti4honlElJeKZoGIPk1Es6T/PBGNU5z/UyJaLR13TnmvwCIrXPfvOZgyey3eXVHeFBEWhUOeXytdfyMt0gQ2vrV8M0665QU8+vZK/UEFw+l4WJ9OMXsLaDXs3VXmx1Op9PVzAVwI4B55IzM/BOAhACCisQCeZOZZmjZuZ+ZbS0qlRckhVlCtbWLan1FNqT7SIpzLK47+Oau2AQDmr92ROR2iX53AlDeUiiI2k/3Ry4uZ5zPzwoTDPgng4XLQY2FhYQ5V3ENrQdva4JRXKfL9AEQ1RzGlK2IzCZ349Jy1+OS9rxszqGJRzTaUSxHPUL5KRLOJ6M9E1L1cRFmUBuH4AIvqRVzcQ7UjjRdUKV2MPQlF2rZpp59K39htWNOuwENvrMCMpZuwskx1XxIZChH1IaILiOgaIvo8ER1NRCbnTSOiuYr/8wzOPQZAAzPP1RxyF4ChAMYBWAvgtpi2riaimUQ0s76+PqlrCwuLBMir3dYmoVTaCypMh6yCk2N6BnbrYNhO6LfmuKaWFJGSRUBrQyGiUwFMBtADwLsANgBoB+B8AEOJ6FEAtzHzdtX5zDypCLouQ4x0wszrJTr/COCpmGPvBXAvAEyYMKGVDf/9B61tpbs/I2iUb+Wo0AWEc24BQE3OX6cfNrBrYe1WmMPHGeXPAfBFZl4R3kFEtQA+CuB0AI9lSZAr/VwC4KSYY/oz81r35wVwjPwWrRDVFblsYQKZ+ZdLN58VkozYZaPDU3mRtK2QUr5mEtfabXswrE9n41YLhVZ1xczfVTETd18zMz/BzAUxE1eFtgrAcQCmENFUafdJAFYx89LQOfdJLsY3E9EcIpoN4FQA1xZCh0Xl0dpUJhat+5lVOjVJGPKCqhDeHKFf08blf3ozfeMFINZtmIjOhKPiGgCH1DVwXHmfLaZTZn4cwOOafS8COFax/Srp++XF9G9RfcjKKD/8h0/j5BG9cd8VR2XSnkUUssF4b3N5dPPZwaH9N5eNwzcemVVBLy+nZ3nUX/3gTH9/oUb5CishtRIKEd0B4BsAXgJwM4Bb3O9fJ6LflIc8C4t0aGphTJu/odJk7NOQ57rrnzAvVVsNELR3bd8m8Lv8hDgfstvw0o27/N2GdFWbxBVrQ2HmEeGNRPQPAIvgMBsLi0xQ6ZXV/oqNO/diyuy1uPzYwcjlzKREWUJZJk2CrQGzVjkZGcREbjLuSmHoVhnlVfsLbbdSiHP/3UNERyu2HwVgT4nosdjPYI3ylcVt/12In/xnHl5ftsn4HHnS6t5BnX69WnHPS45pVgy7Sq3oRb/hyP3w/sR2Qixk/fbKTs1xDOVzAH5HRO8T0X/d//kAfufus7AoGpUW0fd3rNvmTECf+uMbxudUwjV1+cZdmfarm8jLBc+GopVQTOuhBH8vr7DEqFV5MfM7AI4hon5wjPIEx/tqXbmIs9h/YCPlK4NCJtbdjeXNDP3qBxvx6fschrf8pnMzaVNo90ym7bQZik2gipQvqJ3Q7xH9Su8aHIfYiHci6grgZPf/RAAnE1G3chBmkR0aGpvxzUfexYpN5Um/YNF6UMhk+cSsNd73cqRHbyyBJ5l33TH0PzNnnXtIBWwoBXbJDNTv2Jt8YIkQ5+X1WQDvADgFQAcAHeHEfLzt7rNoJXh5UT2emLUGJ93yQqVJsagyGNrhA+jczldsnD66b4bUqNG2TfYpB8VE/qMn52H+2miyj/fXbMebyzdn3q+Az6QI3z9LXRMlXTsOXllcj6NunIbdFaovFOfl9UMA45k5UKjCTcT4BoAHS0mYRXbo4rpIduvQpsKUBDFz+WZs290EwHp5VQqFaHNG9++C5jzj7Q+3oF1tTfZEhVGCoSGr+t5ZsQWH9O8S2N/Q2Jx9pxJkCUX1DAqVihav31k4URkgjvUT1I8yj9ZfSnq/Qt7VGAzrrS7mUylcfPcMLFiXfa0JUzzy5oqKe8VUGoXYUBhAjXteOTKvlKILWTJT2e/KabNXdWWevt75vPDIAQCAmkJEzgwRx1BuBPAOEd1FRD9w/++Gowa7sTzkWWQBoee2Lro+du1txuR/z8Exv5xeaVIqikLGBDN7E1c5JMtSmGlk21HSPSgJ05SM8rsUTg4PzvgQO/Y0GTTjNPSVU4Y57VX4HY/L5fUAgAlwouP3AmgE8CKACcx8fzmIs8gGPkOpXo5Sbi+vFuuvDKCwMZFnQCTGLY+Ekn0nQQlFBX/rQ298mHn/vtswYY/G3vHb6YuN26uwYOIhNpcXM28B8EiZaLEoEcTcWVPFDKXcsPzEQUEqL2bkiJzVcCu9kWkY6ZEHZl+/T3YbHjdI7Tjb1GISxe98VjquRqAg9wkial0JfPZztOSrX+VljfKVQSFDIi9NYmWRUEpilPe/J70Xfbq0zbz/pJry5W4nK8QV2LpQtwtAv9KQY1EKCJVXtaxigORa2MXin2+txIDu7XHCsF7ZNryPoRBVCcMZS4TyxKGUoocu7XyPxySjfCku0XcaLk7ZK7dTDYhTef0DwENQP892pSHHohTIV9kqBih9ltTvPTYbQExktRWIABRmQ2FmEDlMpRy3sRSBhYHLrsB7wRk5ysjtxLV1/rgDiuvIEHEqr9kAbmXmK8P/ALbGnGcEIrqEiOYRUV4qnAUiakNED7gFtOYT0XWa8w8iojeIaDER/YOISpalbk9TCybe+iJmLDFPoFdN4Co0yldbHYf9FY+/uzr1OczwbCitVUKRV/Sqt6LUb4p8TfJrKceKpWWkhMpLKnEM5ZsAlPXi4ZTdLRZzAVwI4OXQ9ksAtGXmsQDGA/h/RDREcf6vAdzOzMMBbAHwhQxoUuKDDTuxdOMu3PDU+6XqoqQQHk011cNPSq7ySuzfMrCCkWdGzl0Rl+W5afrYsGMPVm/dbdzMtgbfDbcpb57OpSQqL43WIO0rKpOWZ31A5o49pQ3UFIhzG34lpgTwTNX2NGDm+cy8ULULQEe3bn17OO7KAcZGzlJ7IoBH3U0PwKksWRL4aX9a5yTkq7yqh6MYVi5NhXInLdwX0KNjesHeGU/kqLzKIqGo+zjv96/ihJueN26nocmfVAd0a+99r+R7Qa5cISDbOY3ubIgxbdcwjukLylN0Li6X1/VE1CNm/0Qi+mgJaHoUwC4AawGsgKN2CyfV6QlgKzOLu7cKTkbkkqCajNmFgD2jfIUJiUEWqpPNDY3Gx7bStUHm6NclvTmUXQkFKFMciqaPtdvSZTkQ7dx04Vi0a+OnjFGqvAITewkjGyP9pm3FTJ09om95smTEGeXnAPhfItoDJzq+Ho4xfjiAcQCmAfhlXONENA1qj7AfMvOTmtOOBtAC4AAA3QG8QkTTmHmp3LTiPOUTIqKrAVwNAAceeGAcuVqQ9/JU7yy0cnMDmIEDe3aI7BP+7FUloZRCRLFIDfm2O8b25DHC7LwTDY0teHbuOvzoo6NLRyCyY/6imfACsRKvhazykvsPMDKD65bjWUb166xNZdSpbWzIYWaIq4fyJIAniWg4gBMA9IejevobgKuZOVF5ycyTCqDpUwCeZeYmABuI6FU4EfsyQ9kIoBsR1bpSykAAa6JNAcx8L4B7AWDChAkFDU1h6KpifoIz73gZDY0tmPezM9ExNHhumboAAFBXk33W1kIRXvVlsQpsrSrJSkK+Z4JRJJ4D9iblcqx8s3qq+bw0+0pQXXN4097mFjAjINkUA136egocY37lDmPSP7xyvRmJMwwzL2bm+5n5V8x8BzNPNWEmRWAFgInkoCOAYwEsCNHEAF4AcLG76QoAOomnaKQpxlMpNLj2g5ueWRDZt367Ux+hX9fq8fYuhdtwmjaq+VmWE/I9M70nedfLa3T/LqjJlX6REmR6HNmmw1vLN+OYX07DgnVB36JCBJKz7ngFR9zwXAFnquFLFkFqcmklFOm7Kg1/mrayQMWWrER0ARGtAnAcgClENNXddSeATnC8wN4C8Bdmnu2e8zQRCYfq7wP4FhF9AMem8qfS0ep8VrPKS4i0hx7QJbJv/ODsU0dkjSq+tfs05FXw4g1mmZ/zzAA5+bzKY5T38cBrywEAS+qT07TPXL4F67fvxdOz1zrtaJxTTAIbl23chd1NLdhukLDRBHIJYLl32c5pcmd1jCl6XHlesIoxFGZ+nJkHMnNbZu7LzGe623cy8yXMfCgzj2bmW6RzzmHmNe73pcx8NDMPc48vYZmy6hZRVm5uwM69jn9CTmF5r0ZVUJikqx4s2nEwnYRShfekEpBvg2k9cpefgEDliUORupi9ehsAoNnAG6CpxXENFod6k3jouCQ1n9zTll3mjh+xbaq1bwFm9/c3lE62wXYSatP7x5UH1aNUr2JUu8rr189G1VwyBN1/+r9lBfdx90tLMGTyFAyZPKXkxYcKhY0tSQ/5jrUxtLEt27gLT81eixyV653wexnco6OzxaBjMceKcZEm75VgRhFKMnYQCNs+Unt5aRiT7rhSI3EEEdHNRNTFjWCfTkQbiegz5SCuWiAeeLWqvA7u1TF2v0x2s+ZFiUNzSz5gm3k+A592efJvk1HEpbWhpIcsqaVxAe7bpS1QgeSQS+p34v8Wb8Q6A5fh8OSs9/KKjr+/v7HS+y4b4rO6XLkEcJAW/3uXdik8sxIllPKMeBOKz2Dm7xHRBXDiPS6BYxD/W0kpqyJ4K50qnYXa1UmeJwoaiyU7PGkUEgwXhnwve3Vqiy0pYki0bRbdwv6HgFHecIC3rc3h/HED8ObyzWVXHf7nvTX4z3tKh04tBIm6QnOdFS61A7r7gY/yeifrRWXUhiIFORoEjj33/nqjfqpGQgEgksucA+BhRZDhPg/xkPcFlUohVxC57gxuw6adPgPJKg16mslN1oWv3NxQfOetFPIdMw0UZDireidSviRkRforBLLkMfHWF3HabS8F9v/z/x0HAKhVSMjt2jhTY/+u7UKxOgUSY4gV0lg06eulRfUAgLY18e7M1cRQ/peIFsCJBZlORL0B7FeFuD0vr/TaorIg6PqpElGKG00RF9+iWnMge5uu3ro7k5WuqoW129Qe7iu3+C/uPS8vKbrv1gr5vv/kP/OMzxEr63Ib5QvFUsnhQDAazzYa88rU5IKLnawkMtn2UUyq/K7tnfV+VymppArlUtebxKFMhuPaO8ENNmwAcF6pCbPIDlmvsLJu44IjBpSMru271Q4EHep8NYcw9O5rWFq/E0/NXoMPYtyBC5JY2ZmM4ySUnXubjewcRv0VuYQJny3m7zgDuAiCXLttDxZKcSyZ2VA0KVP6S7FiJszrwB4dcOrI3hlRVTxSFdgKXfy/S0FQNaK1J4fMmuxsotqdz19dOBZrtu7OaAUVbaNtbbIQvi+oMsNYsakBEyUVj64uTNrb3ticR3OeE9PXX3rPDMxbsx1v/uA09CkgX1gaGo+6cRpe+M4pkfQiT7nxJ2HojPUyhFTSkmcsWu/HvGTm5aWRUG68YAwam/O4+6WlWGoQayNnLYiDLiVL1oh72z7m/n8BTtDgp93/+wDsl15erWHaUYrvGVNerL0jn2e8udwxxbWtzbmqk+Lp2t0Y1Unqmm2lawNj1O80C8tKOzZGXP8MAH8i1N3HeWucVf3t0xanal+FJArrd+zF/yoM9SJyPEyjPwGLlErRHgSjPO7gnhgoGehlBrpu2x786Im52NOUPst1nAvzWWP6Y9ygbkZPJp83dzUOZwwoBeLS14tiWgxgNDNfxMwXATi05FRVGardyysJiTaWFOc7v4u7ETOWbsJ3/vUegKAffrHt3vd/SyPbWqtUWSxMJ5lC7YKeUV4znkSOr8MGdi2sAwkmz7B7jA0hTGNY5aVqXdiJenSsC7oNSwf/9vnF+OvrH+LZuesS6YvSJGgJFgEW34k0hCnaMU36OnvltnREFgATo/wQZpZlx/UARpSInqqEWNFUaxxKINdROfor8nzZQCpUJ0DxDPsjKerHy5NMlT7WqobIkquTLOWa7RVHiEYx3kizH3CK0uWI0KaGAkGO8hwg6u+0FCBeJ5UANq28KJcSSMJRB2mrkWQGE4byIhFNJaLPEdEVAKbAiUPZb9AacnnFIRhrUMD5JXAblpHLSKUoIr1vufgwb5u2zdb5KJU4785XU8dmCBQqweXIrMBWFu9MmjTuZhBSgH4mzjNQQ4Q2NTk0t6gbLyYc14RcI5UXs5b5nDu2f+B3qkDJAmHi5fVVAHcDOBxOHZR7mflrpSasGlGOqOBiobahZNtmljYZZ2Jyvhc7+Qi6Jgzpgd9cNs7ZZqg2aK1YubkB763ciq8//C7++ZYf3W062RV67TlXVRl+J9Zv34Mhk6dg5odbABT/ztTv2ItlhjnGdHhjWTB0LmqUV9tQiBwvL7nMcGbvl2xDUaXPN3yAzEEXfBk9OwUDkMtRDymWoRBRjVvc6nFmvtb9f7zkVFUpkkTbv85YjiGTp6Sqc50FkibNrO0IRcfjSPQ4Kq9sVIpBz5mE7KtF9VQ9kO/Z9x6bnfr8Qm85kbMuDo+tJRtCnklFPtOjbpyG30xPNuzH9TJr5dbAbwp96uJQckTYFEoGGRijGczPceN0595mbExwroiTUCJ9paKsMMQyFGZuAdBARMVb1vYB5BMYyo+edALDvnD/W+Ugp2wIX3WWk7Fcsa5Yvqf2nNlXWIcauntmuhotVNokArbtbsJ7q+INvdUg1fcP1QESKta4cZfPO7aJ88YdENiedRwKoJ7oRSLXyY/NUZ7/yuJ6bNq51zXKO9tuOC/oLxW+rnJUpjSxoewBMIeI/kREvxX/pSasGpG0gh7Uw3EvPO2QPpn1ubWhUZv5VIAD37N/g0UNCoEvPjgTr36wMXU7G3bsibhY5ij7ipiy34yuzWLtStUCmfQjD+zmfTedO/IcPM8UBPJW/pfeM0NJj9N+5W/usD7BqpK+UV5tu2NmvLSoHnmO3kfV9RSiAUjKEiy84/p2aavs7/I/vYmL757hSVLOsfHxPqaSTDEwsdJMcf/3e7QkDJxD+3fFys27UZeQVycNxrlV4nSBaWEsDAUwteQ5ENRUyPsdZiiAw1Tev+Es4zZ27W3G0TdOBxDOLpylDUVqtRzyfZWhkLvHXJhuPUfAKSN748WF9QEbRfgRZiWh9OpUh4079QlE44ZOeFx5DEUTsPzgjA+x2FXdhYMGgxqvwp1JPLdhUt//P11xFI66cRpG9escPdc9ednGXRjcs0PUa00DKkOxEhOj/AOq/2I6JaJLiGgeEeWJaIK0vQ0RPUBEc4hoPhFdpzn/fiJaRkSz3P9xxdBjiqSXo1LeYHJ3D874MPCCrN5SvD3nrDH9Yvs0wYeb/NxZTZLXjEjhAWThPBB1xdS1GXAbbsVqsXBN+AJaKKjfHBH+8rmjklvP6F2oM6zV8vaHm/HY26sC2zbuCDKipJX6KinPW3Su55h95vAlFHUjcW0HNBKShLIiIclpxW0oAEBEw4noUSJ6n4iWiv8i+50L4EIAL4e2XwKgLTOPBTAewP8joiGaNr7LzOPc/1lF0mOEJBuKQKWD6eQ4DzFZikCzLAIbC4Gu5glJcShZMuIkNVqxnkPViEIktPD9ue2/C43O062sw+Mrq0e6JiEvmOj3ortm4Ntu0KzAwvWhtCMJqVfa1joahrevnxS5RuUUUMA17tzrlBJ2sg0UfpMco7yDzSEHgkhAZ6W9vFz8BcBdAJoBnArgQQB/LaZTZp7PzKqRywA6ElEtgPYAGgGUPl+AIZJUXmJ3udnJ7dMWAXDyYgGO/7yAeAFM8v3okEnuLs32HMEz7D71njr3UiF9JF2uaXXCakfgvhaiy0cw39nvnv/A6DzTyamYRULSRBs2mJsiEimvcIuvyRF6dmobCRrMikHe+7KzJg+/l+EaKOHudje24PF3VwfoEc8i6ZFUhYQCoD0zTwdAzPwhM/8UwMQS0fMogF0A1gJYAeDWmPorNxLRbCK6nYiilisXRHQ1Ec0kopn19fVFEZfonus+/koZIkUNh6BI7PwSdpRTb30xdbulvBwiYLdbUnjr7iKLbCm8vHTMUGYoVWA3zgQBhhqaPv751sqIKghwxodstD5nbFS9qYJucopOzoUj6bl87LDCGIrn5SUVCQ4j7Frs05TNYDl7jBN02Ltz20DvItuD7v7e8/ISL20z8t6OAAAgAElEQVSRoMc0Ur5qvLyIKAdgMRF91a3cmOjGRETTiGiu4j8u9f3RAFoAHADgIADfJqKDFcddB2AUgKMA9ADwfV2DzHwvM09g5gm9exeW5jntGCr3BEUEfPXUYZKaxydASCgTRzmPbP12s6SBMrK4nDj31slnjwIAHNC1vfog0z6klODJXl77BhfReauFJ4/vPTYb3/7Xe9jbHPSyE55MS395Dg7s0cHYVtGvq9qjKHxX5dLRpti4cy+mvb8ee5rjky4WWkdEnNfsBlR96W/vBPbLbYUlBlnl5bOj9GOpJkfoLCLX3dNPHtEbNWEJJdR0WLLOs55RRNyGyyCjmIyebwLoAODrcOwanwFwRdJJzDyJmcco/p+MOe1TAJ5l5iZm3gDgVTiFvcJtr2UHe+Go5I42uI6SQzzAcvveO4Y59cASE+cFRwzAoB7tceERAxLbW7R+B26ZusA7l5nRvk0NvnrqsMJp1L10DNTmcvHHpIQT2JhEz76CdM4Fby/fEjyb2Un0mHO87UzH7qRD+qqpyYBRT/jFNFz14Ez88PG5RbelgphYRyo8qLxjNJ5TgfiRIufnuNN1KsUu7YM50tZt34N/zlzlthdPULVIKJuYeSczr3IzEF/EzK+XiJ4VACaSg44AjgUQWeIQUX/3kwCcD8fIXzWoyOpXGi0BlZf7KXIvmajjfv3MAtz5whLU73CkGWagc7tafOfMkTh5RLbFfFZtacisIqYu4ll9sP/1lqkL0ZwQ69MaYDLs9jYHrzNodyJjRltnUGcGMFehyTi4t1PwLOk9IgKuP/eQ1O2L8da2tgbD+3SKpTE8sacaYzEQjFxFV/i4uN9pUC0M5X4iWkJEjxDRV4hobLGdEtEFRLQKTiXIKUQ01d11J4BOcBjEWwD+wsyz3XOeJiKhNH2IiOYAmAOgF4BfFEtTFuDQZ1n6FK6ygW3+d8FA/JKtyW2Go585Rqw2p1O9fdyg7tm5DbufTnOiTXWr4e1/eDF9GeCVmxsw4RfPYVtDU+pzs0LaAE1x3VsbGjFk8hTs2NMciMtImrAGdGuPi44cGNN+EIU4PxwxqDsAYFCPDonHnuJWK0wzdgJRUBRdyMhtlTIOJe6dMrVRAcAPzzFjqlWh8mLmkwAcAuB3ALrDYQA6Q7kR3NxgA5m5LTP3ZeYz3e07mfkSZj6UmUcz8y3SOecw8xr3+0RmHuuq0D7DzMmlzcoAT+VVRp2XnG7EX/FINhT3ZfFKtqZpXHjBwM8XVChj0c1TYwd2zdxtmJCcEj+8vZC6HSfe/AI27mzEBX94NfW5WWBPUwvukApYyZf00Bsfxp77phSM+JdXlwOAm5srvk9RJ8QUzXnGK4vrU62s0zm3FDAgpVN0NV104z0rlRdzYTYY1X0M23l0KIeEkhgpT0QfAXCi+98NwFMAXikxXa0a5ZRQxEuX0xiiA4ZqKmzSVkkoWQYDZlVgK03ePsHz//DpI/GVh94pykff9IXOGve+vBRT5viu1uL+NbXk8fCbK5XnfO/R2di4sxFnHeqreS5w7WqkmVyfnrMWndvV4sThvZ2VdQxN4RX9lNlrMWX2Wlw7aQS+MWm44ZU5mJ2QJ0xecYfHzvKYOKNVUrCvKmNywCgfulj52GfnOYW1RF2UNGAoVF7K4+J/y+dF31H1caWEiTz6Ehw7xb0ATmHmrzDzw6Ulq7Uizcoqyx6DhuiADYXl/cn1K3R9iImi0HiWOAYkXtqik0N6zNP8WL/v9J2f5NqTRvfvkvrcLLBbyos2ScofF5cVW6QvEZMhANx+qZNoIkfqZ/CVh97B5X96E80teazdtgfPxFQo7NGhTrl986703oWvLE7OF6d71qfEuMcPllRpBOClhfV4L5SRWMy+YTWRPE62uqrOG556P5HOMAISiuKem3puyceKU0S6lgNCnnjVEtjYE8ANcOwdz7ruwD8vLVmtE15gYwb8ZOfeZk0fjDtf+AA//c+8QF+OjSQ6YGQJRjdhKHpRtgEUvsqJ61fQXaymMMw84/r1tzvHPTkrfYEqzwsu9ZnZQH4WNTlKNe46tY0qJwjxThvb9zhjcuwAvXpQt3CoLUUgaYGDsXdnP2wtlwMaW/I470612jKy6s/oYatsKMrMA2HpSdFW+Kyzx/TH8pvORfu64DMuhyBtYkPZCmApgGVwAg6HAjipxHS1ahSrunltyUaM+clU/PHlaIab65+Yi1umLsT9ry3Hiws3BFRafv+IfM/lYOzlJeAxKIXKa09THusS0mGYwpMSspqaKXmu8b3fnE85+jgtKhXIKj8Tgq+uMqFHtWChhAWHGNeq3G4CukVBqluU4e0c0C0a2yS/K6pFmDwOI0Z5aZ+QAAqJ2Hfuh37RozOgq+YWneQhtn7u+CFY+IuzqkNCIaIlAG6DE0B4N4CRzHxyqQlrjRCPutiVtsgzdePT8yP7Hnpjhfe9W4c6ZTCbPOg9L68ipAB5NSWPyWN/NR2vFZDGPgLPKF9cM6rT9fEvvuRWcH+i6UqJKBJkZlDofVTZE2R46tWYW3bTM9Ex65ybxihvjqSnp6JVXqnrVu1au4REXHHjNercYDISTdRjqnst8pOVGiZy6HDXw+qXzPwKMxeZH2PfR7Er1i7t2iQfhOBqhaRVuWrQk/DySkGaGJjM+qpwn7rvDSO32bh+vUm9aCOKzzwTvbxE3xloYhiMxet3YGl9ts6Gp976Iu5+Se/OLD8T2eGiUAk5yQlWVinq8PpStQNoGpJM3x+Zjl89vSBSawdQLxgCY1kZ/KE/Px94t5wfT89Zm/qeB728Yo6L/FZIKN6XeDVvOWDyOg0joulENBcAiOgwIrq+xHS1SohB9eayoryqI0OmZ0e1oTPPIS8v5SQq2VByKScb9j/89yr6Au5sVNt7gk3FGeWLs6Hc8L/vY8hkv2SP40KdQI/HaM0klOaWPBpC1+kzXOD021/GxNteMifaAMs27jJOXSIHJRZ6H53xod//3/cdY/zzCzZojxEpfsJI5Tacgn7x/NZt3+MlSZWhKqErP3KthOJuD8fRqNIaNbVwasO8ynMy8Nv9/vNQu8p7Uw5/YEOYMJQ/wsmd1QQAbqDhZaUkqrVCPOvFG3ZmGnkds2YMrRrFGjM66ImALbuaAsW2TPuVV1OqsVts3I1oslDJ7s+vLgMA7FK4b+pa3OUyhw5tzFQBw374DEb/eCo+kGqmZ+mEUQiCNhT4C4CCJZR4G5vIBq2TQgCgQ536fhZrQrnwiAG4+eLDcO2kEd628GJgRJ9oKhVdzjEBlQQj9z9pdB+cdWg//PaTRwAArv7r294++V417E12Hd7T1CI5cvhSv+p56XKqmRjlKwkThtKBmd8MbUteku7n2LQrO83g5l2NOO5X0yMD7+K7Z3ileHWrcjHZ54iweuturN6aXHBL5VkiXrxCC3aZqLyKnZiFuywBsS8rADS7Rb7aGjIUgUn/40shvs0invCfPDkXQyZPwZDJU0qWlkeWUArtgih+4h/Y3TFwd+8QVclucce7aSBpHML36IzRffE/l47DJyYMwtEH9dCep1Jf/vayIyJGczluSDcZi/HTtrYGd18+Hh8/PGp4l597ew0jFdiyqxGjfvSsl5FBllAkxXWgvXZtcjg75AChGmu6vGOVgAlD2UhEQ+FeNxFdDMfba7+BqUExM5dCRUNrt+2JeCIxA9f+w6ktFgzy8o8RxbYKkYqZHfXdc++v80br+2uj5WlMJIvYIzyjfDY3kGSDkga+qjBd2706RdWPSVQ/MOND73sh7sk6hDUkYtwUeh8pwcbG0nFhCNWMaaqbOISPDEhiMYbsa/8RLKwFOIuVCYO7a8+Z+WEwWSYAvL50UyDGR0AkWBVII5nXu6o38Q4zokM0fG0DurWPTf0i8qlFYmWMqcoeJgzlGgD3ABhFRKvhZB/+Ukmp2gdQCldSuYyugOx50+gm/tu22zeSX/fvOc5+EP7fyQcHiinpIKQrBuMT98zAnqa8N2TD6bWB4hmpaLKQiOMgHb6E4m3THiv6NuMootbMySN8G4FsQ4nDEQd2875vaSiNT4ssXRTs5YX4cRu3rykffy+KcRsOOB8EtieDCJGZOumR6yL0w6elsvUozk2KldJlLgCAo4Z0Rx83nkZcT637Ion3qRKmldjZxa2DMoGZJwHoDWAUM3+EmT+MO29/hfzoS5HOS52N1P8uBtT/vuevgkWKjWMPdlQFqd5r6eA447WRhOIeI9QmMsSkfttzi/DSonRF0P41008xIqsZVR5vYXrSvHDieQZq0Xtf46+/Q12N9/KbevAZgYITLStoLLA5NQQTlmYNEYuRdL/TvA/h8RSUUNIxBxVkBnWHmyVAxlmH9sNwqeiYjKAHpTwW0t3zWG8t6XdE/SwWTUSeV5twHPjCiQfh8mMH44snqkpIlQexDIWZ8wC+6n7fxczmFt39EAEPkBJwFFWTspfXCcOdam9yBb6enerQs2Od9yI2Nps7C8jdxb23JpcqDvn5+WMw68enB/bJk8J/UqiEnl+wHt99dLb320tySH5U9zsKlYagWbF4VWLLrkbvvr213DdIm9osmGVPtuzGRWC1Lq1mC+0iya1cHmtyv6pjokih8gpLKAlevnEgAo4f2jOwTRayVRK7KAEcbSz4Ux73plfHPteXrkWoKkPdUfRe3Ppfx5MtR75E39G133Soq8XPzx+DjoosCOWCicrrOSL6DhENIqIe4r/klLVylCR6WtGm7/6qn7TEwJ2xZBMAx2/erDvJXBhr+DO/VtlgLiAHXalsFDrs2KP2DSEQhvV2mKoqNgFwJg0noaZse1Jfx4OSDWTl5t2RTL4mV59VvjIZH27yEyDKq9mmFB6Gsn0hWeXlfCpjOyIG5iAKiX9SIchEzdob2rsTvnfWSGUjSkeWODuStK8lIKGY0eIdL/U9uKdT/0VoEQTiMls4Hnnu92qwxrswYWWfdz+vkbYxgMrJVa0AxQgoadQGfiR8stpB6Ibnrt6Gc8b2T0VHnK3BSEKJEXcCq8GMXo6+XRwVU1eFRxLgl041eRnXhDzjnp+/AZ8+ZrA3eyYtHgL68gxNpgFdv7Sa/civXzBu44rjh3jfVRKKzGQ9e11Me2lsKM0t+UiOr8bmPKbOWx/YFg7g1O1TwUtDH7DDxJ8jS5S6vvY2twSk/aQxEGa4crDwIf27YO7PzlTmV9O1Wlsj21eqh6OY5PI6SPFfFDMhokuIaB4R5YlogrS9joj+QkRziOg9IjpFc34PInqOiBa7n91Vx1USpZBQVG36RnlSFqoqhooTb45OTOoVnUkvvu43bhJPetmbW/K+L7+uW/IncJ3qUUzyMi269mpqgjQJI7upiklWoWSpCT33MGdRcOmEQYH7FlbxnOiqQlUIPDuKPss1Ur42j7lItyPqqqu536HtT85ajWE/fCaSr+4fb61APPzO9zSZS2KBNYv0ffH6aHYDZtZmUGhqyePlRfU45zfBCh5pH6ssoQCaZJ0KBt+hrgZfPPEgAKhKCaUEKUCNMBfAhQBeDm3/IgAw81gApwO4zXUMCGMygOnMPBzAdPd3xSE//FLEG6ha9I10/sCKTvDkHVMo4ozyxXi7RPvR75u7ehuG/fAZfO3hd51+dZMXs5RwUg1xjIhHiTs2em7o0+CcUqi8nJU08OuLD3P17U7jg7p38CQ0APjU0QfGtiFAiF6LzJAV/CRwLuBMciP6dsLPPn6oth/Acc0Fovnq+nWNOm3oVFR7m829AiNBoC7mrol6dOVZnWqICNiwYy8+++c3saQ+WG8ltcqLDd4F50hlP3IpikRfihLFPqlQEYbCzPOZeaFi12g4DALMvAHAVgATFMedB+AB9/sDcOq1VByqCPVi25ERK6FAEqsDjC19/+E6CqJ9Hcy8vPzvXdq1wUVHDsQtFx+Wiq5F6x2fEOEJpqtB361DnTch6J6DmDS6SSox3YsXUQMBGDJ5ihfDkPTClsooLxdpCjODODVRf+n5yrVTcorIxoAnk5TGJ4w8Ax9s2AlmRtvamoAqTeyXMcS1G6iyAQPAz8/zGVLAbhJqM2mRJPbLNMvfB3aPlhlmmMUnyUw7eVkRbNCRUBLUdQqjPNzzHHuX/1sF8Zz7dInPFpAlKucOoMZ7AM4jokcADAIw3v0MR+r3Zea1AMDMa4lInUCogogrclQwFE3KKxZddLj8UrUYTGiqARo39k3mSF8153ze9onDlceZvMiioJWq23GDHHWUkGvjmESOnAll0iF9MG3+htgUNzL+57lozqgk+MzebFz8dvri5IPgT1Py5BNekISfpzyhdu/YRjouyvDkYSzGdMDLy6XgP++twX9cd3VVOeUwTWIcjuwXTZcSpln3veAUMwlqznxYHyXOi/wuQOT3pFs2ktbD5AXd1uMllImj+uLGC8bgoiMHpqezQJikrz9S8T+UiGKZkVuIa67i/7yY0/4MYBWAmQDuAPAaikzzQkRXE9FMIppZX58uxiEtdD7qxbQjI65NWeUVPCwbxuZ7eUWHrxFD8dQlZkbU2LbcT5M0FHEODmJyEkxI21/C9SUa5QGlfSsOJkwrqK4Kug3L82FYopDtA6eO9NdiqglMvjZhLE9y41UtSP79TjTLg0Obk8Bx5WYnaFeosYJMC8rvJuPOZ7hpjPJstLCRj9F5HHp9htpjmTht+2ovL6GNSLKh1NXm8OljBqNdyvRCxcBEQvkDgCMBzIZzLWPc7z2J6EvM/F/VSW4wZCowczOAa8VvInoNgGqptp6I+rvSSX8A2vSnzHwvnPLFmDBhQtmUiaVQW8YJPQS1Ud7ZF/wsBHEvYaqiXUUQ4XUjrfJ08O+F3kgceckNGXn3Dm2wxU3Zf+SB3ZKfNfuebKbxSRceMQD/fne1FxCpadZnoAEJRT8JA0CNe1K/Lu0CE22OoiWikwo6xdlT4iDuw/rtezHhF9MAABePH4hH317l9pHcBoPRnHA/hSQUiD1p43NU1fjQeXk1htyx5fvw1Oy1+P2nkmmWOjayoUS97vx9Ph0p+i0xTGwoywEcwcwTmHk8gCPgGNUnAbg5S2KIqAMRdXS/nw6gmZlVeaH/A+AK9/sVAJ7Mko40YGbMXb0NT81eE3j4okhWGO+u2ILv/uu92Mlw8QZ1XY24iYtINoyqDzQdeKrjcjFcyWSKNFVPmE4kTpuK893PpJT48qSR5NKrsqEImJbepQR6whDBaaeM7K09xjHsimvw6RIebEKHHr6nIjmiSQyGKqTl+nMP8b7HqUflmKJw5LnoZ85q3ygumImz3yfkQKkGvLw9nw/bMaLo7xr5ZSrlFbta5aVWR4XzsBEBv3OzEJvCXxOx8t6FO4iqvADZi9H5WT0cxURCGcXM88QPZn6fiI5g5qWFlpQkogsA/A5OOpcpRDSLmc8E0AfAVCLKA1gN4HLpnPsA3M3MMwHcBOCfRPQFACsAXFIQIRngoOue9r4fNaS7t3r92sPv4pSRvdFZSrXx++cXe5Guk0b3xZmHqkup3vWiuqhSrMoL5kb5RPOh4rGu2BzNI2ZCl4BItmeyKkuCmIxiJTaKpy3PvoSyYbvjGjt71TYcNSQasxtRA0kdy6omHZwgSnVbcecA8YsIDixzSZJQgrr18Gpb/FobKuGcU0xgqvs3SrJ7xEko4tQB3dorbDPxd0KeJL85abh0XrCNDnXRKez4oT1x8fiBOO2Qvuja3nn/0sxVOgnlsS8fh4vumuH9zhHhY4cfgKnz1uHdFVuN2xd9mEkoKpUXBWvSVA8/MZJQFhLRXUR0svv/BwCLiKgt3BopacHMjzPzQGZuy8x9XWYCZl7OzCOZ+RBmniTnDGPmq1xmAmbexMynMfNw97O4ilaJ9Kq3i4lIoDnPGN7Xf9nC1fZe/WCT911V+EdAXpElQbzcJK1aIiovTy3i7ldcT2NzHr9/fjG2NTQpVzxbYqoymjgg3Pys49SnylasJFYBfwXuTraK6dnzelLak6S2pEnjODduY9NOdeLGOAlFDiiMg5+iP50uVJX1VibEtxEEKZPHQ/iWhl1e5XPC9ImJ/57Lx0vHyUaUaDvvuJOraEklxSXdB7mLYPCjf56uCSLgwiMHeswE0Dt7qOjIa2b78YODiw1Bo0l5XZXqytRDLXii4rjE3ssHE4byOQAfwMkyfC2Ape62JgCnloqw1oDwyy5Wrh/ceDaA6ECrrSGlS24YozSeL2JV/bnjh+CvXzgaADBmgONRE1B9cPJLF8Y7K7bg1v8uwuE3/Fc5kIVqQTV473dzaMVBeP4cekDUAygtxCUl2ZQA/cQlqzWE66juBQ8zLrnJXMiQLWd6lo9PG4cijntq9lpc/8Qc7XGyE0LYxhReSITx2JePC7aloM83nsu2lgTivXOdk2trosblJG9DXRcmji/yws1vMIWEArNrFPckp2DEkTbFIkhaDCU7qGjUugTcfNFh0u/qYSkmDOUsAL9n5guY+XxmvpWZG5g5z8zZFtFuBYibrN9btQ1vLtuslQSY/ajreFWGekCLF6hzu1qcOLx3YJvsGhxu208/IdqPdi5PGKo0+Xd9Znzg9/s3nImbLhwLADioV0f9xbg49ADH1XeYJourT6sekQh5jQcM4N8/XTQ1I500I0OeyAhBQ/bhP/tvJJcWw7dbmDowyEfNW6OW6uRj2tTksNutCOhcW3DCU6GuJrjgyRFhzuptgbKzqroxpjp7QV9tjiLMP0mo1TN39fckaCUU1TZOnuyBoHSYdD0RW4iBhOKoIEOLGQTfd5mOaoAJQ/k4HBXXX4no3CR34X0dYU8PFXQTt0hI6HzXg9kvnhPeLgyu8jZAmhyhDkZzjvE2KtqOfyPahHJREAgXj3f8201qrIjWCxLzNW3F21Cchn7/wgeYeNuLgX0tecbf31iBZjcykhBzY6BXAwl6w3SceUc4AUSyk4AKvTq1RYe6Gow/UJ1ZSJ74uneoQ0NjC/Lsb49z9Ra0y5ju6uX/9H/LvG3KhJDS17gcb+I21eRyEbVooW71SRLK9G+fjAU/PyuyXXcP1EZ5dfXHSJvyIi6BvYmKlsIrzeTqiYLBu43NeTS1MNZs3R3K+GzQWJlgksvrSgDDAPwLwKcALHEN5PslmhJSdXRt30a74s3nfZfNOBFFt0JSrRZFBbj6HY5NRhVLICBW67oMvHGQ3VP9beaTpMrdUdlPzBFeN25jcXEo8j1aGrIZiGzLW127UNIjCUs58nEOQ3E2fM6NDt8cKv8sxzU0t+SNXIdlVZPuaHmV66vUfAklLK2FYTIReWWVA8/d/25SFK02p3JHjj9HzwD8E1W3cUC39sq4C1N1pujDpPCa7yUY/w48M2ctLr33dQDAKreEtpFnYMjhY8ocx8usV6e2IQmlejiKUeoVZm4C8AyARwC8DSf1yX6JJinDqDy4RcLA7h3aaI3jeWZP9REroUD9Ajz8plNMSjWAjnT7DyeU++fMVVjnOg/88bNOFhuVisp0vfit00egR8c6tKkhb6JKU2DLJN1EYltem87n3686RtGOvqGerjvr3a4aL0lqHOHatIQdSD5OTiPeo2MdjhrSHVsbmiJMW9Bz23OLlBKMCoIp6O4vQ2IaUvJJYVP2VXrqexGeNH9zWbTYFHuLGGlFLO2PU3cGbSjBfYlMVfP4AqfFLCjC0BvlE/qIgWyjinsFZixV2HSM3IaD9K3Z6rzHnz7mwNYroRDRWUR0PxzD/MUA7gOQnPt8H4WsH5fHUF2N4laGV2UwM84mpWUQbchZZHt39o3m4kV+fkEwDfj4wYUnZd6114kEvvqkoXj7+kmorclpGacKvuqkYBK8RI7i3omJ9sCevlfcXk0BMZVKz3cpDbYXRg8339f9Vx6Ndm1ygbYctYTLLAGcMKxXhA5GUC2oizMKUey1H2fb0RW4kjMp6ydZiv0NAE/MciRgOYW/fNyfP3cUPnqYejoQdDs2lOBFvLZENcn60A0TOfeayrtKJ1noVvHCqUXGrJVb8crijbH0yX0lGeWFK3qOgNNGOZkJjN2Gpd8irqdtm5pWbUP5HIAnAIxg5iuY+Wk3on2/RGCikJ52xJahUD2ZitIstXNwr4544poTAkFiYjWqSni3tzmP/77vMJLP3z8TgF9EKW0shIyWwCQaXB2ZuMJ6NpSE4a9LObJlVyN+8Ljj7RTWx8v3QVcPPLiydWnxbAzuZt3EDf84QnA1SkSQHSNEid9CPO3CIDjPOu7++motya4W2qa74+GhqBqb89c6CTlPCaVoEairzXmqvjAEHbW5XGTVn+Q+rlu9j+rXGd85YwQmDO6OSaP7BvZNOqSvVw432qB682VHDcLpo/uqKzQawpFSkx9y29qaiP0ttt0QR/HVoKH7U0UcxcSGchkzP8HMewGAiE4gojtLT1p1IujBI9tTzPTivgeY/vi8vHohkWsqOmp0eu1lG3cFVC6iIlyczSPpfdDtTyob659vaESBuuKgYCaAo1oC/NVzrTQZ6BwE1AFi7mcCoxVBZI7nVNjLK5j1VSeBpnXtlG+XNtpfolhWPwoJN6wOCyO8WUXirr3NmHRIX/ST3N3Di4KkS6utiTLFU2MyAMSBiPDVicPx6JeP98aBwH1XqBKTO9BKLkQY2Tfopt+9Qxt89rjBmnaibYZdx8PwGbzz/cWFGzB9wQat957cviqDeU4aZwCwZVdB4YAlgZENhYjGEdHNRLQcwC8ALCgpVVUM2SgvI+qqq3Y3FQuo2AGolIflyUPBXEInyMnqxOATp8luoabQM5RscnnJ7sSq5sTKf2jvjmjXJofNuxoxd7XzQsrZCFR2ACCp/reeyX/5b28H1B+7XE8quS3ZIULlHpxWQHlpUT0eeWsldu5tjvUg2rKrETtcVaS4rw2NLXhq9los3bhL6aAQRFjlFT2COciw5b4E9O+Es71GofJKkgiKUY2qkKa5vOr1E+3IWoGcvy1sE7rk7tfwiUfTNDQAACAASURBVHucqHrPDuW6Tz8zZ50ZzRTNDAC4kms1GU4kaBkKEY0goh8T0XwAvwewEgAx86nM/LuyUVhlaNSovOSHDbiGutBEkGffy8t0DlapY1RjKW58iX0dYrKOJklYJrWtTc7XvQjTvnUyvnvmSG1fXdrXon2bGrSvq4m88O3ranBGSP0RR79KPanDM3P9l79Lu2A54Re+cwpeX7LJy3ogV6gP35M0r/8Vf3aqNTQ0tsR6ED0t0Sbu6y+lglWL3GqEzZoJPyqhRKmUU9ToEC6RLCB6Xbm5AVsamnD5n96IbyhAi/GhRkhyAw6qKPUGc1U8Tli9PWvlVry1fAveXLY5IG2L5JtC2pvz0zNiaQrHOHmqVwouH6uJt8Td5gUATgPwMWb+iMtE0vub7mNo1BjlvZde0suH50XZNhIvocSvQHJSHwLhw+WBKNqqrclhzIAuODwhXbuSJs1252UyU/cB8RNr3IshXGRFSm/BIESiwqASKoGWUHBYkg0FAK6dNAK5HOGCIwZ423p2qsNVbjlW0Z4y0WShRhTEexCNG9gtcBzgp4m/49JxXg6sUf264C9XHqVsO/Bb0YczZsPnhY7R0NfguhQLN+pXFm/ErJVbY88RqDUJBEmBONud6npMjPsisWXYrnb+na9634MFzJy2xZbOoQWKiq6A6U8KG6im6HgZcU/tIgDrALxARH8kotNQVeafyiDg5RWjRtEZ5YWoH29DUaRWl74rVV4Rbx/1vs5t26CuJv1jjMtgbGRDkY7XwU+Xotvv/L++dJP3ooYDQHXtKyUU7xwFEwih1r1n4ey/B/f2VXUk9R8eG4W+/3EeRL0613mqwnDz5x8xAN+cNALLbzoXXTu0CdQ9kdsO/o4SqYqJCh+nu2s/+uhoAMDrPzjNs1OE1WcqHNijA04f3RdHDemOuz9zZOLxJki6/8HFoV4qO3tsNKFrWO0btu34x5kZ7wN0yVoQLyaIQpJS9UDLUNwEjpcCGAXgRTh5vPq6iSLjZbV9GEG3YclgJrmOOp/RlWWezQr3hL2InG0Kzb8mNiBMm9FkphnnYgLVTWqqGhrK5r1JPHmlqAw2cz/fW7UNe5rynpHeRLoI7w8zN0H/P99aBR2UakbFMX4SyPhjTRHnQSSb2gqxOcQV3vL6cIgI0RSmQ03fFz5yEJbfdC7a1tZ46kyT+fS7Z45EuzY1+NeXjsdZY5IjFN790elY9IuzY4+JW9FH3x39/fzqqcO878K7LRdKfinSDHntSSpWZpjdBHF8iC64tMlqxmoSVky8vHYx80PM/FEAAwHMAjC55JRVKeSo6eAklTyDqOIGVHASx4W3+VBLKMHfAQkldJxJduAnrjkBy28617P56Oq3O6uuxOYiOcfUbTmfWgmFCJ865kAAwLT5jufVrJVqN2Fd/6EWAQAHuHXN5RiH6JE6FUjwGN+GEpWICoFQk6gg1ICA3pMrDZTXqDBQF6JuEaeI+xJ3S9I2371jnTJVUaDNFO3lY2wociBn9w51XtuBOi3axZfZeyAfj0C7cM8lnDlGXfqi0kilqGTmzcx8DzNPLBVB1Qh5eDQ0qkNwVGOoUKO8zHhU4041GCNqCE0Hry3ZhHdWbPVKrvq0htsL9qU3ymdXsVFMaKr2xKavTxwe2J40kQgEPLNC7YvYhUP6B1eWAdq8++FfQDjFR0BCiZxf2IRPMQxbzlhbSOth1YyKRNnu5x2nOCYJaUsgZ40kr6jA4pD147S2JoejhgTjusKShLz4mrFkE95YttmjIc31O++WTJdvQxGFw/wjqwPZWr72AzRIuYuCXl7OZ4ARhI3yzJ5aIU5fr/IalvtSBazFHa96OeatiV/ZR6sZqmFsQ/G84ExUXpr9MefEHRNuVGfPibsOCn0CDiOS25Cj02U3UhOnBW2/pD9fnvjSMKxTRvbGUUO6e1Uh/b50NpQoTTpaATWTj0goxYhtBSBu0bPRdRrY5HrrOe+f/n568SAuR9nT1ILG5jyW1jtedfLzuvL+t/Dwmyuc4117mOmVEznZnz/p5gGTsyAEjzNssAyoCEMhokuIaB4R5YlogrS9joj+QkRziOg9IjpFc/5PiWg1Ec1y/88pF+0iBQkQHDjhF0RtlPcn6vXb93rnNbf4SRuHTJ6CN5dvxiZ3kIf1/IBGvxtReRVu/JP7zSn6l5EUyR1u32Twx0l7egYST0PgfoSN8u7n3S8twZTZa5XXIycCBJzgtzAIss1L6k6xQDBFHMNOUoPqcP+VR+NfXzo+sn3jjmjRN4binms8Rj7ipp1RM35xX5LHyoi+6npAxSAu7kUw//G/mOZO+PG2znB+sw07nBxbV97/ltNejMrWxNtR4OVF9QD8XGC/e/4Dt1+DkyuESkkocwFcCCCcJe+LAMDMYwGcDuA2ItLReDszj3P/n9Yckznk7KpyQJcw1pO0T7ZVrN66GwvX7/C8XESK8Ov+PQfDfvgMhkyegovvfk3bb2CMhiY3p9+wyku/zwS+FCTUUOrjwmK5Dso06OG2ChBRwl5VupV60LipXuntbmrBNX9/J1DWOdxPhNbQMX56m+gCwwThCTdHhCmz1+KFhRsUx0oScQaTjMqGZMIMwytn1TOOs4/9/apj8PtPObXZTxzeqyQM5ewx/fDvrxyPJb+Mrj1/+vFDve8rNjc46etjbmg4L93vP+l4oh1zkFvRUTN+V2/djZkfbsE7K7YY0fytM0YEfotg5UimAqPWyoOKMBRmns/MCxW7RgOY7h6zAcBWAPp8ChXALpehMHPA3/wzxw4G4L/YLXkO1JUQx76wsD7Q3iNvrfS+i8hvAHjmGycGO5YGaee20ZI0UaO8f8Ijb61QXIl/wpDJU/CZUNCZWNEJFZ3e0JhcC2JrQyNun7Yo1KuqLedTnVI8TLX4bfY6xbsNGzUR6C9sYxLt+aodf3sx2p0PNzWgsSWPK//ylmKvr47KYtV62iF98aWThwZcexkGWXFdiKPiCriJ+yIf0aNTnXdfOynGdhYgIhx5YHelpNKuTY0Xz9TUkk8M5hS0e6lXcoS+XdpK1xj/wF/9YJPRPRUSn0+n8zK2rwvb7qqHpVSbDeU9AOcRUS0RHQRgPIBBmmO/SkSziejPRKRNo0tEVxPRTCKaWV9frzvMGMIo/5vpi71ti35xNo4f2kt3CgA/wv6Ukb1x4vBeXrp71arwpgvHeinWTx4RzXl0/LCekW1xsQFq10vnCF09C/HeTRzlRKAP76NeNW7a1Yi/vb4C3390tnL/nqYWjLvhOb/dmJlP7ImThiLbDN8ldaR89LhJh/RVVpUMSwJqCYUkt+GIWd6IzjTMJ2BDyWidWpMLSXMGEoo4vnM7hxl88cSDI8f4zzZ6gQR9DrRyQRi5vfT/MQPLL8Xgb5NjTFSX0LltLZ762kdS0RR+pqP6dQlkGK9GlIyhENE0Ipqr+I+rpfJnAKsAzARwB4DXAKjcqu4CMBTAOABrAdyma5CZ72XmCcw8oXfvwhLSydi115mARcLAGddNRF1tLrIqO+vQfhjR15+YJgzujsE9O+D+K48OeO4M7d0JHxnWC5859kDv2BwR+nRuhxnXTcTks4OR4E9ecwL6dBaJ+vwBFx7+zIyerhfPLRcfBh0aNenexcR48fiBmP3TMzBSU+de4B8zVyq3yzanpBdKp2d/cMZy3P/acmzf05w4u2l3S02KxYCfOsM/S5dcMmxv8T/9c+UIZm8lnsIIGyITgF/nRh5L8rGCdJ33YVqE44qYYaxT6dS2Fm/+4DR8LeSJB0RtS1GbndnqvlQIOw3EXbIqjVCOCCJETVf4bcyArrjyhCGJ7YdpCm5TLGQM2ioXSsZQmHkSM49R/D8Zc04zM1/r2kXOA9ANwGLFceuZuYWZ8wD+CODoUl1HGLubnBf3FFdyECubQd2dmhziZVKpoEQuKNm/XIjXgdWI+7V/1/aeiC7Xk1ch2p8TLd2pba2ygp1MlwqDevg1RsI5rGS8/N1TtfsAf4K84bxDlbUnZOhMKD9+cp7+nFA/OogJPp9nL2WG3gVcpbJR0ypj3fY9UqCl08ZFd72G+Wu3Y0tDY/QEg77//sVjccbovpoM0b7bsElpahOobGJJ0o9Mcp8u7ZReXnHOHURqVWE5IegTds9YG0o+eA4gnCfEOx09J86+ZIpZK7dig1ssL9h2wU1mjqpSeRFRByLq6H4/HUAzM0dS4xKRrMO5AI6RvywQEkrY+6Vj21osv+lcfOzwA7xtYcciOQdXXlqpye6mYr8O8grlgw07Ivs/f8JBXrsmi70wQ3n/hjOx/KZz9XUlQjiwZwecNqoPxgxQx3AkGctl+JOxev+FUh4tbRtSN10k5nvsr6YDCF6vMpeShsyIygvB30DQO2nNNufFf2eFk7vq7Q/NDLHhS2/XpgZ1tTl1bI7U/wS3iFPRCKtOE+wJDh1qD7xgs0HJLQx/oq0QRxHOJy6ziLsWT0KROEogo3LMgiTN3C/TIBjdgnXRd76aUCm34QuIaBWA4wBMIaKp7q4+AN5xMxx/H8Dl0jn3SS7GN7uuxbMBnAonLUxZsH2PU3sgSbeskhggrVLESyjXG/fPjbasqie/fJMfnCikCBF0ZSK6A9HXtxBdPJE+kp4N6fAagj7u4vBB3aIeLjFv/uyfnok3fnBakB738xunDcfoAzRMUE8aoGAkAscN7ekVjrrpmWCFh7EJ0lkcdDVn5DGY1SI1LGGZaLzOGdMfhx7QBV86eaj2mLDDhXw5JO2vuISieM/05wRVXp4UrLgG2YAPmEkV8liPLbJWRRJKaVwqEsDMjwN4XLF9OYCRmnOukr5frjqmHNi22y9mk7TqDho3fcYhT8AiFbvckmowi5QvzdJoffDzR+Oel5fgt5cdIQ1UX1f9yuJ67GnWJ4hm5kjt80IQjhQO9OEdk9xOcuoV/bm6F65vl3Y4fmhPz1YkGK0uwp4AJUfxJm4StKp02YRLJwzCXS8u8VxI27epwYQh3fHLC8bixJtf0F+Adx3+d5FHLUfqdDksE5QRZDdsEQOT1EX3jnWY8vUTY4/Z0uC8N/PWbMcRB3b3YiwETD2kSgXx3rS4L2bcu62yoRA522WVarB99zMVTf73SsltaVERhtJaUVeTwzb3xUgytUZqGXAw+lzscYKoKDCAVWP5+nMPwS+mzMfA7n7KhZNG9MZJIS8wMSnvbW6JrV2eZxjFW5jAMQmp74fvomui8vKZoa6taDyI83n4oG54YWF9oLKggJzyIk2A5V6ZGYtnF+o32I+TKRfwvfcYjEP6d8GgHh3w+nWn4aSbX8BwhYFdQIyrb5w2HN84bbhHvzodDUv0JF/Qg58/2qNPB+8ZSPRk4UE2wM2XtrR+J8b8ZGpgH0krqkpLKLsbHYYSFwi5cadjD9stLcaEFHnwD9QhcRTiKEbvg8xQYu5LVh5+WaCqbCjVjm4d2mDr7ibHcyfJnTKi8grGDHgGvLxiklS0fNWJB3uZW2O7dRsTRZpO1xSe0rkLF4K4tNwm+nUB3yifflb52sThePabJ+LQA6KqJbF6BPwXM64kLAPYsH0PRl7/rL9dQ6uMnGQLk9Xp4tB+XdvhpBG9EvO4AY4EJaTOcDbbJDpu1nj1nTSiN4ZIyQ1VCBvPTSQUE9TVOo38c6Y6o7Pe3bo8EP0LR41wWhoZor5Lt/a+DS6cwv6CkL3PV00630ycKEwZRSeNo04lYBlKCnTr0AYtecaOvc3KlBRhBFVewehz2Wc9YpQv4gUWK8HtblTt4QPVuntdH4X0ncvFSxWAmU7a198nHyPQr4sjkdTkCKP6qW0icsqLJNuS2D5vbbDe99L6XQD8VenKzbu9tv1+ZJVj8PmqruCFhRswZPIU73+FZBMLxjjoE2aqbCg1RQygsLeVyTgvBIt+cTY6egF6lY9D8SUkh4C4e3juWMcn6KLxA71tjtswY8Lg7jh+aM+I+3k4dY8RSQGVV/TGDO3tLA4mDNaG4ZUd1cPaqhhicujW3onr2NbQlJhALrwnL7398gRsapQ3xegDuqAmR5g4qg+ee3+99rhv/fM95faCjPLQSyj+BG4u4pvMKW1rc9jbnMcXPnKQAX3BCR6Id99k5sDqEwDOcQsrbdgezHf1kpT5wHcNDcahhLtiOPE54ej3u176AA+/6cTzzFiyCV85ZZhHq9rLy49iz2rSF+3t3NuM595d547TLBr32xg7oGu0MJrwsqqwhNKicAkO485PH4k7FefnWXhzkpdRw9/vfIpmzxt3AJKQdNenfvMk1OSoqPkia1gJJQWEXnyra0dJ40LlSCjiNH+CyLuxBPKEW+zwGNLT15OHB9t3QvmBwijIhkJ6JuDND0YqL3O1R11NDrU5Qq2Be3NOoi+pHkV4hf7Hz07AOz863XPLvfToYOKGa071PZvE8w168QUvnVx157KNuyJ9j5Tcjo89uKd0jjqFvU7tmsX8cuTPn8P3H5uDjTv3lsyLSEy6LXmuuIQiLvH7jzkZH9LGiziLRPbiyu64dFxgf4Mr2YpmTUocJ9lQamtyVcVMAMtQUsFjKLsbk43yFK62xgGxVzYS53LBwVPsGJFVPGF8deJwfH3iMPXOAqFzaw3QZNCOkcrLvTk79jYHPN6S6AvbUOJexMbmvMfU2tbmAnVDwkGegRLA0vPVqYxU1zjrx6cDcM7p2r4NPnn0gbhGqgyoKwMs2zcCC5Iixo/q3Ezkk5jx3b1jG/R1nSl0rtylhng3d4rMDikveu7q7Zi+YAPeXbEVO/Y0oyZHGC6l8BGLUFX8kh7VxSxMYFVeKdDNrdC2paEJSDDKR1Ve0ssf0ukH5ZOsvDb0k+23zhiJ37qpsMMopGfZ6B2hwmAC99sREor+mDjvm1j68oKeeBvKqx9sxMadjdjgpnJPs1LNKZ6vo9KMb0OuOcPMSv272m04Gw+scF+AYyeYMmetS1+mXXhY9qtz0NTCqKvNoU9n4N9fOR6HD+xWms4SEB5WaSWUXp3qPO8vUZ5XNYw9JxWDNoU9tDXBSigp0NXVq29raDQzygfchv2JRV5xilVmphKKwTG/uWyccnshInSchCJeoFRG+Rhm2LW9Pg2Mtl1JWgynHhf49UVj8fy3T8YpI/sAgKeSSnM75MJnchqOsMoLCF6j7FmlGlfLNu7y4jhkMMNrPCgFFWGUdz/lBUIWTIsU34koYEvRZQMuCyIMJd3psmddmBn17FiHq1xb3/2vLgcAzF0TdPpQoX1dDc48tC9G9etcOWeFlLAMJQXEZBYWX1UITwqOhCIYCgVUIuEBmMU7laiC0rnNFtAXQS+h+JUsDdox1KOLUr2ihoYJfT4Ddxlc6CZfetSBOLh3J8/dUxRdKsjrLaTuDDcSvj45MaLK2eP/Ptio7EfiJ5lj1sqtEn0l6qSKEH4H016z/D6FDfBfOXUYrv/oaADANa66OZ9CXauSTqsVlqGkQF1tDh3rarBpV6OR4TjoNiwFoQFBozzpXUsLgWyj0b0YOqZVmFE+RkJJ4eVlWndctHlQQkyF3K6vYoRLjxri+oV9Jp3KS7Kh5FmpXiM4Bnv5fskGaZVX2JdOHoo2NQo6JDWqHGTXVESiSMFI1m7zkxDuB/wkco1pVV5BKZS0+8RiyLT5Z+auw+INO72UT9UOy1BS4qDeHbGkfmdiwJezKvZ/O5KIv3PVlt147YONXnyKSi1SSugm+MJUXjGR8l67BjS5x2TtOirbeHTVGgV891FzXbd/rt+GqKshtylo0fUp3E7DhxCpn5dsQxkjBXQWs6I9cXi0xIO4V7rU/iYI3O8qFHnCEmva9yCQ1ysX3qc+zgTHD3W8/R59Wx0QWm2wDCUlRvXrghlLNiWqG8ID0pFEnG1iFfip+97Ais0NYGbsklKpd6iLj4Y3QZIElaWqWvZqitKRvj3Tc0x1+2ovL/WxQofvSSgpbpTs5cVgrYsys1oK81ReSmEk3surfV0NLp2gq0Vnjk8erW9j2rdOxquTJxbdRzVi445gfFHa10NmFCIo0l9MSQuKlO3+5GNOeeJbpqoK3FYfLENJiQ51NWh2y/uGg5fCkCeBuau3e0W5hocqAj4xaw3OGN3P+61aJaaBo1bxvyuPyZChxJcBNlcd+S9e9hKK7KYdR49fF8NNEpiiH1l3LjMNlfQpS2EyLbJ7uXeOpr+wAb+QlDVhOFUnw9ucz0E9OhTseVR9MkkQwhlDILXKK0YKCddNKQTj3Wj4x758XGENlAmWoaTEJwxXgfK42bk3WMjprk+Px+j+vr/9X79wdCTJY7Eo1ChfCDbu3Iv12/fi2blrI/tE/RgjlZf7mbVHiyr1it6GJFRewd+m/QAi9xZrpaHw5Xmp2/PsljmItq26J3KBrSwRZWjZ9lGNzKV9XQ1uveRw73daCT5OoxeUUApjVMKGNrinmd2wUrAMJSWSkuvJEJNAS4vz5Ueup0cuR+jY1lFrdW5bW7REEoa8ItchPGn8/LxD8cw34lOQ6yByaH3pb+94256ZsxZDJk/BeXe+CgDo1altYjuCptNvf7kgOvTtym688U4CgoY/v7rMOa4QJwUIe0jUXiOyUMsMIlB8SuHlpaPBxHW9EETtCdn3Ue1I7eUlPbM4j7FC72VrcRuuWGAjEd0C4GMAGgEsAXAlM291910H4AsAWgB8nZmnKs4/CMAjAHoAeAfA5cxsVmc1I9xz+Xj9TpLydSliMdJFzBYOEy+vH390NC4/bkjBfXzr9BH4/QvBQMkvP/RO4LeoDxKHfl19pjNk8pTI/oYCMySLCR5wAhe9jQqEkwKKOippIFSAze5CYu223UFiIsc7n+z+qW0oim0lmmQiKq8M2mxtTCm9UV7+7r7bod/yNmM63E+V+rQaUUkJ5TkAY5j5MACLAFwHAEQ0GsBlAA4FcBaAPxCRykr9awC3M/NwAFvgMKCSQ36gItun+jj/SD+YLjocSpWLJ9ko7/R7ysje+LxBgsXYtnKE8YO744Rhfv4pOS/VK9871eg6xw/uoQ24BIDBPeNreejw6pJNWLG5Acs37sL3H5sDAN5kH0aYzO4d6pTHyTg/lOhPOCnMdzMWT5ntqwJ3N7ZgSf0uvODa05zjydunys+lk6YYKEqdokOxMRlJaA3MJbVRXuIoXjyV2BcwohVIUCuRUCrGUJj5v8wsjAuvAxC5oM8D8Agz72XmZQA+AHC0fC45b9FEAI+6mx4AcH7paFVvN2UGSm8f93upA4O1zXvG4dL007NTHY4a0h3LbzoXgxKKOsk4b9wA3BKq5zG0d0e8OnkiznHThqdFvevBc8qtL3rbdN5M8mQ68/pJGNmvs/I4Gbd9Yhzm/exM77djs2G0a+Osg268YKy3TzhmhCU6AHhxYT3ymlQtygWCFNsEZGOUB1QMpfhBWk1FoExQTBxKnA2qUBuKP4dU932sFhvK5wE8434fAGCltG+Vu01GTwBbJYakOiZzEOK9OQLHyt43nnE2enzagWuK15ZscvrW7C91QaN8EQZj+T7N+9mZmP7tU4rKa/TWDyd53/t0bovLjhqkj0OR3ggTuw/guBrLBZna5AiNzf59rZVWDecepmaK3Tq0cYz5UBl1dXmhSmRDKbHKq1qnRPldMEgGHEAgUj4X3ud/T7+AdN/TtKdVCCVlKEQ0jYjmKv7Pk475IYBmAA+JTYqmwvfT5BjR/tVENJOIZtbX16sOKRhJYyOc7kMl+ZZiQliwbocX3LZgrTpnULPrxjR/7Y7M+g0EciYEfsYhbrVXCHp3botTR/bG2AFdEyfhYopTCdTW5Dy3YyA42dz5qSOx7Ffn4O9XHRM458IjBmLZxp0alZcauvT1xSJslH9uvr6ujinkBcGHUiGxasWi9fry2SrIw2bu6u2hfaT8ngZJSU2rBSU1yjPzpLj9RHQFgI8COI395cEqALI+YiCANaFTNwLoRkS1rpSiOkbQcC+AewFgwoQJRTP6OG+O4HFS7EPM8aUWYYVnWRijXLflb5yWTSp7VfBe2lWegHye6vYM6tEBC9btUKcj0dInpzzRn2dSmjW5r/gVJRFF0rS3bZPzI9yVKq9oO3KBrXD/xUCM0x4d67B5VyOuPKE4GxvgMKmJo/rg+QUbsGlXWX1nCsJHNZKkDvK7/bnjh4T2+d/FYXIgcxzC9phqRyW9vM4C8H0AJzOzvGT5D4C/E9H/ADgAwHAAb8rnMjMT0QsALobj6XUFgCfLQriEuBdX3hcX+1AKdnLtJKeI1jcmDdceM6Bbeyy/6dwS9O7ACdArjKMk1fb42ccPxfFDe2JYKEA0vk0xKUcrZMoYkoGfv8gdFlvXJfTk5SqPYfpEM43N+UB23pJJKG6j/bu2wzs/Oj2zdgWtkw7pm1mbpcBFRw7EwO7pHEDkZ/bTjx8a2ufvbHK9BkUJaVMkZXioFlSyHsrvAbQF8Jy7ynqdmb/EzPOI6J8A3oejCruGmVsAgIieBnAVM6/B/2/v3IOtqu47/vlygQsIAheulqeAog0mKSJYFIcYRERIpUmdgo2DxmZSWp0+HCaBmk7iVIkmnWgzfYhtUNsaSGKtdUjim8Q2E3lYEFG5ikB4KAKDIAZ1UFb/WGvfu+/hPPa+Z597Hvw+M3vOPmuvvfb67XXO/u211m/9fl4ZrZJ0O7AR+H53C1CqcU9ySJjnTaUScyjFFEkliT9AT7iuy1Zqnmr4oL6p35qjxY0nSgzFxedCCnHZea3MPv+3Ch4Xue7fT+akcXYU8x/W+YwozsZnvrOGXy29nH/6+Ta+/bh3xfFyAjfoaYmul711ly9wcL/0IQi6g3J6AfnmC48Gh47x+3jx2UO45sKRLPrMuITldq5brRs3VE2hOOcKjrU45+4A7siTPie2v50c66/uIPGkfKzhXRELjWqFf8ia3B965EW5/HKzwZvyukxWlz/wpRI/uxJDXgAD+vRiypjBXDHBv63Hfwcv7jncKe+N08awR+CSCQAADclJREFUct0uPjj+MbsPHWtXJrk8vmUfANvzhBfuCvuOfFg6Uwqi30Otv2V3aTFrnnPeftffv5f2HGHexBEhnzqtyE/Ki7sPl85UA9SKlVddUrKHEosrDrmT8gpl1Pi/q4v4SfmuydbJc2tG9yf+plfxxaThQqXMeH+86BK+Mt3HpN97uMNd/PwpnU2ax585gOED+zDzE2fyxoHOk8Vtt89u34+CPM3tonl1xN/O80M2B9/LWKFkWlr2KOczDUc/OHlOZPEsP/Sc255pOMnzdo3fRAsBXAalzIY7wsAWzt9I+iT+AI3Hf0lLFq4q8pUZzWtU+pZHBgDxa5fiN8Hf2y1XnMuVeYbTJO8Sf0Af/5ddccNkpo9vpWdTxzvh7E8Oy2Re7LqpZ7Gm7UDRYb2uUOu/9asnDmfj7sMsnnVe6nPzzefdPGM8N88ob/h57ND+/NHvjuYHa3cBpDJEqQamUMog9aR8HuulSq1D6XZyxPBzKF0tqnwzy3xleiuv/JZRWdJhAJCeXE/U7WUqconvv/dq6tFJmWSJJFbcMCXzcjtcktTmb765ZxPLYotQ09A6oJn1t87kyPvZBsJq6iGWff5TfPnSsew6dIx+vWv7kV3btatxSv0xomdKMYeEjaJPIDegWP4V30moxD3p1EOp8D1PYuWVS6nhsdwy6/FFpA6rnIrWAc20Dki2GDYt41r7M641uVVjtbA5lAREY8m54+/F38Bjk/JRSp789fhgyEeuFCdOZLOwMSuiOO/eO3yFeyjKsfJKcblCeaMyCwXtqge6yyGqUT1MoSRgSXAo+NxrnVfalxo66ZhDidahdOSPusaN+t/KdVyYhooMScWtvLrhpkfKK3H+EpmjHkop9/s1TR1W2UiHKZQEtL3t3ZOs3vxWzkr5wudIvmfjnMu7DiVaP5CViWctEH8mOld8AWExKqRPguVVN03Kx+OdpLpi/rzta1sKLH6sB9rnUOqw7kYybA4lJZ0tkAr/M1at81YZY5f+lAXBbDDeQ7nzC59iySMvcXqfjiZ49KZp7DuSbgVtrSDB2u2HGLPkJ/zJ9HHlOYfMuG4QWV51zxyKLz5bZxmRO5eOl5P6eyrXX42NtJhCScE3fm8CvWKWNcVM+KaMaWHtjkMArFrvnScPiCmPBReNZn6O19uJowbBqEFZV7tbeH77ofb95c9tB/KbUiahEg9Lb3nlCrqHz/Ra7QYAyZVKsXk2iIa8XN5gbfVCHepAIyWmUFKwMEQ1fO32q9h+8L2iJnwP3ngR63YcYuGKDjdk084e2ilPPb5lpuGZV/eXzpSHykzKx+Y1usPKK56Q4Hr5Fr92KkLe0KGI/8iaR3n2jMbCFEoKmsJrYe+ePdrjqBeiT68mpp/bWlEHjLXE9mVzOPz+cVpO683P2/Zzw/3r+bAL4XOh616Ki+EXBrq8Mduzv5af74hcpOxK5K49gdlwWEfjr1F/D+VGsWg0CmMKxciEHj1Ey2k+XO5l553B3y+YyKTRg7tU1pmn98myaoB/J448vL785pGief/nq5/lozJCWQoffTGi0GLFvOcWeeieqPN1KJEer8eqG8kwhWJUhMgZXlc4f/hARgzqy++MGphZfR7ZuLd9/yT/SDmkCVmcjw+Od+6ZXXLO0AI5k7N131G27jvKU6/4YFf1+EyuS1NnIxWmUIya5JdLZlSs7Huvu7BiZUOHmfnnLxjB3fMnJjonrauWeuyh1GGVjZSYQknA4lnnsuKXO6tdDaMMdnxrDh+fcBXzfxUnCpb13T9M7qZ8814/DHf4WP5ohhu+PpOpy55pH4rr19xUfkW7mUi2ve/Up2m8UZqqLGyU9B1JWyVtlvRfkgbFji2VtE1Sm6QrC5z/gKQdkjaFLdlrYBe5ecb4TCPXGd2PpG5RJgDbvzWXnXfOTTVx/i8LJwMw+5P5PfwO7d/MtmVzuG7qaC45ewhn14Ffp1yeDlZ/v8jxOGE0DkpjK5/ZRaVZwLPOuY8k3QXgnPuapAnASnzgrOHA08C5UcTG2PkPAKudcw+nue7kyZPdhg0bshDBMIyU7HnnGJfetYanb5nOOWcMqHZ1jBRIesE5N7lUvqr0UJxzTzrnoog0zwMjw/48YJVz7kPn3A5gG1WIymgYRvaMHNyPnXfONWXSwNSCL68bgZ+F/RHA7tixPSEtH3eEIbO7JVXGZ7RhGIaRmIopFElPS9qSZ5sXy3Mr8BHwUJSUp6h8Y3JLgd8GpgAtwNeK1OMrkjZI2nDggI3dGoZhVIqKWXk552YWOy7peuBzwOWuYyJnDxAPwDwSeDNP2W+F3Q8l3Q8sLlKP+4D7wM+hJBbAMAzDSEW1rLxm43sVVzvn4n4pHgMWSGqWNBYYD6zLc/6w8Cng94Etla+1YRiGUYxqrUP5B6AZeCqYVj7vnFvknHtZ0o+AV/BDYTdFFl6Sfgp82Tn3JvCQpFb8ENkmYFE1hDAMwzA6qIrZcLUws2HDMIz01LTZsGEYhtF4mEIxDMMwMuGUGvKSdAD4dRdPHwoczLA69YDJfGpgMp8alCPzWc651lKZTimFUg6SNiQZQ2wkTOZTA5P51KA7ZLYhL8MwDCMTTKEYhmEYmWAKJTn3VbsCVcBkPjUwmU8NKi6zzaEYhmEYmWA9FMMwDCMTTKEkQNLsEEFym6Ql1a5PV5E0StIaSa9KelnSX4T0FklPSXo9fA4O6ZL0vSD3ZkmTYmVdH/K/Hhx91jSSmiRtlLQ6fB8raW2o/w8l9Q7pzeH7tnB8TKyMktFEawVJgyQ9HCKjvirp4kZvZ0l/FX7XWyStlNSn0dpZ0gpJ+yVtiaVl1q6SLpT0Ujjne8FfYnKcc7YV2YAm4A1gHNAbeBGYUO16dVGWYcCksD8AeA2YAHwbWBLSlwB3hf05+Fg1AqYCa0N6C7A9fA4O+4OrLV8J2W8BfoCP9AnwI2BB2L8X+NOw/2fAvWF/AfDDsD8htH0zMDb8JpqqLVcReR/E+74j/G4HNXI74+Mm7QD6xtr3hkZrZ2A6MAnYEkvLrF3xzngvDuf8DLgqVf2qfYNqfQs394nY96XA0mrXKyPZ/hu4AmgDhoW0YUBb2F8OXBvL3xaOXwssj6V3yldrGz4MwjPADGB1+LMcBHrmtjHwBHBx2O8Z8im33eP5am0DTg8PV+WkN2w70xGcryW022rgykZsZ2BMjkLJpF3Dsa2x9E75kmw25FWaNFEk64bQxb8AWAuc6UKMmfB5RshWSPZ6uyf3AF8FToTvQ4DDriMMdbz+7bKF40dC/nqSeRxwALg/DPP9q6TTaOB2ds7tBf4O2AW8hW+3F2jsdo7Iql1HhP3c9MSYQilN0iiSdYOk/sB/An/pnHu3WNY8aa5Ies0h6XPAfufcC/HkPFldiWN1IzP+jXsS8M/OuQuA3+CHQgpR9zKHeYN5+GGq4cBpwFV5sjZSO5cirYxly24KpTSJokjWC5J64ZXJQ865R0Ly2+oIWjYM2B/SC8leT/dkGnC1pJ3AKvyw1z3AIElRPKB4/dtlC8cHAoeoL5n3AHucc2vD94fxCqaR23kmsMM5d8A5dxx4BLiExm7niKzadU/Yz01PjCmU0qwHxgdrkd74CbzHqlynLhEsNr4PvOqc+27s0GNAZOlxPX5uJUpfGKxFpgJHQpf6CWCWpMHhzXBWSKs5nHNLnXMjnXNj8G33rHPui8Aa4JqQLVfm6F5cE/I7EkYTrQWcc/uA3ZLOC0mX44PWNWw744e6pkrqF37nkcwN284xMmnXcOyopKnhHi6MlZWMak8w1cOGt5Z4DW/xcWu161OGHJfiu7Cb8ZEuNwXZhuAnrV8Pny0hv4B/DHK/BEyOlXUjsC1sX6q2bAnlv4wOK69x+AfFNuDHQHNI7xO+bwvHx8XOvzXcizZSWr9UQdaJwIbQ1o/irXkaup2B24Ct+JDg/4631GqodgZW4ueIjuN7FH+cZbsCk8P9ewMfWVdp6mcr5Q3DMIxMsCEvwzAMIxNMoRiGYRiZYArFMAzDyARTKIZhGEYmmEIxDMMwMsEUimGUgaSPJW2KbUW9UUtaJGlhBtfdKWloueUYRpaY2bBhlIGk95xz/atw3Z34dQUHu/vahlEI66EYRgUIPYi7JK0L2zkh/ZuSFof9P5f0SohVsSqktUh6NKQ9L+nTIX2IpCeDs8flxPwuSbouXGOTpOWSmqogsmGYQjGMMumbM+Q1P3bsXefcRfgVx/fkOXcJcIFz7tPAopB2G7AxpP018G8h/RvA/zrv7PExYDSApE8A84FpzrmJwMfAF7MV0TCS0bN0FsMwivB+eJDnY2Xs8+48xzcDD0l6FO8eBbx7nD8AcM49G3omA/GBlb4Q0n8i6Z2Q/3LgQmB9CK7Xlw7ngIbRrZhCMYzK4QrsR8zFK4qrgb+RdD7FXYjnK0PAg865peVU1DCywIa8DKNyzI99/ip+QFIPYJRzbg0++NcgoD/wHGHIStJlwEHnY9bE06/CO3sE7wzwGklnhGMtks6qoEyGURDroRhGefSVtCn2/XHnXGQ63CxpLf7F7dqc85qA/wjDWQLuds4dlvRNfKTFzcAxOtyS3waslPR/wC/w7tpxzr0i6evAk0FJHQduAn6dtaCGUQozGzaMCmBmvcapiA15GYZhGJlgPRTDMAwjE6yHYhiGYWSCKRTDMAwjE0yhGIZhGJlgCsUwDMPIBFMohmEYRiaYQjEMwzAy4f8BamYdUU7HVtEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(all_avg_rewards)\n",
    "plt.xlabel('Episode')\n",
    "plt.ylabel('Avg rewards (100)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEKCAYAAADaa8itAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XecVNX9//HXh6UrsHQQWAHpRYobwBKDYkE0YowNG1EMmug3GhMVEhNr8osaG8ag2CtgF42KiGCnLNL70hekdxaWLZ/fH3NIJmRhB5jZ2fJ+Ph7z2LnnnjvzuVzgs+eec88xd0dERCQeKiQ7ABERKTuUVEREJG6UVEREJG6UVEREJG6UVEREJG6UVEREJG6UVEREJG6UVEREJG6UVEREJG4qJjuA4lavXj1v3rx5ssMQESlVpk2bttHd6xdVr9wllebNm5ORkZHsMEREShUzWxFLPd3+EhGRuFFSERGRuFFSERGRuFFSERGRuFFSERGRuFFSERGRuFFSERGRuFFSEREp45Zu2MlDYxeQl1+Q8O9KeFIxsxQzm25mH4btFmY22cwWm9loM6scyquE7cywv3nUZwwN5QvN7Oyo8r6hLNPMhiT6XERESpuZq7Yy4JlJjJyyio079yb8+4qjpXIzMD9q+wHgUXdvDWwBBoXyQcAWd28FPBrqYWYdgMuAjkBf4J8hUaUATwLnAB2AAaGuiEi5t3tvPn/51zwueupbKlaowOu/7EmjWlUT/r0JTSpm1hQ4F3g2bBtwOvBWqPIScEF43z9sE/b3CfX7A6PcPcfdlwGZQI/wynT3pe6+FxgV6oqIlGsbd+ZwydPf8cxXy+jftQkf/N8ptGtUs1i+O9Fzfz0G3A7UCNt1ga3unhe2s4Am4X0TYBWAu+eZ2bZQvwkwKeozo49ZtV95z3ifgIhIaTJj1VZuGTWdtdv38NSV3enbqXGxfn/CkoqZnQesd/dpZtZ7X3EhVb2IfQcqL6yV5YWUYWaDgcEAaWlpB4laRKR02pmTx7Dxi3n+62XUPboyrw7qSXrzOsUeRyJbKicD55tZP6AqUJNIyyXVzCqG1kpTYE2onwU0A7LMrCJQC9gcVb5P9DEHKv8v7j4CGAGQnp5eaOIRESmtMpZv5rdvzGDV5t1c2L0Jd5/fkZpVKyUlloT1qbj7UHdv6u7NiXS0f+7uVwATgItCtYHA++H9mLBN2P+5u3sovyyMDmsBtAamAFOB1mE0WeXwHWMSdT4iIiVNQYHz6LhFXPL0d7jDmzecyCOXdE1aQoHkrKdyBzDKzO4HpgPPhfLngFfMLJNIC+UyAHefa2ZvAPOAPOBGd88HMLObgLFACvC8u88t1jMREUmSbbtzuf2tmYydu44LuzXhnv4dqZHEZLKPRRoD5Ud6erprkS4RKc3mrtnGr1/7ntVbdjPknHYMOqUFkcGyiWNm09w9vah65W7lRxGR0srdGT11FX8eM5c61Ssz+vpenHBs8XfGH4ySiohIKbAnN58/vjuHt7/P4set6/HYpV2pe3SVZIf1P5RURERKuE07c7j+lWlMW7mFm/u05jd9WpNSIbG3uw6XkoqISAnl7nw46wfuGjOXnTl5PHl5d/p1Lt6HGQ+VkoqISAm0bvse7h4zl4/nrKXjMTV55JKutG1Uo+gDk0xJRUSkhHlv+mr+9N4cdufmc0ffdgw+tWWJvd21PyUVEZESwt157LPFPD5+MenH1uahi7vQot5RyQ7rkCipiIiUALty8rjj7Vl8OOsHLjqhKf/vws5USil96ygqqYiIJNmmnTlc++JU5qzZzm1nt+XXvY9L+MOMiaKkIiKSRF8v3sgdb89i484cnr7yBM7o0DDZIR2R0te2EhEpI16dtIKBL0yhaqUKjBrcq9QnFFBLRUSk2Lk7/5y4hIfGLuS0tvUZNqBbiZgMMh6UVEREitGe3HyGvjObd6ev5uyODXn8sm5UrZSS7LDiRklFRKSYrNyUza9em8bcNdu59cw2/N/prUpth/yBKKmIiBSDcfPWcesbMzDg2avTy0T/SWGUVEREEigvv4CHPl3I018spVOTmgy/4gSa1ame7LASJmGjv8ysqplNMbOZZjbXzO4J5S+a2TIzmxFeXUO5mdkwM8s0s1lm1j3qswaa2eLwGhhVfoKZzQ7HDLOy1o4UkVLth227GfjCFJ7+YilX9EzjrRtOKtMJBRLbUskBTnf3nWZWCfjazD4O+25z97f2q38OkfXnWwM9geFATzOrA9wFpAMOTDOzMe6+JdQZDEwCPgL6Ah8jIpJkExeuZ8jbs9m6ey8P/vx4LvlRs2SHVCwSllQ8sk7xzrBZKbwOtnZxf+DlcNwkM0s1s8ZAb2Ccu28GMLNxQF8zmwjUdPfvQvnLwAUoqYhIEmVtyeb+D+fzydy1HFOrKu/++mTaN66Z7LCKTUIffjSzFDObAawnkhgmh11/Cbe4HjWzfUuXNQFWRR2eFcoOVp5VSHlhcQw2swwzy9iwYcMRn5eIyP5y8vJ5ZNwi+jz8BV8s2sDvzmzD57/vXa4SCiS4o97d84GuZpYKvGtmnYChwFqgMjACuAO4FyisP8QPo7ywOEaE7yI9Pf1grSURkUO2Zutubnr9e75fuZXzjm/MH/q155jUaskOKymKZfSXu28Nt6v6uvvfQ3GOmb0A/D5sZwHRNx2bAmtCee/9yieG8qaF1BcRKRb5Bc7rk1fwwCcLyS9w/nF5N847/phkh5VUiRz9VT+0UDCzasAZwILQT0IYqXUBMCccMga4OowC6wVsc/cfgLHAWWZW28xqA2cBY8O+HWbWK3zW1cD7iTofEZFom3ftZdBLU/nT+3M5vmktxt5yarlPKJDYlkpj4CUzSyGSvN5w9w/N7HMzq0/k9tUM4IZQ/yOgH5AJZAPXALj7ZjO7D5ga6t27r9Me+BXwIlCNSAe9OulFJOE+mv0D93wwl0079/Ln8zpwzcnNy9yT8YfLIoOtyo/09HTPyMhIdhgiUgpl783jL/+az2uTV9KuUQ3+fnEXOjWpleywioWZTXP39KLq6Yl6EZEYrNyUzTUvTmHpxl0MPrUlt53dtlSuzJhoSioiIkUYP38dv39zJg68NqgnJ7Wql+yQSiwlFRGRA4iet6tD45r84/JutKx/dLLDKtGUVERECrFw7Q5ue2sms7K2MaBHGnef34EqFcvOuieJoqQiIhIlN7+ApyYuYdjni6lRtRJPXt6dc49vnOywSg0lFRGRYM7qbdzx9izmrtnOecc35p7zO1L36CpFHyj/pqQiIuXevjXjH/9sMTWrVeKpK7vTt5NaJ4dDSUVEyrWcvHyGvB1ZM75f50b85YLO1D6qcrLDKrWUVESk3Mpcv5PfvTGDmVnb+P1ZbbjxtLK3ZnxxU1IRkXInN7+AV75bwQOfLKB65RTd7oojJRURKVc27szhhlemkbFiC6e3a8DfLuxMg5pVkx1WmaGkIiLlxsxVW/n1a9+zaVcOj17ahQu6NtHtrjhTUhGRMm9Pbj4PfrKQF79dRqOaVXnz+pPo3LR8TARZ3JRURKRMW7U5m5tHTWf6qq1c3iON2/u2o1a1SskOq8xSUhGRMmviwvX83+vTyStwnry8O/06qzM+0RK58mNVM5tiZjPNbK6Z3RPKW5jZZDNbbGajzaxyKK8StjPD/uZRnzU0lC80s7OjyvuGskwzG5KocxGR0mVvXgFPTsjkupcySKtbnU9/e6oSSjFJ5GIAOcDp7t4F6Ar0DcsEPwA86u6tgS3AoFB/ELDF3VsBj4Z6mFkH4DKgI9AX+KeZpYQVJZ8EzgE6AANCXREpx5Zu2MnPh3/LQ2MXcmaHhrz+y140q1M92WGVGwlLKh6xM2xWCi8HTgfeCuUvEVmnHqB/2Cbs7xPWnu8PjHL3HHdfRmS54R7hlenuS919LzAq1BWRcsjdGT11Jec98TWrtmTz2KVdGX7lCeo/KWYJ7VMJrYlpQCsirYolwFZ3zwtVsoAm4X0TYBWAu+eZ2TagbiifFPWx0ces2q+8ZwJOQ0RKuFWbs7nng3l8Nn8dJx1Xl4cv6ULjWtWSHVa5lNCk4u75QFczSwXeBdoXVi38LGywuB+kvLBWlhdShpkNBgYDpKWlFRG1iJQmY2au4c53Z5OTV8Cd57bn2pNbUKGCnj1JlmIZ/eXuW81sItALSDWziqG10hRYE6plAc2ALDOrCNQCNkeV7xN9zIHK9//+EcAIgPT09EITj4iULnty87nng7mMnLKKrs1SeezSrjSvd1Sywyr3Ejn6q35ooWBm1YAzgPnABOCiUG0g8H54PyZsE/Z/7u4eyi8Lo8NaAK2BKcBUoHUYTVaZSGf+mESdj4iUHEs37OSCJ79h5JRV/Kr3cbx1w4lKKCVEIlsqjYGXQr9KBeANd//QzOYBo8zsfmA68Fyo/xzwipllEmmhXAbg7nPN7A1gHpAH3Bhuq2FmNwFjgRTgeXefm8DzEZESYMzMNfzhndlUTDFeuOZHnNa2QbJDkigWaQyUH+np6Z6RkZHsMETkEG3etZf7P5zHO9NX0z0tlX9c3p1jUtUZX1zMbJq7pxdVT0/Ui0iJN3X5Zn75cgbbd+dyc5/W3HR6KyqlJPIxOzlcSioiUqK9kbGKO9+bQ+NaVRk9+ETaNqqR7JDkIJRURKREcneGjc/k0c8W8ePW9fj7xV1oqHVPSjwlFREpcVZv3c2to2cwedlmzj2+MY9e0pXKFXW7qzQo8iqZ2YNmVtPMKpnZeDPbaGZXFkdwIlL+TF66if7/+JrZq7dx3wWdGHZZNyWUUiSWK3WWu28HziPyIGIb4LaERiUi5c6unDyGvD2LS0dMokrFFD74v1O4qtexpOjp+FIllttf+2Zj6weMdPfNWn5TROJp3prt3Pj69yzftIsbfnIcv+nTiuqVdXe+NIrlqn1gZguA3cCvzaw+sCexYYlIeZBf4Lz47XIe+HgBtapX4vXrenHicXWTHZYcgSKTirsPMbMHgO3unm9m2WiKeRE5Qut37OGWUTP4dskmzmjfgAd+fjx1j66S7LDkCBWZVMysOnAjkEZkpt9jgLbAh4kNTUTKqqnLN/ObkdPZmp3LX3/WmQE9mqHb6mVDLLe/XiCyJspJYTsLeBMlFRE5RDl5+dz34TxenbSSJqnVePOGE+nUpFayw5I4iiWpHOful5rZAAB33236lUJEDtGqzdnc9Pr3zMzaxi9Oas5tZ7flqCrqjC9rYrmie8PU9Q5gZscRWX9eRKRI7s6b07K494N5ADx15Qn07dQoyVFJosSSVO4GPgGamdlrwMnANYkMSkTKhh+27ea2N2fxdeZGerSow8MXd6FZnerJDksSKJbRX5+a2TQiqzYacLO7b0x4ZCJSqn2xaAND3p7Fjj15/Pm8Dgw8qbkeZCwHYpmmZby7b3L3f7n7h+6+0czGx3BcMzObYGbzzWyumd0cyu82s9VmNiO8+kUdM9TMMs1soZmdHVXeN5RlmtmQqPIWZjbZzBab2eiwAqSIJNHmXXv57egZDHx+ClUrpfD6L3ty7SktlFDKiQO2VMysKlAdqGdmtYm0UgBqEhlWXJQ84Hfu/r2Z1QCmmdm4sO9Rd//7ft/Xgchqjx3D539mZm3C7ieBM4mMPJtqZmPcfR7wQPisUWb2FDAIGB5DbCKSANNWbOGW0dNZty2Hm05rxY2ntaJa5ZRkhyXF6GC3v64HbiHyH/w0/pNUthP5T/6g3P0H4IfwfoeZzQeaHOSQ/sAod88BloVlhXuEfZnuvhTAzEYB/cPnnQ5cHuq8RKT/R0lFpJht253LkxMyefarpdSvUYXR1/eiW1rtZIclSXDApOLujwOPm9n/ufsTR/IlZtYc6AZMJtLRf5OZXQ1kEGnNbCGScCZFHZbFf5LQqv3KewJ1ga3unldIfREpBu7Ox3PW8od3Z7M1O5cLuzXhzz/tQGp13Ykur2LpqH/CzDoBHYCqUeUvx/IFZnY08DZwi7tvN7PhwH1EhijfBzwMXMt/WkL/9fUU3u/jB6lfWAyDicwGQFpaWixhi0gRpizbzBOfL+arxRvp0Lgmrw7qqQcZJaZpWu4CehNJKh8B5wBfA0UmFTOrRCShvObu7wC4+7qo/c/wnyfzs4BmUYc3BdaE94WVbwRSzaxiaK1E1/8v7j4CGAGQnp5eaOIRkdi9MmkFf3pvDrWqVeLOc9sz8KTmWjNegNieU7kI6AJMd/drzKwh8GxRB4Wn7p8D5rv7I1HljUN/C8DPgDnh/RjgdTN7hEg/TmtgCpEWSWszawGsJtKZf7m7u5lNCPGNAgYC78dwPiJymLK2ZDP0ndl8tXgjP2lTn+FXdtcU9fJfYvnbsNvdC8wsz8xqAuuBljEcdzJwFTDbzGaEsj8AA8ysK5FbVcuJDAjA3eea2RvAPCIjx25093wAM7sJGAukAM+7+9zweXcAo8zsfmA6kSQmIgkwfeUWfv3a9+zck8eQc9rxyx+31DBh+R+xJJUMM0sFniEyCmwnkRbEQbn71xTe7/HRQY75C/CXQso/Kuy4MCKsx/7lIhI/7s4Tn2fyyLhFpFavxOjrT6TDMTWTHZaUULF01P86vH3KzD4Barr7rMSGJSIlQU5ePkPfns0701dzQddjuO+CTtSoWqnoA6XciqWjfry79wFw9+X7l4lI2TQrayu3vTmLhet28Nsz2nDT6a10u0uKlMgn6kWklHrp2+XcNWYuDWtWYcRVJ3BWR80qLLFJ2BP1IlL6bNiRw9B3ZvPZ/HWc0b4Bf7+4ix5klENSLE/Ui0jJ99rkFTz86SJ25eRx29ltGXxqSz17IocsltFfa82sRpi/606gO3C/u3+f4NhEpJi8+M0y7v5gHj1a1OH+CzrRpmGNZIckpVQsv4b8KSSUU4CziUzcqEkbRcqA3PwC7v1gHnd/MI8z2jfk9et6KqHIEYklqeSHn+cCw939fUA3WUVKuXXb93DR8G95/ptl/OKk5gy/sjsVdbtLjlAst79Wm9nTwBnAA2ZWhdiSkYiUQHn5BbwzfTUPjV1Idk4e/7i8G+d2bkxkZiWRIxNLUrkE6Av83d23mllj4LbEhiUiiTBvzXZufWMGC9buoFOTmjx8cU/aNtLtLomfWJ6ozwbeidr+9+JbIlJ6vDs9izvenk1qtUoMv6I7fTs1UutE4k7Ti4qUcRt35vDwp4sYOWUlvVrW4cnLu1P36CrJDkvKKCUVkTJswsL13Dp6Btv35HHdKS24vW87KldUl6gkjpKKSBmUvTePO9+bwzvfr+bYutV584YTadVAfSeSeAeb+2sHB1ieF8DdNfe1SAmUsXwzQ9+ZzZINO7nptFbcdHorqlZKSXZYUk4csB3s7jVC4ngMGAI0IbJk7x3A/UV9sJk1M7MJZjbfzOaa2c2hvI6ZjTOzxeFn7VBuZjbMzDLNbJaZdY/6rIGh/mIzGxhVfoKZzQ7HDDP1Oko5lpOXz/CJS7jk6e/YtjuXF6/pwe/PbquEIsUqlpurZ7v7P919h7tvd/fhwM9jOC4P+J27twd6ATeaWQciCWq8u7cGxodtgHOILCHcGhhMeGrfzOoAdwE9iSzIdde+RBTqDI46rm8McYmUOdNWbObcYV/zwCcL+Emb+nz2u59wapv6yQ5LyqGYnqg3syvMLMXMKpjZFfznKfsDcvcf9s0P5u47gPlEWjv9iUz1Qvh5QXjfH3jZIyYBqeGZmLOBce6+2d23AOOAvmFfTXf/zt0deDnqs0TKBXdn5JSVDHhmMnty83luYDovXNODmlpIS5Iklo76y4HHw8uBb0JZzMysOdANmAw0DM+64O4/mFmDUK0JsCrqsKxQdrDyrELKRcqFnLx87vtwHq9OWskpreoxbEA36hylGZQkuWJ5+HE5kVbEYTGzo4G3gVvcfftBuj0K2+GHUV5YDIOJ3CYjLS2tqJBFSryvFm/gD+/OZtXm3Vx/aktuO7ut5u2SEqHIv4Vm1sbMxpvZnLB9fJgCv0hmVolIQnnN3fc9lb8u3Loi/FwfyrOAZlGHNwXWFFHetJDy/+HuI9w93d3T69fXfWYpvXLy8vnDu7O56rkpGMZzA9MZ2q+9EoqUGLH8TXwGGArkArj7LOCyog4KI7GeA+a7+yNRu8YA+0ZwDQTejyq/OowC6wVsC7fJxgJnmVnt0EF/FjA27NthZr3Cd10d9VkiZc7srG1c8tR3vD55Jdef2pJxt55Kn/YNkx2WyH+JpU+lurtP2e+2VV4Mx50MXAXMNrMZoewPwN+AN8xsELASuDjs+wjoB2QC2cA1AO6+2czuA6aGeve6++bw/lfAi0A14OPwEilT3J1XJ6/knjFzqV45haeu7E7fTo2THZZIoWJJKhvN7DhCf4WZXUQME0q6+9cU3u8B0KeQ+g7ceIDPeh54vpDyDKBTUbGIlFazs7bx5zFzmL5yK6e0qscTA7pRW53xUoLFklRuBEYA7cxsNbAMuCKhUYmUcwUFziPjFvHkxExSq1Xibxd25uL0ZqRU0PO9UrIdNKmYWQUg3d3PMLOjgArhmRMRSZC12/Zw21sz+WrxRi5Jb8of+3WgVnU9dyKlw0GTirsXmNlNwBvuvquYYhIpl/bk5vPoZ4t44Zvl4PD/LuzMZT9qpjVPpFSJ5fbXODP7PTAa+HdiieosF5EjlLUlmxtf+56ZWdu4sHsTbu7TmmPrHpXssEQOWSxJ5drwM7oT3YGW8Q9HpHxxd96alsVdY+ZSwYynrzqBszs2SnZYIoctlifqWxRHICLlzbw127l7zFymLN9Mj+Z1eOji49U6kVKvyKQSnor/FXBqKJoIPO3uuQmMS6TMcnee+DyTx8cvpmbVivz1Z5259Eca2SVlQyy3v4YDlYB/hu2rQtl1iQpKpKyat2Y7f/tkAV8u2sB5xzfmnvM7ar14KVNiSSo/cvcuUdufm9nMRAUkUhbl5OXz4CcLeeGbZdSoWok/ndeBa05qTgW1TqSMiSWp5JvZce6+BMDMWhLDeioiErFkw05ufO17FqzdwWU/asbQc9rruRMps2JJKrcBE8xsKZFpV44lzMslIge2N6+AEV8uYdj4TKpVTuHZq9M5o4MmgJSyLZbRX+PNrDXQlkhSWeDuOQmPTKSUyssv4LXJK3n+m2Ws2JTNGe0b8tcLO9GgRtVkhyaScAdMKmZ24QF2HWdmRK2PIiLBonU7uPWNGcxZvZ1uaanc/dOOnNauQdEHipQRB2up/PQg+xxQUhEJ9uTm8/rklTw4dgFVKqbwyCVd+Fm3JppiRcqdAyYVd1e/iUgRcvLy+eu/5jM6YxV7cgs4pVU9Hr6kCw1r6laXlE+xPPxYF7gLOIVIC+VrIgtlbUpwbCIl2qJ1O7h51Azm/7Cdi09oyvldj+GUVvXUOpFyLZblhEcBG4CfAxeF96OLOsjMnjez9fvWtg9ld5vZajObEV79ovYNNbNMM1toZmdHlfcNZZlmNiSqvIWZTTazxWY22sy0cpEUi7z8Ap4Yv5jznvia9dv3MOKqE3jo4i78uHV9JRQp92JJKnXc/T53XxZe9wOpMRz3ItC3kPJH3b1reH0EYGYdiKx73zEc808zSzGzFOBJ4BygAzAg1AV4IHxWa2ALMCiGmESOyKysrVw4/FseHreIM9s35JNbTuUsTQAp8m+xJJUJZnaZmVUIr0uAfxV1kLt/CcQ6PX5/YJS757j7MiLr1PcIr0x3X+rue4m0mvpb5NfB04G3wvEvARfE+F0ihyw3tE4u/Oe3rN6ymycGdOPJK7pTv4amWBGJFsvDj9cDtwKvhO0UYJeZ3Upkafmah/idN5nZ1UAG8Dt33wI0ASZF1ckKZQCr9ivvCdQFtrp7XiH1/4eZDQYGA6SlpR1iuFLezVy1lT+/P4eZWds4q0NDHrzoeFKr626rSGGKbKm4ew13r+DulcKrQiircRgJZThwHNAV+AF4OJQXdiPaD6O8UO4+wt3T3T29fv36hxaxlFu7cvK4873Z9H/yG5Zu2MWwAd0YcXW6EorIQcQy+muQuz8XtZ0C3Onu9xzql7n7uqjPeQb4MGxmAc2iqjYF1oT3hZVvBFLNrGJorUTXFzki7s6kpZu5e8xcFq3fwTUnN+e3Z7ahZlXN1yVSlFj6VPqY2Udm1tjMOhO5TVXjcL7MzBpHbf4M2DcybAxwmZlVMbMWQGtgCjAVaB1GelUm0pk/xt0dmEBkNBrAQOD9w4lJJNr3K7fws39+y4BnJrFjTy7PXJXOXT/tqIQiEqNY5v663MwuBWYD2cAAd/+mqOPMbCTQG6hnZllEnnXpbWZdidyqWk6kvwZ3n2tmbwDzgDzgRnfPD59zEzCWSF/O8+4+N3zFHcAoM7sfmA78uzUlcqg27czhLx/N553vV9OgRhXu7d+Rn3dvylFVYul2FJF9LPJL/0EqRCaTfIlIUmlP5D/+W909O/HhxV96erpnZGQkOwwpIQoKImvEPzh2Adt35zHoxy246bRWSiYi+zGzae6eXlS9WP7lfECk5TA+DOW9lchtqY5HGKNIUs3K2srtb81iwdoddE9L5S8/60z7xoc69kREosWSVHq4+3aIjB8GHjazMYkNSyRx8gucp79cwiOfLiK1emWGDejGT49vrKfhReLggB31ZnY7gLtvN7OL99utySalVPps3jrOHfYVD36ykN5t6/PxzT/m/C7HKKGIxMnBRn9dFvV+6H77Cpt+RaTEytqSzbUvTuW6lzPYk5vPPy7vxjNXp+uJeJE4O9jtLzvA+8K2RUokd+eFb5bzyLhFANzety3XndKSyhVjGU0vIofqYEnFD/C+sG2REmfFpl3cNWYuExdu4JRW9fjrzzqTVrd6ssMSKdMOllS6mNl2Iq2SauE9YVsrEEmJtSc3n+ETlzD8iyVUqmD8sV97Bp3SggoV1MAWSbSDrfyYUpyBiMTDonU7GPxyBss3ZXN+l2P447nttQqjSDHSE15SJuzJzeelb5fzxOeZVK2UwmvX9eTkVvWSHZZIuaOkIqVaXn4BozNWMeLLpazYlE2vlnV45JKuHJNaLdmhiZRLSipSai1cu4MbXp3Gso276NC4Ji9e8yN6t22Q7LBEyjUlFSl19uYVMOLLJQz7PJOaVSvy1JXdObvqAd5iAAARwUlEQVRjIz3AKFICKKlIqfJt5kbu/XAeC9bu4JxOjbi3fyc9wChSgiipSKmwbvse7vlgLh/NXkuT1GqMuOoEzurYKNlhich+lFSkRNuTm8+rk1bwjwmZZOfkc+uZbRh8akuqVtKId5GSKGFzVZjZ82a23szmRJXVMbNxZrY4/Kwdys3MhplZppnNMrPuUccMDPUXm9nAqPITzGx2OGaY6YZ6mfPp3LWc8sAE7v/XfI5vmsqHvzmF3/RprYQiUoIlcgKkF/nfiSeHAOPdvTUwPmwDnENkCeHWwGBgOESSEJEVI3sCPYC79iWiUGdw1HGa5LKMyNqSzS9fzmDwK9OoX6MKrw7qycvX9qBNw8NaxVpEilHCbn+5+5dm1ny/4v5ElhiGyGqSE4ksC9wfeDms1zLJzFLDeva9gXHuvhnAzMYBfc1sIlDT3b8L5S8DFwAfJ+p8JPF25eTx5IRMnv16GRUrGLf3bcsvf9ySSima/FGktCjuPpWG7v4DgLv/YGb7HipoAqyKqpcVyg5WnlVIeaHMbDCRVg1paWlHeAoSb3ty83nluxU8/eVSNu7MoV/nRgw9pz3N6mjyR5HSpqR01BfWH+KHUV4odx8BjIDIGvWHE6AkxpzV27h51HSWbNhFjxZ1ePLybvRsWTfZYYnIYSrupLLOzBqHVkpjYH0ozwKaRdVrCqwJ5b33K58YypsWUl9Kidz8AkZNWcl9H86n9lGVePnaHpzapn6ywxKRI1TcN6vHAPtGcA0E3o8qvzqMAusFbAu3ycYCZ5lZ7dBBfxYwNuzbYWa9wqivq6M+S0q4zPU7uPq5Kfzp/bl0S0vl45tPVUIRKSMS1lIxs5FEWhn1zCyLyCiuvwFvmNkgYCVwcaj+EdAPyASygWsA3H2zmd0HTA317t3XaQ/8isgIs2pEOujVSV/Cbd+TywMfL2DklJVUr1yR+y/oxBU90zS9ikgZYpEBV+VHenq6Z2RkJDuMcmX99j0M/2IJb03LIntvPhd1b8rtfdtS92hNryJSWpjZNHdPL6peSemolzJo8bodPDJuEZ/MXYsB53c5hqtPak73tNpFHisipZOSisTdrpw8nv5iCU9MyOToyhUZdHILBvRM47j6Ryc7NBFJMCUViZudOXm89O1ynv1qKVuyc+nXuRF3/7QjDbScr0i5oaQiR2zb7lxenbSCZ75aytbsXHq3rc/NfVrTTbe5RModJRU5bHty83n+m2U8NXEJ2/fkcXq7BvymT2u6NktNdmgikiRKKnLIcvLyeX3ySv7xeSabdu3lpOPqckffdnRRMhEp95RUJGabdubw0rfLGZ2xinXbc+iWlsrDl3ThJ23q61kTEQGUVCQGu/fm8/J3y3n0s0Xk5BXQu019Hvh5c3q3bVDksSJSviipyAGt3rqbkZNX8vw3y8jem8/p7Rrwu7Pa0PGYWskOTURKKCUV+R8FBc5rU1by4McL2JGTR592DRh4UnN+3LqebnOJyEEpqch/mbhwPQ+NXcjcNdvp0aIOf7uwMy310KKIxEhJRQDYvGsvd4+Zy5iZa2hauxoPXXQ8F53QVC0TETkkSirl3MK1Oxg5ZSUjp6wkN7+AW85ozQ0/OY6qlVKSHZqIlEJKKuXUtuxcHvp0Aa9PXklKBeOM9g259cw2tG5YI9mhiUgppqRSzuQXOG9NW8XDny5i0669XNXrWG45ow21j6qc7NBEpAxISlIxs+XADiAfyHP3dDOrA4wGmgPLgUvcfUtY2fFxIot4ZQO/cPfvw+cMBO4MH3u/u79UnOdRmmzcmcPIySt5b8ZqlmzYRbe0VJ4dmM7xTfUUvIjETzJbKqe5+8ao7SHAeHf/m5kNCdt3AOcArcOrJzAc6BmS0F1AOuDANDMb4+5bivMkSrqcvHyenLCEZ79aSvbefLo0rcXjl3Xl/C7HqBNeROKuJN3+6k9k+WGAl4CJRJJKf+BljyxROcnMUs2scag7bt/ywmY2DugLjCzesEumggLng1lr+OtH81m3PYd+nRtxc582tG2kPhMRSZxkJRUHPjUzB5529xFAQ3f/AcDdfzCzfXOANAFWRR2bFcoOVF7ufbloA0Pfmc3qrbtp37gmf/v58ZymKVVEpBgkK6mc7O5rQuIYZ2YLDlK3sHs0fpDy//0As8HAYIC0tLRDjbXU2Jq9l4fGLmTklJW0qHcUwwZ0o1+nRlRMqZDs0ESknEhKUnH3NeHnejN7F+gBrDOzxqGV0hhYH6pnAc2iDm8KrAnlvfcrn3iA7xsBjABIT08vNPGUZu7Od0s2cfPoGWzamcPVJzbnt2e2oVa1SskOTUTKmWL/FdbMjjKzGvveA2cBc4AxwMBQbSDwfng/BrjaInoB28JtsrHAWWZW28xqh88ZW4ynknTuzjeZG7n4qe+4/NnJVKuUwvs3nsLd53dUQhGRpEhGS6Uh8G4YeVQReN3dPzGzqcAbZjYIWAlcHOp/RGQ4cSaRIcXXALj7ZjO7D5ga6t27r9O+rMsvcD6ctYbHxy9m6YZdNKhRhTvPbc+AHmkcVaUkjb0QkfLGIoOqyo/09HTPyMhIdhiHZdPOHN6alsU7369m4bodtGtUg1+c1JwLujXRtCoiklBmNs3d04uqp19rS4G9eQW89G1kkazsvfl0aFyTxy6NPGtSoYKeNRGRkkNJpQTLzS/gkzlreeCTBWRt2U2fdg24rW9b2jWqmezQREQKpaRSQk1bsYXb3prJ0g27aFnvKF74xY/o3VZrwYtIyaakUoK4O4vW7eShsQv5bP46GtaswlNXnsCZHRqSottcIlIKKKmUAHvzCnhvxmqGjV9M1pbdVK1UgZtOa8V1P25BanXNHiwipYeSShLtzStg9NSVPPXFUlZv3U2XprW4/ifH0bdjI+rXqJLs8EREDpmSShJsy85l7Ly1/26ZdGmWyv0XdOInbeprNJeIlGpKKsVozdbdPPjJAv41+wdy853OTWr9O5moA15EygIllWKwKyePp75YwovfLie/wBnQI42+HRtx4nF1lUxEpExRUkmw75ZsYug7s1ixOZs+7Roy5Jx2tGpwdLLDEhFJCCWVBFm4dgfDJ2by/sw1NK1djVG/7EXPlnWTHZaISEIpqcTZll17eXDsAkZPXUX1yhX5xUnNue3stlSvrD9qESn79D9dnKzeuptXvlvBW9Oy2LQrh4EnNufmPq2pfZSeMxGR8kNJ5Qhtzd7LU18s5flvllFQ4Jx4XF1+f1Y6XZqlJjs0EZFip6RyBMbPX8cdb89m484c+rRrwD39O9K0dvVkhyUikjSlPqmYWV/gcSAFeNbd/5bo79yavZe7xszl/RlraNeoBi9d+yM6HlMr0V8rIlLileqkYmYpwJPAmUTWrJ9qZmPcfV6ivnP6yi386tXv2bQrh5v7tObXpx1HlYpaIEtEBEp5UgF6AJnuvhTAzEYB/YG4J5Xc/AIe+2wRT3+xlGNSq/HWDSep30REZD+lPak0AVZFbWcBPeP9Jbn5BQx+OYMJCzdwYfcm/LFfe+oerQkfRUT2V9qTSmFznPj/VDIbDAwGSEtLO+QvqZRSgZb1j+bMDo24vOehHy8iUl6U9qSSBTSL2m4KrNm/kruPAEYApKen/0/SicWfzutwOIeJiJQrFZIdwBGaCrQ2sxZmVhm4DBiT5JhERMqtUt1Scfc8M7sJGEtkSPHz7j43yWGJiJRbpTqpALj7R8BHyY5DRERK/+0vEREpQZRUREQkbpRUREQkbpRUREQkbpRUREQkbsz9sJ4FLLXMbAOw4jAPrwdsjGM4pYHOuXwob+dc3s4Xjvycj3X3+kVVKndJ5UiYWYa7pyc7juKkcy4fyts5l7fzheI7Z93+EhGRuFFSERGRuFFSOTQjkh1AEuicy4fyds7l7XyhmM5ZfSoiIhI3aqmIiEjcKKnEwMz6mtlCM8s0syHJjudImFkzM5tgZvPNbK6Z3RzK65jZODNbHH7WDuVmZsPCuc8ys+5RnzUw1F9sZgOTdU6xMrMUM5tuZh+G7RZmNjnEPzosn4CZVQnbmWF/86jPGBrKF5rZ2ck5k9iYWaqZvWVmC8L1PrGsX2cz+234ez3HzEaaWdWydp3N7HkzW29mc6LK4nZdzewEM5sdjhlmZoUthnhg7q7XQV5EptRfArQEKgMzgQ7JjusIzqcx0D28rwEsAjoADwJDQvkQ4IHwvh/wMZFVNnsBk0N5HWBp+Fk7vK+d7PMr4txvBV4HPgzbbwCXhfdPAb8K738NPBXeXwaMDu87hOtfBWgR/l6kJPu8DnK+LwHXhfeVgdSyfJ2JLC++DKgWdX1/UdauM3Aq0B2YE1UWt+sKTAFODMd8DJxzSPEl+w+opL/CH+7YqO2hwNBkxxXH83sfOBNYCDQOZY2BheH908CAqPoLw/4BwNNR5f9Vr6S9iKwKOh44Hfgw/IPZCFTc/zoTWZ/nxPC+Yqhn+1/76Hol7QXUDP/B2n7lZfY6h6SyKvxHWTFc57PL4nUGmu+XVOJyXcO+BVHl/1UvlpdufxVt31/UfbJCWakXmvvdgMlAQ3f/ASD8bBCqHej8S9ufy2PA7UBB2K4LbHX3vLAdHf+/zy3s3xbql6ZzbglsAF4It/yeNbOjKMPX2d1XA38HVgI/ELlu0yjb13mfeF3XJuH9/uUxU1IpWmH3E0v9kDkzOxp4G7jF3bcfrGohZX6Q8hLHzM4D1rv7tOjiQqp6EftKzTkT+c27OzDc3bsBu4jcFjmQUn/OoR+hP5FbVscARwHnFFK1LF3nohzqOR7xuSupFC0LaBa13RRYk6RY4sLMKhFJKK+5+zuheJ2ZNQ77GwPrQ/mBzr80/bmcDJxvZsuBUURugT0GpJrZvtVPo+P/97mF/bWAzZSuc84Cstx9cth+i0iSKcvX+QxgmbtvcPdc4B3gJMr2dd4nXtc1K7zfvzxmSipFmwq0DiNIKhPp0BuT5JgOWxjJ8Rww390fido1Btg3AmQgkb6WfeVXh1EkvYBtoXk9FjjLzGqH3xDPCmUljrsPdfem7t6cyPX73N2vACYAF4Vq+5/zvj+Li0J9D+WXhVFDLYDWRDo1Sxx3XwusMrO2oagPMI8yfJ2J3PbqZWbVw9/zfedcZq9zlLhc17Bvh5n1Cn+GV0d9VmyS3eFUGl5ERlAsIjIK5I/JjucIz+UUIs3ZWcCM8OpH5F7yeGBx+Fkn1DfgyXDus4H0qM+6FsgMr2uSfW4xnn9v/jP6qyWR/ywygTeBKqG8atjODPtbRh3/x/BnsZBDHBWThHPtCmSEa/0ekVE+Zfo6A/cAC4A5wCtERnCVqesMjCTSZ5RLpGUxKJ7XFUgPf35LgH+w32CPol56ol5EROJGt79ERCRulFRERCRulFRERCRulFRERCRulFRERCRulFREjpCZ5ZvZjKjXQWeyNrMbzOzqOHzvcjOrd6SfIxJPGlIscoTMbKe7H52E711O5LmDjcX93SIHopaKSIKElsQDZjYlvFqF8rvN7Pfh/W/MbF5Y62JUKKtjZu+Fsklmdnwor2tmn4YJIp8map4mM7syfMcMM3vazFKScMoiSioicVBtv9tfl0bt2+7uPYg8mfxYIccOAbq5+/HADaHsHmB6KPsD8HIovwv42iMTRI4B0gDMrD1wKXCyu3cF8oEr4nuKIrGpWHQVESnC7vCfeWFGRv18tJD9s4DXzOw9IlOpQGQqnZ8DuPvnoYVSi8jiTBeG8n+Z2ZZQvw9wAjA1LNJXjf9MKChSrJRURBLLD/B+n3OJJIvzgT+ZWUcOPv14YZ9hwEvuPvRIAhWJB93+EkmsS6N+fhe9w8wqAM3cfQKRBcRSgaOBLwm3r8ysN7DRI2veRJefQ2SCSIhMIHiRmTUI++qY2bEJPCeRA1JLReTIVTOzGVHbn7j7vmHFVcxsMpFf4Absd1wK8Gq4tWXAo+6+1czuJrJi4ywgm/9MaX4PMNLMvge+IDLVO+4+z8zuBD4NiSoXuBFYEe8TFSmKhhSLJIiG/Ep5pNtfIiISN2qpiIhI3KilIiIicaOkIiIicaOkIiIicaOkIiIicaOkIiIicaOkIiIicfP/AXeqvK1PrUGKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(all_qtable_rows)\n",
    "plt.xlabel('Episode')\n",
    "plt.ylabel('Explored states')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd8VuX9//HXJ3cSNmErJGEvkU1kCAq4wFGWKGBxFC0FldbZyq/9ttZWrdYKLhDcW4ZWkbqVvSQoIJsQVgAhgCxZGdfvj/uQphiSALlzcud+Px+P+5Ezrtz5nBy43znnXOc65pxDREQEIMrvAkREpORQKIiISA6FgoiI5FAoiIhIDoWCiIjkUCiIiEgOhYKIiORQKIiISA6FgoiI5Ij2u4DTVaNGDVe/fn2/yxARCStLlizZ7ZyrWVC7sAuF+vXrk5yc7HcZIiJhxcw2F6adTh+JiEiOkIWCmb1sZrvMbMUp1puZPW1mKWa23Mzah6oWEREpnFAeKbwK9M5n/ZVAE+81HBgfwlpERKQQQhYKzrnZwN58mvQFXndBC4EqZlY7VPWIiEjB/LymEA9szTWf5i0TERGf+BkKlseyPJ/4Y2bDzSzZzJLT09NDXJaISOTyMxTSgMRc8wnA9rwaOucmOueSnHNJNWsW2M1WRETOkJ+hMA24yeuF1BnY75zbEaoftmn3Tzz26Rqys/X4URGRUwnZzWtm9g7QA6hhZmnAX4AYAOfc88DHwFVACnAY+FWoagH4fNUPjJ+5gX2HM3i4X0uiovI6eyUiEtlCFgrOuSEFrHfAHaH6+Sf79UUN2Xc4g3EzNxBl8Pd+LTFTMIiI5BZ2w1ycKTPj/l7NyHbw/KwNRJnxUN/zFQwiIrlETChAMBj+0LsZ2c4xcXYqgSjjL79ooWAQEfFEVChAMBhGX9mcrGzHS3M3EmXG/11znoJBRIQIDAUIBsOfrj6PbOd4ed5Gogz+eLWCQUQkIkMBgsHw52takJ3teHHuRgJRxgNXNlcwiEhEi9hQgGAwPNjnfLKcY8LsVKKijN/3aqZgEJGIFdGhAMFgeKhPS7KyYfzMDQTMuPeKpgoGEYlIER8KAFFRxsP9WuKc49kZKUQZ3H25gkFEIo9CwRMVZTzSvxXZzvH01ylkZjvu16kkEYkwCoVcoqKMfwxoTSDKGDdzA5nZjtG6+CwiEUShcJLgqaRWREdFMXF2KhlZ2fz5Gt3gJiKRQaGQh6io4BAY0QHjlXmbyMjK5qE+GkRPREo/hcIpnLiPITYQxYTZqWRmOR7p30rBICKlmkIhH2bBG9qiA8ZzM4LXGB67NnjNQUSkNFIoFMDMuO+KZsQEohj75Xoys7J54ro2RAf8fD6RiEhoKBQKwcy467KmREcZT3y+jsxsx5hBbYlRMIhIKaNQOA13XtKEmEAUj36yhswsx9ND2hEbrWAQkdJDn2in6TfdG/F/17Tg05U/MPLNJRzNyPK7JBGRIqNQOAO3dmvA3/q15Ks1uxj26mJ+Opbpd0kiIkVCoXCGbuxcjyevb8PC1D0MfWkR+w9n+F2SiMhZUyichQHtExj3y/as2LafIS8sZPehY36XJCJyVhQKZ6l3y9q8ePMFpO4+xKAJC9ix/4jfJYmInDGFQhHo3rQmr/2qIzsPHOO65xewZc9hv0sSETkjIQ0FM+ttZmvNLMXMHshjfT0z+8rMlpvZTDNLCGU9odSpYXXeuq0Th45lMvD5+azfedDvkkRETlvIQsHMAsBzwJVAC2CImbU4qdkTwOvOudbAQ8CjoaqnOLRJrMKk4V3IdjBo4kJWbNvvd0kiIqcllEcKHYEU51yqc+448C7Q96Q2LYCvvOkZeawPO83OrcSUEV0oFxNgyMSFJG/a63dJIiKFFspQiAe25ppP85bltgy41pvuD1Qys+ohrKlYNKhRgckjulCjUhlufOkb5qxP97skEZFCCWUo5DWUqDtp/j6gu5l9B3QHtgE/uxPMzIabWbKZJaenh8cHbHyVckz+TRfqVS/PsFcXM335dr9LEhEpUChDIQ1IzDWfAPzPJ6NzbrtzboBzrh3wR2/Zz07EO+cmOueSnHNJNWvWDGHJRatmpTJMGt6FNglVGPXOd7yxcLPfJYmI5CuUobAYaGJmDcwsFhgMTMvdwMxqmNmJGkYDL4ewHl/ElY/hjVs70bNZLf7vgxU89eV6nDv5gElEpGQIWSg45zKBO4HPgNXAZOfcSjN7yMz6eM16AGvNbB1wDvBwqOrxU7nYABNu7MCA9vGM+XIdD05bSXa2gkFESp6QDp3tnPsY+PikZX/ONT0VmBrKGkqKmEAUTwxsQ/UKsbwwZyM/Hs7gievaaOhtESlR9DyFYhQVZfy/q86jWoUyPPbpGvYfyWD80PaUj9VuEJGSQX+mFjMzY2SPRjx2bSvmrE9n6IuL2Hf4uN9liYgACgXfDLqgLuN+2YEV2w9w/YQF/LD/qN8liYgoFPzUu+W5vParjmzfd5Rrx89nQ/ohv0sSkQinUPBZl0bVeXd4Z45lZjFw/HyWbP7R75JEJIIpFEqAlvFxvDfyQuLKxXDDCwv5fOUPfpckIhFKoVBC1KtegfdGXkjz2pUZ8eYS3f0sIr5QKJQg1SuW4Z1f//fu58c/XaO7n0WkWCkUSpjysdFMuLEDQzomMm7mBu6dvIzjmdl+lyUiEUJ3TZVA0YEoHunfitpx5Xjyi3XsOniM8UPbU6lsjN+liUgppyOFEsrM+O2lTXh8YGsWpO5h0ISF7DygexlEJLQUCiXc9UmJvHRzEpv2/MSAcfNJ2aVnP4tI6CgUwkCPZrWYNLwLxzKzuHb8AhbrEZ8iEiIKhTDRKiGO90d2pXqFWH75wiI+XLrN75JEpBRSKISRutXL897IC2mbWIXfvbuUZ77SA3tEpGgpFMJM1QqxvHFbR/q3i+dfX6zjvinL1WVVRIqMuqSGoTLRAZ68vg31qpdn7Jfr2bbvMBOGJhFXXl1WReTs6EghTJkZd13WlDGD2rBk84/0Hz+PLXsO+12WiIQ5hUKY698ugTdv7cTen47Tb9w8lmxWzyQROXMKhVKgU8PqvD/yQiqXjWbIC4v4aNl2v0sSkTClUCglGtasyPu3d6VNQhyj3vmO52akqGeSiJw2hUIpUq1CLG/e1om+bevwz8/W8vup6pkkIqdHvY9KmTLRAcYOakv96hV46qv1bN5zmPFD21O9Yhm/SxORMBDSIwUz621ma80sxcweyGN9XTObYWbfmdlyM7sqlPVECjPj7sub8syQdixL20efZ+ex5ocDfpclImEgZKFgZgHgOeBKoAUwxMxanNTsT8Bk51w7YDAwLlT1RKJftKnD5N90ISMrm2vHzefLVTv9LklESrhQHil0BFKcc6nOuePAu0Dfk9o4oLI3HQeo20wRa5NYhWl3dqNRrYr8+o1knp+1QRegReSUQhkK8cDWXPNp3rLcHgSGmlka8DEwKq83MrPhZpZsZsnp6emhqLVUOzeuLJOGd+HqVrX5xydruHfKMo5mZPldloiUQKEMBctj2cl/og4BXnXOJQBXAW+Y2c9qcs5NdM4lOeeSatasGYJSS79ysQGeGdKOey5vyvvfbuOGFxay66Ae2iMi/yuUoZAGJOaaT+Dnp4duBSYDOOcWAGWBGiGsKaKdeJrb+F+2Z/WOg/R7dh4rt+/3uywRKUFCGQqLgSZm1sDMYgleSJ52UpstwKUAZnYewVDQ+aEQu7JVbaaM6ALAwPEL+HTFDp8rEpGSImSh4JzLBO4EPgNWE+xltNLMHjKzPl6ze4Ffm9ky4B3gFqeroMWiZXwcH9zZlea1KzHizW8Z88U6srP1qxeJdBZun8FJSUkuOTnZ7zJKjaMZWfzpgxVMXZLGZeedw5hBbahUVkNwi5Q2ZrbEOZdUUDsNcxHhysYE+OfA1vy1z/nMWLuLfs/NY0P6Ib/LEhGfKBQEM+PmC+vz1m2d2Hc4g37PztONbiIRSqEgOTo3rM60Ud2oX6MCt72ezNNfrdd1BpEIo1CQ/xFfpRxTRnRhQLt4nvxiHSPeXMKhY5l+lyUixUShID9TNibAv65vw5+vacFXa3bR/7l5bNz9k99liUgxUChInsyMYd0a8MatHdl96Bh9np3LjDW7/C5LREJMoSD5urBRDT4a1Y261coz7LXFus4gUsopFKRACVXLM3XEhfRrG7zOcNvryew/nOF3WSISAgoFKZRysQGevL4Nf+t7PnPWp3PNs3NYsU3jJomUNgoFKTQz48Yu9Zn0my5kZjkGjJ/PpMVb/C5LRIqQQkFOW/u6VZk+qhsd61fjD+99zx+mLtfzGURKCYWCnJHqFcvw2rCO3NmzMZOSt3Lt+Pls2XPY77JE5CwpFOSMBaKM+3o146Wbk9i69zDXPDOHr9doeAyRcKZQkLN26XnnMH3URSRULc+wV5N58vO1ZKnbqkhYUihIkahbvTzv334h13VI4OmvU7jllW/YfeiY32WJyGlSKEiRKRsT4PGBrXl0QCsWbdzLVU/NYWHqHr/LEpHToFCQImVmDOlYl3/ffiEVykRzwwsLefZr3QUtEi4UChIS59eJ46NR3bimdR2e+HwdN+t0kkhYKFQomNkAM1tvZvvN7ICZHTSzA6EuTsJbxTLRPDW4rU4niYSRwh4pPA70cc7FOecqO+cqOecqh7IwKR1OnE764PauVPROJz3z1Xr1ThIpoQobCjudc6tDWomUai3qVGbaqG78ok0d/vXFOm5++RvSD+p0kkhJU9hQSDazSWY2xDuVNMDMBoS0Mil1KpaJZuygtvxjQCsWb9rLVU/PYf6G3X6XJSK5FDYUKgOHgSuAX3ivawr6JjPrbWZrzSzFzB7IY/0YM1vqvdaZ2b7TKV7Cj5kxuGNdPrijK5XKRjP0xUWM/XKdTieJlBDmXGj+M5pZAFgHXA6kAYuBIc65VadoPwpo55wblt/7JiUlueTk5KIuV3zw07FM/vTBCv793TY61q/GmMFtia9Szu+yREolM1vinEsqqF1hex8lmNm/zWyXme00s/fMLKGAb+sIpDjnUp1zx4F3gb75tB8CvFOYeqR0qFAmmjGD2vLk9W1YuX0/V46dzcff7/C7LJGIVtjTR68A04A6QDzwkbcsP/HA1lzzad6ynzGzekAD4OtC1iOlyID2CXz8u4toULMit7/1LaPfX87h45l+lyUSkQobCjWdc6845zK916tAzQK+x/JYdqpzVYOBqc65PAflN7PhZpZsZsnp6emFLFnCSb3qFZg6ogsjezTi3cVb+cUzc1m1XbfCiBS3wobCbjMbamYB7zUUKOgupDQgMdd8ArD9FG0Hk8+pI+fcROdcknMuqWbNgrJIwlVMIIo/9G7Om7d24uDRTPo9N4+X524kVNe9ROTnChsKw4DrgR+AHcBAb1l+FgNNzKyBmcUS/OCfdnIjM2sGVAUWFLZoKd26Nq7Bp3ddzMVNa/DQ9FUMe3WxhsgQKSaFCgXn3BbnXB/nXE3nXC3nXD/n3OYCvicTuBP4DFgNTHbOrTSzh8ysT66mQ4B3nf4clFyqVYjlhZuSeKjv+czbsIfeY+cwe51OHYqEWr5dUs3sGU59HQDn3G9DUVR+1CU18qz54QC/fec71u08xK3dGnB/r2aUjQn4XZZIWClsl9ToAtbr01d81/zcyky7sxuPfLyal+ZuZO763YwZ1JYWdTT8lkhRC9nNa6GiI4XINnPtLu6fupz9hzO494qm3HZRQwJReXV0E5HciuTmNTMb6339yMymnfwqqmJFCqtHs1p8dtfFXNK8Fo9+soYhLywk7cfDfpclUmoUdE2hg3NuiZl1z2u9c25WyCo7BR0pCIBzjve+3caD01ZiwIN9zmdA+3jMdNQgkpfCHimc9ukjM6sKJDrnlp9pcWdDoSC5bd17mHsmL2Xxph+5qtW5PNyvFVUrxPpdlkiJU9RjH800s8pmVg1YBrxiZk+ebZEiZyuxWnneHd6FP/RuzherdtJr7GxmqeuqyBkr7M1rcc65A8AA4BXnXAfgstCVJVJ4gShjZI9G/Pv2rsSVi+Hml7/hLx+u4MjxPEdNEZF8FDYUos2sNsG7mqeHsB6RM9YyPo6PRnVjWNcGvLZgM1c9PYclm/f6XZZIWClsKDxE8M7kDc65xWbWEFgfurJEzkzZmAB//kUL3v51JzKyshn4/AIe+Xg1RzN01CBSGLpPQUqtQ8cyeeTj1by9aAuNalbgX9e3pW1iFb/LEvFFUV9obujdq5DuPWjnQzNrcPZlioROxTLRPNK/Fa8P68iR41kMGDePxz9dw7FMHTWInEphTx+9DUwGahN80M4Ugk9SEynxLm5ak0/vvpiBHRIYN3MDfZ6Zx4pt+/0uS6REKmwomHPujVwP2XmTfAbKEylpKpeN4fGBbXjllgvYd+Q4fZ+bx5NfrON4ZrbfpYmUKIUNhRlm9oCZ1Tezemb2e+A/ZlbNu3dBJCz0bF6Lz+/qTt+2dXj6q/X0e26envAmkkuhLjSb2cZ8VjvnXMOiKyl/utAsReWLVTsZ/f737D9ynDt6Nub2Ho2JjS7s30ki4aWohs4GwDmni8pS6lze4hyS6lXlrx+tZOyX6/nk+x94bGBr9VCSiFbQKKm/zzV93UnrHglVUSLFpWqFWMYObsfLtyRx4GgGA8bN4+/TV+luaIlYBR0rD841Pfqkdb2LuBYR31zS/Bw+v/tihnSsy4tzN9Jr7Gzmb9jtd1kixa6gULBTTOc1LxLWKpWN4eH+rXh3eGeiDG54YRGj31/O/iMZfpcmUmwKCgV3ium85kVKhc4Nq/PpXRfzm+4NmbR4K1eMmcUXq3b6XZZIsSgoFNqY2QEzOwi09qZPzLcqhvpEfFE2JsDoK8/jgzu6UrV8LL9+PZk73/6W3YeO+V2aSEjlGwrOuYBzrrJzrpJzLtqbPjEfU1xFivildUIVPhrVjfuuaMrnK3dy2ZOzmLokjXAbM0yksNQpW6QAMYEo7rykCR//rhuNalbkvinLuOGFRaSmH/K7NJEiF9JQMLPeZrbWzFLM7IFTtLnezFaZ2UozezuU9Yicjca1KjHlN114uH9LVmzfT++xc3jqy/UaYE9KlZANnW1mAWAdcDmQBiwGhjjnVuVq04TgQHuXOOd+NLNazrld+b2v7miWkmDXwaP8bfpqPlq2nYY1K/BI/1Z0bljd77JETqlIh84+Qx2BFOdcqnPuOMFRVfue1ObXwHPOuR8BCgoEkZKiVqWyPDOkHa/+6gIysrIZPHEh909Zxo8/Hfe7NJGzEspQiAe25ppP85bl1hRoambzzGyhmeV5Q5yZDTezZDNLTk/XQ9ml5OjRLDjA3sgejfj3d9u49MlZvP+tLkRL+AplKOR1c9vJ/1OigSZAD2AI8KKZ/WzgGefcROdcknMuqWbNmkVeqMjZKBcb4A+9mzP9t92oX70890xexi9f1IVoCU+hDIU0IDHXfAKwPY82HzrnMpxzG4G1BENCJOw0P7cyU0dcyN/7teT7bfvp/VTwQrSeDy3hJJShsBhoYmYNzCyW4DhK005q8wHQE8DMahA8nZQawppEQioqyhjauR5f3dudK1qcw5gv19Fr7GxmrNXlMgkPIQsF51wmcCfwGbAamOycW2lmD5lZH6/ZZ8AeM1sFzADud87tCVVNIsWlVqWyPHtDe966rRPRUcavXlnM8NeTSfvxsN+lieQrZF1SQ0VdUiXcHM/M5qW5G3n6q/U4HKMuacJtFzWgTHTA79IkgpSELqkiAsRGRzGyRyO+urc7PZvV4p+fraX32DnMXqeedFLyKBREikmdKuUYP7QDrw3rCMBNL3/DyDeXsH3fEZ8rE/kvhYJIMevetCaf3nUR9/dqxoy1u7j0X7MYP3MDxzOz/S5NRKEg4ocy0QHu6NmYL+7uzkVNavDYp2u48qnZOqUkvlMoiPgosVp5Jt6UxCu3XEBWtuOml7/httcWs2n3T36XJhFKoSBSAvRsXovP7r6YB65szoINe7h8zCwe/WQ1h45l+l2aRBiFgkgJUSY6wIjujZhxfw/6tY1nwqxUej4xk6lL0sjODq+u4xK+FAoiJUytSmX553Vt+PCOriRULcd9U5bRf/x8vtvyo9+lSQRQKIiUUG0Sq/DeiAsZM6gNP+w/Qv9x87ln0lJ2Hjjqd2lSiikUREqwqCijf7sEvr63B3f0bMT073fQ84mZPDcjRQPtSUgoFETCQIUy0dzfqzlfel1Y//nZWi4fM4vpy7fr2Q1SpBQKImGkbvXyTLgxibdu60SF2GjufPs7rh0/nyWbdb1BioZCQSQMdW1cg//89iIev7Y1aT8e4drx87nj7W/ZulejsMrZ0SipImHup2OZTJydysTZqWRlO27pWp87ejYmrlyM36VJCaJRUkUiRIUy0dx9eVNm3NeDvm3r8MKcVHr8cwavzttIRpbGU5LTo1AQKSXOjQve3zB9VDda1KnMgx+t4ooxs/ls5Q+6GC2FplAQKWXOrxPHm7d24uVbkghEGb95YwmDJy5kedo+v0uTMKBQECmFzIxLmp/Dp7+7iL/1a0nKrkP0eXYed7z1LRs12J7kQxeaRSLAwaMZvDBnIy/OSeVYZjaDL0jkd5c2oVblsn6XJsWksBeaFQoiEST94DGe+Xo9by/aQkwgilu7NWB494ZULqueSqWdQkFETmnT7p/41xfr+GjZdqqWj+GOno25sUs9ykQH/C5NQkRdUkXklOrXqMAzQ9oxfVQ3WsbH8ff/rOaSJ2bx3pI0sjRMd0QLaSiYWW8zW2tmKWb2QB7rbzGzdDNb6r1uC2U9IvK/WsbH8catnXjrtk5UqxDLvVOWcfXTc/h6zU51Y41QIQsFMwsAzwFXAi2AIWbWIo+mk5xzbb3Xi6GqR0ROrWvjGnx4R1eevaEdRzOyGPZqMtdPWMDC1D1+lybFLJRHCh2BFOdcqnPuOPAu0DeEP09EzkJUlHFN6zp8cU93/t6vJVv2HmbwxIUMfXER3+oBPxEjlKEQD2zNNZ/mLTvZtWa23MymmlliCOsRkUKICUQxtHM9Zt3fkz9dfR6rdxxgwLj53PrqYlZs2+93eRJioQwFy2PZyScpPwLqO+daA18Cr+X5RmbDzSzZzJLT09OLuEwRyUvZmAC3XdSQ2b/vyf29mrF4016ueWYut7+1hPU7D/pdnoRIyLqkmlkX4EHnXC9vfjSAc+7RU7QPAHudc3H5va+6pIr4Y/+RDF6au5GX5qRyOCOLfm3jueuyJtSrXsHv0qQQSkKX1MVAEzNrYGaxwGBgWu4GZlY712wfYHUI6xGRsxBXLoZ7Lm/KnD9cwvCLG/LJih1c8q9ZjH5/Odv2HfG7PCkiIb15zcyuAsYCAeBl59zDZvYQkOycm2ZmjxIMg0xgLzDSObcmv/fUkYJIybDrwFHGzdzA24u2ADC4YyIjezSidlw5nyuTvOiOZhEpFtv2HeHZr9czJTmNKDOuvyCBkT0aE19F4VCSKBREpFht3XuYcTM3MHVJsNPhdUmJ3N6jEQlVy/tcmYBCQUR8sm3fEcbPTGHy4jQcjoEdEri9R2MSqykc/KRQEBFfbd93hOdnbeDdb7aS7RwD2sdzZ88m1K2ucPCDQkFESoQf9h/l+VkbePubLWRlO/q3i+fOno2pX0NdWYuTQkFESpSdB7xwWLSFzGxH3zZ1uL1nIxrXquR3aRFBoSAiJdKuA0eZMDuVtxZt5lhmNr1anMvtPRvROqGK36WVagoFESnR9hw6xqvzN/Hq/E0cPJrJRU1qcEfPxnRqUA2zvEbJkbOhUBCRsHDwaAZvLtzCS3NT2X3oOB3qVeWOno3o2ayWwqEIKRREJKwczchicvJWJsxKZdu+IzQ/txK392zM1a1qE4hSOJwthYKIhKWMrGymLd3OuJkpbEj/ifrVyzOieyP6t4/XM6TPgkJBRMJadrbj81U7eW5GCt9v28+5lctya7cGDO6YSKWyMX6XF3YUCiJSKjjnmJuym+dmpLAwdS+VykZzQ6e6DOvagHMql/W7vLChUBCRUmd52j4mzE7lk+93EIgy+raNZ/jFDWl6ju51KIhCQURKrS17DvPS3FQmJW/laEY2PZvVZPjFjejcUN1ZT0WhICKl3o8/HeeNhZt5bf4m9vx0nNYJcQy/uCG9zz+X6EAonyEWfhQKIhIxjmZk8d63abw4ZyMbd/9EYrVy3Nq1AddfkEj52Gi/yysRFAoiEnGysh1frNrJxNkb+HbLPqqUj+GGjnW5qUt9zo2L7IvSCgURiWjJm/YycXYqX6zeScCMq1vX5lddG9A2MTLHWCpsKOi4SkRKpaT61UiqX40tew7z2oJNTFq8lQ+XbqdDvaoM69qAXuefo+sOedCRgohEhINHM5i6JI1X5m1iy97D1Ikry80X1mfwBXWJK1/6b4bT6SMRkTxkZTu+XrOLl+duZEHqHsrFBBjYIYFbutanUc2KfpcXMgoFEZECrNp+gFfmbeTDpds5nhW832FYtwZ0a1yj1N3voFAQESmk9IPHeHvRFt5YuJndh47RqGYFbuxcj2s7JJSacZZKRCiYWW/gKSAAvOic+8cp2g0EpgAXOOfy/cRXKIhIqBzLzOI/y3fw+oLNLN26j/KxAQa0j+emLvXDfigN30PBzALAOuByIA1YDAxxzq06qV0l4D9ALHCnQkFESoLlaft4fcFmpi3bzvHMbDo1qMbNF9bn8hbnEBOGvZYKGwqh3LKOQIpzLtU5dxx4F+ibR7u/AY8DR0NYi4jIaWmdUIUnrmvDwtGX8sCVzdm27wi3v/Ut3R77mqe+XM+ug6XzIyuUoRAPbM01n+Yty2Fm7YBE59z0ENYhInLGqlWIZUT3Rsy6vycv3pREs3MrM+bLdXT9x9eMeuc7Fm/aS7hdm81PKG9ey+vSfc5vzsyigDHALQW+kdlwYDhA3bp1i6g8EZHCC0QZl7U4h8tanENq+iHeXLiFKUu28tGy7ZxXuzJDO9elb9t4KpYJ73uCQ3lNoQvwoHOulzc/GsA596g3HwdsAA5533IusBfok991BV1TEJGS4vDxTD5cup3XF2xm9Y4DlI8N0LdtHW7oWI9WCXF+l/c/SsKF5miCF5oZphKFAAAIkklEQVQvBbYRvNB8g3Nu5SnazwTu04VmEQk3zjmWbt3H24u28NHy7RzNyKZVfBxDOtalT9s6JeLowfdQ8Iq4ChhLsEvqy865h83sISDZOTftpLYzUSiISJjbfySDD5du4+1FW1jzw0EqxAbo2y6eGzrWpWW8f0cPJSIUQkGhICLhwDnHt1uCRw/Tl2/nWGY2rRPiuKFjXX7Rpg4VivnoQaEgIlJC7D+cwb+/S+Ptb7awbuchKpaJpm/bOgwpxqMHhYKISAnjnGPJ5h+DRw/f7+B4Zjbn16nMoAsS6dsmPqSjtSoURERKsH2Hj/PBd9uYlJzG6h0HiI2Oovf55zLogkS6NKxOVFTRDsinUBARCRMrtu1ncvJWPvhuGweOZpJQtRzXdUhkYFIC8VXKFcnPUCiIiISZoxlZfLbyByYnb2Veyh7MoFvjGgy6IJHLW5xDmejAGb+3QkFEJIxt3XuYKclbmbokje37j1KlfAx/7XM+fdvGF/zNedAzmkVEwlhitfLcc0UzfndZU+am7GZy8lYSqhbNqaT8KBREREqwQJTRvWlNujetWSw/L/wGBRcRkZBRKIiISA6FgoiI5FAoiIhIDoWCiIjkUCiIiEgOhYKIiORQKIiISI6wG+bCzNKBzWf47TWA3UVYTjjQNkcGbXNkOJttruecK/AOuLALhbNhZsmFGfujNNE2RwZtc2Qojm3W6SMREcmhUBARkRyRFgoT/S7AB9rmyKBtjgwh3+aIuqYgIiL5i7QjBRERyUfEhIKZ9TaztWaWYmYP+F3PmTKzRDObYWarzWylmf3OW17NzL4ws/Xe16recjOzp73tXm5m7XO9181e+/VmdrNf21RYZhYws+/MbLo338DMFnn1TzKzWG95GW8+xVtfP9d7jPaWrzWzXv5sSeGYWRUzm2pma7z93aW072czu9v7d73CzN4xs7KlbT+b2ctmtsvMVuRaVmT71cw6mNn33vc8bWZ2WgU650r9CwgAG4CGQCywDGjhd11nuC21gfbedCVgHdACeBx4wFv+APCYN30V8AlgQGdgkbe8GpDqfa3qTVf1e/sK2PZ7gLeB6d78ZGCwN/08MNKbvh143pseDEzyplt4+74M0MD7NxHwe7vy2d7XgNu86VigSmnez0A8sBEol2v/3lLa9jNwMdAeWJFrWZHtV+AboIv3PZ8AV55WfX7/goppJ3QBPss1PxoY7XddRbRtHwKXA2uB2t6y2sBab3oCMCRX+7Xe+iHAhFzL/6ddSXsBCcBXwCXAdO8f/G4g+uR9DHwGdPGmo712dvJ+z92upL2Ayt4HpJ20vNTuZy8UtnofdNHefu5VGvczUP+kUCiS/eqtW5Nr+f+0K8wrUk4fnfjHdkKatyyseYfL7YBFwDnOuR0A3tdaXrNTbXu4/U7GAr8Hsr356sA+51ymN5+7/pxt89bv99qH0zY3BNKBV7xTZi+aWQVK8X52zm0DngC2ADsI7rcllO79fEJR7dd4b/rk5YUWKaGQ1zm1sO52ZWYVgfeAu5xzB/Jrmscyl8/yEsfMrgF2OeeW5F6cR1NXwLqw2WaCf/m2B8Y759oBPxE8rXAqYb/N3nn0vgRP+dQBKgBX5tG0NO3ngpzuNp71tkdKKKQBibnmE4DtPtVy1swshmAgvOWce99bvNPManvrawO7vOWn2vZw+p10BfqY2SbgXYKnkMYCVcws2muTu/6cbfPWxwF7Ca9tTgPSnHOLvPmpBEOiNO/ny4CNzrl051wG8D5wIaV7P59QVPs1zZs+eXmhRUooLAaaeL0YYglelJrmc01nxOtJ8BKw2jn3ZK5V04ATPRBuJnit4cTym7xeDJ2B/d7h6WfAFWZW1fsL7QpvWYnjnBvtnEtwztUnuO++ds79EpgBDPSanbzNJ34XA732zls+2Ou10gBoQvCiXInjnPsB2GpmzbxFlwKrKMX7meBpo85mVt77d35im0vtfs6lSPart+6gmXX2foc35XqvwvH7gksxXti5imBPnQ3AH/2u5yy2oxvBw8HlwFLvdRXBc6lfAeu9r9W89gY8523390BSrvcaBqR4r1/5vW2F3P4e/Lf3UUOC/9lTgClAGW95WW8+xVvfMNf3/9H7XazlNHtl+LCtbYFkb19/QLCXSanez8BfgTXACuANgj2IStV+Bt4heM0kg+Bf9rcW5X4Fkrzf3wbgWU7qrFDQS3c0i4hIjkg5fSQiIoWgUBARkRwKBRERyaFQEBGRHAoFERHJoVCQiGdmWWa2NNcr31F0zWyEmd1UBD93k5nVONv3ESlK6pIqEc/MDjnnKvrwczcR7He+u7h/tsip6EhB5BS8v+QfM7NvvFdjb/mDZnafN/1bM1vljXX/rresmpl94C1baGatveXVzexzb4C7CeQap8bMhno/Y6mZTTCzgA+bLKJQEAHKnXT6aFCudQeccx0J3hk6No/vfQBo55xrDYzwlv0V+M5b9v+A173lfwHmuuAAd9OAugBmdh4wCOjqnGsLZAG/LNpNFCmc6IKbiJR6R7wP47y8k+vrmDzWLwfeMrMPCA5FAcGhSK4FcM597R0hxBF8uMoAb/l/zOxHr/2lQAdgsfeQrHL8d0A0kWKlUBDJnzvF9AlXE/yw7wP8n5mdT/7DF+f1Hga85pwbfTaFihQFnT4Syd+gXF8X5F5hZlFAonNuBsEHAFUBKgKz8U7/mFkPYLcLPvMi9/IrCQ5wB8EB0AaaWS1vXTUzqxfCbRI5JR0piHjXFHLNf+qcO9EttYyZLSL4B9SQk74vALzpnRoyYIxzbp+ZPUjwiWnLgcP8d0jkvwLvmNm3wCyCQ0XjnFtlZn8CPveCJgO4A9hc1BsqUhB1SRU5BXUZlUik00ciIpJDRwoiIpJDRwoiIpJDoSAiIjkUCiIikkOhICIiORQKIiKSQ6EgIiI5/j8WAH+w7gr9vQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(all_epsilons)\n",
    "plt.xlabel('Episode')\n",
    "plt.ylabel('Epsilon')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"create_an_agent\"></a>\n",
    "# Create an Agent\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmp_dict_q_table = q_table.table.copy()\n",
    "dict_q_table = dict()\n",
    "\n",
    "for k in tmp_dict_q_table:\n",
    "    if np.count_nonzero(tmp_dict_q_table[k]) > 0:\n",
    "        dict_q_table[k] = int(np.argmax(tmp_dict_q_table[k]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_agent = '''def my_agent(observation, configuration):\n",
    "    from random import choice\n",
    "\n",
    "    q_table = ''' \\\n",
    "    + str(dict_q_table).replace(' ', '') \\\n",
    "    + '''\n",
    "\n",
    "    board = observation.board[:]\n",
    "    board.append(observation.mark)\n",
    "    state_key = list(map(str, board))\n",
    "    state_key = hex(int(''.join(state_key), 3))[2:]\n",
    "\n",
    "    if state_key not in q_table.keys():\n",
    "        return choice([c for c in range(configuration.columns) if observation.board[c] == 0])\n",
    "\n",
    "    action = q_table[state_key]\n",
    "\n",
    "    if observation.board[action] != 0:\n",
    "        return choice([c for c in range(configuration.columns) if observation.board[c] == 0])\n",
    "\n",
    "    return action\n",
    "    '''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('submission.py', 'w') as f:\n",
    "    f.write(my_agent)"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"evaluate_the_agent\"></a>\n",
    "# Evaluate the agent\n",
    "[Back to Table of Contents](#ToC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from submission import my_agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My Agent vs Random Agent: 0.4\n",
      "My Agent vs Negamax Agent: 0.0\n"
     ]
    }
   ],
   "source": [
    "def mean_reward(rewards):\n",
    "    return sum(r[0] for r in rewards) / sum(r[0] + r[1] for r in rewards)\n",
    "\n",
    "# Run multiple episodes to estimate agent's performance.\n",
    "print(\"My Agent vs Random Agent:\", mean_reward(evaluate(\"connectx\", [my_agent, \"random\"], num_episodes=10)))\n",
    "print(\"My Agent vs Negamax Agent:\", mean_reward(evaluate(\"connectx\", [my_agent, \"negamax\"], num_episodes=10)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "0f0f043967bb40b3a5d848c028977045": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_8a9bbbc8298f40268e68c686065466a4",
        "IPY_MODEL_c23bcea5b79c450e9233e1607e882a7a"
       ],
       "layout": "IPY_MODEL_baea61270cdd4495a2a82a9e6ef25673"
      }
     },
     "38b8aa2352b14877a540222285da0e94": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "789876113059481baf597e3d96fd47a8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7dcf549cac8442dea877f5cf32716b55": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "8a9bbbc8298f40268e68c686065466a4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "IntProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "IntProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_789876113059481baf597e3d96fd47a8",
       "max": 10000,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_38b8aa2352b14877a540222285da0e94",
       "value": 10000
      }
     },
     "baea61270cdd4495a2a82a9e6ef25673": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "c23bcea5b79c450e9233e1607e882a7a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_c6eac9c61e5946d8a46356dd1e678932",
       "placeholder": "​",
       "style": "IPY_MODEL_7dcf549cac8442dea877f5cf32716b55",
       "value": " 10000/10000 [5:01:19&lt;00:00,  1.81s/it]"
      }
     },
     "c6eac9c61e5946d8a46356dd1e678932": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}