Optical Character Recognition (OCR)/OCR License Plates/OCR_license_plate_text_recognition.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OCR of License Plates\n",
    "\n",
    "Dataset Source: https://www.kaggle.com/datasets/nickyazdani/license-plate-text-recognition-dataset"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Install Necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pip in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (22.3.1)\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Requirement already satisfied: torch in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (1.12.1)\n",
      "Requirement already satisfied: torchvision in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (0.13.1)\n",
      "Requirement already satisfied: torchaudio in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (0.12.1)\n",
      "Requirement already satisfied: typing-extensions in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from torch) (4.3.0)\n",
      "Requirement already satisfied: numpy in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from torchvision) (1.23.3)\n",
      "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from torchvision) (9.2.0)\n",
      "Requirement already satisfied: requests in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from torchvision) (2.28.1)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from requests->torchvision) (1.26.12)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from requests->torchvision) (3.3)\n",
      "Requirement already satisfied: charset-normalizer<3,>=2 in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from requests->torchvision) (2.1.1)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages (from requests->torchvision) (2022.6.15.1)\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade pip\n",
    "%pip install torch torchvision torchaudio\n",
    "%pip install -q datasets jiwer"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Import Necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, sys, itertools\n",
    "os.environ['TOKENIZERS_PARALLELISM']='false'\n",
    "\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "from PIL import Image\n",
    "\n",
    "import torch\n",
    "from torch.utils.data import Dataset\n",
    "\n",
    "import datasets\n",
    "from datasets import load_dataset\n",
    "\n",
    "import transformers\n",
    "from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer\n",
    "from transformers import VisionEncoderDecoderModel, TrOCRProcessor, default_data_collator\n",
    "\n",
    "import evaluate"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Display Versions of Relevant Software & Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        Python: 3.9.7 \n",
      "        Pandas: 1.4.4\n",
      "      Datasets: 2.4.0\n",
      "  Transformers: 4.21.3\n",
      "         Torch: 1.12.1\n"
     ]
    }
   ],
   "source": [
    "print(\"Python:\".rjust(15), sys.version[0:6])\n",
    "print(\"Pandas:\".rjust(15), pd.__version__)\n",
    "print(\"Datasets:\".rjust(15), datasets.__version__)\n",
    "print(\"Transformers:\".rjust(15), transformers.__version__)\n",
    "print(\"Torch:\".rjust(15), torch.__version__)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ingest, Preprocess, & Split Dataset (into Training & Testing Datasets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 16000 entries, 0 to 15999\n",
      "Data columns (total 2 columns):\n",
      " #   Column     Non-Null Count  Dtype \n",
      "---  ------     --------------  ----- \n",
      " 0   file_name  16000 non-null  object\n",
      " 1   text       16000 non-null  object\n",
      "dtypes: object(2)\n",
      "memory usage: 250.1+ KB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4000 entries, 0 to 3999\n",
      "Data columns (total 2 columns):\n",
      " #   Column     Non-Null Count  Dtype \n",
      "---  ------     --------------  ----- \n",
      " 0   file_name  4000 non-null   object\n",
      " 1   text       4000 non-null   object\n",
      "dtypes: object(2)\n",
      "memory usage: 62.6+ KB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "path = \"/Users/leedunn/Desktop/Projects to Train/OCR LP Text Recognition/lpr.csv\"\n",
    "\n",
    "dataset = pd.read_csv(path)\n",
    "\n",
    "dataset.drop(columns=[\"Unnamed: 0\"], inplace=True)\n",
    "dataset.rename(columns={\"images\" : \"file_name\", \"labels\" : \"text\"}, inplace=True)\n",
    "\n",
    "# train/test split\n",
    "train_dataset, test_dataset = train_test_split(dataset, train_size=0.80, random_state=42)\n",
    "\n",
    "train_dataset.reset_index(drop=True, inplace=True)\n",
    "test_dataset.reset_index(drop=True, inplace=True)\n",
    "\n",
    "print(train_dataset.info())\n",
    "print(test_dataset.info())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Show First Samples in DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>file_name</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19994.jpg</td>\n",
       "      <td>2730UB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16385.jpg</td>\n",
       "      <td>2124SA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>25271.jpg</td>\n",
       "      <td>D56501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>23041.jpg</td>\n",
       "      <td>NK8918</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>20167.jpg</td>\n",
       "      <td>5362HT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>19736.jpg</td>\n",
       "      <td>HW1723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>12907.jpg</td>\n",
       "      <td>9683PA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>21581.jpg</td>\n",
       "      <td>5172QB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>21036.jpg</td>\n",
       "      <td>N58328</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1821.jpg</td>\n",
       "      <td>0574EU</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>36852.jpg</td>\n",
       "      <td>2889HS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1037.jpg</td>\n",
       "      <td>YC1987</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    file_name    text\n",
       "0   19994.jpg  2730UB\n",
       "1   16385.jpg  2124SA\n",
       "2   25271.jpg  D56501\n",
       "3   23041.jpg  NK8918\n",
       "4   20167.jpg  5362HT\n",
       "5   19736.jpg  HW1723\n",
       "6   12907.jpg  9683PA\n",
       "7   21581.jpg  5172QB\n",
       "8   21036.jpg  N58328\n",
       "9    1821.jpg  0574EU\n",
       "10  36852.jpg  2889HS\n",
       "11   1037.jpg  YC1987"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_dataset.head(12)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Create Dataset Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "class License_Plates_OCR_Dataset(Dataset):\n",
    "    \n",
    "    def __init__(self, root_dir, df, processor, max_target_length=128):\n",
    "        self.root_dir = root_dir\n",
    "        self.df = df\n",
    "        self.processor = processor\n",
    "        self.max_target_length = max_target_length\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.df)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        # get file name + text\n",
    "        file_name = self.df['file_name'][idx]\n",
    "        text = self.df['text'][idx]\n",
    "        # prepare image (i.e. resize + normalize)\n",
    "        image = Image.open(self.root_dir + file_name).convert(\"RGB\")\n",
    "        pixel_values = self.processor(image, return_tensors=\"pt\").pixel_values\n",
    "        # add labels (input_ids) by encoding the text\n",
    "        labels = self.processor.tokenizer(text, padding=\"max_length\", max_length=self.max_target_length).input_ids\n",
    "        # important: make sure that PAD tokens are ignored by the loss function\n",
    "        labels = [label if label != self.processor.tokenizer.pad_token_id \n",
    "                  else -100 for label in labels]\n",
    "        \n",
    "        encoding = {\"pixel_values\" : pixel_values.squeeze(), \"labels\" : torch.tensor(labels)}\n",
    "        return encoding"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Basic Values/Constants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL_CKPT = \"microsoft/trocr-base-printed\"\n",
    "MODEL_NAME =  MODEL_CKPT.split(\"/\")[-1] + \"_license_plates_ocr\"\n",
    "NUM_OF_EPOCHS = 2"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Instantiate Processor, Create Training, & Testing Dataset Instances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "loading feature extractor configuration file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/preprocessor_config.json from cache at /Users/leedunn/.cache/huggingface/transformers/832950d3437b5e524332931523cb1fc015003d2d55692b5f8ef437d253ad17dd.3ce3b627e98330dd48cd2a0c208fd2bda7a8a77f12b4253b922d1c1ce40e01a3\n",
      "Feature extractor ViTFeatureExtractor {\n",
      "  \"do_normalize\": true,\n",
      "  \"do_resize\": true,\n",
      "  \"feature_extractor_type\": \"ViTFeatureExtractor\",\n",
      "  \"image_mean\": [\n",
      "    0.5,\n",
      "    0.5,\n",
      "    0.5\n",
      "  ],\n",
      "  \"image_std\": [\n",
      "    0.5,\n",
      "    0.5,\n",
      "    0.5\n",
      "  ],\n",
      "  \"resample\": 2,\n",
      "  \"size\": 384\n",
      "}\n",
      "\n",
      "loading file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/vocab.json from cache at /Users/leedunn/.cache/huggingface/transformers/fda4022f9ad7b6db6decb3cef7525279bd3702e076ff5d3a435fb5ae63b107d6.647b4548b6d9ea817e82e7a9231a320231a1c9ea24053cc9e758f3fe68216f05\n",
      "loading file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/merges.txt from cache at /Users/leedunn/.cache/huggingface/transformers/8ec658daff7d6596364ed0266adcc1aa7808de9af3a345d47a1533f17d1a45f6.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b\n",
      "loading file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/tokenizer.json from cache at None\n",
      "loading file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/added_tokens.json from cache at None\n",
      "loading file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/special_tokens_map.json from cache at /Users/leedunn/.cache/huggingface/transformers/525c89cbf7be250f1734dd9ffc81403fdb59520f008dc95338d0ec2eb166117a.cb2244924ab24d706b02fd7fcedaea4531566537687a539ebb94db511fd122a0\n",
      "loading file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/tokenizer_config.json from cache at /Users/leedunn/.cache/huggingface/transformers/986b031569a78b816741fcacc718b9fbe8e978442ed3fe70fd6fc5c422e8d585.3e1c000f87cbb0f076c5db660ba1bd5ce4c4f903a5d2fb381ab952e53953fc4a\n"
     ]
    }
   ],
   "source": [
    "processor = TrOCRProcessor.from_pretrained(MODEL_CKPT)\n",
    "\n",
    "train_ds = License_Plates_OCR_Dataset(root_dir=\"/Users/leedunn/Desktop/Projects to Train/OCR LP Text Recognition/cropped_lps/cropped_lps/\",\n",
    "                             df=train_dataset,\n",
    "                             processor=processor)\n",
    "\n",
    "test_ds = License_Plates_OCR_Dataset(root_dir=\"/Users/leedunn/Desktop/Projects to Train/OCR LP Text Recognition/cropped_lps/cropped_lps/\",\n",
    "                             df=test_dataset,\n",
    "                             processor=processor)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Print Length of Training & Testing Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The training dataset has 16000 samples in it.\n",
      "The testing dataset has 4000 samples in it.\n"
     ]
    }
   ],
   "source": [
    "print(f\"The training dataset has {len(train_ds)} samples in it.\")\n",
    "print(f\"The testing dataset has {len(test_ds)} samples in it.\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Example of Input Data Shapes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pixel_values  :  torch.Size([3, 384, 384])\n",
      "labels  :  torch.Size([128])\n"
     ]
    }
   ],
   "source": [
    "encoding = train_ds[0]\n",
    "\n",
    "for k,v in encoding.items():\n",
    "    print(k, \" : \", v.shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Show Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAI4AAABCCAIAAAAOtXmQAABG/klEQVR4nE29Z3NkS5Id6B76ilRIaJSuev3UdM8MucsdWxr31y8/7HJJm57pfvqVQBVQUCmvDOn7IfCKxAdYWRky894bEe7Hzznuid/8n/8XImomtJSCcQlMcC4ZF4yHEEKKKLhSiguREFIK07p489Wr1Wr1+fPnFCHGSER933MuvPecS2NMjLHveyHEZD4JjIaxC9ZxQIFMM1GV5aSqg3XGmKKuiqJQRldVdbBczudzJbmUkiOLMXLOpZScwHuPibTW0QfnnOAcABCxKHQY+9l8ok1prXWBtNZC6pBijNFUZaQw+lFKab3dbtfO2pms7DjGlCIlAFCFiZQ2m804jkKIdt9sNhtyARELY2bzeUhRGp0SxRhjoPV6vVqtUkpEfBzH/X4fQuBMeu8R2Ww2ffPqWUqBMSal5JynlLz3MUbvPQBUVXVwcCClbJpmu91aNxyfnc7ns4vTs77vC6Wrqlrf3iul3l9+WO+2m83GBq+1NlUpAICCJ8mLQldFzSlRBEYJEkrFJSqhhdYFExgShOAYY/v9vus6YwxD0bYtEdV1XRRlCAGRa62JqCiKGCMipuC11oezxWw6LYRKPsQQkGAxnQkhuJJCCME4Edlx3O12ZaG99ylExhgAIBHnXEvV7vb5zpWQWmvvfYqxLI0d2vVWK2UiQUqAnAGxBMSVZIxZ75wbEyZCopgYpC7s8lIloISgtQ4pbjabYRiEEGM/dF3HCDgyZ20iavuunk2lVERkR7/dbtu2FUKFMDoXrLUxRq04IiIiAOz3+5QCIgohOOdE5L0PIeR/AwDnPD/D/X7vnOvtOJtNu65zw6iENFpvV2tjzG63a4d+GEcfg4thcFZgitaPgvO6Ls9PzwopICFjIFAgI84kV5xzGZLvR2fdsJhOleRKyLqupdSr1YoxZozRyhDRH5coQwjDMIxu8BCRQ6HNpK5LqSEm71z0wUjFGGOMEUPGGJdCKSWEQILgPMXIpCQi570QQinFpEhEEQgYcikiJSLiUoAXow+RnNQKGA699T4mIFUYa4dIKUY/DB0xqMtSMO598tYzwRnnMcUQYkwJCMuiEkJw5FIqgSyFCIkY4xEgJuJEREQITHBTVkVRWGt1jMhYSsmYEhGJqCy0MQaREPHx7oiklDHGcRwBIKWUUgKA/NCk0YSJceF98D6M/biyq/V6bYxZrVZEFCkRQkJIRCIGRzFqLY9Pjv701evlfG6UMYXmwBCBcwEcCTAE31vrx3F5MGeMjeM4nU4ZE7vdrigKrXXfDZw/bi7OJWPMOdePnaOQUuCMFUorITlBcD6EgImUUlJKAAAAJoWUkjG2Wt1XRWGtTSnlhc+R5Gh5qLWOMcYYtdb5zBmjvB2stYyxqpykBJvNLh905NJaW5eVUPzhLoXkjTKIKAROJpPJbKqU6sYhEXHOnXNd13HOg/cpJcn4MAxutFLK+dEyEeVjIaWu67rvR611vjZnw5elcs5xjkCx1EpKmVIKIeTtm+8xpSSEKMsSAIZhCCEwKQLGoigk413bblfr65vPd7e3UsrtdiullPpxj6aUhLdjXZcvnz/7/ttv/uG7fzg8mFdFXRTajx4goZCcIzKBjFIEghh9KAodQhJCOOe897PZwjn37t07zjklyNfHOZZlqYwc3Cgk01pXpii0UYynEKPzQFRVVU5sIQRAzCsdvfXW3Y/3q/sHqdX5+bkxpus6H8PR/Fhrba3VWldVlW8eUuy6DhGrauJ9tJ6sjyklImibnqFQSrnB+eTVXAGmsR+qqjo9PZ1MJru2sc4ZY1Dwtm0558kHBOCA+/2+a1ouhanKzXa737d1XR8dHTnn7u9X1trDw8PZbMaZBICiKBBxGGxKASFVVSGlHIah7/t8qjjnxpicrhhj1tqu62KMyuh6MTHGhBD6trt89/7j9ZWPIaToYwDJkRJ4H0LwKQpI8eL89D/80z/+8z/95cXTF8YoyZVSAvLCEBJFYJwxYEwg0n67K8syJcgJ0xiTt8DHjx9DCHZ01toQAgAIoYRgTDCuuZGq0OZgNj86PFzM5uWBafb7qqqUUtbaHF2C9845o7Qdxru7u99//30ym85mMyJar9dd1/kYj4+PldG6Kk1dee+jc8iBG4XAE+MBUqQUUgrOO+c+fLicVvXBweLt2/dK8OX0QCp+c3MnhGCMVdMJcWat1WVhjDk/P0dEP1pr7dD3fd+nlCDGoijWm421tq7rfMG7XTOOo1JqsVjU1RQRi6IQQjgXYvRSMCFYzmHDMGR8wRjLce/LCUNErXVRlsvlsqrrEAIdJWutMBqlSCnJwnDOCcDHMNrROSeq0rx68fzP//Dd8ycXWsuUAjFOgEwIIiAfQ3TREmBiXDLGENGNY94sAhklGtrm7vP15bv3u91uv9/HGGNMIQREFErpQmsjMxY4OTwKr18X2lRVlYic9znlMgKGOAzjdrslipuH1c3NzX6/L6rSe79v29Vm8/DwMAa/3e+evXi+WB74GHwMXIoUotIFImeMkfVMSCl0DLTdPHz48KEuyq5t3v3+tirN6cmJlPzm5sYY03cjZ7Kup0o5abSUUghBRB68c2G3bT5f3z7c33MhTD1Z7/bXN3dtPwITjLGm6xLA6BwhcsWJiBgxyRRXmGTwdrfbffz48d27d/v9Pr9zDtpElON5BoTGmOl8lgSbx8QYq4sSuOgH2w5jCEEpRQSYyIcUIiVCMaur06Pj06NDrfUwdDFGrEBp0fUNEaWUMhxnjAEiQ14aMwxDRn0MxTiOzrm+77uuu7+/32w2iMgYz9tKKemimy3my8UBY8x7P45j07Wcc+AsxkgpAYAxhVIqpdQ0zXq9uX243+12XIrJZIKcj+PY9l0Curm7td5NFvNT7yNRSmk6nTp0mjGGgoikS6YsrPWI2Lbt6mETKmu02q53Yz9sHlbA2X6/3+337dATkdaaSZEDb0op55W8HYEzLoTWerfbPdyvr66ubm5u2rYty1IIMZlMQgj5Jd77DO3yC4vKbDaru7u7y8vLpmmMMTm1ZwSYVy6EEEIoy7JezP5pMYsxMsZ0WZiqlFpxJQlgGEfGGOf8C2gUDLEqy7qqBOcIIDgPIWw2m0QEAJB/A+SM4hCNVJwjYzwEp40hiLe3t+M4brfrzWa12eyklEqpXEzEGIBD9AERpZQhhPVmM18slstlPmd906aUiroahmG9Xo/j+Nu7tx+vPlnvptOpEGKz2cQYq6rqum7f7IuiCCHs9/t6OinLMgFVkxoJiHCz2SBj0+m03Xc5+p+dne3Xq6urKyEEIt7e3k4X86ZpvPdlWYYQRme5kgAwnc1yPEgIGelkfNSNg1vR9fV113W5GOr73lp7cXGRz4r3HhHn83lKqe/76aymmHbN/u7hftfsu67b7LbDMHRdV1VVXddCiPzmIYTNblt1zTfffxdTOjs6EkLElJgU7dAzAuSMcY6A3nvKCEszIQh5Ai0kr2rgLIOZru9jjAiglDLGEFH+yMX5hdY6pTQMQwh9XsJ8zkJ4rDMeQR1jQoiiLrTWzrnNZuO9r6aThEAA/ThwzoVWHFBIyZ1jjAFnOdlMZrPTs7PJZAIMx3F0o0fO6rqezGZa64zsuZIJoW1bIYRRhRACIuSMPY5jWVdlZexggJJXggtMKVlrjTHEMB/uQIlbHlNq2jZDyhRj13WDHYkoxtgPY3O3HgYL8IgF6rrOief4+HixWBhjcq7lAoVkKaXgXC5v27bNAclau9/vDw4OlFL5lGTIzjlXSu3aZj4M1tpcULddNwwDANRFKaXUUqWUKCXOuaCUvHPBeyTIAYGIElFZlvm5CykzwsFEwbrV/cNisWCM2WEUQlRVdXpy5OxwenLirE0xAoBWRV4tLsVkOlFGc84jJSZ4WVdlXQFnShUoeHKPKw0A8/m8qKtI5GOYzWYnZ6eMMR+DUNKl2H7+rJQqisIYk08JxORDAEgZPEopI8WUovfWeztfTGezWbAupZhSIIo+xdDt68nMmFIIpXVRCKG1zjnPe6+EhESUMMyC97Ft+9v1+rd3b4GhLoxzzsc0Om/twNcrZbRQkgkMgx9sr7VORP3YITEmhalKU5XjOIYQUHBpdASywXNK+SQ0baO1nqYkmdTSKKGttUQohJLKxBi51FoXSsjgvPc2pSR872w/2N5651iUSImhQEQpNQCEkIhoHBwXKISazWYMMIcImUtU5xhjs9nsn//5n4+Pj29vb2OMgqsQgvc+AaHAalJLKYGz5XL59MXz2cGCBONCBMr5AXyKApkqTLLYj8PDdgOC77p2GAbvvdY6R6dMzBwfHy+Xyy8gCpGEEIxDSsF7N46jCw6QptPpbDGLMSBFpVnwlnOeT0bXdd3Qj+OojJFaEQIAWGsRkWLy3ksp83G5vLzcbHaT6fTwcJpPQ9d1Dw93TdNsNpu2befzaX70jLGYvPdeS7NYLF6/fs05v7297bpOa310dFSWpZQy50LnnBCiKIqTk5PZbFaWZT7TUsqMMx8RAIBFZocxWkdEIvkUfYqBgk8UPAXQmjEumn5QSpZKem+HwRLFUpdFWWkpfLBEVJQ6RchF+MHBQVEUZ2cnTdNxzoUQGUE4720KpiqFEAQwmUxOT0+rqgoh2OAZY6Y0WiqJLPrg2vZ+vSom9WQ2rSa1Mnp0FhNjguddj4gxxqIotFRN0+Rr4JwxJIHAkXywfd92fQMAk8OT+XyqtdRKdF0T3BhC2G631kdjjFIKEb33OI4hxUdChLFxtN57JeV8Pp9Op0qpqqq01oWptNaj7bfbbX764zhmFCeECNEhIyFERuRlWR4fH6eUyrLMpVWOdZm8yLxG3/fGmPOLi9lsxhGHYRjHMZfVGeOM/TD2PUaClBRyKaUgIkwIRIicgCBCSolzEUIoiqIsSueEt8Fan2RCxHEc224PAHVdI/AYIyLP92OMOT5+5CBijNbaEKModGLovbfeSSl1XTLOkVJhquA8cBZStOMARCi41Gp5dPgC4nJxcHFx4b2HmDjn2+222e0555mwyJtOMEYhxhRRA2e8KLT3XkiWEV1VFYcnh5LzyaRyNtej/d3dfYpweHh8cHBQVFUC0lpHShkKAUAK0WlVFuWBUl9//XXbDb+9e5+IJpOJUkob6ZwDSMfHx5l1zS/sh/Q/Sb8I+cScnZ2dnp7m6tN7XxRFxsAppcxlCCFms1k9mSFnwbmUkmA8gz0lJOoEiRJEznghlDFGBBfHwdnRQwQhZQTyPjof26YPPnW8s9Z2bRtCGPU49D1AGG1PROM4MhTOOQDo+3a5PBJCEHnnxr7vM5ERU6rlwjpnrfUxAGe7rs28eFVVth84oBvGdrOrq2q5OJhOp/PlAS90YYwuTFGVDNA5Z7x79ea1YHw2mzW7nRtHpRTXxlrrw1gEXUiFAEbp+XRycnJkjJlPZ4hU1/XiYDZ27b7ZrdfrEIIdAwDsdrtIhJxlGCmEyAcrI+8MB+q6fvLkCTGeCIrCpJSk4qenp+v1ejKplstlCG69XiPiaHshRFWVnIsYk5RyPp9/OSI5F2Q80nWdcy7v7MycJcLBjiGESOS9H7reW8c558YwQIwphSiRC87FYPt23HdDa4M1ilFKznvr3cPtXQjBjqPtB++9YLw0pTZqu30QkhljqqqazWaT6VTrIofgjBJ3u912u2/b1jkXgMx1NXoXY+RK5v3OUAjJKESKaVLX5NxmtZrWE47Apfj2zZ86N2aKDIBBTH70wYYQEpdiv91uVivG2NHR0XQ6HYZuupia0QBxRA4AyhTLo8ODgwMkYALn8/l8MW12MlDcd3vkrKjq4NNm17SjBcSqLnyMiDibzabTaaG1YQVnPKTEBK+q6vDoSBldaDPYvipLbcz9/X2iIJTcNft92+TFKEtDAMYYa73WxbSaAKQQUoreGFPXddd1jLG+7/u+zwWTtbYfBmNM07WZc9psV/cPt5vtqjIFESkhBXCAFBLw5IVj1qZBVkyXzPo+hLDZNb/++uvd3YMfbQpkx3G33lJMRpcE0fmBCby4uMhE3MvXr4+PTxeLxTBYRN52wy+//n55eblarUIIZlIxo0bvKKHWWggRvQ8uABBC6pt2MZuWSgJFxcXt9Yfvv/uH0+MTFmA6Lb33N5/vMvLe3K8xQNu3D7e9dYMLAX7+IVfl08XB4uDw6ZMnFxcXm83m5OTo8PSEiIK3xbQu6+rD5aWz4/X19Xa7fffhUvDa+XR991BPJ8BxsD1wKOtiPp8X2swn0+Ojo7Pjk7IoEtBhitViVtQVJnLRaSl9jMDBObdv2/v7+7uH+xjjZDI5OTmSRWnqWYzoEBlA9MH2VgqWgt+uPzdNc319zRiUk3q73Z5fXJhCXX3+lAs+pZTW+reff3r326/r+7tOqazVaWkUZ4wJm0DU01IoHslbb1cP6893t58+Xf/444/ttm+aJtgQrOvbnhETQnhv69lEG2kHp7Sez+eRcL9vnz5/Np/PE9Fut7u+vv7tt98+390652RppNE2eCCmpWTIYwgpRA7gndvtVovJtCglJFICz8/PS6P/8g//2BIwQsbE6uHhr//6b8MwcClub29HZ/u+s3ZAgWVltDFCiKvbh+n07u7uYb3ZIdJ8Pp/PpZFqGDDF2A7t7cP9drV+/+Hddr159/4yeB4jAaLQAhhEjEyQ1OLk5ORoeWiPTjP5crw8nFTTk5OT0VllTEZJnHNDxKWw1v7666+/v/39w4cPLobZbPZ6fD1dHJxeSMMkABCDBEhEzoX9dnP96dPb336/urrikp2enj57+aKu67IyQoib+9tf3/7Wt63W+ubmZhj7cRj2u11ZloiY8XbOgsLHMNjRueCcW203v//++2+/vf3l11/H3vdNSxEYIoUkGGfEuq7ZtXtT6n3TAMBisfh8d3t4eBhS/PbbbxnnMcbRu96OzrkYI08JADggIKYQQwrRh5QSB7DjuFqtut1eKvTWMU6jdy9fvvx8ey2ECcH98vNvf/3rXz9cftg37W63y7hosEPft4GSlJwxFlKsZ3MptDHm44fL+WJaacOBlosDZOBHu91uP7x99+njx/fv3+/3+/Vqu90MjIkE0UXPBHLFgVGMfjabnZwenx6fHi4OLs6e4Fc0qeq6noR9FIwJrTliAshM0hD62883P/3w4y+//BIpzWYzCvHV8xeIWGgdQiCImaj1zu12u8vLy//xP/7Hx48fieKTZ0+Bs2+/+44gfvr06d/+/rcffvhh87DKRMF+1wAxINbsuxgjZGVVa865uLu7W61W+/1+u99fX1//+PNPv/369urqqu8dEiymi+l0LrlUQlAAaWSCxCTbNvv9fr/abvTN9cHBwexg8er168l8dnB0eHFxMQzDcrkkBFWYDC4oZmoqRudjjAC03+1ijKumSdEOwyAEzhcLG3yInnN5d3fz73/766+//tw03Xqz/fTpExEdHh5O57OiKAIlxsB7PwzD/ef7QMmP9uH27mA5r5SBFNlXXy1ms67r3r1798tPv/7y60+Xl5fDMAhuKPG6NkwodIBIxugxjPumXW9Wn6+vplV9cnL6H/48nh4dj+OYS7qiKIqi4JxDjIEoOLderzer1eXl5afLS2Bsv93NJtPtemP7oa40JoKUODJljEBGMW6320+fPl1dXVk7ZF5qv9vxFj+8f/+3v/7bx+urbt8g8nEcY4xG6bqu9/uWcwaJgMC7GDAJY8qqmpiyJKKH9erd5YdPn6/X203fu6OD5enF+avnLwpdCMaRqO/7bbMb3XBzc9P2XdO1gLjb759dfrhfr+rZdLlcPnvxHDnLBERG7USU/lgqiimEEKJ///795eXb/X4/9p33fmqqalLPD2Yn5yeru9Xnu8+r7erh4e633942bS+lUsocHh6/ePlSKu69jzG0Q980TfPrr24Y16tVirFtdkfzg+V88frlSwZoB/fxw+Xb33//+cdfrq6uQghnpxdPn706OTmpJqUNFhkJxbfNTt3w+/v7tm3WD3ebzaYQ6ujo6Gh5WJalUZojk1wQUXCeITBAb23fdtv1er/dIWd2GPbbrbc2OU8qBO8xRc6YFJIJIqKxH/abbd+0Nti2bcd+oBA5l0jQ96NzmftNw2BTSoJJzmE6nXPOkcA591iccM65lIjYj8Nqu8mCPwCbTuvTi/Nvv/vu22++qYtKMM4AnbdXnz/d3t057z9cfWo2W2JIDN+/f//zzz9Pp9PT87PDw0POc72FiKilYgREmFJCZITonBvtAACXl293u53zfaHV8dnpyfnZwXK52+0ixBD8/f3t9edPm80KUE6ns++///P333//4uVLIhrGLpOQm/0WEW9ubsa2w0Ttbv/u3buL83M3/ku24ny+urm/v99ut957IcTJycnzF0+fPn16eLwESBEiIW22q/l8Op9Or66urq+vN+uHv//wN2PM2cn5ixcvDg4OQghCiEzT5aqRI0NEzIR2TG60SFCbsjRFcB5CjBQSAU8QQgjOW2szf53FihBC9AG0zOxt3/dt02VkwTkvTZUZO8EVQxz6kdLWuyQ2u/1ut1+t1j7GvhuNLhfLQ1MOk8nk+fOnT148efbiaWkqxQXn3Hu/OFkWv//e2/Hdp8v9fo8MfQwfrz69ff/u1ZvXZxfns9lMGp3lO0aQrAeiGIiIhJDAsO/7/Z5leWV01no3n8+PTo6PT0/q2fRhfV+XE8A0ulFp/eTZUyXLqqpfvnz54sWLly9e5BxbVpX3frNdTav673//9/XdLQC07fD5+vrT5cd237j5YujGh4f1ftd6F7Uqqqp69uzZn/705unzJ6enp5yjTyFRaJrm/Pz0XxGGrl3dPwxdf3t7+8uvP7+/fPcfuv9YFFX2TmUeXUopGEcCJJBcVEWZCWslpFEKEsUQKCUgSiHaRJQSxISJco2VbVuZ2o4hcMAQwtDbrh3YRBSFNtrMpou6rmOMQkgkQBAxEiIXy+VysVjMZrPpbHZ+fu6csz70fV8UxcXFxYsXL86eXFS6UlJqIW3wB2PXj8Nqsz44OLi5uclcXDZnfRGnUfDMd4GPQhcUU5ZEBJeE4EZLRLvdLteDnPPDw8M3b968+dOfXrx6Wevi4f6+6/vz83Mp1d3tmgifXDz705/+9OrVqydPnuR0XZZl13Up+ifnF58+fshsN8XUt13btl+UpLZt+37MxJhSBhg+e/H0yZPz09NTLpiLLqU0jv3R0fL3n34BgKxfp5Tatl2tVg8PD1VR54vMxZCUEgkyAxBjzDuSYoSUxnHcbTZlMUUAJAqZZQL4YjTLFHautaWUWuuiKBgTmXbinIcQxnFkCzafz3e7PSIy5ErhZDKTUgs/+v2+TQmmk/k333wzn88f1pv84qOjoydPnsznc8WFEIJiCqMbhgE4SwgRKJukYoxPnz51zs1ms/v7e0Q8Ojra7XYhhEIbrQwgIMchxmxDWK/Xv//++88//5z1C8YNk+Lk7Ozbb79FxL7vOeeZ8XTOHx+dF0V1enL+l7/8Uy57vyjfUsqD+eLN69e//PxjaYr7+3si6rouU/VKmQ8fPmpdcC6H0U1ndYgUIx0fH79580YIYd2oUTLG+GK+2+1evXx5eXn5TirLOACjBDESEU1m02EYRmdTSlVVCSG6rtOFyY+eYhyHYbk4SCEG55WQFBPnvCorG3z2FpaTWhWmnk3lgx6bcfQuX3w2w2SqV0gGmGIkrbXWWkrF2CPZP5tNg0/r9VooITkgxCSFOJjNOefz+YHUaj6fl2WZKWEKMaXk/jBvNE2zWq1y+T2ZTLTWz58/Xy6XmZDOQVwJWWijhUzWc8aEEIYIGI8+rNfrjx8ut6v1fr93ozWVPDpYnpyc1HWNiKYqhVDPnj2bTueL+SEiF9xUVb1YLDh/xEiPVk5kWuuqKmazWXYGZjaorut8MrgQnHOplDFG6QISEcAXzTfGCJCIiCENfT+fz6uiZoDOhRiScyGlTBVzYCyf2gSEiF84XMZYPgdfTlhVVRh5lqMoxC9GJR9DPw4hxUw9A8MYo3POWotIRVFQjCklxtgXa1DmlClC27beRUQU0aexHbumhwjTejabLWx4JIwBgDGwdgjWBee7prm5u9vum5/+/sOvP/2cmSdjzMF8cXF2/uT8QnJRaJNSGrseUtLaMMTBOSEEIsUYETDGuN1ur6+v27bNjq3lcvnVV1999fr1YjazduBcRkEHh0faVGfnT72PRlaIbDKZ5RTNkSGlFDykJBh67xkDAPDeM8ayu3YYnXWOCZ4IgDFlDBMSUgoEypTSaIGMCyYYIqLgXHI1ra/qui6KSom9o8i5ZMIwofZt249DCjGEkNOVc87HoAsjtULOE0BI0QWf/2Bez+w4pj/qKi4EMIxETdf5GJkUWRGNlCCQDR4gcYGIFKMXArNnZhzH7AkjIGtt8ElKJZLzQ9f1bQeJZpOJ1NoFz5VUSmUzhkA2+Dj4frPZXF9f//jjz3/74e+5TNFaz2azs7Ozly9fPn/+XHGhhYwh7NtOCMEQKSZVGIjJ+zCOY0jknMs5IMaopTRKPXvy9Ptvv3vx4kWhzdC1KaXgk+KSCKtq4mxYzA4AMMZHPyhjkFJy3iYfANI49jmGZF0jUrLWWu9GZ4UQPobRuZBSSJEBT5GGYWjbXiCklJQUDCkJkY3WDLgSyugSWRBcAbAYiQkpTcrKqveeAUYiY8zobN5qWe3MJ+xLqkuQgDMOWdemkOLobESSUiBjBJAQpBSZzrd2GMcegAEkH+wwdlLovh+UMoJJIQRnSASCA0+BIIISelLVyuh92wKgQIaMS8Yppn5o726uf/31l59++e3HH3++/PRx6PqqKI+Pj58+ffr1m6/+5X//T6enp1IIzlgigERaKskFMcp7hGJKCMMwPDw8fPr06fPnz+2+iSlMp9Pzk9OXz54fzBeScc4lJUQUTCrkEojlxMQY2+02RCQYZ1xwBA7EBOOCG2OEkrowk9nUWjtaO9iRMZYS6aKMhNYHHxP6WCiFnD2ak0OK0acYUwgcsWmaFIhCipFCSMEn58I42GG0gRIhKK0kF9y56DxZiER5Q0RKXAouJTDGBBdCjONorWUCmRQAEFPyFD2lxBCz+YdSSBEZM8aUdVWUxnvnvK2riVIqR8XCVFJK7z0KVhRFitA0jUDE5IN3jkKERBRTyoS8dQnIjXYYhquPl2/fvv3pp59++eWX3357t28bLdV8Pn/69Om333773dfffPvtt87aGKMdRmstI5CMR+c55/t275xjTGitu7bfbDb39/dt2z48PGjDF4vF8uDgcLnM3uNpVVufOJNArCgqRJ7zU7bPMcYAKaUgkJlCZfdgVoNijPnaH8sXhoRYTydcikgUiUIIUDAmlDKFEAIwCcalYMQEQ4wmlGVlVCFQQMK8iEQIAKO1PrpElJQKMfjgvfchxfnBwWQy2a433ntrbd/3GSNoXgCilJIp6WJIMSDnpiy4FEzwRMkGb4MHhkJJrVVR6KyxaaOkEAAJkbSRZVk2TUsREDHG4JwTkMg51zdt17ZjN6YEIQRgrKoq7330drt+uLy8fPv27fWnq/v7+4eHByHE8mjx/NnzP3/3/XfffXd+dlaVpUDW932wzvaDFIIhdm2rjRnHcbBjYSpjjI/hfvXw8PCQXRucq2k9OVoeTqraWwdEk+nUN0NCPgw2m7CMKUIIDFBrzQBjCjF6xhOFuF1v7lcPf//x58vLy/V2lePG7GBRTScEECmZqhRKIWeEzIWYgBKQtXboLaTAkSEoCoEzlhIBMc65ksaYkoAraYQQwIQy2g3eBo+IQJT9ttMQ5vN5NZkoo/tx8DF0Q79v227oZ9Uiw/HE0LvgvEfEbEkDzkIINit5McaUQooRSEouBIsxElmtzBene1mW3oYQgnMOEQXjMu/NcRydtUopiVxpXZuii2nd9Z+vrn/66acffvjhw4ePnz9/lozPp7Pnz5599+23//iXv7x58yaXGpPJBAAEMgDI4rS1lgsxnU/SPgnBCFPXtTd3N3d3t/u2qSflbDZfLA/mi6UpKudiSvFAGqO5c2E37OuCex9mk9lII+ecYkIGIUbvrQ9p7Ppff/v57z/9/O79+8vLj7lmmB9MDg4ODo+Oco2Si2VEzGJhjDEmbypjChEdIQCjFFLyMQbrun0zDBYBjFQEmLNIiM67sWt2IYSMOZWQAEAUrbXRewCQXHDOBeOIiMBdDCFFFlMIoeu63tmsviIiowQxxeRDcCG4EL33NuurDHDoWsaEEpLAj2NLFIVQUjHX9jE5rkDc73dnz57s+6EqJ1LKFKNMAC40q00C+vXHX/7+tx/+v//3v3+8+vSw3vrRPX3y5M2r1//457/8x//4H1+9eqWkEkIIZH3TYiJitFgsiKgfBm1MhOhGxyRTmu+75tPnDx8+vdvuNz754P1sPn/+4tXFkxf1ZBFC8N5ay/wYKMK0mkAiI0zX9AySdx6AECl6PwzDevPw/sOHv/3tb7/99tu+ba4/XwMkraXW8ptv/vR//Kf/7fDoABH2+50plJI8BseZ9HbUUo793hTnfbBaSBaxKMvdZocESkpBGJ2/v71zFHVh+r4tjaTk2mb98PCwfljN5wcC2cHBUjBOEIVgbrSQKBEdLg6n1VQVxkESRo0pIGJ2EyqlnB0YUooek1eMSQ5AIUXLgTBF2w/BW0QcrVWS77fb7Xo9WywAYDFfutgCh8/XV0IZ3Tu33W73bbO0i9wExxjfbrefPl//9a9//dd/+7cPHz42Q2+tPTo+enJ+8eblq9cvXp4eHU/KKu+XL/0OuT7NjVDOOR88UywmbwN6b110iYJP3nuri2K6mD97+eri2XOpy8HuBxtWmy0jllKiRIgkkXGBDDgADWMnhEBGIYSmae7v7z/f3tzc3Q7D0Lb7DEez/+TZs2evX77q+pERWGsBYDqdSim9i9vd2nvHORZFIZAN+zY6n1LSUi0XB3VdS8mREUQgSgSJKA5Df3f7+f3791efPh/M50aXr1/GxeIgOu+c89aFEDhQBvHOhXIimZRAkXOuU4ECJ1VVFyUH0lwkxEIrrQRDohhCcH3TumHERMYUAM451w8tJWSCp5SGoXfO+xC6bidSin3fZV/uZDKRUtp+GLu+aZqff/75xx9+/vnnX1erFUoRQnj27NnX337z/V/+/PV3356cnpZl+WiTlgJjSCkCQEJgnEFiESi/bRjd2PXr+4f7m9v1/UO73499X6iiUPrkcLmczygGSFFLEb1DIQkSAAEAYELkDBEZZK7o0TRvQ9N0u13TNM3Dw0PO548GEqWMMZPJhHFZFIVkjBEhIufco/fep0QxUozEBRNKKxRIyICntH583MH6RNYOXdfs9/vJory9uf/733789dffp1Vd1zUkePNGpJSyjxuRMxSMCUSe7WnEIqVIBBATpJRCjD4wRCSgmGIIwXkKkQNXXEmpiRCASamtj+PgGB84F+NqDQA+BKWMcw6RCSFEDIGIMg2VLaib/W69Xn28+rRt9j4GpZQpihDC06dPv/766xevXi6XSyZ4jJEQEDCkyDnnUkAiYJhSihl0KzmbzTjH3W63Wq0uLy+vr6+bpsnGq7Is67rmnA/DgIjT6TSEwBjECClGyveaCP7INgApe/YygWaMKYpKiG1VVV8cW7e3t58+fTo+Pp7OFmVZFkWRCb3sCy+LWnBlXRyaYVLxqpiYspK7vR1cFqXqul4ulxFhcbiYTCZVVTDGx9E9PKw/ffqkhayq+vT4fLk8AmBZ2UHkgI+G2RQpP5MYQ4zRO+esH4Zh6HuOIjttu33X7lprrRBiMpkdHh5+/HiVyREiiDGFEBnjKYKUMqUghOi6oSyNqKuCc4zRd12z3myMMQkhpfT77+8+vP/YNE3ekkVRqMI8e/bszVdfHR4dAcOu76UQjHME8CkarbkQCJCIXPDOey549lPk38653W7XNE2MXkq+PFwcnxzO53POOVEwRhmjhiFygYABKMUYCZCAESEASsljjCHG7Mk6PT19un4KAIvZfBiGT9dXm81mt9m8f/v292cvnj9/rrWOwT32dAJIxlAIzjnnmrOCYSDiLiTto3fRh9B23a7Zb5v9rmkIgRuR46qU0phiOp3NZnMOKIUGgBTh8R8JgBIgDcPYtl3XddP5jAEQoGA823UlF5JxLaQWMgjBILchomBSKWVMiVxEggSYT5iShjMupCjLUplCa71vx0gocmopiqIsS6VUbni6u7//17/+9efffr2/v7feNW17cnr64vWr169fv379Ovf5knOM81zlMaKYeWREIgoxEhEXwhiz3W6dG9u23e/3TdNkuixnuOl0WpSPrSxZqmEciDxBIAiACYEzBoIzBD6fm3EcrQ8quyRTJISjo6PdZvvp06e7h3tr7Wazef/+/ZO3b//85z9PJpOu67JMNZlM6roevbfWxQCFqsRUIFHTNN2+i9Zxzg8ODmazWVGV2sgQo2TcFGoyrbI76ptvvpFScuIppadPns/nB2M3GlNy9mjTjIG8i9HFFCICARFnLMsLVVEYU0opjSkZolJSa82YSAmiT303pASMCSEU56R0kSOB9Y6hKIxCxLqeco6CKDIGutSmMroshJTb6/ZvP/70y+9vP3++Xa/XRVnGGA8PD//lX/7l9evXRV0JISAEI0VuzUw+MSls8F+aXoAzRCCG2W7W7pub69vrT59X9w92GCEREgiGy8W8KkzwVkoJFGMgyZmLiSgSRQACZIwjY8gYZuMcME6InMvjY+JCXZw//Xx9ba2XXHEUCHy92r777ff3v7+9uLhgjBVGGS0zPwsxMoLoExEicACKIRFQAialaPuut2NKIYtJw9B1XZPnBJyfPTG6PD46lUyO43h+ejGfz1d3K8Y4IjImOKAQOUua6D1FBEjEGSZiBJlYyHeNESFSdCFYH13I/IiSpiwiZ9IFXxTlfL5Axvq+l1ILray1J8cTgiSquhCSWTfsdjshVFmWu93u09XVfteM1jkf5lonoOPj4++///7k5KTruqySffESZ1t57tLN9H7uWAYA632hy13cbrfbu7u7rIzkl5+dnT179myxWACkEFzuEtCGMwLGICESPd4lESNKWbQOiQgxxsQ5n81ms9mMUnr//n1RFLnJZ7ff397eXl5e/meAqqoWi4VSqm3bLODmDTsM1g2jUkIXpVEyOItILiYmeDWdLI8ORmu1UVIKxtBItVwuZ7PZZDIRKNq2nU8XQohxtMMwjKNjjCFXmZ/M1RgiEsGX7rSsPYaQrLV917GB7Xdt3/cpRIaiKKrJZEJEPgbnQlFAVddFUZycnHnvmZDr9Xo6nVprxbOL87quz85OTKEWy/nNzd3nm5vPt7f9aHVhqlgPdvz666//8pd/Ws4XjCC7f7+0yH0xAOdL/ILTsmHfubFU2jmXEXYOR8656XS6OJhdPDmbzSeZ9XJ+VNI0TW9KNQwDRZhMJkbIP1rq1H7XIaJUOhKNo8vYcj6fBxdPj89ePn91d/vwsLrz3ndd17bt2PVZumWMScFSCoLrcei9j3b0gnPnXFWWMfiEMAx923X9OGbu3Lqxqo02Mov0y+Uyyz3JJ0Tcb5ubm5txHA8ODo6Pj3e73dAN2+02wyWlpDEqb+XMSuToF0JYr7dj10+nNSIqaYqiklIyIdbb3Wq1yvw4AQihANjofG4QFUJ0wyiEEC9fPdO6ODxc5h0xjO7m7u7Dhw/b3c5amyJMJvXR0dFyuZxMJkVRAM+GQB/C4yCNbOj90k2eA0jWIwqld9t13/e73S4rxdkqlbFZDuX5J0NzAGKMcS4TxZSS99G5wJGllP+fi0ytA+NSaGPKspzUdW7x10oppYiIIWamQAjR9302rmaW1jk3DJ0Plj+OYgAQLAUGDIUWRWWQQwieKHrv+76PwWWhMrMeGR9BQu/9fD4zxgDkLRuFENlxbIxRSnx5FH8crMfWUjCmKCqlTLZA+0T5zMUYU0rxf+mcLLQCgLzLXYhEJP7Lf/nP1lomzDj2m81mv9/vdrt+dN1oiaLU6uDw8Pmrl2dPLuq61loDgFE691oxAYUp8lKpqs6X5b0PzhNjTEouZVmWRhXB+65tvXMIwJAYUlWUgnGKEYkEY0gkOQeQeaoDIqQEIUVKILQ0umyb/rFbLSWAJARjHPJHI7AMU7U0QAwRg/e5GZlB4khCsCzyeW/Hvo1+SAKIUkiCKBIm5MAEFpUhpMH2jDHv7dA2WeJLEMMjFOIx+hg95xyREEEIUZZGS66U8N623V5XihgRkQCKQMCZVEobI5WRyhAhVxq4iIA+0aPJIvqQvAQBAF86i/uhY1mmYCgEU0qJ129eNk3Tj8HH0LZt27bb/Z4QYkqMcSn1fH5wfnaRRdi8TVJKwfthGDB3WaXkQphOJs5771ye05IY44wlxrRU2e7Ttm1W/77InYxhdlVl7CcV5yS8DwxlpGDH4IfRWluaAoGPo9MaE1gfY+7c8jF4G3L9m1Flbr3OsTdPiKjrOlv48qIyBkQBgIgyB0qMQz6oyEFpmc+T1jKzLYxBUZjR+3zlgCkGShQYA8ZYjD7ERzCVKIy23zXNZDED9jiPJDeG5jiUU3jm/sdxHEcXY+RC5aI+Z/eE6XGUCeJ0OmWMhRjHUeSuVtG3bRZzheBE1DTN1dXn3baJOZ0DmLLI0S8H/WHwKUaKiXIdHgLFFIIfun501g5jAlJCKq0lFwi4Wq9Xq1XGFLnTiHOczWaLxVwpFWMETIiMEqWUCBiQYIiBaL/fr27udrudknI6rcuynk7rsq5yrGCCf1l1LaWRWgs1SpknIQ29pZS0Unl8AaSQQkCOkiPjIBVygXG0gw8SOFeF0FwaJbRgUgB/bARGRoho3eBDijEiCkSQiiOUQoh6UiJi2+73+xYTzWZTRChKYwrFhYgxAmKkBAQu+NE7FDxQGpwHGPZN13ZDTCC04lIwgVJypUSCkDARRYBk/ZijLucMBYvRi3Ec274X3DofCeTtw/3n2xsbfEKQXEqp62q6XB7Vda2kNEYjMaNNtsDlUTsBQsE5AGSDdR5Jkzd47g7u+75pGveo3KMQ8vDwMLs2kFGMKQflGD2QCFHEhM7G3ba5urq5ufnMAOtJmeWxUymEUlprZTSXgiK40ZdlaYzJh/5LesgOqi+Djh7JhBiHoYvRci4ZA0BCBM4ZAXGBMcbcA/kY0pUCSNYOhByAnHMQkxAi76oQXEweEaXkjBgixhhDcCklRvTY+M0eIzMTHNhj6vIp2uCdczmc5J+MHkOCLzzqer3OPFn2rAOAEJL1fUdpUD4JWeauEq31MFhlNFcy/6m1NoyD90UKRCHGGDFR8qHrB+fcF4CghZRScsYfyS7vlRAUo8/9W8goJiHF4eJgPp0apTARxASUorMJkTBZGwHFOLr1enP54dPbd79564zR9aTs+55LNlsstJEicYjkbRgHa0c/jmPf90M/ppSKQpZFkZvGjdKlKYqiAICUwjB0nz9f7fZbU2ghOXLOBRJL3od929yv7vJkEefGotSPlh5EU2oi6rvR+RHAQCJrByLiHKuqkJJHF1MKTbNbrVZHpycCIgCTDBMCIgAiIAJyxmXOW0JKYhgJKE9wgcQYSMl5QGREEAHTwcE8hJAQkCGEFKITdvRjPypdlWVV1fOyLGP0jwYuAojB+bHrmtXdve27sjQUSCtFfwxJyH1UeRxK5tCgKBLneQunFOzQd92u3e+s7RhiTIkzPZnWWfhJIaaUGHAiQmLICBkRJOfG7Xb7/uP7H374YbfbCcGPjpamKA6Pl8BYEUyIkRDs4Fbr3cPmYbXZrHebtu+1VkrrelqVZVmWpizLqqomVY0Ew9jZcby9udltdgfzBWNMIENg5FK0YfewXd08dPshjsGOgSZMi5KDFExWpkJEChR8qooSAAFwPp/Xdc2liMPQDj1w1o2D9Y4xhvh4jp1zKYWub9puD5CYQG1kWZmiKBjn9LjnKaUEwBB5VvvyD0JKKRAhKKYFx0KLdjeUxdT60LedHeNqda+kSGGsCmnH1oiq2az/n//6X98dLcs8EA5ZtgdlX9x0OkXE674XQmTzXjGbxhhj8CmErmu26/u7u0/73T1jwUjVjQ4pFlodHsw5glEaGAvOaWmMNPuuFRxiisvDWmkcff/bu1+zkLHv9vPlganM6747Pz91ftxut0Lqm88P//1v//rLh98jJ13ootBHJ4eco9bSe//sydPzsxOtVJPSpKrvrj5fHV3/t//7vxk0h4eHre+EkiGE+/v7z+9vbj7cjjvLg5jq2eHkSJGmEfyQGhgYY8FBCKkPA2OMEavr6fn5k/MnF+uHlS4aIlpvV+8/fCinkxevXmb8CSlud+vr2+tts725/bRv1kqZ2WJ6enZ8fHKoC50TBEUodGmHcej6siyDdWPXa605A+e8bTvBuDJa9N0ojRZc5tSCFDnngjMgDAgp+P12e/Xpcr9ecQYCWS6hsvOi0MaUBSSy3iHBi1cv67KaTqdaKq11oY3W0oWuMppzghgcpeSDEKw0ajabVXnGAnDgoAUqpZRXKIULnnMxnZUnJ0cnZ8fb7b7rul2z/+GnH9u+uXu4ffnypSnUfr/rh3G97T/f3Fg/MsERUZfFfDE9PD7SWkspq7o4PDxcLpdD28XoGROr+4effvqlUMXhybEWUhUm+XD7cP/Tv/94d7OKY1CqnE+mFxfPXz57fXryhAGPLkaIDNmknOQRHs6509PT4+Pjuq7v7u7avnPOfbz69O8//DtXuutHpUVVFSnE27vPNzfXTbPLcnQIjijmDmIppTYyJQghpRQevdbWjeNojPn06VJrjYKHEELwiCh8irXWBJgQEkHuhsxDV2KSOcRdXV0JZJRCjt3W2uSD0KoyBZNCIAPO2t2eafnn4c8RKI8JEcgSgvW+t84n4kpzzrksFsuj5dFJJBysd84BYGZfXIxt1x+eHBGCUjq7o7/77ruPH6+ur69z+9Hd3R1Aur+/L0odgt/u9nfr5mG9bbqWEExhDg6Xz1++/Oa777gUhCCVOj45uXjypGmaptmlBC74t+/eDcNwcno6nUxMUaQYN9vtDz//tNvvUfB5tbg4P3/95s3LN68PDg60MaOzWQ/L04WGYdjv94yx+Xx+dHS0Xq+HYWjbtmma3377Tevq5vZ2OqsPDw/caN9/ePvwcHd/f599Z5KL0hTZ55oRR7bGdl2XUiLCvh83m01G8Cml6KL33nuXUhJ1XU+n0xCT9x6YOj48ujg98d53+6bUhgEWRnFAgpjhfx76AIkyCkwj5ePJlTR/TAmKMY3OMkDC5H0ExMV8eX7xVHIBjD+9eHJ8elZVlVQ6JWKcIzEfnZbGh0AICYgxNl8s3rx5s9qsp9O5VCp433adUmrfNh8/fiRMVVk4HxLTiFjXtZRyPpm+fPnym2+++eabb6TWKSUuxdnF+bfffhtjvL373O67obc393fr9fr67rYuSqEVI+jGwVpbTmot5HQxf/3i5auv3pyenUWgL/NbctjP5uLdbmeMWSwW33zzjRBiOp3e399779v9/urq6uOnT/WkPDk5yku1220QMfcAnBydfvPNN3/5y1/Ozs7qup7PDnIhNAxDSikTRnnYQgjOe9/bMYTAORNCiKOjZV1P264noqKanJ+fv3r5Ugix3zZEhIy0VIoLZCQY5xwH6/pxhJS4lAxgsBaJhFLz6fT49LSa1MBY8N7HaJQChovlwenFk6++bsrpDFICxs+OT2aLA0AWAQG50EaggMAFV8wp6/zonJJaGn12cf51++10MtdVeX9zu2sbJZiLbhgGG6zUShbFdHaUgCkt6ro+ODh4/vz566/eLI8OAcBbxyU7Pjv98z/+xZTF9fWn7Xp3e3t/e3vvRgsM26GnsZdcAMPDk2OhpJbq6OT4zavXr968Xh4f+RiyWz3XsLkCUUplufzFixdFUeQJFx8+fFiv17nS32w2Wsv9dpNSeljd+3FYLBaqqs7Ozr79+rt/+Id/ePHizWQyoZgE51VVTSaPE56qqkop9WM3juPd3Z2UEjjkxBRCFAcHB5zzfhwYY1VdCHH81es3i8XCu0hECAkRM6zlCMhZ248JiCNDzlKILniKiQk+m0yPTo6FUm3f9W3nYwghIKMEWFfT0/MLQhZCKIrq+Ph4Mplt9k3hUoyxJOQ8WGstj8PoFPA8qRARi6p68uTJfH5QTScPDw/r9Xoc+5j8xcWFtQMiAGNK1sqU88X08PDw+Pj49PT0/OT4caAKQjZ2nT254EoenhxuV9vdrrm/W9lxTER91znvq7JUWhutq7qWQpydn794/vzk9HR5eDham6ufTDFmaxdjrCzL3LWeK8hMvmSSc7vvU0pSPvLXs9kMJ/Xp6SkRXVxc/PnPf/7qq6/m80Mi6vsBEZfL5evXr8vS5ObBvu+H6zEfstlsNl1Mvfc5xorcyvPYCaNNWdSvXuGLFy9yJZh8SCkBJsYYA0pAl9efTVloqQiBYsq/I6WD+QIYuuBv7+/afRNS1FIRUYx+dK6qp4sDH2OczRbz+TwXGUzIBMiEBMQEKJiQUiI+GihijJyJyWRycHBY1/V6vW6aZrtdh+gQseuaYeiVMUZPTVnP5pPZbDafz2ezWV2XKB6ZJGSMcW6K4uj4uJ5W7WG337UnZ2103sXQ7vaB0rSqpdGMYL48EMiOz06X84U0uihLAsicRdYTsjszUyRf6LGLi4vcNbPb7bz31zcPTdNoLXe7Tdc0RDFGf3JyUhbF6enp4XIphbDD2Ho/DDbGuFwuv//++xcvXuz3e2vt/f2t9S7TBbPZrCirvu/zRz/O1cgsg1JCSq1PjopqUmqTzYJ5TA9jj0tSTKb1dGKMyUNEACBTA7mbpenacRzbts1cRoyRYkgRptMpAISQcujoum65PHqkE0OghM4GhiERSs4EifwsEJKUcjab5eFDiLjfb0N0Usr9fuucPTw8ErISyggGEUggy3s/u8lz/0uuhauqquqirqaHy5j7OHPWyWxbHhR6dHQEAMvlMis1CQE4Y/joonkk8WLMfps8SCjribmgzKeqrOfe+6LQ79+//Xx1FYIbhi67InP/9sPDAyXunMvzq4Tix8fHWVQbhmGxWPgYckUPkGzwuf9MSimAEZdMKdF1vmvb+UzVZaW0rMqybVspWFlMXfBNs0spqcI8ffqUCZ7XKXfp5PCdJZndbjcMw5f5wYgYEiklEXE+P/jjnnVK8DijSJcxEFECgL4fmIC+d6rQeXQTEZVlOQxD0zS5t6kotHUDY+z8/DSl2PdDXR0QMO/GfhxC9IjIQQFjuYsmpZRLwKyDTKfTqqj7fsxT8U5PT7PelovCPOc566XDMOTZQ5JjHp6Lf4w0zFxwJprzypVleX5+fnp6Oo7jw/3WWnt3d0Mxbjab+/tbwXHoewb4+eqaMcEYk6iNMWVZ67Ko6vq77757+vRpCCFE97e//e3d+/fZ6uO958HmobfW2jzHnuf+JADwwYqgNOlskHvUUSgwxjhHIVj2NeSzn4NM3vhd1xVFkR/oIzf4WN+lL6LwH01FHgCOjo5ylM8fAY+zJCkrSRCJc86BhRCiiwAQrCMiBgkAJONCCCQWdfKjRSEFMi0VcmJCIOXxw+S9B0iZcZdSPk5K/oMPzBEsH5F8zRmA5Vbz3FIopWSQ8izTfKdZQCKiPGgzN3V9YceFEHVdLxYLKfnV9Ufn3H6/N1oqpe5v77K9QkopUXPOldpzJbM4eXFxUZZlIpVHMY2DQ0YxRiCWx2wzxkTCJKXO6vswDBATJzmt6hS8QEiALoQEUUqJAjOBGMbhy7JnVvTLmLTcGparkBw/o8+t4Tx7RbJ2TESFNpPJ4zDk3EvEOU8QQ+wpJSDQQiKJSAkBtJDBewAABowy45UQgQO6EBSXKnOaj4Y0733Ik1t8CACQKU8fnbUuuOhcyKbwDOpyQZorm0zq0B8DTqWUdujylsrMWR6tR0TZt5MJgf91qfLcZQRodvuHu/vNaq2VcKPN+B4ToqGEAADDYIk9HlkAMMaklIwuc5PdF42RMcaQGAohpZSSe44pxL7pB+yQ2KSq83ZjgJgiADGJefy7tUMIARIFAEjEGINEfd8zwPz31lo32gTEADOGjIzlMpkDCiG1lEwIChERMZG1NtiQjNZCuuikYpEiZ1woyUi64BNSURRD1wslGVAkzhARgGJiBIJxLaSSIlc/kRK5wACM0grIOcz+nBhjCtE7RxwQsVBaGq24CJQgJuecH61XCmICKRkBEWGi3FP1hf/+spBfmPu8ujn8ZNl3sVg8jjZIqSzLw8NDoJhS+jLU5YvqjYgcH5uIcsHb931ue3k8EkH65OGPqZwiG+fZHzOm+7Zfx5Vg3JiSZ+NxjMBZAkYxxBibpuNCMUDkzCg9m0xTiP0w5KbSg/kCEbumdcEzQJ6nZFCklAhAcK50URWlLozgyLPdNLjgPWPgYnDBMhAREucIiYBSChEiEEChNArOIAUCTPlbI0ggi4koxLxPiQgQODLBOMWUDcdokYgYgeKCtCmKKgZCAGQshuC9ByJkj9+EwrgwSpOQRCSF8CHkTrf8ZHMUyV22WbHM5ynH9hxUPHM5O5yenr569Wq7nTs7DMOQfJBSSqkZYwwk51wKLZQ8Oz1fLA7qeiKEHIZhGIaUICWo69p7nxvLc5gVfd8DgABeT0qeWHA+e1GLomKMccm4EFwgcYYMCJlzQUmTn4s1JlsP1ut1jDH3kddlJbnIfj8Gqa6KDPMys6KUEEJwZFluQMRxHL2PrOsAwHvLeOKSK6EAkSLzMVCEmFJZFMgYQooUIBFhyg7poY9KGskxAiEi4xwZRaKd39Z1LSTLQ3hNWQjBYqSh672P+WJyoNNaGymdc3Ycs40HAJy1QDSOY4L4he3Orpi88TN6ysRdPlL5NlfjdrFYlGX5/PnzYRju7io79t77SVkxxhgTAIAkhBBGl1Kr49Oz4+PjnPizwFYURc5H+Md46nx8H5N/QsE5n0wm1trgdtH5PrQ2eOSgjeECbQyMo1CaEeup/xLNMyfW931GXF90sPy23tvgaw6Qgy8RDM7bfmCMtU3/aCUgInqkAWP0plBFXWqhXAzkETiDCKN3pTYRiFEKFCjExJLiXAhpR5JCM6SQIhGh4IxBAvLeV5MJ49C07eM4TMmczcEJHhU/xr7YVT9//iylzHOIOefWWqXUMAyEKU+myjDki5nOOZeXKrNN8KXNIonpdJrdTlVRLuZz50xKqVCacy6EQkQkwRjLp4pzvlqtfv31V+/9avWQp4emlJqmyVRAVtq89yJD58E7JaSRqiwrO1hM6Jzz3kaXiIgwdeOAHKuyBmDjODLgWksi7LomhMQlCyF1YxcDISPFFUByo3dDv1vdK/2YkBG+6Fh5rkQgIiUk4iP6J6KujbMwd1pb+ziaLgIOw9B1TUqJgo/Jh+ATBSmlksaPxLkkitlhQQiIFChxzrO6v96tx9FlF1Hf9yky5MIoLbWCRD4GBsilGPsBOWv3jdRKcuFjMEr3YyelLMtHNj1GyiWwKQs3Wqn/51JlxM8YQ+ApxpQSIjHMhiQXY5SMK6W0LoRQlBCAISIwrsxmvV5Lqa3NJpGQwYgxxoXHL0bxyYcQ8Kef/71rWu+9UUowkXxIIXJkeVgY5AE1CBEon6QMgSBAij64GILziTx5G5P1Y4iEkCgCxiBBSM6cH7hk8/ncOZcgKqXaoeecC2Tex7EfkNjB7ODTp6uqKJngwClSSkAMBQqOyGLKmSD7jb33FlJgDFKI3ns3kpI6pdQN/Rcc1fYd51xoBQDD0Pf94IIjIkpgbUwA07rOTKZQaug65NwOw2yx2G+3Uut2v5dac8RxHIUAY8y22Rtjcpc/Y8yHkGJ03s/nB7nsy6V3nj+X40oKHoiEYJI9DsPnnKOQnEnkgjPJpeRMWmu3261zoSrNMAyC4du3b5HBdDrdbFbjOOpCPb5tjCmklBJFApmn9hPkIR9CCKmVUooJDgCBUg53EBPFlEIkH1zwLkUPqQ/OOu1TZBlJx6SIKcFdNIhUlgaRAqHS0oBUSimurLWYIiXUShRaT8qKK8E0jxRSxpNCAcMYkk/ROQuYyHHnEybBOOaoQNEqLWOMKohEJASPEQRjwzgYIJ6NE5TyqOcYUz9YACYYG8cxpaRCyN/ckX2D+/0+f2NBttFZO4xDZ4ze7vJSbUMIKGQIoe/7EMJ0tskKSPZpM8GllGVdV4UOIbj8HQlC5m4XzjlyzQRHkExw5IpzPgy2aRqIaT6fU/BlWXJEwYVAJv7A6hwZY+z/B3hHhkp0QyTvAAAAAElFTkSuQmCC",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=142x66>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "image = Image.open(train_ds.root_dir + train_dataset['file_name'][0]).convert(\"RGB\")\n",
    "\n",
    "image"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Show Label for Above Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2730UB\n"
     ]
    }
   ],
   "source": [
    "labels = encoding['labels']\n",
    "labels[labels == -100] = processor.tokenizer.pad_token_id\n",
    "label_str = processor.decode(labels, skip_special_tokens=True)\n",
    "print(label_str)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Instantiate Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "loading configuration file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/config.json from cache at /Users/leedunn/.cache/huggingface/transformers/212ce944385c924d9d9f37e30c9e9c23414582ec39b2d99b6acc7e19fc372f86.d59848aae48f9ec6eaf5caa5ded0a5e1660ddc8202561ca823851ada5344c39f\n",
      "Model config VisionEncoderDecoderConfig {\n",
      "  \"architectures\": [\n",
      "    \"VisionEncoderDecoderModel\"\n",
      "  ],\n",
      "  \"decoder\": {\n",
      "    \"_name_or_path\": \"\",\n",
      "    \"activation_dropout\": 0.0,\n",
      "    \"activation_function\": \"gelu\",\n",
      "    \"add_cross_attention\": true,\n",
      "    \"architectures\": null,\n",
      "    \"attention_dropout\": 0.0,\n",
      "    \"bad_words_ids\": null,\n",
      "    \"bos_token_id\": 0,\n",
      "    \"chunk_size_feed_forward\": 0,\n",
      "    \"classifier_dropout\": 0.0,\n",
      "    \"cross_attention_hidden_size\": 768,\n",
      "    \"d_model\": 1024,\n",
      "    \"decoder_attention_heads\": 16,\n",
      "    \"decoder_ffn_dim\": 4096,\n",
      "    \"decoder_layerdrop\": 0.0,\n",
      "    \"decoder_layers\": 12,\n",
      "    \"decoder_start_token_id\": 2,\n",
      "    \"diversity_penalty\": 0.0,\n",
      "    \"do_sample\": false,\n",
      "    \"dropout\": 0.1,\n",
      "    \"early_stopping\": false,\n",
      "    \"encoder_no_repeat_ngram_size\": 0,\n",
      "    \"eos_token_id\": 2,\n",
      "    \"exponential_decay_length_penalty\": null,\n",
      "    \"finetuning_task\": null,\n",
      "    \"forced_bos_token_id\": null,\n",
      "    \"forced_eos_token_id\": null,\n",
      "    \"id2label\": {\n",
      "      \"0\": \"LABEL_0\",\n",
      "      \"1\": \"LABEL_1\"\n",
      "    },\n",
      "    \"init_std\": 0.02,\n",
      "    \"is_decoder\": true,\n",
      "    \"is_encoder_decoder\": false,\n",
      "    \"label2id\": {\n",
      "      \"LABEL_0\": 0,\n",
      "      \"LABEL_1\": 1\n",
      "    },\n",
      "    \"layernorm_embedding\": true,\n",
      "    \"length_penalty\": 1.0,\n",
      "    \"max_length\": 20,\n",
      "    \"max_position_embeddings\": 512,\n",
      "    \"min_length\": 0,\n",
      "    \"model_type\": \"trocr\",\n",
      "    \"no_repeat_ngram_size\": 0,\n",
      "    \"num_beam_groups\": 1,\n",
      "    \"num_beams\": 1,\n",
      "    \"num_return_sequences\": 1,\n",
      "    \"output_attentions\": false,\n",
      "    \"output_hidden_states\": false,\n",
      "    \"output_scores\": false,\n",
      "    \"pad_token_id\": 1,\n",
      "    \"prefix\": null,\n",
      "    \"problem_type\": null,\n",
      "    \"pruned_heads\": {},\n",
      "    \"remove_invalid_values\": false,\n",
      "    \"repetition_penalty\": 1.0,\n",
      "    \"return_dict\": true,\n",
      "    \"return_dict_in_generate\": false,\n",
      "    \"scale_embedding\": false,\n",
      "    \"sep_token_id\": null,\n",
      "    \"task_specific_params\": null,\n",
      "    \"temperature\": 1.0,\n",
      "    \"tf_legacy_loss\": false,\n",
      "    \"tie_encoder_decoder\": false,\n",
      "    \"tie_word_embeddings\": true,\n",
      "    \"tokenizer_class\": null,\n",
      "    \"top_k\": 50,\n",
      "    \"top_p\": 1.0,\n",
      "    \"torch_dtype\": null,\n",
      "    \"torchscript\": false,\n",
      "    \"transformers_version\": \"4.21.3\",\n",
      "    \"typical_p\": 1.0,\n",
      "    \"use_bfloat16\": false,\n",
      "    \"use_cache\": false,\n",
      "    \"use_learned_position_embeddings\": true,\n",
      "    \"vocab_size\": 50265\n",
      "  },\n",
      "  \"encoder\": {\n",
      "    \"_name_or_path\": \"\",\n",
      "    \"add_cross_attention\": false,\n",
      "    \"architectures\": null,\n",
      "    \"attention_probs_dropout_prob\": 0.0,\n",
      "    \"bad_words_ids\": null,\n",
      "    \"bos_token_id\": null,\n",
      "    \"chunk_size_feed_forward\": 0,\n",
      "    \"cross_attention_hidden_size\": null,\n",
      "    \"decoder_start_token_id\": null,\n",
      "    \"diversity_penalty\": 0.0,\n",
      "    \"do_sample\": false,\n",
      "    \"early_stopping\": false,\n",
      "    \"encoder_no_repeat_ngram_size\": 0,\n",
      "    \"encoder_stride\": 16,\n",
      "    \"eos_token_id\": null,\n",
      "    \"exponential_decay_length_penalty\": null,\n",
      "    \"finetuning_task\": null,\n",
      "    \"forced_bos_token_id\": null,\n",
      "    \"forced_eos_token_id\": null,\n",
      "    \"hidden_act\": \"gelu\",\n",
      "    \"hidden_dropout_prob\": 0.0,\n",
      "    \"hidden_size\": 768,\n",
      "    \"id2label\": {\n",
      "      \"0\": \"LABEL_0\",\n",
      "      \"1\": \"LABEL_1\"\n",
      "    },\n",
      "    \"image_size\": 384,\n",
      "    \"initializer_range\": 0.02,\n",
      "    \"intermediate_size\": 3072,\n",
      "    \"is_decoder\": false,\n",
      "    \"is_encoder_decoder\": false,\n",
      "    \"label2id\": {\n",
      "      \"LABEL_0\": 0,\n",
      "      \"LABEL_1\": 1\n",
      "    },\n",
      "    \"layer_norm_eps\": 1e-12,\n",
      "    \"length_penalty\": 1.0,\n",
      "    \"max_length\": 20,\n",
      "    \"min_length\": 0,\n",
      "    \"model_type\": \"vit\",\n",
      "    \"no_repeat_ngram_size\": 0,\n",
      "    \"num_attention_heads\": 12,\n",
      "    \"num_beam_groups\": 1,\n",
      "    \"num_beams\": 1,\n",
      "    \"num_channels\": 3,\n",
      "    \"num_hidden_layers\": 12,\n",
      "    \"num_return_sequences\": 1,\n",
      "    \"output_attentions\": false,\n",
      "    \"output_hidden_states\": false,\n",
      "    \"output_scores\": false,\n",
      "    \"pad_token_id\": null,\n",
      "    \"patch_size\": 16,\n",
      "    \"prefix\": null,\n",
      "    \"problem_type\": null,\n",
      "    \"pruned_heads\": {},\n",
      "    \"qkv_bias\": false,\n",
      "    \"remove_invalid_values\": false,\n",
      "    \"repetition_penalty\": 1.0,\n",
      "    \"return_dict\": true,\n",
      "    \"return_dict_in_generate\": false,\n",
      "    \"sep_token_id\": null,\n",
      "    \"task_specific_params\": null,\n",
      "    \"temperature\": 1.0,\n",
      "    \"tf_legacy_loss\": false,\n",
      "    \"tie_encoder_decoder\": false,\n",
      "    \"tie_word_embeddings\": true,\n",
      "    \"tokenizer_class\": null,\n",
      "    \"top_k\": 50,\n",
      "    \"top_p\": 1.0,\n",
      "    \"torch_dtype\": null,\n",
      "    \"torchscript\": false,\n",
      "    \"transformers_version\": \"4.21.3\",\n",
      "    \"typical_p\": 1.0,\n",
      "    \"use_bfloat16\": false\n",
      "  },\n",
      "  \"is_encoder_decoder\": true,\n",
      "  \"model_type\": \"vision-encoder-decoder\",\n",
      "  \"tie_word_embeddings\": false,\n",
      "  \"torch_dtype\": \"float32\",\n",
      "  \"transformers_version\": null\n",
      "}\n",
      "\n",
      "loading weights file https://huggingface.co/microsoft/trocr-base-printed/resolve/main/pytorch_model.bin from cache at /Users/leedunn/.cache/huggingface/transformers/12e56e9d71761048853a854c9b4c055164dfcc0cd03837d5893ca04818d39702.8f6ef3e46bd2801b75ad3599a96da1c4b0153cfd058a50ecbe6f58764eb0ce2d\n",
      "All model checkpoint weights were used when initializing VisionEncoderDecoderModel.\n",
      "\n",
      "Some weights of VisionEncoderDecoderModel were not initialized from the model checkpoint at microsoft/trocr-base-printed and are newly initialized: ['encoder.pooler.dense.bias', 'encoder.pooler.dense.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "model = VisionEncoderDecoderModel.from_pretrained(MODEL_CKPT)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Model Configuration Modifications"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.config.decoder_start_token_id = processor.tokenizer.cls_token_id\n",
    "model.config.pad_token_id = processor.tokenizer.pad_token_id\n",
    "\n",
    "model.config.vocab_size = model.config.decoder.vocab_size\n",
    "\n",
    "model.config.eos_token_id = processor.tokenizer.sep_token_id\n",
    "model.config.max_length = 64\n",
    "model.config.early_stopping = True\n",
    "model.config.no_repeat_ngram_size = 3\n",
    "model.config.length_penalty = 2.0\n",
    "model.config.num_beams = 4"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Define Metrics Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "cer_metric = evaluate.load(\"cer\")\n",
    "\n",
    "def compute_metrics(pred):\n",
    "    label_ids = pred.label_ids\n",
    "    pred_ids = pred.predictions\n",
    "    \n",
    "    pred_str = processor.batch_decode(pred_ids, skip_special_tokens=True)\n",
    "    label_ids[label_ids == -100] = processor.tokenizer.pad_token_id\n",
    "    label_str = processor.batch_decode(label_ids, skip_special_tokens=True)\n",
    "    \n",
    "    cer = cer_metric.compute(predictions=pred_str, references=label_str)\n",
    "    \n",
    "    return {\"cer\" : cer}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Define Training Arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "PyTorch: setting up devices\n",
      "The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).\n"
     ]
    }
   ],
   "source": [
    "args = Seq2SeqTrainingArguments(\n",
    "    output_dir = MODEL_NAME,\n",
    "    num_train_epochs=NUM_OF_EPOCHS,\n",
    "    predict_with_generate=True,\n",
    "    evaluation_strategy=\"epoch\",\n",
    "    save_strategy=\"epoch\",\n",
    "    per_device_train_batch_size=8,\n",
    "    per_device_eval_batch_size=8,\n",
    "    logging_first_step=True,\n",
    "    hub_private_repo=True,\n",
    "    push_to_hub=True\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Define Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Cloning https://huggingface.co/DunnBC22/trocr-base-printed_license_plates_ocr into local empty directory.\n"
     ]
    }
   ],
   "source": [
    "trainer = Seq2SeqTrainer(\n",
    "    model=model,\n",
    "    tokenizer=processor.feature_extractor,\n",
    "    args=args,\n",
    "    compute_metrics=compute_metrics,\n",
    "    train_dataset=train_ds,\n",
    "    eval_dataset=test_ds,\n",
    "    data_collator=default_data_collator\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Fit/Train Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n",
      "***** Running training *****\n",
      "  Num examples = 16000\n",
      "  Num Epochs = 2\n",
      "  Instantaneous batch size per device = 8\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 8\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 4000\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "aa21424688c24922a4d567f4d6409d27",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/4000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 9.7097, 'learning_rate': 4.99875e-05, 'epoch': 0.0}\n",
      "{'loss': 0.7791, 'learning_rate': 4.375e-05, 'epoch': 0.25}\n",
      "{'loss': 0.5107, 'learning_rate': 3.7500000000000003e-05, 'epoch': 0.5}\n",
      "{'loss': 0.3557, 'learning_rate': 3.125e-05, 'epoch': 0.75}\n",
      "{'loss': 0.3144, 'learning_rate': 2.5e-05, 'epoch': 1.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running Evaluation *****\n",
      "  Num examples = 4000\n",
      "  Batch size = 8\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c887da46f999474b8124b8d4d04aba6d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/500 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to trocr-base-printed_license_plates_ocr/checkpoint-2000\n",
      "Configuration saved in trocr-base-printed_license_plates_ocr/checkpoint-2000/config.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.24631145596504211, 'eval_cer': 0.04733333333333333, 'eval_runtime': 55301.7054, 'eval_samples_per_second': 0.072, 'eval_steps_per_second': 0.009, 'epoch': 1.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Model weights saved in trocr-base-printed_license_plates_ocr/checkpoint-2000/pytorch_model.bin\n",
      "Feature extractor saved in trocr-base-printed_license_plates_ocr/checkpoint-2000/preprocessor_config.json\n",
      "Feature extractor saved in trocr-base-printed_license_plates_ocr/preprocessor_config.json\n",
      "Adding files tracked by Git LFS: ['.DS_Store']. This may take a bit of time if the files are large.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.2086, 'learning_rate': 1.8750000000000002e-05, 'epoch': 1.25}\n",
      "{'loss': 0.1776, 'learning_rate': 1.25e-05, 'epoch': 1.5}\n",
      "{'loss': 0.1802, 'learning_rate': 6.25e-06, 'epoch': 1.75}\n",
      "{'loss': 0.143, 'learning_rate': 0.0, 'epoch': 2.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running Evaluation *****\n",
      "  Num examples = 4000\n",
      "  Batch size = 8\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9c70f9961f4e41868f9081aef7b47648",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/500 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to trocr-base-printed_license_plates_ocr/checkpoint-4000\n",
      "Configuration saved in trocr-base-printed_license_plates_ocr/checkpoint-4000/config.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.15811479091644287, 'eval_cer': 0.036833333333333336, 'eval_runtime': 54648.5441, 'eval_samples_per_second': 0.073, 'eval_steps_per_second': 0.009, 'epoch': 2.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Model weights saved in trocr-base-printed_license_plates_ocr/checkpoint-4000/pytorch_model.bin\n",
      "Feature extractor saved in trocr-base-printed_license_plates_ocr/checkpoint-4000/preprocessor_config.json\n",
      "Feature extractor saved in trocr-base-printed_license_plates_ocr/preprocessor_config.json\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'train_runtime': 774109.7161, 'train_samples_per_second': 0.041, 'train_steps_per_second': 0.005, 'train_loss': 0.3358841931819916, 'epoch': 2.0}\n"
     ]
    }
   ],
   "source": [
    "train_results = trainer.train()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Save Model & Training Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to trocr-base-printed_license_plates_ocr\n",
      "Configuration saved in trocr-base-printed_license_plates_ocr/config.json\n",
      "Model weights saved in trocr-base-printed_license_plates_ocr/pytorch_model.bin\n",
      "Feature extractor saved in trocr-base-printed_license_plates_ocr/preprocessor_config.json\n",
      "Saving model checkpoint to trocr-base-printed_license_plates_ocr\n",
      "Configuration saved in trocr-base-printed_license_plates_ocr/config.json\n",
      "Model weights saved in trocr-base-printed_license_plates_ocr/pytorch_model.bin\n",
      "Feature extractor saved in trocr-base-printed_license_plates_ocr/preprocessor_config.json\n",
      "Several commits (2) will be pushed upstream.\n",
      "The progress bars may be unreliable.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "61cc96209a084af9b59a84e56425e372",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload file pytorch_model.bin:   0%|          | 32.0k/1.24G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "remote: Scanning LFS files for validity, may be slow...        \n",
      "remote: LFS file scan complete.        \n",
      "To https://huggingface.co/DunnBC22/trocr-base-printed_license_plates_ocr\n",
      "   2ea0ec3..f8584da  main -> main\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** train metrics *****\n",
      "  epoch                    =                 2.0\n",
      "  train_loss               =              0.3359\n",
      "  train_runtime            = 8 days, 23:01:49.71\n",
      "  train_samples_per_second =               0.041\n",
      "  train_steps_per_second   =               0.005\n"
     ]
    }
   ],
   "source": [
    "trainer.save_model()\n",
    "trainer.log_metrics(\"train\", train_results.metrics)\n",
    "trainer.save_metrics(\"train\", train_results.metrics)\n",
    "trainer.save_state()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Evaluate Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running Evaluation *****\n",
      "  Num examples = 4000\n",
      "  Batch size = 8\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c18353ad07c74078868a7a359e7f1488",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/500 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** eval metrics *****\n",
      "  epoch                   =         2.0\n",
      "  eval_cer                =      0.0368\n",
      "  eval_loss               =      0.1581\n",
      "  eval_runtime            = 15:19:38.93\n",
      "  eval_samples_per_second =       0.072\n",
      "  eval_steps_per_second   =       0.009\n"
     ]
    }
   ],
   "source": [
    "metrics = trainer.evaluate()\n",
    "trainer.log_metrics(\"eval\", metrics)\n",
    "trainer.save_metrics(\"eval\", metrics)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Push Model to Hub (My Profile!!!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to trocr-base-printed_license_plates_ocr\n",
      "Configuration saved in trocr-base-printed_license_plates_ocr/config.json\n",
      "Model weights saved in trocr-base-printed_license_plates_ocr/pytorch_model.bin\n",
      "Feature extractor saved in trocr-base-printed_license_plates_ocr/preprocessor_config.json\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cfbd7970c9064219a54214fc4c7bd8ee",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload file .DS_Store: 100%|##########| 6.00k/6.00k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "remote: Scanning LFS files for validity, may be slow...        \n",
      "remote: LFS file scan complete.        \n",
      "To https://huggingface.co/DunnBC22/trocr-base-printed_license_plates_ocr\n",
      "   f8584da..3b058e3  main -> main\n",
      "\n"
     ]
    }
   ],
   "source": [
    "kwargs = {\n",
    "    \"finetuned_from\" : model.config._name_or_path,\n",
    "    \"tasks\" : \"image-to-text\",\n",
    "    \"tags\" : [\"image-to-text\"],\n",
    "}\n",
    "\n",
    "if args.push_to_hub:\n",
    "    trainer.push_to_hub(\"All Dunn!!!\")\n",
    "else:\n",
    "    trainer.create_model_card(**kwargs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notes & Other Takeaways From This Project\n",
    "****\n",
    "- The results were pretty good. I was pondering whether to train for 2 or 3 epochs. Ultimately, I trained this model for 2 epochs. If this were a work project (where multiprocessing and other options are available), I would have trained for 3, if not 4, epochs.\n",
    "****"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Citations\n",
    "\n",
    "##### For Transformer Checkpoint\n",
    "- @misc{li2021trocr,\n",
    "      title={TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models}, \n",
    "      author={Minghao Li and Tengchao Lv and Lei Cui and Yijuan Lu and Dinei Florencio and Cha Zhang and Zhoujun Li and Furu Wei},\n",
    "      year={2021},\n",
    "      eprint={2109.10282},\n",
    "      archivePrefix={arXiv},\n",
    "      primaryClass={cs.CL}\n",
    "}\n",
    "\n",
    "##### For CER Metric\n",
    "- @inproceedings{morris2004,\n",
    "author = {Morris, Andrew and Maier, Viktoria and Green, Phil},\n",
    "year = {2004},\n",
    "month = {01},\n",
    "pages = {},\n",
    "title = {From WER and RIL to MER and WIL: improved evaluation measures for connected speech recognition.}\n",
    "}"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "nlp",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "41bc52750e0704433c7c40a5c68d8f60e760babe95f2dffc82e8c3790208ff57"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}