Ch09_Modern_Convolutional_Networks/Networks_with_Parallel_Concatenations_(GoogLeNet).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oHO37YNdaYHl"
   },
   "source": [
    "# Networks with Parallel Concatenations (GoogLeNet)\n",
    "\n",
    "In 2014, [Szegedy et al., 2014](https://arxiv.org/abs/1409.4842)\n",
    "won the ImageNet Challenge, proposing a structure \n",
    "that combined the strengths of the NiN and repeated blocks paradigms. \n",
    "One focus of the paper was to address the question \n",
    "of which sized convolutional kernels are best. \n",
    "After all, previous popular networks employed choices\n",
    "as small as $1 \\times 1$ and as large as $11 \\times 11$. \n",
    "One insight in this paper was that sometimes\n",
    "it can be advantageous to employ a combination of variously-sized kernels. \n",
    "In this section, we will introduce GoogLeNet, \n",
    "presenting a slightly simplified version of the original model—we \n",
    "omit a few ad hoc features that were added to stabilize training\n",
    "but are unnecessary now with better training algorithms available. \n",
    "\n",
    "## Inception Blocks\n",
    "\n",
    "The basic convolutional block in GoogLeNet is called an Inception block, \n",
    "likely named due to a quote from the movie Inception (\"We Need To Go Deeper\"),\n",
    "which launched a viral meme.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"128pt\" version=\"1.1\" viewBox=\"0 0 330 128\" width=\"330pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol id=\"glyph0-0\" overflow=\"visible\">\n",
       "<path d=\"M 1.125 0 L 1.125 -5.625 L 5.625 -5.625 L 5.625 0 Z M 1.265625 -0.140625 L 5.484375 -0.140625 L 5.484375 -5.484375 L 1.265625 -5.484375 Z M 1.265625 -0.140625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-1\" overflow=\"visible\">\n",
       "<path d=\"M 3.359375 0 L 2.5625 0 L 2.5625 -5.046875 C 2.375 -4.859375 2.125 -4.671875 1.8125 -4.484375 C 1.5 -4.304688 1.222656 -4.175781 0.984375 -4.09375 L 0.984375 -4.859375 C 1.421875 -5.054688 1.804688 -5.300781 2.140625 -5.59375 C 2.472656 -5.894531 2.707031 -6.1875 2.84375 -6.46875 L 3.359375 -6.46875 Z M 3.359375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-2\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-3\" overflow=\"visible\">\n",
       "<path d=\"M 0.0625 0 L 1.765625 -2.421875 L 0.1875 -4.671875 L 1.1875 -4.671875 L 1.90625 -3.578125 C 2.03125 -3.367188 2.132812 -3.191406 2.21875 -3.046875 C 2.351562 -3.242188 2.472656 -3.414062 2.578125 -3.5625 L 3.359375 -4.671875 L 4.3125 -4.671875 L 2.703125 -2.46875 L 4.4375 0 L 3.46875 0 L 2.5 -1.453125 L 2.25 -1.84375 L 1.03125 0 Z M 0.0625 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-4\" overflow=\"visible\">\n",
       "<path d=\"M 5.296875 -2.265625 L 6.140625 -2.046875 C 5.960938 -1.347656 5.640625 -0.8125 5.171875 -0.4375 C 4.710938 -0.0703125 4.144531 0.109375 3.46875 0.109375 C 2.78125 0.109375 2.21875 -0.03125 1.78125 -0.3125 C 1.34375 -0.59375 1.007812 -1 0.78125 -1.53125 C 0.5625 -2.070312 0.453125 -2.648438 0.453125 -3.265625 C 0.453125 -3.941406 0.578125 -4.53125 0.828125 -5.03125 C 1.085938 -5.53125 1.453125 -5.90625 1.921875 -6.15625 C 2.398438 -6.414062 2.921875 -6.546875 3.484375 -6.546875 C 4.128906 -6.546875 4.671875 -6.378906 5.109375 -6.046875 C 5.554688 -5.722656 5.863281 -5.265625 6.03125 -4.671875 L 5.1875 -4.484375 C 5.039062 -4.953125 4.828125 -5.289062 4.546875 -5.5 C 4.265625 -5.71875 3.90625 -5.828125 3.46875 -5.828125 C 2.976562 -5.828125 2.566406 -5.707031 2.234375 -5.46875 C 1.898438 -5.226562 1.664062 -4.90625 1.53125 -4.5 C 1.394531 -4.101562 1.328125 -3.695312 1.328125 -3.28125 C 1.328125 -2.726562 1.40625 -2.242188 1.5625 -1.828125 C 1.726562 -1.421875 1.976562 -1.117188 2.3125 -0.921875 C 2.644531 -0.722656 3.007812 -0.625 3.40625 -0.625 C 3.882812 -0.625 4.285156 -0.757812 4.609375 -1.03125 C 4.941406 -1.3125 5.171875 -1.722656 5.296875 -2.265625 Z M 5.296875 -2.265625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-5\" overflow=\"visible\">\n",
       "<path d=\"M 0.296875 -2.328125 C 0.296875 -3.191406 0.535156 -3.832031 1.015625 -4.25 C 1.421875 -4.59375 1.910156 -4.765625 2.484375 -4.765625 C 3.128906 -4.765625 3.65625 -4.554688 4.0625 -4.140625 C 4.46875 -3.722656 4.671875 -3.144531 4.671875 -2.40625 C 4.671875 -1.800781 4.578125 -1.328125 4.390625 -0.984375 C 4.210938 -0.640625 3.953125 -0.367188 3.609375 -0.171875 C 3.265625 0.015625 2.890625 0.109375 2.484375 0.109375 C 1.828125 0.109375 1.296875 -0.0976562 0.890625 -0.515625 C 0.492188 -0.941406 0.296875 -1.546875 0.296875 -2.328125 Z M 1.109375 -2.328125 C 1.109375 -1.734375 1.238281 -1.285156 1.5 -0.984375 C 1.757812 -0.691406 2.085938 -0.546875 2.484375 -0.546875 C 2.878906 -0.546875 3.207031 -0.691406 3.46875 -0.984375 C 3.726562 -1.285156 3.859375 -1.742188 3.859375 -2.359375 C 3.859375 -2.929688 3.726562 -3.367188 3.46875 -3.671875 C 3.207031 -3.972656 2.878906 -4.125 2.484375 -4.125 C 2.085938 -4.125 1.757812 -3.972656 1.5 -3.671875 C 1.238281 -3.378906 1.109375 -2.929688 1.109375 -2.328125 Z M 1.109375 -2.328125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-6\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 0 L 0.59375 -4.671875 L 1.3125 -4.671875 L 1.3125 -4 C 1.644531 -4.507812 2.140625 -4.765625 2.796875 -4.765625 C 3.078125 -4.765625 3.332031 -4.710938 3.5625 -4.609375 C 3.800781 -4.515625 3.976562 -4.382812 4.09375 -4.21875 C 4.207031 -4.0625 4.289062 -3.867188 4.34375 -3.640625 C 4.375 -3.492188 4.390625 -3.238281 4.390625 -2.875 L 4.390625 0 L 3.59375 0 L 3.59375 -2.84375 C 3.59375 -3.164062 3.5625 -3.40625 3.5 -3.5625 C 3.4375 -3.71875 3.328125 -3.84375 3.171875 -3.9375 C 3.015625 -4.039062 2.832031 -4.09375 2.625 -4.09375 C 2.289062 -4.09375 2 -3.984375 1.75 -3.765625 C 1.507812 -3.554688 1.390625 -3.148438 1.390625 -2.546875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-7\" overflow=\"visible\">\n",
       "<path d=\"M 1.890625 0 L 0.109375 -4.671875 L 0.953125 -4.671875 L 1.953125 -1.875 C 2.054688 -1.570312 2.15625 -1.257812 2.25 -0.9375 C 2.320312 -1.175781 2.421875 -1.46875 2.546875 -1.8125 L 3.578125 -4.671875 L 4.390625 -4.671875 L 2.625 0 Z M 1.890625 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-8\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -1.703125 L 1.171875 -1.8125 C 1.265625 -1.363281 1.414062 -1.039062 1.625 -0.84375 C 1.84375 -0.644531 2.113281 -0.546875 2.4375 -0.546875 C 2.800781 -0.546875 3.109375 -0.671875 3.359375 -0.921875 C 3.617188 -1.179688 3.75 -1.503906 3.75 -1.890625 C 3.75 -2.253906 3.628906 -2.550781 3.390625 -2.78125 C 3.160156 -3.019531 2.863281 -3.140625 2.5 -3.140625 C 2.34375 -3.140625 2.15625 -3.109375 1.9375 -3.046875 L 2.03125 -3.75 C 2.082031 -3.738281 2.125 -3.734375 2.15625 -3.734375 C 2.488281 -3.734375 2.789062 -3.820312 3.0625 -4 C 3.332031 -4.175781 3.46875 -4.445312 3.46875 -4.8125 C 3.46875 -5.101562 3.367188 -5.34375 3.171875 -5.53125 C 2.972656 -5.71875 2.71875 -5.8125 2.40625 -5.8125 C 2.101562 -5.8125 1.847656 -5.710938 1.640625 -5.515625 C 1.441406 -5.328125 1.3125 -5.039062 1.25 -4.65625 L 0.453125 -4.796875 C 0.554688 -5.328125 0.773438 -5.738281 1.109375 -6.03125 C 1.453125 -6.320312 1.878906 -6.46875 2.390625 -6.46875 C 2.742188 -6.46875 3.066406 -6.390625 3.359375 -6.234375 C 3.660156 -6.085938 3.890625 -5.882812 4.046875 -5.625 C 4.203125 -5.363281 4.28125 -5.085938 4.28125 -4.796875 C 4.28125 -4.515625 4.203125 -4.257812 4.046875 -4.03125 C 3.898438 -3.800781 3.679688 -3.617188 3.390625 -3.484375 C 3.773438 -3.398438 4.070312 -3.21875 4.28125 -2.9375 C 4.488281 -2.664062 4.59375 -2.320312 4.59375 -1.90625 C 4.59375 -1.34375 4.382812 -0.863281 3.96875 -0.46875 C 3.5625 -0.0820312 3.046875 0.109375 2.421875 0.109375 C 1.859375 0.109375 1.390625 -0.0546875 1.015625 -0.390625 C 0.640625 -0.722656 0.425781 -1.160156 0.375 -1.703125 Z M 0.375 -1.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-9\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -1.6875 L 1.203125 -1.765625 C 1.265625 -1.359375 1.40625 -1.050781 1.625 -0.84375 C 1.851562 -0.644531 2.125 -0.546875 2.4375 -0.546875 C 2.820312 -0.546875 3.144531 -0.6875 3.40625 -0.96875 C 3.675781 -1.257812 3.8125 -1.640625 3.8125 -2.109375 C 3.8125 -2.566406 3.679688 -2.925781 3.421875 -3.1875 C 3.171875 -3.445312 2.84375 -3.578125 2.4375 -3.578125 C 2.175781 -3.578125 1.941406 -3.515625 1.734375 -3.390625 C 1.535156 -3.273438 1.375 -3.128906 1.25 -2.953125 L 0.515625 -3.046875 L 1.140625 -6.359375 L 4.34375 -6.359375 L 4.34375 -5.59375 L 1.765625 -5.59375 L 1.421875 -3.875 C 1.804688 -4.132812 2.210938 -4.265625 2.640625 -4.265625 C 3.203125 -4.265625 3.675781 -4.070312 4.0625 -3.6875 C 4.445312 -3.300781 4.640625 -2.800781 4.640625 -2.1875 C 4.640625 -1.601562 4.472656 -1.097656 4.140625 -0.671875 C 3.722656 -0.148438 3.15625 0.109375 2.4375 0.109375 C 1.851562 0.109375 1.375 -0.0507812 1 -0.375 C 0.632812 -0.707031 0.425781 -1.144531 0.375 -1.6875 Z M 0.375 -1.6875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-10\" overflow=\"visible\">\n",
       "<path d=\"M 3.640625 -1.703125 L 4.421875 -1.609375 C 4.335938 -1.066406 4.117188 -0.644531 3.765625 -0.34375 C 3.410156 -0.0390625 2.976562 0.109375 2.46875 0.109375 C 1.832031 0.109375 1.320312 -0.0976562 0.9375 -0.515625 C 0.550781 -0.929688 0.359375 -1.53125 0.359375 -2.3125 C 0.359375 -2.820312 0.441406 -3.265625 0.609375 -3.640625 C 0.773438 -4.015625 1.023438 -4.296875 1.359375 -4.484375 C 1.703125 -4.671875 2.078125 -4.765625 2.484375 -4.765625 C 2.984375 -4.765625 3.394531 -4.632812 3.71875 -4.375 C 4.039062 -4.125 4.25 -3.765625 4.34375 -3.296875 L 3.578125 -3.171875 C 3.503906 -3.484375 3.375 -3.71875 3.1875 -3.875 C 3 -4.039062 2.773438 -4.125 2.515625 -4.125 C 2.109375 -4.125 1.78125 -3.976562 1.53125 -3.6875 C 1.289062 -3.40625 1.171875 -2.957031 1.171875 -2.34375 C 1.171875 -1.707031 1.289062 -1.25 1.53125 -0.96875 C 1.769531 -0.6875 2.082031 -0.546875 2.46875 -0.546875 C 2.78125 -0.546875 3.039062 -0.640625 3.25 -0.828125 C 3.457031 -1.015625 3.585938 -1.304688 3.640625 -1.703125 Z M 3.640625 -1.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-11\" overflow=\"visible\">\n",
       "<path d=\"M 0.671875 0 L 0.671875 -6.4375 L 1.953125 -6.4375 L 3.46875 -1.875 C 3.613281 -1.457031 3.71875 -1.140625 3.78125 -0.921875 C 3.851562 -1.160156 3.96875 -1.503906 4.125 -1.953125 L 5.671875 -6.4375 L 6.8125 -6.4375 L 6.8125 0 L 6 0 L 6 -5.390625 L 4.125 0 L 3.359375 0 L 1.484375 -5.484375 L 1.484375 0 Z M 0.671875 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-12\" overflow=\"visible\">\n",
       "<path d=\"M 3.640625 -0.578125 C 3.347656 -0.328125 3.066406 -0.148438 2.796875 -0.046875 C 2.523438 0.0546875 2.234375 0.109375 1.921875 0.109375 C 1.410156 0.109375 1.015625 -0.015625 0.734375 -0.265625 C 0.460938 -0.515625 0.328125 -0.835938 0.328125 -1.234375 C 0.328125 -1.460938 0.378906 -1.671875 0.484375 -1.859375 C 0.585938 -2.046875 0.722656 -2.195312 0.890625 -2.3125 C 1.054688 -2.425781 1.242188 -2.515625 1.453125 -2.578125 C 1.609375 -2.609375 1.84375 -2.644531 2.15625 -2.6875 C 2.800781 -2.757812 3.273438 -2.851562 3.578125 -2.96875 C 3.578125 -3.070312 3.578125 -3.140625 3.578125 -3.171875 C 3.578125 -3.492188 3.503906 -3.71875 3.359375 -3.84375 C 3.148438 -4.03125 2.847656 -4.125 2.453125 -4.125 C 2.078125 -4.125 1.800781 -4.054688 1.625 -3.921875 C 1.445312 -3.796875 1.316406 -3.566406 1.234375 -3.234375 L 0.46875 -3.328125 C 0.53125 -3.660156 0.640625 -3.925781 0.796875 -4.125 C 0.960938 -4.332031 1.195312 -4.488281 1.5 -4.59375 C 1.8125 -4.707031 2.164062 -4.765625 2.5625 -4.765625 C 2.96875 -4.765625 3.289062 -4.71875 3.53125 -4.625 C 3.78125 -4.53125 3.960938 -4.410156 4.078125 -4.265625 C 4.203125 -4.128906 4.285156 -3.953125 4.328125 -3.734375 C 4.359375 -3.597656 4.375 -3.359375 4.375 -3.015625 L 4.375 -1.953125 C 4.375 -1.222656 4.390625 -0.757812 4.421875 -0.5625 C 4.453125 -0.363281 4.519531 -0.175781 4.625 0 L 3.796875 0 C 3.710938 -0.164062 3.660156 -0.359375 3.640625 -0.578125 Z M 3.578125 -2.34375 C 3.285156 -2.226562 2.851562 -2.128906 2.28125 -2.046875 C 1.957031 -1.992188 1.726562 -1.9375 1.59375 -1.875 C 1.457031 -1.820312 1.351562 -1.738281 1.28125 -1.625 C 1.207031 -1.507812 1.171875 -1.382812 1.171875 -1.25 C 1.171875 -1.039062 1.25 -0.863281 1.40625 -0.71875 C 1.5625 -0.582031 1.796875 -0.515625 2.109375 -0.515625 C 2.410156 -0.515625 2.679688 -0.582031 2.921875 -0.71875 C 3.160156 -0.851562 3.335938 -1.035156 3.453125 -1.265625 C 3.535156 -1.441406 3.578125 -1.703125 3.578125 -2.046875 Z M 3.578125 -2.34375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-13\" overflow=\"visible\">\n",
       "<path d=\"M 0.6875 0 L 0.6875 -6.4375 L 3.125 -6.4375 C 3.550781 -6.4375 3.878906 -6.414062 4.109375 -6.375 C 4.421875 -6.320312 4.679688 -6.222656 4.890625 -6.078125 C 5.109375 -5.929688 5.28125 -5.722656 5.40625 -5.453125 C 5.539062 -5.191406 5.609375 -4.898438 5.609375 -4.578125 C 5.609375 -4.023438 5.429688 -3.5625 5.078125 -3.1875 C 4.734375 -2.8125 4.109375 -2.625 3.203125 -2.625 L 1.546875 -2.625 L 1.546875 0 Z M 1.546875 -3.375 L 3.21875 -3.375 C 3.757812 -3.375 4.144531 -3.476562 4.375 -3.6875 C 4.613281 -3.894531 4.734375 -4.179688 4.734375 -4.546875 C 4.734375 -4.816406 4.664062 -5.046875 4.53125 -5.234375 C 4.394531 -5.429688 4.210938 -5.5625 3.984375 -5.625 C 3.847656 -5.664062 3.582031 -5.6875 3.1875 -5.6875 L 1.546875 -5.6875 Z M 1.546875 -3.375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-14\" overflow=\"visible\">\n",
       "<path d=\"M 0.578125 0 L 0.578125 -6.4375 L 1.359375 -6.4375 L 1.359375 0 Z M 0.578125 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-15\" overflow=\"visible\">\n",
       "<path d=\"M 0.84375 0 L 0.84375 -6.4375 L 1.6875 -6.4375 L 1.6875 0 Z M 0.84375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-16\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 1.78125 L 0.59375 -4.671875 L 1.3125 -4.671875 L 1.3125 -4.0625 C 1.476562 -4.300781 1.664062 -4.476562 1.875 -4.59375 C 2.09375 -4.707031 2.359375 -4.765625 2.671875 -4.765625 C 3.066406 -4.765625 3.414062 -4.660156 3.71875 -4.453125 C 4.019531 -4.253906 4.25 -3.96875 4.40625 -3.59375 C 4.5625 -3.21875 4.640625 -2.8125 4.640625 -2.375 C 4.640625 -1.894531 4.550781 -1.460938 4.375 -1.078125 C 4.207031 -0.691406 3.960938 -0.394531 3.640625 -0.1875 C 3.316406 0.0078125 2.972656 0.109375 2.609375 0.109375 C 2.347656 0.109375 2.113281 0.0507812 1.90625 -0.0625 C 1.695312 -0.175781 1.523438 -0.316406 1.390625 -0.484375 L 1.390625 1.78125 Z M 1.3125 -2.3125 C 1.3125 -1.707031 1.429688 -1.257812 1.671875 -0.96875 C 1.921875 -0.6875 2.21875 -0.546875 2.5625 -0.546875 C 2.90625 -0.546875 3.203125 -0.691406 3.453125 -0.984375 C 3.710938 -1.285156 3.84375 -1.75 3.84375 -2.375 C 3.84375 -2.96875 3.71875 -3.410156 3.46875 -3.703125 C 3.226562 -4.003906 2.9375 -4.15625 2.59375 -4.15625 C 2.257812 -4.15625 1.960938 -3.992188 1.703125 -3.671875 C 1.441406 -3.359375 1.3125 -2.90625 1.3125 -2.3125 Z M 1.3125 -2.3125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-17\" overflow=\"visible\">\n",
       "<path d=\"M 3.65625 0 L 3.65625 -0.6875 C 3.289062 -0.15625 2.796875 0.109375 2.171875 0.109375 C 1.898438 0.109375 1.644531 0.0546875 1.40625 -0.046875 C 1.164062 -0.160156 0.984375 -0.296875 0.859375 -0.453125 C 0.742188 -0.609375 0.664062 -0.800781 0.625 -1.03125 C 0.59375 -1.1875 0.578125 -1.4375 0.578125 -1.78125 L 0.578125 -4.671875 L 1.359375 -4.671875 L 1.359375 -2.078125 C 1.359375 -1.660156 1.378906 -1.382812 1.421875 -1.25 C 1.460938 -1.039062 1.5625 -0.875 1.71875 -0.75 C 1.882812 -0.632812 2.085938 -0.578125 2.328125 -0.578125 C 2.566406 -0.578125 2.789062 -0.632812 3 -0.75 C 3.207031 -0.875 3.351562 -1.039062 3.4375 -1.25 C 3.519531 -1.457031 3.5625 -1.765625 3.5625 -2.171875 L 3.5625 -4.671875 L 4.359375 -4.671875 L 4.359375 0 Z M 3.65625 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-18\" overflow=\"visible\">\n",
       "<path d=\"M 2.328125 -0.703125 L 2.4375 -0.015625 C 2.207031 0.0351562 2.007812 0.0625 1.84375 0.0625 C 1.550781 0.0625 1.328125 0.015625 1.171875 -0.078125 C 1.015625 -0.171875 0.898438 -0.289062 0.828125 -0.4375 C 0.765625 -0.582031 0.734375 -0.890625 0.734375 -1.359375 L 0.734375 -4.046875 L 0.15625 -4.046875 L 0.15625 -4.671875 L 0.734375 -4.671875 L 0.734375 -5.828125 L 1.53125 -6.296875 L 1.53125 -4.671875 L 2.328125 -4.671875 L 2.328125 -4.046875 L 1.53125 -4.046875 L 1.53125 -1.328125 C 1.53125 -1.097656 1.539062 -0.953125 1.5625 -0.890625 C 1.59375 -0.828125 1.640625 -0.773438 1.703125 -0.734375 C 1.765625 -0.691406 1.851562 -0.671875 1.96875 -0.671875 C 2.0625 -0.671875 2.179688 -0.679688 2.328125 -0.703125 Z M 2.328125 -0.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-19\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 0 L 0.59375 -6.4375 L 1.390625 -6.4375 L 1.390625 -4.125 C 1.753906 -4.550781 2.21875 -4.765625 2.78125 -4.765625 C 3.125 -4.765625 3.421875 -4.695312 3.671875 -4.5625 C 3.929688 -4.425781 4.113281 -4.238281 4.21875 -4 C 4.332031 -3.757812 4.390625 -3.410156 4.390625 -2.953125 L 4.390625 0 L 3.609375 0 L 3.609375 -2.953125 C 3.609375 -3.347656 3.519531 -3.632812 3.34375 -3.8125 C 3.175781 -4 2.9375 -4.09375 2.625 -4.09375 C 2.382812 -4.09375 2.160156 -4.03125 1.953125 -3.90625 C 1.742188 -3.789062 1.597656 -3.628906 1.515625 -3.421875 C 1.429688 -3.210938 1.390625 -2.921875 1.390625 -2.546875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-20\" overflow=\"visible\">\n",
       "<path d=\"M 3.78125 -1.5 L 4.609375 -1.40625 C 4.472656 -0.925781 4.226562 -0.550781 3.875 -0.28125 C 3.53125 -0.0195312 3.085938 0.109375 2.546875 0.109375 C 1.867188 0.109375 1.328125 -0.0976562 0.921875 -0.515625 C 0.523438 -0.941406 0.328125 -1.535156 0.328125 -2.296875 C 0.328125 -3.078125 0.53125 -3.679688 0.9375 -4.109375 C 1.34375 -4.546875 1.867188 -4.765625 2.515625 -4.765625 C 3.140625 -4.765625 3.644531 -4.550781 4.03125 -4.125 C 4.425781 -3.707031 4.625 -3.113281 4.625 -2.34375 C 4.625 -2.289062 4.625 -2.21875 4.625 -2.125 L 1.140625 -2.125 C 1.171875 -1.613281 1.316406 -1.222656 1.578125 -0.953125 C 1.835938 -0.679688 2.164062 -0.546875 2.5625 -0.546875 C 2.851562 -0.546875 3.097656 -0.617188 3.296875 -0.765625 C 3.503906 -0.921875 3.664062 -1.164062 3.78125 -1.5 Z M 1.1875 -2.78125 L 3.796875 -2.78125 C 3.765625 -3.175781 3.664062 -3.472656 3.5 -3.671875 C 3.25 -3.972656 2.921875 -4.125 2.515625 -4.125 C 2.148438 -4.125 1.84375 -4 1.59375 -3.75 C 1.351562 -3.507812 1.21875 -3.1875 1.1875 -2.78125 Z M 1.1875 -2.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-21\" overflow=\"visible\">\n",
       "<path d=\"M 0.28125 -1.390625 L 1.0625 -1.515625 C 1.101562 -1.203125 1.222656 -0.960938 1.421875 -0.796875 C 1.628906 -0.628906 1.910156 -0.546875 2.265625 -0.546875 C 2.628906 -0.546875 2.898438 -0.617188 3.078125 -0.765625 C 3.253906 -0.910156 3.34375 -1.082031 3.34375 -1.28125 C 3.34375 -1.46875 3.265625 -1.609375 3.109375 -1.703125 C 2.992188 -1.773438 2.722656 -1.867188 2.296875 -1.984375 C 1.710938 -2.128906 1.304688 -2.253906 1.078125 -2.359375 C 0.859375 -2.460938 0.691406 -2.609375 0.578125 -2.796875 C 0.460938 -2.984375 0.40625 -3.191406 0.40625 -3.421875 C 0.40625 -3.628906 0.453125 -3.820312 0.546875 -4 C 0.640625 -4.175781 0.769531 -4.328125 0.9375 -4.453125 C 1.0625 -4.535156 1.226562 -4.609375 1.4375 -4.671875 C 1.65625 -4.734375 1.882812 -4.765625 2.125 -4.765625 C 2.488281 -4.765625 2.804688 -4.710938 3.078125 -4.609375 C 3.359375 -4.503906 3.566406 -4.363281 3.703125 -4.1875 C 3.835938 -4.007812 3.929688 -3.769531 3.984375 -3.46875 L 3.203125 -3.359375 C 3.171875 -3.597656 3.066406 -3.785156 2.890625 -3.921875 C 2.722656 -4.054688 2.488281 -4.125 2.1875 -4.125 C 1.820312 -4.125 1.5625 -4.0625 1.40625 -3.9375 C 1.25 -3.820312 1.171875 -3.679688 1.171875 -3.515625 C 1.171875 -3.410156 1.203125 -3.320312 1.265625 -3.25 C 1.328125 -3.15625 1.429688 -3.082031 1.578125 -3.03125 C 1.648438 -3 1.878906 -2.929688 2.265625 -2.828125 C 2.828125 -2.679688 3.21875 -2.5625 3.4375 -2.46875 C 3.664062 -2.375 3.84375 -2.234375 3.96875 -2.046875 C 4.09375 -1.867188 4.15625 -1.644531 4.15625 -1.375 C 4.15625 -1.101562 4.078125 -0.851562 3.921875 -0.625 C 3.765625 -0.394531 3.539062 -0.210938 3.25 -0.078125 C 2.96875 0.046875 2.640625 0.109375 2.265625 0.109375 C 1.660156 0.109375 1.195312 -0.015625 0.875 -0.265625 C 0.5625 -0.523438 0.363281 -0.898438 0.28125 -1.390625 Z M 0.28125 -1.390625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<rect height=\"128\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"330\" x=\"0\" y=\"0\"/>\n",
       "<rect height=\"128\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"330\" x=\"0\" y=\"0\"/>\n",
       "<path d=\"M 40.5 54 L 103.5 54 L 103.5 72 L 40.5 72 Z M 40.5 54 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.900818%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"10.48877\" xlink:href=\"#glyph0-1\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"15.49457\" xlink:href=\"#glyph0-2\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"17.99477\" xlink:href=\"#glyph0-3\" y=\"66.852783\"/>\n",
       "  <use x=\"22.49477\" xlink:href=\"#glyph0-2\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"24.99497\" xlink:href=\"#glyph0-1\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"32.50097\" xlink:href=\"#glyph0-4\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"39.00077\" xlink:href=\"#glyph0-5\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"44.00657\" xlink:href=\"#glyph0-6\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"49.01237\" xlink:href=\"#glyph0-7\" y=\"66.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 121.5 36 L 189 36 L 189 54 L 121.5 54 Z M 121.5 36 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.900818%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"93.73877\" xlink:href=\"#glyph0-8\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"98.74457\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"101.24477\" xlink:href=\"#glyph0-3\" y=\"48.852783\"/>\n",
       "  <use x=\"105.74477\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"108.24497\" xlink:href=\"#glyph0-8\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"113.25077\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"115.75097\" xlink:href=\"#glyph0-4\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"122.25077\" xlink:href=\"#glyph0-5\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"127.25657\" xlink:href=\"#glyph0-6\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"132.26237\" xlink:href=\"#glyph0-7\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 121.5 72 L 189 72 L 189 90 L 121.5 90 Z M 121.5 72 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"93.73877\" xlink:href=\"#glyph0-1\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"98.74457\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"101.24477\" xlink:href=\"#glyph0-3\" y=\"84.852783\"/>\n",
       "  <use x=\"105.74477\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"108.24497\" xlink:href=\"#glyph0-1\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"113.25077\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"115.75097\" xlink:href=\"#glyph0-4\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"122.25077\" xlink:href=\"#glyph0-5\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"127.25657\" xlink:href=\"#glyph0-6\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"132.26237\" xlink:href=\"#glyph0-7\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 202.5 72 L 270 72 L 270 90 L 202.5 90 Z M 202.5 72 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"174.73877\" xlink:href=\"#glyph0-1\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"179.74457\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"182.24477\" xlink:href=\"#glyph0-3\" y=\"84.852783\"/>\n",
       "  <use x=\"186.74477\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"189.24497\" xlink:href=\"#glyph0-1\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"194.25077\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"196.75097\" xlink:href=\"#glyph0-4\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"203.25077\" xlink:href=\"#glyph0-5\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"208.25657\" xlink:href=\"#glyph0-6\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"213.26237\" xlink:href=\"#glyph0-7\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 202.5 36 L 270 36 L 270 54 L 202.5 54 Z M 202.5 36 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.900818%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"175.73853\" xlink:href=\"#glyph0-9\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"180.74433\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"183.24453\" xlink:href=\"#glyph0-3\" y=\"48.852783\"/>\n",
       "  <use x=\"187.74453\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"190.24473\" xlink:href=\"#glyph0-9\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"195.25053\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"197.75073\" xlink:href=\"#glyph0-10\" y=\"48.852783\"/>\n",
       "  <use x=\"202.25073\" xlink:href=\"#glyph0-5\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"207.25653\" xlink:href=\"#glyph0-6\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"212.26233\" xlink:href=\"#glyph0-7\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 297 36 L 360 36 L 360 54 L 297 54 Z M 297 36 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"266.98877\" xlink:href=\"#glyph0-1\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.99457\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"274.49477\" xlink:href=\"#glyph0-3\" y=\"48.852783\"/>\n",
       "  <use x=\"278.99477\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"281.49497\" xlink:href=\"#glyph0-1\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"286.50077\" xlink:href=\"#glyph0-2\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"289.00097\" xlink:href=\"#glyph0-4\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"295.50077\" xlink:href=\"#glyph0-5\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"300.50657\" xlink:href=\"#glyph0-6\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"305.51237\" xlink:href=\"#glyph0-7\" y=\"48.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 288 72 L 369 72 L 369 90 L 288 90 Z M 288 72 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.900818%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"258.73584\" xlink:href=\"#glyph0-8\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"263.74164\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"266.24184\" xlink:href=\"#glyph0-3\" y=\"84.852783\"/>\n",
       "  <use x=\"270.74184\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"273.24204\" xlink:href=\"#glyph0-8\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"278.24784\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"280.74804\" xlink:href=\"#glyph0-11\" y=\"84.852783\"/>\n",
       "  <use x=\"288.24504\" xlink:href=\"#glyph0-12\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"293.25084\" xlink:href=\"#glyph0-3\" y=\"84.852783\"/>\n",
       "  <use x=\"297.75084\" xlink:href=\"#glyph0-2\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"300.25104\" xlink:href=\"#glyph0-13\" y=\"84.852783\"/>\n",
       "  <use x=\"306.25404\" xlink:href=\"#glyph0-5\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"311.25984\" xlink:href=\"#glyph0-5\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"316.26564\" xlink:href=\"#glyph0-14\" y=\"84.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 162 108 L 229.5 108 L 229.5 126 L 162 126 Z M 162 108 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"145.74146\" xlink:href=\"#glyph0-15\" y=\"120.852783\"/>\n",
       "  <use x=\"148.24166\" xlink:href=\"#glyph0-6\" y=\"120.852783\"/>\n",
       "  <use x=\"153.24386\" xlink:href=\"#glyph0-16\" y=\"120.852783\"/>\n",
       "  <use x=\"158.24606\" xlink:href=\"#glyph0-17\" y=\"120.852783\"/>\n",
       "  <use x=\"163.24826\" xlink:href=\"#glyph0-18\" y=\"120.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 157.5 0 L 238.5 0 L 238.5 18 L 157.5 18 Z M 157.5 0 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"124.47852\" xlink:href=\"#glyph0-4\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.97832\" xlink:href=\"#glyph0-5\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"135.98412\" xlink:href=\"#glyph0-6\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"140.98992\" xlink:href=\"#glyph0-10\" y=\"12.852783\"/>\n",
       "  <use x=\"145.48992\" xlink:href=\"#glyph0-12\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"150.49572\" xlink:href=\"#glyph0-18\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"152.99592\" xlink:href=\"#glyph0-2\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"155.49612\" xlink:href=\"#glyph0-10\" y=\"12.852783\"/>\n",
       "  <use x=\"159.99612\" xlink:href=\"#glyph0-19\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"165.00192\" xlink:href=\"#glyph0-12\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"170.00772\" xlink:href=\"#glyph0-6\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"175.01352\" xlink:href=\"#glyph0-6\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"180.01932\" xlink:href=\"#glyph0-20\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"185.02512\" xlink:href=\"#glyph0-14\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"187.02492\" xlink:href=\"#glyph0-21\" y=\"12.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 205.875 108 L 221.714844 93.917969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 224.703125 91.261719 L 221.714844 93.917969 M 220.71875 92.796875 L 224.703125 91.261719 L 222.710938 95.039062 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 185.625 108 L 169.785156 93.917969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 166.796875 91.261719 L 169.785156 93.917969 M 168.789062 95.039062 L 166.796875 91.261719 L 170.78125 92.796875 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 155.25 72 L 155.25 59.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 155.25 55.898438 L 155.25 59.898438 M 153.75 59.898438 L 155.25 55.898438 L 156.75 59.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 236.25 72 L 236.25 59.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 236.25 55.898438 L 236.25 59.898438 M 234.75 59.898438 L 236.25 55.898438 L 237.75 59.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 328.5 72 L 328.5 59.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 328.5 55.898438 L 328.5 59.898438 M 327 59.898438 L 328.5 55.898438 L 330 59.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 162 117 L 75 117 C 73.34375 117 72 115.65625 72 114 L 72 77.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 72 73.898438 L 72 77.898438 M 70.5 77.898438 L 72 73.898438 L 73.5 77.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 72 54 L 72 12 C 72 10.34375 73.34375 9 75 9 L 151.601562 9 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 155.601562 9 L 151.601562 9 M 151.601562 7.5 L 155.601562 9 L 151.601562 10.5 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 165.9375 36 L 182.800781 21.800781 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 185.859375 19.222656 L 182.800781 21.800781 M 181.832031 20.652344 L 185.859375 19.222656 L 183.765625 22.949219 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 226.6875 36 L 211.859375 22.042969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 208.945312 19.300781 L 211.859375 22.042969 M 210.832031 23.136719 L 208.945312 19.300781 L 212.886719 20.953125 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 328.5 36 L 328.5 12 C 328.5 10.34375 327.15625 9 325.5 9 L 244.398438 9 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 240.398438 9 L 244.398438 9 M 244.398438 10.5 L 240.398438 9 L 244.398438 7.5 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 229.5 117 L 325.5 117 C 327.15625 117 328.5 115.65625 328.5 114 L 328.5 95.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "<path d=\"M 328.5 91.898438 L 328.5 95.898438 M 327 95.898438 L 328.5 91.898438 L 330 95.898438 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-40,1)\"/>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import SVG\n",
    "SVG(filename=\"../img/inception.svg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As depicted in the figure above, \n",
    "the inception block consists of four parallel paths. \n",
    "The first three paths use convolutional layers \n",
    "with window sizes of $1\\times 1$, $3\\times 3$, and $5\\times 5$ \n",
    "to extract information from different spatial sizes. \n",
    "The middle two paths perform a $1\\times 1$ convolution on the input \n",
    "to reduce the number of input channels, reducing the model's complexity. \n",
    "The fourth path uses a $3\\times 3$ maximum pooling layer, \n",
    "followed by a $1\\times 1$ convolutional layer \n",
    "to change the number of channels. \n",
    "The four paths all use appropriate padding to give the input and output the same height and width. \n",
    "Finally, the outputs along each path are concatenated \n",
    "along the channel dimension and comprise the block's output. \n",
    "The commonly-tuned parameters of the Inception block\n",
    "are the number of output channels per layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "1"
    },
    "colab": {},
    "colab_type": "code",
    "id": "0AClS9IlaYHo"
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.insert(0, '..')\n",
    "\n",
    "import d2l\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Inception(nn.Module):\n",
    "    # c1 - c4 are the number of output channels for each layer in the path\n",
    "    def __init__(self, in_channels, c1, c2, c3, c4, **kwargs):\n",
    "        super(Inception, self).__init__(**kwargs)\n",
    "        # Path 1 is a single 1 x 1 convolutional layer\n",
    "        self.p1_1 = nn.Conv2d(in_channels, c1, kernel_size=1)\n",
    "        # Path 2 is a 1 x 1 convolutional layer followed by a 3 x 3\n",
    "        # convolutional layer\n",
    "        self.p2_1 = nn.Conv2d(in_channels, c2[0], kernel_size=1)\n",
    "        self.p2_2 = nn.Conv2d(c2[0], c2[1], kernel_size=3, padding=1)\n",
    "        # Path 3 is a 1 x 1 convolutional layer followed by a 5 x 5\n",
    "        # convolutional layer\n",
    "        self.p3_1 = nn.Conv2d(in_channels, c3[0], kernel_size=1)\n",
    "        self.p3_2 = nn.Conv2d(c3[0], c3[1], kernel_size=5, padding=2)\n",
    "        # Path 4 is a 3 x 3 maximum pooling layer followed by a 1 x 1\n",
    "        # convolutional layer\n",
    "        self.p4_1 = nn.MaxPool2d(kernel_size=3, stride=1, padding=1)\n",
    "        self.p4_2 = nn.Conv2d(in_channels, c4, kernel_size=1)\n",
    "\n",
    "    def forward(self, x):\n",
    "        p1 = F.relu(self.p1_1(x))\n",
    "        p2 = F.relu(self.p2_2(F.relu(self.p2_1(x))))\n",
    "        p3 = F.relu(self.p3_2(F.relu(self.p3_1(x))))\n",
    "        p4 = F.relu(self.p4_2(self.p4_1(x)))\n",
    "        # Concatenate the outputs on the channel dimension\n",
    "        return torch.cat((p1, p2, p3, p4), dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EJfqpHLGaYHw"
   },
   "source": [
    "To gain some intuition for why this network works so well, \n",
    "consider the combination of the filters. \n",
    "They explore the image in varying ranges. \n",
    "This means that details at different extents \n",
    "can be recognized efficiently by different filters. \n",
    "At the same time, we can allocate different amounts of parameters \n",
    "for different ranges (e.g. more for short range \n",
    "but not ignore the long range entirely).\n",
    "\n",
    "## GoogLeNet Model\n",
    "\n",
    "GoogLeNet uses a stack of a total of 9 inception blocks \n",
    "and global average pooling to generate its estimates.\n",
    "Maximum pooling between inception blocks reduced the dimensionality. \n",
    "The first part is identical to AlexNet and LeNet, \n",
    "the stack of blocks is inherited from VGG \n",
    "and the global average pooling avoids \n",
    "a stack of fully-connected layers at the end. \n",
    "The architecture is depicted below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"319pt\" version=\"1.1\" viewBox=\"0 0 110 319\" width=\"110pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol id=\"glyph0-0\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-1\" overflow=\"visible\">\n",
       "<path d=\"M 2.765625 -6.546875 C 2.15625 -6.546875 1.65625 -6.375 1.28125 -6.03125 C 0.875 -5.671875 0.640625 -5.1875 0.59375 -4.578125 L 1.328125 -4.578125 C 1.375 -5 1.515625 -5.34375 1.765625 -5.578125 C 2 -5.8125 2.34375 -5.90625 2.765625 -5.90625 C 3.1875 -5.90625 3.515625 -5.8125 3.75 -5.59375 C 3.96875 -5.40625 4.09375 -5.125 4.09375 -4.765625 C 4.09375 -4.40625 3.96875 -4.125 3.75 -3.921875 C 3.515625 -3.734375 3.1875 -3.640625 2.765625 -3.640625 L 2.265625 -3.640625 L 2.265625 -3.0625 L 2.796875 -3.0625 C 3.234375 -3.0625 3.578125 -2.96875 3.828125 -2.75 C 4.09375 -2.53125 4.21875 -2.234375 4.21875 -1.84375 C 4.21875 -1.46875 4.078125 -1.15625 3.828125 -0.90625 C 3.546875 -0.640625 3.171875 -0.5 2.71875 -0.5 C 2.3125 -0.5 1.984375 -0.625 1.703125 -0.84375 C 1.390625 -1.09375 1.21875 -1.484375 1.203125 -1.984375 L 0.453125 -1.984375 C 0.515625 -1.265625 0.765625 -0.71875 1.21875 -0.34375 C 1.609375 -0.03125 2.109375 0.125 2.71875 0.125 C 3.390625 0.125 3.921875 -0.0625 4.34375 -0.4375 C 4.75 -0.8125 4.953125 -1.28125 4.953125 -1.875 C 4.953125 -2.25 4.84375 -2.5625 4.625 -2.8125 C 4.421875 -3.0625 4.140625 -3.25 3.765625 -3.359375 C 4.46875 -3.59375 4.8125 -4.09375 4.8125 -4.8125 C 4.8125 -5.34375 4.625 -5.765625 4.25 -6.078125 C 3.875 -6.40625 3.375 -6.546875 2.765625 -6.546875 Z M 2.765625 -6.546875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-2\" overflow=\"visible\">\n",
       "<path d=\"M 0.25 -4.65625 L 1.890625 -2.46875 L 0.046875 0 L 0.90625 0 L 2.296875 -1.9375 L 3.6875 0 L 4.53125 0 L 2.6875 -2.46875 L 4.34375 -4.65625 L 3.484375 -4.65625 L 2.296875 -3 L 1.09375 -4.65625 Z M 0.25 -4.65625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-3\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-4\" overflow=\"visible\">\n",
       "<path d=\"M 0.640625 -6.421875 L 0.640625 0 L 1.390625 0 L 1.390625 -5.109375 L 1.40625 -5.109375 L 3.640625 0 L 4.296875 0 L 6.53125 -5.109375 L 6.546875 -5.109375 L 6.546875 0 L 7.296875 0 L 7.296875 -6.421875 L 6.375 -6.421875 L 3.984375 -0.9375 L 3.953125 -0.9375 L 1.5625 -6.421875 Z M 0.640625 -6.421875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-5\" overflow=\"visible\">\n",
       "<path d=\"M 2.578125 -4.78125 C 2.03125 -4.78125 1.578125 -4.671875 1.234375 -4.453125 C 0.859375 -4.21875 0.625 -3.859375 0.515625 -3.359375 L 1.234375 -3.3125 C 1.3125 -3.609375 1.453125 -3.828125 1.703125 -3.984375 C 1.921875 -4.125 2.203125 -4.1875 2.53125 -4.1875 C 3.34375 -4.1875 3.75 -3.796875 3.75 -3.015625 L 3.75 -2.765625 L 2.640625 -2.75 C 1.9375 -2.75 1.390625 -2.609375 1.015625 -2.359375 C 0.578125 -2.09375 0.375 -1.6875 0.375 -1.15625 C 0.375 -0.78125 0.515625 -0.46875 0.8125 -0.21875 C 1.09375 0.015625 1.453125 0.125 1.921875 0.125 C 2.359375 0.125 2.75 0.03125 3.09375 -0.15625 C 3.375 -0.328125 3.625 -0.53125 3.796875 -0.78125 L 3.796875 0 L 4.46875 0 L 4.46875 -2.984375 C 4.46875 -3.53125 4.328125 -3.953125 4.0625 -4.25 C 3.734375 -4.609375 3.25 -4.78125 2.578125 -4.78125 Z M 3.75 -2.234375 L 3.75 -1.84375 C 3.75 -1.46875 3.578125 -1.15625 3.25 -0.875 C 2.90625 -0.578125 2.515625 -0.4375 2.078125 -0.4375 C 1.796875 -0.4375 1.5625 -0.515625 1.375 -0.640625 C 1.203125 -0.796875 1.109375 -0.96875 1.109375 -1.171875 C 1.109375 -1.859375 1.640625 -2.203125 2.6875 -2.21875 Z M 3.75 -2.234375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-6\" overflow=\"visible\">\n",
       "<path d=\"M 0.640625 -6.421875 L 0.640625 0 L 1.390625 0 L 1.390625 -2.625 L 3.265625 -2.625 C 4.75 -2.625 5.484375 -3.265625 5.484375 -4.53125 C 5.484375 -5.796875 4.75 -6.421875 3.28125 -6.421875 Z M 1.390625 -5.78125 L 3.234375 -5.78125 C 3.75 -5.78125 4.125 -5.6875 4.375 -5.46875 C 4.625 -5.28125 4.75 -4.96875 4.75 -4.53125 C 4.75 -4.109375 4.625 -3.78125 4.390625 -3.578125 C 4.125 -3.375 3.75 -3.265625 3.234375 -3.265625 L 1.390625 -3.265625 Z M 1.390625 -5.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-7\" overflow=\"visible\">\n",
       "<path d=\"M 2.640625 -4.78125 C 1.9375 -4.78125 1.390625 -4.546875 0.96875 -4.0625 C 0.578125 -3.59375 0.375 -3.03125 0.375 -2.328125 C 0.375 -1.625 0.578125 -1.046875 0.96875 -0.609375 C 1.390625 -0.109375 1.9375 0.125 2.640625 0.125 C 3.328125 0.125 3.890625 -0.109375 4.3125 -0.609375 C 4.703125 -1.046875 4.890625 -1.625 4.890625 -2.328125 C 4.890625 -3.03125 4.703125 -3.59375 4.296875 -4.0625 C 3.890625 -4.546875 3.328125 -4.78125 2.640625 -4.78125 Z M 2.640625 -4.171875 C 3.125 -4.171875 3.515625 -3.984375 3.796875 -3.609375 C 4.046875 -3.265625 4.171875 -2.84375 4.171875 -2.328125 C 4.171875 -1.8125 4.046875 -1.390625 3.796875 -1.046875 C 3.515625 -0.671875 3.125 -0.484375 2.640625 -0.484375 C 2.140625 -0.484375 1.75 -0.671875 1.484375 -1.046875 C 1.234375 -1.390625 1.109375 -1.8125 1.109375 -2.328125 C 1.109375 -2.84375 1.234375 -3.265625 1.484375 -3.609375 C 1.75 -3.984375 2.140625 -4.171875 2.640625 -4.171875 Z M 2.640625 -4.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-8\" overflow=\"visible\">\n",
       "<path d=\"M 0.703125 -6.546875 L 0.703125 0 L 1.40625 0 L 1.40625 -6.546875 Z M 0.703125 -6.546875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-9\" overflow=\"visible\">\n",
       "<path d=\"M 0.3125 -6.421875 L 0.3125 -5.75 L 3.765625 -5.75 L 1.375 0 L 2.15625 0 L 4.53125 -5.828125 L 4.53125 -6.421875 Z M 0.3125 -6.421875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-10\" overflow=\"visible\">\n",
       "<path d=\"M 3.421875 -6.546875 C 2.4375 -6.546875 1.65625 -6.21875 1.109375 -5.53125 C 0.625 -4.921875 0.375 -4.140625 0.375 -3.1875 C 0.375 -2.234375 0.609375 -1.453125 1.09375 -0.859375 C 1.625 -0.203125 2.390625 0.125 3.390625 0.125 C 4.078125 0.125 4.671875 -0.078125 5.171875 -0.453125 C 5.703125 -0.875 6.03125 -1.453125 6.171875 -2.203125 L 5.46875 -2.203125 C 5.34375 -1.640625 5.078125 -1.21875 4.703125 -0.9375 C 4.34375 -0.671875 3.90625 -0.53125 3.390625 -0.53125 C 2.640625 -0.53125 2.0625 -0.78125 1.671875 -1.28125 C 1.296875 -1.75 1.109375 -2.390625 1.109375 -3.1875 C 1.109375 -3.984375 1.3125 -4.625 1.6875 -5.109375 C 2.09375 -5.640625 2.65625 -5.890625 3.40625 -5.890625 C 3.921875 -5.890625 4.34375 -5.78125 4.6875 -5.53125 C 5.046875 -5.28125 5.28125 -4.921875 5.375 -4.453125 L 6.078125 -4.453125 C 6 -5.09375 5.703125 -5.609375 5.21875 -5.984375 C 4.734375 -6.359375 4.125 -6.546875 3.421875 -6.546875 Z M 3.421875 -6.546875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-11\" overflow=\"visible\">\n",
       "<path d=\"M 2.734375 -4.78125 C 2.4375 -4.78125 2.15625 -4.703125 1.90625 -4.5625 C 1.640625 -4.421875 1.4375 -4.234375 1.28125 -3.96875 L 1.28125 -4.65625 L 0.5625 -4.65625 L 0.5625 0 L 1.28125 0 L 1.28125 -2.8125 C 1.3125 -3.234375 1.4375 -3.5625 1.6875 -3.8125 C 1.921875 -4.046875 2.203125 -4.171875 2.53125 -4.171875 C 3.34375 -4.171875 3.75 -3.71875 3.75 -2.8125 L 3.75 0 L 4.46875 0 L 4.46875 -2.859375 C 4.46875 -4.140625 3.890625 -4.78125 2.734375 -4.78125 Z M 2.734375 -4.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-12\" overflow=\"visible\">\n",
       "<path d=\"M 0.046875 -4.65625 L 1.796875 0 L 2.53125 0 L 4.296875 -4.65625 L 3.5 -4.65625 L 2.171875 -0.90625 L 0.84375 -4.65625 Z M 0.046875 -4.65625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-13\" overflow=\"visible\">\n",
       "<path d=\"M 2.0625 -6.421875 C 1.859375 -6.1875 1.609375 -5.96875 1.296875 -5.765625 C 0.984375 -5.59375 0.6875 -5.46875 0.390625 -5.375 L 0.390625 -4.640625 C 1 -4.828125 1.5 -5.109375 1.90625 -5.53125 L 1.90625 0 L 2.640625 0 L 2.640625 -6.421875 Z M 2.0625 -6.421875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-14\" overflow=\"visible\">\n",
       "<path d=\"M 3.390625 -6.546875 C 2.453125 -6.546875 1.703125 -6.21875 1.15625 -5.546875 C 0.625 -4.921875 0.375 -4.140625 0.375 -3.1875 C 0.375 -2.234375 0.640625 -1.453125 1.15625 -0.84375 C 1.71875 -0.203125 2.515625 0.125 3.53125 0.125 C 4.0625 0.125 4.546875 0.046875 5.015625 -0.09375 C 5.46875 -0.234375 5.84375 -0.421875 6.140625 -0.671875 L 6.140625 -3.296875 L 3.40625 -3.296875 L 3.40625 -2.65625 L 5.421875 -2.65625 L 5.421875 -1.03125 C 5.1875 -0.875 4.921875 -0.75 4.609375 -0.671875 C 4.28125 -0.578125 3.953125 -0.515625 3.59375 -0.515625 C 2.765625 -0.515625 2.140625 -0.78125 1.703125 -1.265625 C 1.3125 -1.734375 1.109375 -2.375 1.109375 -3.1875 C 1.109375 -4.015625 1.3125 -4.671875 1.734375 -5.15625 C 2.140625 -5.65625 2.6875 -5.890625 3.390625 -5.890625 C 3.953125 -5.890625 4.421875 -5.765625 4.765625 -5.484375 C 5.109375 -5.21875 5.3125 -4.84375 5.421875 -4.359375 L 6.15625 -4.359375 C 6.03125 -5.0625 5.765625 -5.59375 5.3125 -5.96875 C 4.828125 -6.359375 4.1875 -6.546875 3.390625 -6.546875 Z M 3.390625 -6.546875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-15\" overflow=\"visible\">\n",
       "<path d=\"M 0.5625 -6.546875 L 0.5625 0 L 1.234375 0 L 1.234375 -0.765625 C 1.546875 -0.171875 2.0625 0.125 2.78125 0.125 C 3.453125 0.125 3.984375 -0.125 4.375 -0.609375 C 4.71875 -1.0625 4.890625 -1.640625 4.890625 -2.34375 C 4.890625 -3.03125 4.71875 -3.578125 4.375 -4.03125 C 4 -4.53125 3.46875 -4.78125 2.8125 -4.78125 C 2.15625 -4.78125 1.640625 -4.453125 1.28125 -3.796875 L 1.28125 -6.546875 Z M 2.65625 -4.1875 C 3.171875 -4.1875 3.546875 -4 3.8125 -3.640625 C 4.046875 -3.328125 4.15625 -2.890625 4.15625 -2.34375 C 4.15625 -1.78125 4.046875 -1.34375 3.828125 -1.015625 C 3.578125 -0.65625 3.1875 -0.46875 2.6875 -0.46875 C 2.21875 -0.46875 1.859375 -0.640625 1.609375 -1.015625 C 1.375 -1.328125 1.25 -1.75 1.25 -2.28125 L 1.25 -2.359375 C 1.25 -2.90625 1.375 -3.34375 1.640625 -3.6875 C 1.90625 -4.015625 2.234375 -4.1875 2.65625 -4.1875 Z M 2.65625 -4.1875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-16\" overflow=\"visible\">\n",
       "<path d=\"M 2.53125 -6.421875 L 0 0 L 0.796875 0 L 1.46875 -1.796875 L 4.421875 -1.796875 L 5.109375 0 L 5.90625 0 L 3.390625 -6.421875 Z M 1.703125 -2.421875 L 2.9375 -5.640625 L 2.984375 -5.640625 L 4.1875 -2.421875 Z M 1.703125 -2.421875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-17\" overflow=\"visible\">\n",
       "<path d=\"M 2.46875 -4.78125 C 1.828125 -4.78125 1.3125 -4.546875 0.9375 -4.09375 C 0.5625 -3.65625 0.375 -3.109375 0.375 -2.4375 C 0.375 -1.75 0.5625 -1.203125 0.9375 -0.78125 C 1.3125 -0.328125 1.828125 -0.109375 2.46875 -0.109375 C 3.171875 -0.109375 3.703125 -0.421875 4.046875 -1.046875 L 4.046875 -0.21875 C 4.046875 0.796875 3.578125 1.328125 2.640625 1.328125 C 2.21875 1.328125 1.90625 1.234375 1.6875 1.09375 C 1.484375 0.9375 1.34375 0.71875 1.296875 0.390625 L 0.578125 0.390625 C 0.65625 0.90625 0.859375 1.28125 1.203125 1.53125 C 1.515625 1.78125 2 1.90625 2.640625 1.90625 C 4.046875 1.90625 4.765625 1.15625 4.765625 -0.328125 L 4.765625 -4.65625 L 4.046875 -4.65625 L 4.046875 -3.859375 C 3.703125 -4.46875 3.171875 -4.78125 2.46875 -4.78125 Z M 2.578125 -4.1875 C 3.046875 -4.1875 3.40625 -4.015625 3.671875 -3.6875 C 3.921875 -3.375 4.046875 -2.953125 4.046875 -2.4375 C 4.046875 -1.921875 3.921875 -1.515625 3.671875 -1.203125 C 3.40625 -0.875 3.046875 -0.71875 2.578125 -0.71875 C 2.0625 -0.71875 1.703125 -0.890625 1.453125 -1.234375 C 1.21875 -1.53125 1.109375 -1.9375 1.109375 -2.4375 C 1.109375 -2.953125 1.234375 -3.375 1.5 -3.6875 C 1.75 -4.015625 2.109375 -4.1875 2.578125 -4.1875 Z M 2.578125 -4.1875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-18\" overflow=\"visible\">\n",
       "<path d=\"M 0.640625 -6.421875 L 0.640625 0 L 2.875 0 C 3.890625 0 4.65625 -0.296875 5.203125 -0.875 C 5.71875 -1.4375 5.96875 -2.21875 5.96875 -3.21875 C 5.96875 -4.21875 5.734375 -5 5.25 -5.546875 C 4.734375 -6.140625 3.96875 -6.421875 2.953125 -6.421875 Z M 1.390625 -5.78125 L 2.828125 -5.78125 C 3.6875 -5.78125 4.296875 -5.578125 4.6875 -5.125 C 5.0625 -4.71875 5.25 -4.078125 5.25 -3.21875 C 5.25 -2.375 5.046875 -1.734375 4.640625 -1.3125 C 4.21875 -0.859375 3.59375 -0.640625 2.75 -0.640625 L 1.390625 -0.640625 Z M 1.390625 -5.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-19\" overflow=\"visible\">\n",
       "<path d=\"M 2.46875 -4.78125 C 1.796875 -4.78125 1.265625 -4.53125 0.875 -4.046875 C 0.484375 -3.59375 0.28125 -3.015625 0.28125 -2.328125 C 0.28125 -1.578125 0.484375 -0.984375 0.90625 -0.53125 C 1.296875 -0.09375 1.84375 0.125 2.5 0.125 C 3.109375 0.125 3.59375 -0.046875 3.984375 -0.390625 C 4.296875 -0.6875 4.515625 -1.046875 4.609375 -1.5 L 3.890625 -1.5 C 3.78125 -1.171875 3.625 -0.921875 3.421875 -0.765625 C 3.1875 -0.578125 2.875 -0.484375 2.5 -0.484375 C 2.0625 -0.484375 1.703125 -0.625 1.453125 -0.90625 C 1.203125 -1.203125 1.078125 -1.609375 1.03125 -2.125 L 4.703125 -2.125 C 4.6875 -2.9375 4.515625 -3.578125 4.171875 -4.015625 C 3.78125 -4.53125 3.21875 -4.78125 2.46875 -4.78125 Z M 2.5 -4.171875 C 3.375 -4.171875 3.875 -3.6875 3.953125 -2.6875 L 1.046875 -2.6875 C 1.09375 -3.140625 1.234375 -3.5 1.5 -3.765625 C 1.75 -4.046875 2.078125 -4.171875 2.5 -4.171875 Z M 2.5 -4.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-20\" overflow=\"visible\">\n",
       "<path d=\"M 2.203125 -4.78125 C 1.6875 -4.78125 1.25 -4.65625 0.921875 -4.421875 C 0.578125 -4.1875 0.40625 -3.890625 0.40625 -3.484375 C 0.40625 -3.09375 0.578125 -2.796875 0.9375 -2.578125 C 1.15625 -2.4375 1.53125 -2.296875 2.09375 -2.171875 C 2.59375 -2.046875 2.921875 -1.9375 3.078125 -1.875 C 3.359375 -1.734375 3.515625 -1.53125 3.515625 -1.265625 C 3.515625 -0.75 3.09375 -0.484375 2.265625 -0.484375 C 1.875 -0.484375 1.578125 -0.546875 1.375 -0.6875 C 1.171875 -0.859375 1.046875 -1.109375 0.984375 -1.484375 L 0.28125 -1.484375 C 0.390625 -0.40625 1.0625 0.125 2.28125 0.125 C 3.59375 0.125 4.25 -0.359375 4.25 -1.3125 C 4.25 -1.71875 4.0625 -2.03125 3.703125 -2.265625 C 3.453125 -2.421875 3.0625 -2.5625 2.5 -2.703125 C 2.03125 -2.828125 1.703125 -2.921875 1.53125 -3 C 1.265625 -3.140625 1.140625 -3.3125 1.140625 -3.515625 C 1.140625 -3.71875 1.234375 -3.875 1.4375 -4 C 1.625 -4.125 1.875 -4.171875 2.203125 -4.171875 C 2.5625 -4.171875 2.859375 -4.109375 3.046875 -3.953125 C 3.234375 -3.828125 3.34375 -3.625 3.40625 -3.328125 L 4.109375 -3.328125 C 3.984375 -4.296875 3.359375 -4.78125 2.203125 -4.78125 Z M 2.203125 -4.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-0\" overflow=\"visible\">\n",
       "<path d=\"M 3.6875 -6 L 0.921875 -6 L 0.921875 -0.421875 L 3.6875 -0.421875 Z M 4.140625 -6.40625 L 4.140625 -0.015625 L 0.453125 -0.015625 L 0.453125 -6.40625 Z M 4.140625 -6.40625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-1\" overflow=\"visible\">\n",
       "<path d=\"M 0.390625 -4.125 L 1.15625 -4.125 C 1.15625 -4.3125 1.175781 -4.5 1.21875 -4.6875 C 1.257812 -4.882812 1.332031 -5.054688 1.4375 -5.203125 C 1.539062 -5.347656 1.671875 -5.46875 1.828125 -5.5625 C 1.984375 -5.65625 2.175781 -5.703125 2.40625 -5.703125 C 2.570312 -5.703125 2.726562 -5.675781 2.875 -5.625 C 3.03125 -5.570312 3.160156 -5.492188 3.265625 -5.390625 C 3.378906 -5.285156 3.46875 -5.160156 3.53125 -5.015625 C 3.601562 -4.878906 3.640625 -4.722656 3.640625 -4.546875 C 3.640625 -4.328125 3.601562 -4.132812 3.53125 -3.96875 C 3.457031 -3.800781 3.351562 -3.644531 3.21875 -3.5 C 3.09375 -3.351562 2.925781 -3.207031 2.71875 -3.0625 C 2.507812 -2.925781 2.269531 -2.773438 2 -2.609375 C 1.78125 -2.472656 1.566406 -2.332031 1.359375 -2.1875 C 1.160156 -2.039062 0.976562 -1.867188 0.8125 -1.671875 C 0.65625 -1.472656 0.523438 -1.238281 0.421875 -0.96875 C 0.316406 -0.695312 0.25 -0.375 0.21875 0 L 4.390625 0 L 4.390625 -0.671875 L 1.109375 -0.671875 C 1.140625 -0.867188 1.210938 -1.046875 1.328125 -1.203125 C 1.453125 -1.359375 1.59375 -1.5 1.75 -1.625 C 1.914062 -1.757812 2.097656 -1.882812 2.296875 -2 C 2.492188 -2.113281 2.691406 -2.234375 2.890625 -2.359375 C 3.085938 -2.484375 3.28125 -2.613281 3.46875 -2.75 C 3.65625 -2.894531 3.820312 -3.050781 3.96875 -3.21875 C 4.113281 -3.382812 4.226562 -3.578125 4.3125 -3.796875 C 4.40625 -4.015625 4.453125 -4.265625 4.453125 -4.546875 C 4.453125 -4.847656 4.394531 -5.109375 4.28125 -5.328125 C 4.175781 -5.554688 4.035156 -5.75 3.859375 -5.90625 C 3.679688 -6.0625 3.46875 -6.175781 3.21875 -6.25 C 2.976562 -6.332031 2.722656 -6.375 2.453125 -6.375 C 2.109375 -6.375 1.804688 -6.316406 1.546875 -6.203125 C 1.285156 -6.085938 1.066406 -5.929688 0.890625 -5.734375 C 0.710938 -5.535156 0.582031 -5.296875 0.5 -5.015625 C 0.414062 -4.742188 0.378906 -4.445312 0.390625 -4.125 Z M 0.390625 -4.125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-2\" overflow=\"visible\">\n",
       "<path d=\"M 1.828125 -2.453125 L 0.078125 0 L 1.015625 0 L 2.296875 -1.921875 L 3.59375 0 L 4.578125 0 L 2.78125 -2.515625 L 4.390625 -4.65625 L 3.46875 -4.65625 L 2.296875 -3.03125 L 1.1875 -4.65625 L 0.203125 -4.65625 Z M 1.828125 -2.453125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-3\" overflow=\"visible\">\n",
       "<path d=\"M 4.234375 -5.59375 L 4.234375 -6.265625 L 1.078125 -6.265625 L 0.484375 -2.96875 L 1.140625 -2.921875 C 1.296875 -3.097656 1.46875 -3.242188 1.65625 -3.359375 C 1.851562 -3.472656 2.078125 -3.53125 2.328125 -3.53125 C 2.546875 -3.53125 2.742188 -3.492188 2.921875 -3.421875 C 3.097656 -3.347656 3.25 -3.242188 3.375 -3.109375 C 3.5 -2.984375 3.59375 -2.832031 3.65625 -2.65625 C 3.726562 -2.476562 3.765625 -2.285156 3.765625 -2.078125 C 3.765625 -1.828125 3.726562 -1.609375 3.65625 -1.421875 C 3.59375 -1.234375 3.5 -1.070312 3.375 -0.9375 C 3.25 -0.8125 3.101562 -0.71875 2.9375 -0.65625 C 2.769531 -0.59375 2.597656 -0.5625 2.421875 -0.5625 C 2.234375 -0.5625 2.054688 -0.585938 1.890625 -0.640625 C 1.734375 -0.703125 1.597656 -0.785156 1.484375 -0.890625 C 1.367188 -0.992188 1.273438 -1.113281 1.203125 -1.25 C 1.128906 -1.394531 1.085938 -1.550781 1.078125 -1.71875 L 0.3125 -1.71875 C 0.320312 -1.425781 0.378906 -1.164062 0.484375 -0.9375 C 0.585938 -0.71875 0.734375 -0.53125 0.921875 -0.375 C 1.109375 -0.21875 1.320312 -0.0976562 1.5625 -0.015625 C 1.8125 0.0546875 2.070312 0.09375 2.34375 0.09375 C 2.71875 0.09375 3.046875 0.0351562 3.328125 -0.078125 C 3.609375 -0.191406 3.835938 -0.347656 4.015625 -0.546875 C 4.203125 -0.742188 4.34375 -0.96875 4.4375 -1.21875 C 4.53125 -1.476562 4.578125 -1.742188 4.578125 -2.015625 C 4.578125 -2.378906 4.519531 -2.695312 4.40625 -2.96875 C 4.300781 -3.25 4.15625 -3.476562 3.96875 -3.65625 C 3.789062 -3.84375 3.570312 -3.976562 3.3125 -4.0625 C 3.0625 -4.15625 2.796875 -4.203125 2.515625 -4.203125 C 2.304688 -4.203125 2.085938 -4.164062 1.859375 -4.09375 C 1.640625 -4.019531 1.460938 -3.90625 1.328125 -3.75 L 1.3125 -3.765625 L 1.65625 -5.59375 Z M 4.234375 -5.59375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<path d=\"M 635.5 377.695312 L 653.644531 377.695312 L 653.644531 381.929688 L 635.5 381.929688 Z M 635.5 377.695312 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 656.5 381.28125 L 674.640625 381.28125 L 674.640625 385.515625 L 656.5 385.515625 Z M 656.5 381.28125 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 656.5 374.113281 L 674.640625 374.113281 L 674.640625 378.347656 L 656.5 378.347656 Z M 656.5 374.113281 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.570312 381.28125 L 665.570312 381.046875 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.570312 380.246094 L 665.570312 381.046875 M 665.269531 381.046875 L 665.570312 380.246094 L 665.871094 381.046875 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 677.496094 381.28125 L 695.640625 381.28125 L 695.640625 385.515625 L 677.496094 385.515625 Z M 677.496094 381.28125 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 677.496094 374.113281 L 695.640625 374.113281 L 695.640625 378.347656 L 677.496094 378.347656 Z M 677.496094 374.113281 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.570312 381.28125 L 686.570312 381.046875 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.570312 380.246094 L 686.570312 381.046875 M 686.269531 381.046875 L 686.570312 380.246094 L 686.871094 381.046875 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 698.355469 381.058594 L 716.5 381.058594 L 716.5 385.289062 L 698.355469 385.289062 Z M 698.355469 381.058594 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 698.355469 373.890625 L 716.5 373.890625 L 716.5 378.121094 L 698.355469 378.121094 Z M 698.355469 373.890625 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.429688 381.058594 L 707.429688 380.824219 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.429688 380.023438 L 707.429688 380.824219 M 707.128906 380.824219 L 707.429688 380.023438 L 707.726562 380.824219 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 388.449219 L 685.058594 388.449219 L 685.058594 392.683594 L 666.914062 392.683594 Z M 666.914062 388.449219 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 366.945312 L 685.058594 366.945312 L 685.058594 371.179688 L 666.914062 371.179688 Z M 666.914062 366.945312 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 679.113281 388.449219 L 681.207031 387.027344 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 681.871094 386.578125 L 681.039062 386.78125 L 681.375 387.277344 Z M 681.871094 386.578125 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 672.910156 388.449219 L 670.871094 387.042969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 670.210938 386.589844 L 670.699219 387.292969 L 671.039062 386.796875 Z M 670.210938 386.589844 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 668.644531 374.113281 L 670.6875 372.710938 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 671.34375 372.257812 L 670.515625 372.460938 L 670.855469 372.957031 Z M 671.34375 372.257812 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 683.445312 374.113281 L 681.347656 372.691406 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 680.683594 372.246094 L 681.179688 372.941406 L 681.515625 372.445312 Z M 680.683594 372.246094 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 685.058594 390.859375 C 685.0625 390.859375 685.066406 390.859375 685.066406 390.859375 L 702.429688 390.859375 C 704.445312 390.859375 706.179688 389.667969 706.972656 387.949219 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.105469 387.164062 L 706.675781 387.902344 L 707.265625 388 Z M 707.105469 387.164062 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 390.804688 C 666.90625 390.808594 666.898438 390.808594 666.890625 390.808594 L 649.570312 390.808594 C 646.808594 390.808594 644.570312 388.570312 644.570312 385.808594 L 644.570312 384.628906 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 644.570312 383.832031 L 644.273438 384.628906 L 644.871094 384.628906 Z M 644.570312 383.832031 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 644.570312 377.695312 L 644.570312 374.171875 C 644.570312 371.410156 646.808594 369.171875 649.570312 369.171875 L 664.214844 369.171875 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.015625 369.171875 L 664.214844 368.871094 L 664.214844 369.472656 Z M 665.015625 369.171875 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.394531 373.890625 C 707.109375 371.398438 704.996094 369.464844 702.429688 369.464844 L 687.757812 369.464844 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.957031 369.460938 L 687.757812 369.765625 L 687.757812 369.164062 Z M 686.957031 369.460938 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 635.5 475.9375 L 716.5 475.9375 L 716.5 490.9375 L 635.5 490.9375 Z M 635.5 475.9375 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"40.8155\" xlink:href=\"#glyph0-1\" y=\"289.93634\"/>\n",
       "  <use x=\"46.2155\" xlink:href=\"#glyph0-2\" y=\"289.93634\"/>\n",
       "  <use x=\"50.7965\" xlink:href=\"#glyph0-1\" y=\"289.93634\"/>\n",
       "  <use x=\"56.1965\" xlink:href=\"#glyph0-3\" y=\"289.93634\"/>\n",
       "  <use x=\"59.1935\" xlink:href=\"#glyph0-4\" y=\"289.93634\"/>\n",
       "  <use x=\"67.1315\" xlink:href=\"#glyph0-5\" y=\"289.93634\"/>\n",
       "  <use x=\"72.1625\" xlink:href=\"#glyph0-2\" y=\"289.93634\"/>\n",
       "  <use x=\"76.7435\" xlink:href=\"#glyph0-6\" y=\"289.93634\"/>\n",
       "  <use x=\"82.5215\" xlink:href=\"#glyph0-7\" y=\"289.93634\"/>\n",
       "  <use x=\"87.7955\" xlink:href=\"#glyph0-7\" y=\"289.93634\"/>\n",
       "  <use x=\"93.0695\" xlink:href=\"#glyph0-8\" y=\"289.93634\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 500.5 L 716.5 500.5 L 716.5 515.5 L 635.5 515.5 Z M 635.5 500.5 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"48.6905\" xlink:href=\"#glyph0-9\" y=\"314.5\"/>\n",
       "  <use x=\"53.6135\" xlink:href=\"#glyph0-2\" y=\"314.5\"/>\n",
       "  <use x=\"58.1945\" xlink:href=\"#glyph0-9\" y=\"314.5\"/>\n",
       "  <use x=\"63.1175\" xlink:href=\"#glyph0-3\" y=\"314.5\"/>\n",
       "  <use x=\"66.1145\" xlink:href=\"#glyph0-10\" y=\"314.5\"/>\n",
       "  <use x=\"72.6665\" xlink:href=\"#glyph0-7\" y=\"314.5\"/>\n",
       "  <use x=\"77.9405\" xlink:href=\"#glyph0-11\" y=\"314.5\"/>\n",
       "  <use x=\"82.9715\" xlink:href=\"#glyph0-12\" y=\"314.5\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 451.371094 L 716.5 451.371094 L 716.5 466.371094 L 635.5 466.371094 Z M 635.5 451.371094 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.0045\" xlink:href=\"#glyph0-13\" y=\"265.37268\"/>\n",
       "  <use x=\"53.6135\" xlink:href=\"#glyph0-2\" y=\"265.37268\"/>\n",
       "  <use x=\"58.1945\" xlink:href=\"#glyph0-13\" y=\"265.37268\"/>\n",
       "  <use x=\"61.8035\" xlink:href=\"#glyph0-3\" y=\"265.37268\"/>\n",
       "  <use x=\"64.8005\" xlink:href=\"#glyph0-10\" y=\"265.37268\"/>\n",
       "  <use x=\"71.3525\" xlink:href=\"#glyph0-7\" y=\"265.37268\"/>\n",
       "  <use x=\"76.6265\" xlink:href=\"#glyph0-11\" y=\"265.37268\"/>\n",
       "  <use x=\"81.6575\" xlink:href=\"#glyph0-12\" y=\"265.37268\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 426.808594 L 716.5 426.808594 L 716.5 441.808594 L 635.5 441.808594 Z M 635.5 426.808594 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"48.2135\" xlink:href=\"#glyph0-1\" y=\"240.80902\"/>\n",
       "  <use x=\"53.6135\" xlink:href=\"#glyph0-2\" y=\"240.80902\"/>\n",
       "  <use x=\"58.1945\" xlink:href=\"#glyph0-1\" y=\"240.80902\"/>\n",
       "  <use x=\"63.5945\" xlink:href=\"#glyph0-3\" y=\"240.80902\"/>\n",
       "  <use x=\"66.5915\" xlink:href=\"#glyph0-10\" y=\"240.80902\"/>\n",
       "  <use x=\"73.1435\" xlink:href=\"#glyph0-7\" y=\"240.80902\"/>\n",
       "  <use x=\"78.4175\" xlink:href=\"#glyph0-11\" y=\"240.80902\"/>\n",
       "  <use x=\"83.4485\" xlink:href=\"#glyph0-12\" y=\"240.80902\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 402.246094 L 716.5 402.246094 L 716.5 417.246094 L 635.5 417.246094 Z M 635.5 402.246094 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"40.8155\" xlink:href=\"#glyph0-1\" y=\"216.2454\"/>\n",
       "  <use x=\"46.2155\" xlink:href=\"#glyph0-2\" y=\"216.2454\"/>\n",
       "  <use x=\"50.7965\" xlink:href=\"#glyph0-1\" y=\"216.2454\"/>\n",
       "  <use x=\"56.1965\" xlink:href=\"#glyph0-3\" y=\"216.2454\"/>\n",
       "  <use x=\"59.1935\" xlink:href=\"#glyph0-4\" y=\"216.2454\"/>\n",
       "  <use x=\"67.1315\" xlink:href=\"#glyph0-5\" y=\"216.2454\"/>\n",
       "  <use x=\"72.1625\" xlink:href=\"#glyph0-2\" y=\"216.2454\"/>\n",
       "  <use x=\"76.7435\" xlink:href=\"#glyph0-6\" y=\"216.2454\"/>\n",
       "  <use x=\"82.5215\" xlink:href=\"#glyph0-7\" y=\"216.2454\"/>\n",
       "  <use x=\"87.7955\" xlink:href=\"#glyph0-7\" y=\"216.2454\"/>\n",
       "  <use x=\"93.0695\" xlink:href=\"#glyph0-8\" y=\"216.2454\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 342.382812 L 716.5 342.382812 L 716.5 357.382812 L 635.5 357.382812 Z M 635.5 342.382812 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"40.8155\" xlink:href=\"#glyph0-1\" y=\"156.3817\"/>\n",
       "  <use x=\"46.2155\" xlink:href=\"#glyph0-2\" y=\"156.3817\"/>\n",
       "  <use x=\"50.7965\" xlink:href=\"#glyph0-1\" y=\"156.3817\"/>\n",
       "  <use x=\"56.1965\" xlink:href=\"#glyph0-3\" y=\"156.3817\"/>\n",
       "  <use x=\"59.1935\" xlink:href=\"#glyph0-4\" y=\"156.3817\"/>\n",
       "  <use x=\"67.1315\" xlink:href=\"#glyph0-5\" y=\"156.3817\"/>\n",
       "  <use x=\"72.1625\" xlink:href=\"#glyph0-2\" y=\"156.3817\"/>\n",
       "  <use x=\"76.7435\" xlink:href=\"#glyph0-6\" y=\"156.3817\"/>\n",
       "  <use x=\"82.5215\" xlink:href=\"#glyph0-7\" y=\"156.3817\"/>\n",
       "  <use x=\"87.7955\" xlink:href=\"#glyph0-7\" y=\"156.3817\"/>\n",
       "  <use x=\"93.0695\" xlink:href=\"#glyph0-8\" y=\"156.3817\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 317.832031 L 653.644531 317.832031 L 653.644531 322.066406 L 635.5 322.066406 Z M 635.5 317.832031 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 656.5 321.417969 L 674.640625 321.417969 L 674.640625 325.652344 L 656.5 325.652344 Z M 656.5 321.417969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 656.5 314.25 L 674.640625 314.25 L 674.640625 318.484375 L 656.5 318.484375 Z M 656.5 314.25 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.570312 321.417969 L 665.570312 321.183594 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.570312 320.382812 L 665.570312 321.183594 M 665.269531 321.183594 L 665.570312 320.382812 L 665.871094 321.183594 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 677.496094 321.417969 L 695.640625 321.417969 L 695.640625 325.652344 L 677.496094 325.652344 Z M 677.496094 321.417969 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 677.496094 314.25 L 695.640625 314.25 L 695.640625 318.484375 L 677.496094 318.484375 Z M 677.496094 314.25 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.570312 321.417969 L 686.570312 321.183594 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.570312 320.382812 L 686.570312 321.183594 M 686.269531 321.183594 L 686.570312 320.382812 L 686.871094 321.183594 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 698.355469 321.191406 L 716.5 321.191406 L 716.5 325.425781 L 698.355469 325.425781 Z M 698.355469 321.191406 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 698.355469 314.027344 L 716.5 314.027344 L 716.5 318.257812 L 698.355469 318.257812 Z M 698.355469 314.027344 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.429688 321.191406 L 707.429688 320.957031 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.429688 320.160156 L 707.429688 320.957031 M 707.128906 320.957031 L 707.429688 320.160156 L 707.726562 320.957031 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 328.585938 L 685.058594 328.585938 L 685.058594 332.816406 L 666.914062 332.816406 Z M 666.914062 328.585938 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 307.082031 L 685.058594 307.082031 L 685.058594 311.316406 L 666.914062 311.316406 Z M 666.914062 307.082031 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 679.113281 328.585938 L 681.207031 327.164062 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 681.871094 326.714844 L 681.039062 326.914062 L 681.375 327.414062 Z M 681.871094 326.714844 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 672.910156 328.585938 L 670.871094 327.179688 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 670.210938 326.726562 L 670.699219 327.429688 L 671.039062 326.933594 Z M 670.210938 326.726562 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 668.644531 314.25 L 670.6875 312.847656 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 671.34375 312.390625 L 670.515625 312.597656 L 670.855469 313.09375 Z M 671.34375 312.390625 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 683.445312 314.25 L 681.347656 312.828125 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 680.683594 312.378906 L 681.179688 313.078125 L 681.515625 312.582031 Z M 680.683594 312.378906 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 685.058594 330.996094 C 685.0625 330.996094 685.066406 330.996094 685.066406 330.996094 L 702.429688 330.996094 C 704.445312 330.996094 706.179688 329.804688 706.972656 328.085938 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.105469 327.300781 L 706.675781 328.035156 L 707.265625 328.136719 Z M 707.105469 327.300781 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 330.941406 C 666.90625 330.945312 666.898438 330.945312 666.890625 330.945312 L 649.570312 330.945312 C 646.808594 330.945312 644.570312 328.707031 644.570312 325.945312 L 644.570312 324.765625 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 644.570312 323.964844 L 644.273438 324.765625 L 644.871094 324.765625 Z M 644.570312 323.964844 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 644.570312 317.832031 L 644.570312 314.308594 C 644.570312 311.546875 646.808594 309.308594 649.570312 309.308594 L 664.214844 309.308594 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.015625 309.308594 L 664.214844 309.007812 L 664.214844 309.609375 Z M 665.015625 309.308594 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.394531 314.027344 C 707.109375 311.535156 704.996094 309.601562 702.429688 309.601562 L 687.757812 309.601562 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.957031 309.597656 L 687.757812 309.902344 L 687.757812 309.300781 Z M 686.957031 309.597656 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 635.5 257.96875 L 653.644531 257.96875 L 653.644531 262.203125 L 635.5 262.203125 Z M 635.5 257.96875 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 656.5 261.554688 L 674.640625 261.554688 L 674.640625 265.785156 L 656.5 265.785156 Z M 656.5 261.554688 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 656.5 254.386719 L 674.640625 254.386719 L 674.640625 258.617188 L 656.5 258.617188 Z M 656.5 254.386719 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.570312 261.554688 L 665.570312 261.320312 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.570312 260.519531 L 665.570312 261.320312 M 665.269531 261.320312 L 665.570312 260.519531 L 665.871094 261.320312 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 677.496094 261.554688 L 695.640625 261.554688 L 695.640625 265.785156 L 677.496094 265.785156 Z M 677.496094 261.554688 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 677.496094 254.386719 L 695.640625 254.386719 L 695.640625 258.617188 L 677.496094 258.617188 Z M 677.496094 254.386719 \" style=\"fill-rule:nonzero;fill:rgb(39.99939%,74.902344%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.570312 261.554688 L 686.570312 261.320312 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.570312 260.519531 L 686.570312 261.320312 M 686.269531 261.320312 L 686.570312 260.519531 L 686.871094 261.320312 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 698.355469 261.328125 L 716.5 261.328125 L 716.5 265.5625 L 698.355469 265.5625 Z M 698.355469 261.328125 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 698.355469 254.160156 L 716.5 254.160156 L 716.5 258.394531 L 698.355469 258.394531 Z M 698.355469 254.160156 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.429688 261.328125 L 707.429688 261.09375 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.429688 260.296875 L 707.429688 261.09375 M 707.128906 261.09375 L 707.429688 260.296875 L 707.726562 261.09375 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 268.722656 L 685.058594 268.722656 L 685.058594 272.953125 L 666.914062 272.953125 Z M 666.914062 268.722656 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 247.21875 L 685.058594 247.21875 L 685.058594 251.453125 L 666.914062 251.453125 Z M 666.914062 247.21875 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 679.113281 268.722656 L 681.207031 267.300781 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 681.871094 266.851562 L 681.039062 267.050781 L 681.375 267.550781 Z M 681.871094 266.851562 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 672.910156 268.722656 L 670.871094 267.316406 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 670.210938 266.863281 L 670.699219 267.566406 L 671.039062 267.070312 Z M 670.210938 266.863281 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 668.644531 254.386719 L 670.6875 252.980469 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 671.34375 252.527344 L 670.515625 252.734375 L 670.855469 253.230469 Z M 671.34375 252.527344 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 683.445312 254.386719 L 681.347656 252.964844 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 680.683594 252.515625 L 681.179688 253.214844 L 681.515625 252.71875 Z M 680.683594 252.515625 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 685.058594 271.132812 C 685.0625 271.132812 685.066406 271.132812 685.066406 271.132812 L 702.429688 271.132812 C 704.445312 271.132812 706.179688 269.941406 706.972656 268.222656 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.105469 267.433594 L 706.675781 268.171875 L 707.265625 268.273438 Z M 707.105469 267.433594 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 666.914062 271.078125 C 666.90625 271.078125 666.898438 271.082031 666.890625 271.082031 L 649.570312 271.082031 C 646.808594 271.082031 644.570312 268.84375 644.570312 266.082031 L 644.570312 264.902344 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 644.570312 264.101562 L 644.273438 264.902344 L 644.871094 264.902344 Z M 644.570312 264.101562 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 644.570312 257.96875 L 644.570312 254.445312 C 644.570312 251.683594 646.808594 249.445312 649.570312 249.445312 L 664.214844 249.445312 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 665.015625 249.445312 L 664.214844 249.144531 L 664.214844 249.746094 Z M 665.015625 249.445312 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 707.394531 254.160156 C 707.109375 251.671875 704.996094 249.738281 702.429688 249.738281 L 687.757812 249.738281 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 686.957031 249.734375 L 687.757812 250.035156 L 687.757812 249.4375 Z M 686.957031 249.734375 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 635.5 282.519531 L 716.5 282.519531 L 716.5 297.519531 L 635.5 297.519531 Z M 635.5 282.519531 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"40.8155\" xlink:href=\"#glyph0-1\" y=\"96.5181\"/>\n",
       "  <use x=\"46.2155\" xlink:href=\"#glyph0-2\" y=\"96.5181\"/>\n",
       "  <use x=\"50.7965\" xlink:href=\"#glyph0-1\" y=\"96.5181\"/>\n",
       "  <use x=\"56.1965\" xlink:href=\"#glyph0-3\" y=\"96.5181\"/>\n",
       "  <use x=\"59.1935\" xlink:href=\"#glyph0-4\" y=\"96.5181\"/>\n",
       "  <use x=\"67.1315\" xlink:href=\"#glyph0-5\" y=\"96.5181\"/>\n",
       "  <use x=\"72.1625\" xlink:href=\"#glyph0-2\" y=\"96.5181\"/>\n",
       "  <use x=\"76.7435\" xlink:href=\"#glyph0-6\" y=\"96.5181\"/>\n",
       "  <use x=\"82.5215\" xlink:href=\"#glyph0-7\" y=\"96.5181\"/>\n",
       "  <use x=\"87.7955\" xlink:href=\"#glyph0-7\" y=\"96.5181\"/>\n",
       "  <use x=\"93.0695\" xlink:href=\"#glyph0-8\" y=\"96.5181\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 222.65625 L 716.5 222.65625 L 716.5 237.65625 L 635.5 237.65625 Z M 635.5 222.65625 \" style=\"fill-rule:nonzero;fill:rgb(69.804382%,85.098267%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"36.2255\" xlink:href=\"#glyph0-14\" y=\"36.6544\"/>\n",
       "  <use x=\"42.9575\" xlink:href=\"#glyph0-8\" y=\"36.6544\"/>\n",
       "  <use x=\"45.0725\" xlink:href=\"#glyph0-7\" y=\"36.6544\"/>\n",
       "  <use x=\"50.3465\" xlink:href=\"#glyph0-15\" y=\"36.6544\"/>\n",
       "  <use x=\"55.6205\" xlink:href=\"#glyph0-5\" y=\"36.6544\"/>\n",
       "  <use x=\"60.6515\" xlink:href=\"#glyph0-8\" y=\"36.6544\"/>\n",
       "  <use x=\"62.7665\" xlink:href=\"#glyph0-3\" y=\"36.6544\"/>\n",
       "  <use x=\"65.7635\" xlink:href=\"#glyph0-16\" y=\"36.6544\"/>\n",
       "  <use x=\"71.6765\" xlink:href=\"#glyph0-12\" y=\"36.6544\"/>\n",
       "  <use x=\"76.0145\" xlink:href=\"#glyph0-17\" y=\"36.6544\"/>\n",
       "  <use x=\"81.3335\" xlink:href=\"#glyph0-6\" y=\"36.6544\"/>\n",
       "  <use x=\"87.1115\" xlink:href=\"#glyph0-7\" y=\"36.6544\"/>\n",
       "  <use x=\"92.3855\" xlink:href=\"#glyph0-7\" y=\"36.6544\"/>\n",
       "  <use x=\"97.6595\" xlink:href=\"#glyph0-8\" y=\"36.6544\"/>\n",
       "</g>\n",
       "<path d=\"M 635.5 198.089844 L 716.5 198.089844 L 716.5 213.089844 L 635.5 213.089844 Z M 635.5 198.089844 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"55.04\" xlink:href=\"#glyph0-18\" y=\"12.0908\"/>\n",
       "  <use x=\"61.394\" xlink:href=\"#glyph0-19\" y=\"12.0908\"/>\n",
       "  <use x=\"66.389\" xlink:href=\"#glyph0-11\" y=\"12.0908\"/>\n",
       "  <use x=\"71.42\" xlink:href=\"#glyph0-20\" y=\"12.0908\"/>\n",
       "  <use x=\"75.965\" xlink:href=\"#glyph0-19\" y=\"12.0908\"/>\n",
       "</g>\n",
       "<path d=\"M 676 500.5 L 676 496.835938 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 492.835938 L 674.5 496.835938 L 677.5 496.835938 Z M 676 492.835938 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 475.9375 L 676 472.273438 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 468.273438 L 674.5 472.273438 L 677.5 472.273438 Z M 676 468.273438 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 451.371094 L 676 447.710938 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 443.710938 L 674.5 447.710938 L 677.5 447.710938 Z M 676 443.710938 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 426.808594 L 676 423.144531 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 419.144531 L 674.5 423.144531 L 677.5 423.144531 Z M 676 419.144531 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 402.246094 L 676 400.15625 L 675.992188 397.914062 L 675.984375 400.15625 L 675.984375 398.582031 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.984375 394.582031 L 674.484375 398.582031 L 677.484375 398.582031 Z M 675.984375 394.582031 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 222.65625 L 676 218.992188 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 214.992188 L 674.5 218.992188 L 677.5 218.992188 Z M 676 214.992188 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"5.667\" xlink:href=\"#glyph1-1\" y=\"186.187501\"/>\n",
       "  <use x=\"10.671\" xlink:href=\"#glyph1-2\" y=\"186.187501\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"5.667\" xlink:href=\"#glyph1-3\" y=\"126.323901\"/>\n",
       "  <use x=\"10.671\" xlink:href=\"#glyph1-2\" y=\"126.323901\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"5.667\" xlink:href=\"#glyph1-1\" y=\"66.460201\"/>\n",
       "  <use x=\"10.671\" xlink:href=\"#glyph1-2\" y=\"66.460201\"/>\n",
       "</g>\n",
       "<path d=\"M 675.984375 366.945312 L 675.984375 363.28125 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.996094 359.28125 L 674.484375 363.277344 L 677.484375 363.285156 Z M 675.996094 359.28125 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 342.382812 L 676 340.292969 L 675.992188 338.050781 L 675.984375 340.292969 L 675.984375 338.71875 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.984375 334.71875 L 674.484375 338.71875 L 677.484375 338.71875 Z M 675.984375 334.71875 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.984375 307.082031 L 675.984375 303.417969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.996094 299.417969 L 674.484375 303.414062 L 677.484375 303.421875 Z M 675.996094 299.417969 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 676 282.519531 L 676 280.429688 L 675.992188 278.1875 L 675.984375 280.429688 L 675.984375 278.855469 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.984375 274.855469 L 674.484375 278.855469 L 677.484375 278.855469 Z M 675.984375 274.855469 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.984375 247.21875 L 675.984375 243.554688 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "<path d=\"M 675.996094 239.554688 L 674.484375 243.550781 L 677.484375 243.558594 Z M 675.996094 239.554688 \" style=\"fill-rule:nonzero;fill:rgb(0%,0%,0%);fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-608,-197)\"/>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SVG(filename=\"../img/inception-full.svg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now implement GoogLeNet piece by piece. \n",
    "The first component uses a 64-channel 7×7 convolutional layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "2"
    },
    "colab": {},
    "colab_type": "code",
    "id": "db8Crmm1aYHy"
   },
   "outputs": [],
   "source": [
    "b1 = nn.Sequential(nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3),\n",
    "                   nn.ReLU(),\n",
    "                   nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Je6F1MK4aYH3"
   },
   "source": [
    "The second component uses two convolutional layers: \n",
    "first, a 64-channel $1\\times 1$ convolutional layer, \n",
    "then a $3\\times 3$ convolutional layer that triples the number of channels. This corresponds to the second path in the Inception block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "3"
    },
    "colab": {},
    "colab_type": "code",
    "id": "3aA7LlBAaYH4"
   },
   "outputs": [],
   "source": [
    "b2 = nn.Sequential(nn.Conv2d(64, 64, kernel_size=1),\n",
    "                   nn.ReLU(),\n",
    "                   nn.Conv2d(64, 192, kernel_size=3, padding=1),\n",
    "                   nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Oo_br7MdaYH-"
   },
   "source": [
    "The third component connects two complete Inception blocks in series. \n",
    "The number of output channels of the first Inception block is \n",
    "$64+128+32+32=256$, and the ratio to the output channels \n",
    "of the four paths is $64:128:32:32=2:4:1:1$. \n",
    "The second and third paths first reduce the number of input channels \n",
    "to $96/192=1/2$ and $16/192=1/12$, respectively, \n",
    "and then connect the second convolutional layer. \n",
    "The number of output channels of the second Inception block \n",
    "is increased to $128+192+96+64=480$, and the ratio to the number of output channels per path is $128:192:96:64 = 4:6:3:2$. \n",
    "The second and third paths first reduce the number of input channels \n",
    "to $128/256=1/2$ and $32/256=1/8$, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "4"
    },
    "colab": {},
    "colab_type": "code",
    "id": "4KnU5BDNaYIA"
   },
   "outputs": [],
   "source": [
    "b3 = nn.Sequential(Inception(192, 64, (96, 128), (16, 32), 32),\n",
    "                    Inception(256, 128, (128, 192), (32, 96), 64),\n",
    "                    nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FGUk2ZDEaYIE"
   },
   "source": [
    "The fourth block is more complicated. \n",
    "It connects five Inception blocks in series, \n",
    "and they have $192+208+48+64=512$, $160+224+64+64=512$, \n",
    "$128+256+64+64=512$, $112+288+64+64=528$, \n",
    "and $256+320+128+128=832$ output channels, respectively. \n",
    "The number of channels assigned to these paths is similar \n",
    "to that in the third module: \n",
    "the second path with the $3\\times 3$ convolutional layer \n",
    "outputs the largest number of channels, \n",
    "followed by the first path with only the $1\\times 1$ convolutional layer, \n",
    "the third path with the $5\\times 5$ convolutional layer, \n",
    "and the fourth path with the $3\\times 3$ maximum pooling layer. \n",
    "The second and third paths will first reduce \n",
    "the number of channels according the ratio. \n",
    "These ratios are slightly different in different Inception blocks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "5"
    },
    "colab": {},
    "colab_type": "code",
    "id": "pUG_m7Y9aYIG"
   },
   "outputs": [],
   "source": [
    "b4 = nn.Sequential(Inception(480, 192, (96, 208), (16, 48), 64),\n",
    "                   Inception(512, 160, (112, 224), (24, 64), 64),\n",
    "                   Inception(512, 128, (128, 256), (24, 64), 64),\n",
    "                   Inception(512, 112, (144, 288), (32, 64), 64),\n",
    "                   Inception(528, 256, (160, 320), (32, 128), 128),\n",
    "                   nn.MaxPool2d(kernel_size=3, stride=2, padding=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HRyamUaJaYIL"
   },
   "source": [
    "The fifth block has two Inception blocks with $256+320+128+128=832$ \n",
    "and $384+384+128+128=1024$ output channels. \n",
    "The number of channels assigned to each path\n",
    "is the same as that in the third and fourth modules, \n",
    "but differs in specific values. \n",
    "It should be noted that the fifth block is followed by the output layer. \n",
    "This block uses the global average pooling layer \n",
    "to change the height and width of each channel to 1, just as in NiN. \n",
    "Finally, we turn the output into a two-dimensional array \n",
    "followed by a fully-connected layer \n",
    "whose number of outputs is the number of label classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "6"
    },
    "colab": {},
    "colab_type": "code",
    "id": "INwnSJaTaYIM"
   },
   "outputs": [],
   "source": [
    "class Flatten(nn.Module):\n",
    "    def forward(self, input):\n",
    "        return input.view(input.size(0), -1)\n",
    "\n",
    "b5 = nn.Sequential(Inception(832, 256, (160, 320), (32, 128), 128),\n",
    "                   Inception(832, 384, (192, 384), (48, 128), 128),\n",
    "                   nn.AdaptiveMaxPool2d((1,1)),\n",
    "                   Flatten())\n",
    "\n",
    "net = nn.Sequential(b1, b2, b3, b4, b5, nn.Linear(1024, 10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Q7Ij51d-aYIT"
   },
   "source": [
    "The GoogLeNet model is computationally complex, \n",
    "so it is not as easy to modify the number of channels as in VGG. \n",
    "To have a reasonable training time on Fashion-MNIST,\n",
    "we reduce the input height and width from 224 to 96. \n",
    "This simplifies the computation. \n",
    "The changes in the shape of the output \n",
    "between the various modules is demonstrated below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "7"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 119
    },
    "colab_type": "code",
    "id": "dpXeK72caYIV",
    "outputId": "964084c4-28f5-43c7-d853-7462d0dfad05"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequential output shape:\t torch.Size([1, 64, 24, 24])\n",
      "Sequential output shape:\t torch.Size([1, 192, 12, 12])\n",
      "Sequential output shape:\t torch.Size([1, 480, 6, 6])\n",
      "Sequential output shape:\t torch.Size([1, 832, 3, 3])\n",
      "Sequential output shape:\t torch.Size([1, 1024])\n",
      "Linear output shape:\t torch.Size([1, 10])\n"
     ]
    }
   ],
   "source": [
    "X = torch.rand(size=(1, 1, 96, 96))\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape:\\t', X.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IGNGhC-taYIf"
   },
   "source": [
    "## Data Acquisition and Training\n",
    "\n",
    "As before, we train our model using the Fashion-MNIST dataset.\n",
    " We transform it to $96 \\times 96$ pixel resolution \n",
    " before invoking the training procedure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "8"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 119
    },
    "colab_type": "code",
    "id": "O1TOK4NyaYIh",
    "outputId": "dae0a7c9-2033-43c3-97c8-7f18193f42b8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training on cuda:0\n",
      "epoch 1, loss 0.0125, train acc 0.409, test acc 0.766, time 47.9 sec\n",
      "epoch 2, loss 0.0050, train acc 0.764, test acc 0.793, time 48.6 sec\n",
      "epoch 3, loss 0.0042, train acc 0.807, test acc 0.838, time 48.5 sec\n",
      "epoch 4, loss 0.0028, train acc 0.863, test acc 0.850, time 48.5 sec\n",
      "epoch 5, loss 0.0025, train acc 0.878, test acc 0.875, time 48.5 sec\n"
     ]
    }
   ],
   "source": [
    "lr, num_epochs, batch_size, device = 0.1, 5, 128, d2l.try_gpu()\n",
    "\n",
    "#Initialization of Weights\n",
    "def init_weights(m):\n",
    "    if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "        torch.nn.init.xavier_uniform_(m.weight)\n",
    "net.apply(init_weights)\n",
    "\n",
    "#Loss Function Criterion\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "\n",
    "#Loading Fashion-MNIST Dataset\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=96)\n",
    "\n",
    "d2l.train_ch5(net, train_iter, test_iter, criterion, num_epochs, batch_size, device, lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MlOhZzW_aYIn"
   },
   "source": [
    "## Summary\n",
    "\n",
    "* The Inception block is equivalent to a subnetwork with four paths. It extracts information in parallel through convolutional layers of different window shapes and maximum pooling layers. $1 \\times 1$ convolutions reduce channel dimensionality on a per-pixel level. Max-pooling reduces the resolution.\n",
    "* GoogLeNet connects multiple well-designed Inception blocks with other layers in series. The ratio of the number of channels assigned in the Inception block is obtained through a large number of experiments on the ImageNet data set.\n",
    "* GoogLeNet, as well as its succeeding versions, was one of the most efficient models on ImageNet, providing similar test accuracy with lower computational complexity.\n",
    "\n",
    "## Exercises\n",
    "\n",
    "1. There are several iterations of GoogLeNet. Try to implement and run them. Some of them include the following:\n",
    "    * Add a batch normalization layer, as described later in this chapter [2].\n",
    "    * Make adjustments to the Inception block [3].\n",
    "    * Use \"label smoothing\" for model regularization [3].\n",
    "    * Include it in the residual connection, as described later in this chapter [4].\n",
    "1. What is the minimum image size for GoogLeNet to work?\n",
    "1. Compare the model parameter sizes of AlexNet, VGG, and NiN with GoogLeNet. How do the latter two network architectures significantly reduce the model parameter size?\n",
    "1. Why do we need a large range convolution initially?\n",
    "\n",
    "\n",
    "\n",
    "## References\n",
    "\n",
    "[1] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., & Anguelov, D. & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).\n",
    "\n",
    "[2] Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167.\n",
    "\n",
    "[3] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2818-2826).\n",
    "\n",
    "[4] Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017, February). Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 4, p. 12)."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "googlenet.ipynb",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}