TensorFlow_MNIST.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "TensorFlow MNIST.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyNmjNfJ10TclBOq3VU54i74",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/RonakMehta21/FastAI-Keras-Colabs/blob/master/TensorFlow_MNIST.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zNpnb1aTfp5C",
        "colab_type": "text"
      },
      "source": [
        "# TensorFlow 2 with MNIST dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tf9ZPOP6a9C3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "try:\n",
        "  # %tensorflow_version only exists in Colab.\n",
        "  %tensorflow_version 2.x\n",
        "except Exception:\n",
        "  pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NX1SWxiMbJmo",
        "colab_type": "text"
      },
      "source": [
        "Import TensorFlow in your program"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HR469RzobG0W",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "from tensorflow.keras.layers import Dense, Flatten, Conv2D\n",
        "from tensorflow.keras import Model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OqHwGfF5bdvs",
        "colab_type": "text"
      },
      "source": [
        "Load and prepare the [MNIST dataset](http://yann.lecun.com/exdb/mnist/)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BlejjXlybMfi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "e97fb9e1-3eb3-4cbb-b2ad-b129c152a2c1"
      },
      "source": [
        "mnist = tf.keras.datasets.mnist\n",
        "\n",
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
        "\n",
        "# Add a channels dimension\n",
        "x_train = x_train[..., tf.newaxis]\n",
        "x_test = x_test[..., tf.newaxis]"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11493376/11490434 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lqgcPIMwbn1z",
        "colab_type": "text"
      },
      "source": [
        "Use `tf.data` to batch and shuffle the dataset:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5Bd82Wm6bhUN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_ds = tf.data.Dataset.from_tensor_slices(\n",
        "    (x_train, y_train)).shuffle(10000).batch(32)\n",
        "\n",
        "test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k54FC372bwmI",
        "colab_type": "text"
      },
      "source": [
        "Build the `tf.keras` model using the Keras [model subclassing API](https://www.tensorflow.org/guide/keras#model_subclassing):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E9xpckzibmJl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class MyModel(Model):\n",
        "  def __init__(self):\n",
        "    super(MyModel, self).__init__()\n",
        "    self.conv1 = Conv2D(32, 3, activation='relu')\n",
        "    self.flatten = Flatten()\n",
        "    self.d1 = Dense(128, activation='relu')\n",
        "    self.d2 = Dense(10)\n",
        "\n",
        "  def call(self, x):\n",
        "    x = self.conv1(x)\n",
        "    x = self.flatten(x)\n",
        "    x = self.d1(x)\n",
        "    return self.d2(x)\n",
        "\n",
        "# Create an instance of the model\n",
        "model = MyModel()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "asBgVrNYb1hL",
        "colab_type": "text"
      },
      "source": [
        "Choose an optimizer and loss function for training: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uFGUc0mMbuFL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n",
        "\n",
        "optimizer = tf.keras.optimizers.Adam()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_EKFh-Kib7mP",
        "colab_type": "text"
      },
      "source": [
        "Select metrics to measure the loss and the accuracy of the model. These metrics accumulate the values over epochs and then print the overall result."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q43uwNK_b36v",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_loss = tf.keras.metrics.Mean(name='train_loss')\n",
        "train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')\n",
        "\n",
        "test_loss = tf.keras.metrics.Mean(name='test_loss')\n",
        "test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='test_accuracy')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nSNtXe_dcAJN",
        "colab_type": "text"
      },
      "source": [
        "Use `tf.GradientTape` to train the model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QLjKdoasb7L2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "@tf.function\n",
        "def train_step(images, labels):\n",
        "  with tf.GradientTape() as tape:\n",
        "    # training=True is only needed if there are layers with different\n",
        "    # behavior during training versus inference (e.g. Dropout).\n",
        "    predictions = model(images, training=True)\n",
        "    loss = loss_object(labels, predictions)\n",
        "  gradients = tape.gradient(loss, model.trainable_variables)\n",
        "  optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
        "\n",
        "  train_loss(loss)\n",
        "  train_accuracy(labels, predictions)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "idydMCewcDsJ",
        "colab_type": "text"
      },
      "source": [
        "Test the model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I9Waa86JcB9Y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "@tf.function\n",
        "def test_step(images, labels):\n",
        "  # training=False is only needed if there are layers with different\n",
        "  # behavior during training versus inference (e.g. Dropout).\n",
        "  predictions = model(images, training=False)\n",
        "  t_loss = loss_object(labels, predictions)\n",
        "\n",
        "  test_loss(t_loss)\n",
        "  test_accuracy(labels, predictions)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7afK1QHecFrq",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "outputId": "622e7a29-0ed6-4c11-9162-d17236a175cc"
      },
      "source": [
        "EPOCHS = 5\n",
        "\n",
        "for epoch in range(EPOCHS):\n",
        "  # Reset the metrics at the start of the next epoch\n",
        "  train_loss.reset_states()\n",
        "  train_accuracy.reset_states()\n",
        "  test_loss.reset_states()\n",
        "  test_accuracy.reset_states()\n",
        "\n",
        "  for images, labels in train_ds:\n",
        "    train_step(images, labels)\n",
        "\n",
        "  for test_images, test_labels in test_ds:\n",
        "    test_step(test_images, test_labels)\n",
        "\n",
        "  template = 'Epoch {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}'\n",
        "  print(template.format(epoch+1,\n",
        "                        train_loss.result(),\n",
        "                        train_accuracy.result()*100,\n",
        "                        test_loss.result(),\n",
        "                        test_accuracy.result()*100))"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Layer my_model is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.\n",
            "\n",
            "If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
            "\n",
            "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
            "\n",
            "Epoch 1, Loss: 0.14781925082206726, Accuracy: 95.62332916259766, Test Loss: 0.07046515494585037, Test Accuracy: 97.69999694824219\n",
            "Epoch 2, Loss: 0.04532306641340256, Accuracy: 98.55999755859375, Test Loss: 0.06059658154845238, Test Accuracy: 98.06999969482422\n",
            "Epoch 3, Loss: 0.02410200797021389, Accuracy: 99.23500061035156, Test Loss: 0.05059795081615448, Test Accuracy: 98.3499984741211\n",
            "Epoch 4, Loss: 0.014695691876113415, Accuracy: 99.51166534423828, Test Loss: 0.05588207766413689, Test Accuracy: 98.36000061035156\n",
            "Epoch 5, Loss: 0.009750019758939743, Accuracy: 99.65166473388672, Test Loss: 0.06029528006911278, Test Accuracy: 98.44999694824219\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5kukHsiLd1rA",
        "colab_type": "text"
      },
      "source": [
        "The image classifier is trained to give ~99% accuracy on the dataset."
      ]
    }
  ]
}