dcgan.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_jQ1tEQCxwRx"
   },
   "source": [
    "##### Copyright 2019 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "id": "V_sgB_5dx1f1"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rF2x3qooyBTI"
   },
   "source": [
    "# Deep Convolutional Generative Adversarial Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0TD5ZrvEMbhZ"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://www.tensorflow.org/beta/tutorials/generative/dcgan.ipynb\">\n",
    "    <img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />\n",
    "    View on TensorFlow.org</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/r2/tutorials/generative/dcgan.ipynb\">\n",
    "    <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />\n",
    "    Run in Google Colab</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/r2/tutorials/generative/dcgan.ipynb\">\n",
    "    <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />\n",
    "    View source on GitHub</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://storage.googleapis.com/tensorflow_docs/site/en/r2/tutorials/generative/dcgan.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ITZuApL56Mny"
   },
   "source": [
    "This tutorial demonstrates how to generate images of handwritten digits using a [Deep Convolutional Generative Adversarial Network](https://arxiv.org/pdf/1511.06434.pdf) (DCGAN). The code is written using the [Keras Sequential API](https://www.tensorflow.org/guide/keras) with a `tf.GradientTape` training loop."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2MbKJY38Puy9"
   },
   "source": [
    "## What are GANs?\n",
    "[Generative Adversarial Networks](https://arxiv.org/abs/1406.2661) (GANs) are one of the most interesting ideas in computer science today. Two models are trained simultaneously by an adversarial process. A *generator* (\"the artist\") learns to create images that look real, while a *discriminator* (\"the art critic\") learns to tell real images apart from fakes.\n",
    "\n",
    "![A diagram of a generator and discriminator](https://tensorflow.org/beta/tutorials/generative/images/gan1.png)\n",
    "\n",
    "During training, the *generator* progressively becomes better at creating images that look real, while the *discriminator* becomes better at telling them apart. The process reaches equilibrium when the *discriminator* can no longer distinguish real images from fakes.\n",
    "\n",
    "![A second diagram of a generator and discriminator](https://tensorflow.org/beta/tutorials/generative/images/gan2.png)\n",
    "\n",
    "This notebook demonstrates this process on the MNIST dataset. The following animation shows a series of images produced by the *generator* as it was trained for 50 epochs. The images begin as random noise, and increasingly resemble hand written digits over time.\n",
    "\n",
    "![sample output](https://tensorflow.org/images/gan/dcgan.gif)\n",
    "\n",
    "To learn more about GANs, we recommend MIT's [Intro to Deep Learning](http://introtodeeplearning.com/) course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "e1_Y75QXJS6h"
   },
   "source": [
    "### Import TensorFlow and other libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "J5oue0oqCkZZ"
   },
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 428
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 40677,
     "status": "ok",
     "timestamp": 1554225081967,
     "user_tz": 420
    },
    "id": "g5RstiiB8V-z",
    "outputId": "74cde9cf-239e-4e62-c7fd-0218c59e1d1a"
   },
   "outputs": [],
   "source": [
    "!pip install -q tensorflow-gpu==2.0.0-beta1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WZKbyU2-AiY-"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 41826,
     "status": "ok",
     "timestamp": 1554225083141,
     "user_tz": 420
    },
    "id": "wx-zNbLqB4K8",
    "outputId": "471ecaf1-a318-46ad-d4ee-d393403efadc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.0.0-beta1'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 44426,
     "status": "ok",
     "timestamp": 1554225085752,
     "user_tz": 420
    },
    "id": "YzTlj4YdCip_",
    "outputId": "aa2e380a-a135-41c1-930b-c791893b522a"
   },
   "outputs": [],
   "source": [
    "# To generate GIFs\n",
    "!pip install -q imageio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YfIk2es3hJEd"
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "import imageio\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import PIL\n",
    "from tensorflow.keras import layers\n",
    "import time\n",
    "\n",
    "from IPython import display"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iYn4MdZnKCey"
   },
   "source": [
    "### Load and prepare the dataset\n",
    "\n",
    "You will use the MNIST dataset to train the generator and the discriminator. The generator will generate handwritten digits resembling the MNIST data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "a4fYMGxGhrna"
   },
   "outputs": [],
   "source": [
    "(train_images, train_labels), (_, _) = tf.keras.datasets.mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NFC2ghIdiZYE"
   },
   "outputs": [],
   "source": [
    "train_images = train_images.reshape(train_images.shape[0], 28, 28, 1).astype('float32')\n",
    "train_images = (train_images - 127.5) / 127.5 # Normalize the images to [-1, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "S4PIDhoDLbsZ"
   },
   "outputs": [],
   "source": [
    "BUFFER_SIZE = 60000\n",
    "BATCH_SIZE = 256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-yKCCQOoJ7cn"
   },
   "outputs": [],
   "source": [
    "# Batch and shuffle the data\n",
    "train_dataset = tf.data.Dataset.from_tensor_slices(train_images).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "THY-sZMiQ4UV"
   },
   "source": [
    "## Create the models\n",
    "\n",
    "Both the generator and discriminator are defined using the [Keras Sequential API](https://www.tensorflow.org/guide/keras#sequential_model)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-tEyxE-GMC48"
   },
   "source": [
    "### The Generator\n",
    "\n",
    "The generator uses `tf.keras.layers.Conv2DTranspose` (upsampling) layers to produce an image from a seed (random noise). Start with a `Dense` layer that takes this seed as input, then upsample several times until you reach the desired image size of 28x28x1. Notice the `tf.keras.layers.LeakyReLU` activation for each layer, except the output layer which uses tanh."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6bpTcDqoLWjY"
   },
   "outputs": [],
   "source": [
    "def make_generator_model():\n",
    "    model = tf.keras.Sequential()\n",
    "    model.add(layers.Dense(7*7*256, use_bias=False, input_shape=(100,)))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU())\n",
    "\n",
    "    model.add(layers.Reshape((7, 7, 256)))\n",
    "    assert model.output_shape == (None, 7, 7, 256) # Note: None is the batch size\n",
    "\n",
    "    model.add(layers.Conv2DTranspose(128, (5, 5), strides=(1, 1), padding='same', use_bias=False))\n",
    "    assert model.output_shape == (None, 7, 7, 128)\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU())\n",
    "\n",
    "    model.add(layers.Conv2DTranspose(64, (5, 5), strides=(2, 2), padding='same', use_bias=False))\n",
    "    assert model.output_shape == (None, 14, 14, 64)\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU())\n",
    "\n",
    "    model.add(layers.Conv2DTranspose(1, (5, 5), strides=(2, 2), padding='same', use_bias=False, activation='tanh'))\n",
    "    assert model.output_shape == (None, 28, 28, 1)\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GyWgG09LCSJl"
   },
   "source": [
    "Use the (as yet untrained) generator to create an image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 286
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 49698,
     "status": "ok",
     "timestamp": 1554225091093,
     "user_tz": 420
    },
    "id": "gl7jcC7TdPTG",
    "outputId": "6368572a-2cb9-4537-89a3-15294dc5ee7e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f615fc30cf8>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAGQNJREFUeJzt3Xlw1dX5BvDnZd9RQAFZBCOLCBQkA24VFBe0WmSkFqgOLpXaSvuzY6sWp4o6wzAubZ3WZUBwrQWroChWQQpqBytbZXNjiyxlk30TCLy/P3LtROQ8J03Cvdee5zPDkOTJyT25yZub3PM95zV3h4ikp0quJyAiuaHiF0mUil8kUSp+kUSp+EUSpeIXSZSKXyRRKn6RRKn4RRJVLZs3Vr9+fW/cuHEwNzM6nl2NWL16dTr28OHDND948CDNq1atWu7bPnDgAM0r8nnHVKnCf74fOnSoQrddo0aNcn/82Nz27dtH83r16tGcfc1jn3fsaxL7vHft2kXz2rVrl/u22ffqli1bsGvXLv4BMipU/GbWD8AjAKoCeNLdR7P3b9y4Me65557wZKrx6ezfvz+YNWvWjI7du3cvzTds2EDzBg0aBLOWLVvSsatWraJ57Bsp9oOJFRH7JgOAHTt20Ly4uJjmrVu3pvnWrVuDWZ06dejYpUuX0vyss86iOfvhsXv3bjo2VoCxz/vdd9+leZcuXYJZ7Ifipk2bgtn9999Px37tdsr8nkcws6oAHgVwKYBOAAabWafyfjwRya6K/M3fE8Byd1/p7gcATADQv3KmJSLHWkWKvwWANaVeX5t529eY2TAzm2dm82K/aolI9hzzZ/vdfYy7F7p7YewJGhHJnooU/zoArUq93jLzNhH5FqhI8c8F0M7M2ppZDQCDAEypnGmJyLFW7qU+dy82s+EA3kLJUt94d6drM9WqVUPDhg2D+cKFC+lttmrVKpjt3LmTjq1bty7N2TIiADRq1CiYFRUV0bEnnngizV944QWaX3fddTRnSz979uyhY2Nr6ew+B+JLpCeddFIwe/HFF+nYPn360Dz2NWfLnNOnT6djb7zxRpovWLCA5m3atKE5W0Jdv349HcuuvYhdz1Jahdb53f0NAG9U5GOISG7o8l6RRKn4RRKl4hdJlIpfJFEqfpFEqfhFEpXV/fyHDx+mW2tPOOEEOp5tq43tO49tTY3d9hdffBHMYmu6s2fPpvnJJ59M8yVLltC8oKAgmMW2E3fo0IHmbEsuALRr147mixcvDmZ9+/alYz/55BOaDxo0iOZsLX/IkCF07FtvvUXzyy67jOZr166lec2aNYNZ7PyHzp07B7NatWrRsaXpkV8kUSp+kUSp+EUSpeIXSZSKXyRRKn6RRGV1qa9atWp0e2tsuY0tE8ZO/o0t9cXGs9OBY0d3X3/99TR/5513aB6bGzsOffXq1XQs2w4MxE+pjW2N7devXzBbvnw5HXvTTTfRPLatlt325s2b6diePXvS/N///jfNY6dWse+ZCy64gI6dNm1aMIudUl2aHvlFEqXiF0mUil8kUSp+kUSp+EUSpeIXSZSKXyRRWV3n379/f3SLKdOixTe6gf1HrEvvnDlzaN68eXOaL1q0KJh17dqVjh03bhzNY2vGPXr0oDnbbjxr1iw69owzzqD5li1baN69e3eaz58/P5jFvmZ///vfaR67ToBtpT7ttNPo2C+//JLmsa95rEtvkyZNglmsczK7riN2TUhpeuQXSZSKXyRRKn6RRKn4RRKl4hdJlIpfJFEqfpFEVWid38yKAOwCcAhAsbsXsvevUqUKPVo4dg0Aa5PN9jgDwPHHH0/zv/71rzRnRzUvW7aMjmX7ygHgqaeeonnsaPDRo0cHs5///Od07EsvvUTzX//61zQfOXIkze+///5gFrvPf/rTn9L8+eefp/mpp54azGJHa//iF7+g+TXXXEPzoUOH0pyt1cdauj/22GPBLNZqvrTKuMjnfHcPX2UiInlJv/aLJKqixe8AppnZfDMbVhkTEpHsqOiv/ee6+zozOxHAdDP7xN2/dlFz5ofCMID/nSMi2VWhR353X5f5fxOAyQC+ceqhu49x90J3L6xfv35Fbk5EKlG5i9/M6ppZ/a9eBnAxAN5RUkTyRkV+7W8KYLKZffVxXnD3NytlViJyzFmstXVlKigocLYm/emnn9LxdevWDWasfTcQP6c9ds769u3bg1nHjh3p2BkzZtD87LPPpvnHH39M83vvvTeYjRo1io5lraKB+P3K1tIB4JVXXglmrVq1omNjeWxNe9u2bcEsNu/XX3+d5nfccQfNH3/8cZqzHhWxPhCsl8JvfvMbrFixwugHyNBSn0iiVPwiiVLxiyRKxS+SKBW/SKJU/CKJyurR3WZGlzHYUh4AnH766cFs8uTJdGxsOS12RPWFF14YzKZOnUrHnn/++TSfOXMmzW+55RaaT5w4MZj17t2bjmVLmABQu3Ztmk+YMIHmrN30cccdR8fGluOmTJlC84KCgmDWtGlTOvYPf/gDzWPtwSvSZrt///50LFuejS0TlqZHfpFEqfhFEqXiF0mUil8kUSp+kUSp+EUSpeIXSVRW1/kPHz6M3bt3B/MuXbrQ8StWrAhm55xzDh27YcMGmnfq1InmL774YjCLrUdXqcJ/xg4YMIDm06dPpznbuhpbS49dB/DAAw/QPHbMNMO+F4B4m+uLL76Y5qx1eWw9/MEHH6R5r169aL5nzx6aX3311cHsvffeo2MvueSSYBb7Xvva+5b5PUXkf4qKXyRRKn6RRKn4RRKl4hdJlIpfJFEqfpFE5dV+/tj6ZsOGDYNZbF21a9euNH/55ZdpfsUVVwSzWIvuq666iuaxds6DBw+m+auvvhrMYkeWx46Y/sEPfkDzWCvrHj16BLNYe/A+ffrQnB0LDgBDhgwJZh988AEd27JlS5rH7rfYdQLsPIBYO3l2v7FrPo6kR36RRKn4RRKl4hdJlIpfJFEqfpFEqfhFEqXiF0lUtEW3mY0HcDmATe7eOfO2RgAmAmgDoAjA1e4eXWBs27atjxw5Mpg3btyYjl+zZk0wi+3Xb9SoEc1POeUUmq9cuTKYxc4CePLJJ2n+wx/+kOaxtVt2zvvtt99Ox5588sk0j+0Pj50HMGfOnGDGztUH4l/TatX4ZSqHDh0KZjt37qRjY9cYxM5weOGFF2jerVu3YMbuM4B/zW6//fZKbdH9NIB+R7ztTgAz3L0dgBmZ10XkWyRa/O7+LoCtR7y5P4BnMi8/A+DKSp6XiBxj5f2bv6m7r8+8vAEA730kInmnwk/4ecmTBsEnDsxsmJnNM7N5u3btqujNiUglKW/xbzSz5gCQ+X9T6B3dfYy7F7p7Yf369ct5cyJS2cpb/FMAfLUVbSiA8LYyEclL0eI3s78AeB9ABzNba2Y3AhgN4CIzWwbgwszrIvItEt3P7+6hzeR9/9sbMzNUrVo1mH/55Zd0PDuDPrbm2717d5r/61//onmLFi2C2dq1a+nYe+65h+axXu/79u2j+V133RXMOnbsSMe2adOG5uwcAwAYO3YszdnXZfbs2XTs3XffTfPY9REjRowIZnPnzqVjY9duxM6eiF0fwb5nTjrpJDr24MGDwSx23U5pusJPJFEqfpFEqfhFEqXiF0mUil8kUSp+kURl9ejugwcPYtOm4MWA6Ny5Mx3PWjofOHCAjmXHWwPADTfcQHPWJju2PXT48OE0/9WvfkXz2LIRO2a6Tp06dOzq1atp/stf/pLmsas2a9euHcxi22Kfe+45mj/99NM0f+KJJ4LZRRddRMfOmDGD5uzzKot27doFs40bN9KxZuEdu2wp/Uh65BdJlIpfJFEqfpFEqfhFEqXiF0mUil8kUSp+kURldZ2/evXqOPHEE4P5rFmz6Hi2LTe2Jffss8+meazlcpMmTYLZokWL6Ng//elPNB8zZgzNY8dv/+hHPwpmU6dOpWNj1xBs3Xrk2a1f17cv39n93e9+N5iNHz+ejmXfKwDwve99j+Zsy3DsaG3WDh4A5s2bR/Mf//jHNGct4dmWXQDYv39/MItt/y5Nj/wiiVLxiyRKxS+SKBW/SKJU/CKJUvGLJErFL5KoaIvuylRQUOCjRo0K5ocPH6bjWcvlmKKiIpq3bt2a5mw9PNbe+/PPP6f5Z599RvNY6/KmTcOtEmNf39h+/Ng1DLH1cLa/fPPmzXRs7GvSrFkzmr///vvBbO/evXRs7ByEq666iuaxazfYkeaxo7uz2aJbRP4HqfhFEqXiF0mUil8kUSp+kUSp+EUSpeIXSVR0P7+ZjQdwOYBN7t4587aRAG4C8NVC7Qh3f6MsN8jWy2vVqkXHFhcXB7PYumxsvfuhhx6i+aRJk4JZ7Iz3WPvwWGvy2FnsLF++fDkdu27dOpoPHhzq0F4idr+zveexcw4KCwtp/uijj9K8X79+weyDDz6gY2P9Cp566imax3oSsF4LXbp0oWPnzJkTzCq7RffTAI52L/7e3btl/pWp8EUkf0SL393fBcCPcxGRb52K/M0/3MwWmdl4Mzu+0mYkIllR3uJ/HEABgG4A1gN4OPSOZjbMzOaZ2bxYTzsRyZ5yFb+7b3T3Q+5+GMBYAD3J+45x90J3L2zQoEF55ykilaxcxW9mzUu9OgDAksqZjohkS1mW+v4CoA+AJma2FsA9APqYWTcADqAIwE+O4RxF5BjIq/38sb3l7Kz0gQMH0rELFiygeb169WjO+tjXqFGDjt2+fTvN27dvT/OFCxfSnK1nx/oZtGnThuZvv/02zYcPH05ztpbfu3dvOjZ2bv/u3btp/tprrwWz2J+gHTt2pDk7QwEA5s+fX+6Pv2PHDjqWXStz9913Y9WqVdrPLyJhKn6RRKn4RRKl4hdJlIpfJFEqfpFEZbVFd3FxMbZt2xbMV65cScf36tUrmL3++ut0bKxFd2w5ji0Nxdo133rrrTSPbSeOHVF94MCBYBZrFc2WCQHghhtuoPkTTzxB85tvvjmYjR07lo6NbfFmy68A8MgjjwSz6dOn07GxY8Nfeuklmo8ePZrmEydODGaxz4u18GZbqI+kR36RRKn4RRKl4hdJlIpfJFEqfpFEqfhFEqXiF0lUVrf0nnLKKX7fffcF89g2S3Zk8fXXX0/Hxo557t69O82Zyy+/nOaxds2xI6qvueYamrM169gR1bFjxZctW0bzSy65hOas/XinTp3oWLb9GwB++9vf0nzChAnB7MILL6RjY23VY1+z2P3OjmuP1SRrCa8W3SISpeIXSZSKXyRRKn6RRKn4RRKl4hdJlIpfJFFZXefv0KGDP/bYY8F83759dPyGDRuC2emnn07HLlq0iOZTp06lOZv3Rx99RMfG2mAfOnSI5scfz1shsnbPsePQN23aRPPY8dixo8HZ9RMtWrSgY6tXr07z2JHp7BqD2bNn07GxcwzeeecdmseuI2BHnl955ZV0bJMmTYLZbbfdhuXLl2udX0TCVPwiiVLxiyRKxS+SKBW/SKJU/CKJUvGLJCp6br+ZtQLwLICmABzAGHd/xMwaAZgIoA2AIgBXu3v4UH6UrOMvXbo0mPfs2ZPOhV0HEGuJXFBQQPPYeQBsX3ujRo3o2Fib6zPOOIPmsfbh7FqNe++9l44dMmQIzWPnwJ911lk0X7NmTTDr1q0bHTtt2jSax1p4M7FrBGJfs1hr8zfffJPmbJ0/dt0Iu+2qVavSsaWV5ZG/GMBt7t4JwJkAbjGzTgDuBDDD3dsBmJF5XUS+JaLF7+7r3X1B5uVdAD4G0AJAfwDPZN7tGQD8siQRySv/1d/8ZtYGQHcAHwBo6u7rM9EGlPxZICLfEmUufjOrB+BlALe6+87SmZf80XnUPzzNbJiZzTOzebHrxEUke8pU/GZWHSWF/2d3n5R580Yza57JmwM46g4Rdx/j7oXuXhh74kpEsida/GZmAMYB+Njdf1cqmgJgaObloQBerfzpicixUpYW3ecAuBbAYjP7MPO2EQBGA3jRzG4E8DmAq2MfyMxo2+XYttp27doFs9jW5Ni22NhRyzt27AhmCxcupGN79+5N81deeYXm48aNo/nAgQOD2YMPPkjHbt26lea7du2i+WuvvUbzwYMHB7OZM2fSsWeeeSbNJ02aRHO2rfY73/kOHXvcccfRPNaa/IEHHqD5s88+G8yaNuVPn7Ft2Kx995Gixe/u/wAQ2h/ct8y3JCJ5RVf4iSRKxS+SKBW/SKJU/CKJUvGLJErFL5KorB7d3b59e//jH/8YzOvWrUvHr1ixIpjFjoFeu3Ytzbt27UpztkUzduT4gQMHaM5aLgNAs2bNaF5UVBTM2rdvT8fGtrZu2bKF5rGjv9kW7ptvvpmOffVVft1YrD34ww8/HMzYlloAqFmzJs1j6+lz586lOft+i11zwq53ufPOO9WiW0Q4Fb9IolT8IolS8YskSsUvkigVv0iiVPwiiSrLfv5KVaVK+OcNW8cHgFatWgWzkjNHwho3bkzz2FHNbdu2DWaxNd/Y/uznn3+e5sOGDaM5W0uPrRnH1ukHDRpE89gZDD169Ahm7733Hh0bOx577969NGfXOCxZsoSOjZ3/wO7zsoxnn/v3v/99OnblypU0Lys98oskSsUvkigVv0iiVPwiiVLxiyRKxS+SKBW/SKKyus6/Z88e/POf/wzmHTp0oOM3bNgQzGKtiWNr8S1btqQ5O2vg888/p2Nj1y9ccMEFNP/b3/5G8wEDBgSzxYsX07GdOnWi+ZNPPknzSy+9lOb169cPZrEW3Xv27KH5tm20Izw94yHWsj22lt6rVy+ar1+/nubs+oedO3cGMwC09wW7juYb71vm9xSR/ykqfpFEqfhFEqXiF0mUil8kUSp+kUSp+EUSFV3nN7NWAJ4F0BSAAxjj7o+Y2UgANwHYnHnXEe7+BvtYtWrVQseOHYP57Nmz6VzY+fY7duygY5s3b07z2Ln+bHyTJk3o2IEDB9I8tibcsGFDmm/fvj2YxT7vTz/9lOY/+9nPaB67xqFevXrBLLanfvny5TSP9RS49tprg9n8+fPp2MLCQppPmTKF5j179qT5zJkzg9kJJ5xAx7I+EbEeEqWV5SKfYgC3ufsCM6sPYL6ZTc9kv3f3h8p8ayKSN6LF7+7rAazPvLzLzD4GwNvjiEje+6/+5jezNgC6A/jqbKjhZrbIzMab2VHPLTKzYWY2z8zmxS5bFJHsKXPxm1k9AC8DuNXddwJ4HEABgG4o+c3gqI3R3H2Muxe6e2GDBg0qYcoiUhnKVPxmVh0lhf9nd58EAO6+0d0PufthAGMB8Gc4RCSvRIvfSo7FHQfgY3f/Xam3l34aeQAA/tStiOSVaItuMzsXwHsAFgM4nHnzCACDUfIrvwMoAvCTzJODQQUFBT5q1KhgHjuqmbWiZtscgfiSVGxLL/v4+/fvp2O/+OILmjdq1IjmtWvXpvnq1auD2datW+nYIUOG0Hzz5s00jy0txT53JraUF3sOiY0/7bTT6Ng6derQPNZ2PbaV+oorrghmy5Yto2OLi4uD2X333YeioqIyteguy7P9/wBwtA9G1/RFJL/pCj+RRKn4RRKl4hdJlIpfJFEqfpFEqfhFEpXVo7vNDDVr1gzmbL0aANjlwbGWyaeeeirN2bZYAFi3bl0w69u3Lx0bO1Y8ts7Pjg2PffzYOvuqVatoPnnyZJqPGDGC5uxzmzVrFh0ba6veunVrmrO19tix36y9NxBvL37eeefRnF1fE9uGXb169WDG6utIeuQXSZSKXyRRKn6RRKn4RRKl4hdJlIpfJFEqfpFERffzV+qNmW0GUHpjfRMA5d/wfWzl69zydV6A5lZelTm3k92dn/2dkdXi/8aNm81zd35Aeo7k69zydV6A5lZeuZqbfu0XSZSKXyRRuS7+MTm+fSZf55av8wI0t/LKydxy+je/iOROrh/5RSRHclL8ZtbPzD41s+Vmdmcu5hBiZkVmttjMPjSzeTmey3gz22RmS0q9rZGZTTezZZn/j9omLUdzG2lm6zL33YdmdlmO5tbKzGaa2UdmttTM/i/z9pzed2ReObnfsv5rv5lVBfAZgIsArAUwF8Bgd/8oqxMJMLMiAIXunvM1YTM7D8BuAM+6e+fM2x4AsNXdR2d+cB7v7nfkydxGAtid687NmYYyzUt3lgZwJYDrkMP7jszrauTgfsvFI39PAMvdfaW7HwAwAUD/HMwj77n7uwCO7LrRH8AzmZefQck3T9YF5pYX3H29uy/IvLwLwFedpXN635F55UQuir8FgDWlXl+L/Gr57QCmmdl8MxuW68kcRdNSnZE2AGiay8kcRbRzczYd0Vk6b+678nS8rmx6wu+bznX3MwBcCuCWzK+3eclL/mbLp+WaMnVuzpajdJb+j1zed+XteF3ZclH86wC0KvV6y8zb8oK7r8v8vwnAZORf9+GNXzVJzfy/Kcfz+Y986tx8tM7SyIP7Lp86Xuei+OcCaGdmbc2sBoBBAKbkYB7fYGZ1M0/EwMzqArgY+dd9eAqAoZmXhwJ4NYdz+Zp86dwc6iyNHN93edfx2t2z/g/AZSh5xn8FgLtyMYfAvE4BsDDzb2mu5wbgLyj5NfAgSp4buRFAYwAzACwD8DaARnk0t+dQ0s15EUoKrXmO5nYuSn6lXwTgw8y/y3J935F55eR+0xV+IonSE34iiVLxiyRKxS+SKBW/SKJU/CKJUvGLJErFL5IoFb9Iov4fwXVtSCKDP0gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "generator = make_generator_model()\n",
    "\n",
    "noise = tf.random.normal([1, 100])\n",
    "generated_image = generator(noise, training=False)\n",
    "\n",
    "plt.imshow(generated_image[0, :, :, 0], cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "D0IKnaCtg6WE"
   },
   "source": [
    "### The Discriminator\n",
    "\n",
    "The discriminator is a CNN-based image classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dw2tPLmk2pEP"
   },
   "outputs": [],
   "source": [
    "def make_discriminator_model():\n",
    "    model = tf.keras.Sequential()\n",
    "    model.add(layers.Conv2D(64, (5, 5), strides=(2, 2), padding='same',\n",
    "                                     input_shape=[28, 28, 1]))\n",
    "    model.add(layers.LeakyReLU())\n",
    "    model.add(layers.Dropout(0.3))\n",
    "\n",
    "    model.add(layers.Conv2D(128, (5, 5), strides=(2, 2), padding='same'))\n",
    "    model.add(layers.LeakyReLU())\n",
    "    model.add(layers.Dropout(0.3))\n",
    "\n",
    "    model.add(layers.Flatten())\n",
    "    model.add(layers.Dense(1))\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QhPneagzCaQv"
   },
   "source": [
    "Use the (as yet untrained) discriminator to classify the generated images as real or fake. The model will be trained to output positive values for real images, and negative values for fake images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 50156,
     "status": "ok",
     "timestamp": 1554225091568,
     "user_tz": 420
    },
    "id": "gDkA05NE6QMs",
    "outputId": "b64c1639-21a4-4f6c-9e97-b51610efa8ba"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([[-0.00218262]], shape=(1, 1), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "discriminator = make_discriminator_model()\n",
    "decision = discriminator(generated_image)\n",
    "print (decision)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0FMYgY_mPfTi"
   },
   "source": [
    "## Define the loss and optimizers\n",
    "\n",
    "Define loss functions and optimizers for both models.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "psQfmXxYKU3X"
   },
   "outputs": [],
   "source": [
    "# This method returns a helper function to compute cross entropy loss\n",
    "cross_entropy = tf.keras.losses.BinaryCrossentropy(from_logits=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PKY_iPSPNWoj"
   },
   "source": [
    "### Discriminator loss\n",
    "\n",
    "This method quantifies how well the discriminator is able to distinguish real images from fakes. It compares the discriminator's predictions on real images to an array of 1s, and the discriminator's predictions on fake (generated) images to an array of 0s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wkMNfBWlT-PV"
   },
   "outputs": [],
   "source": [
    "def discriminator_loss(real_output, fake_output):\n",
    "    real_loss = cross_entropy(tf.ones_like(real_output), real_output)\n",
    "    fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)\n",
    "    total_loss = real_loss + fake_loss\n",
    "    return total_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Jd-3GCUEiKtv"
   },
   "source": [
    "### Generator loss\n",
    "The generator's loss quantifies how well it was able to trick the discriminator. Intuitively, if the generator is performing well, the discriminator will classify the fake images as real (or 1). Here, we will compare the discriminators decisions on the generated images to an array of 1s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "90BIcCKcDMxz"
   },
   "outputs": [],
   "source": [
    "def generator_loss(fake_output):\n",
    "    return cross_entropy(tf.ones_like(fake_output), fake_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MgIc7i0th_Iu"
   },
   "source": [
    "The discriminator and the generator optimizers are different since we will train two networks separately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "iWCn_PVdEJZ7"
   },
   "outputs": [],
   "source": [
    "generator_optimizer = tf.keras.optimizers.Adam(1e-4)\n",
    "discriminator_optimizer = tf.keras.optimizers.Adam(1e-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mWtinsGDPJlV"
   },
   "source": [
    "### Save checkpoints\n",
    "This notebook also demonstrates how to save and restore models, which can be helpful in case a long running training task is interrupted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CA1w-7s2POEy"
   },
   "outputs": [],
   "source": [
    "checkpoint_dir = './training_checkpoints'\n",
    "checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt\")\n",
    "checkpoint = tf.train.Checkpoint(generator_optimizer=generator_optimizer,\n",
    "                                 discriminator_optimizer=discriminator_optimizer,\n",
    "                                 generator=generator,\n",
    "                                 discriminator=discriminator)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Rw1fkAczTQYh"
   },
   "source": [
    "## Define the training loop\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NS2GWywBbAWo"
   },
   "outputs": [],
   "source": [
    "EPOCHS = 50\n",
    "noise_dim = 100\n",
    "num_examples_to_generate = 16\n",
    "\n",
    "# We will reuse this seed overtime (so it's easier)\n",
    "# to visualize progress in the animated GIF)\n",
    "seed = tf.random.normal([num_examples_to_generate, noise_dim])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jylSonrqSWfi"
   },
   "source": [
    "The training loop begins with generator receiving a random seed as input. That seed is used to produce an image. The discriminator is then used to classify real images (drawn from the training set) and fakes images (produced by the generator). The loss is calculated for each of these models, and the gradients are used to update the generator and discriminator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3t5ibNo05jCB"
   },
   "outputs": [],
   "source": [
    "# Notice the use of `tf.function`\n",
    "# This annotation causes the function to be \"compiled\".\n",
    "@tf.function\n",
    "def train_step(images):\n",
    "    noise = tf.random.normal([BATCH_SIZE, noise_dim])\n",
    "\n",
    "    with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:\n",
    "      generated_images = generator(noise, training=True)\n",
    "\n",
    "      real_output = discriminator(images, training=True)\n",
    "      fake_output = discriminator(generated_images, training=True)\n",
    "\n",
    "      gen_loss = generator_loss(fake_output)\n",
    "      disc_loss = discriminator_loss(real_output, fake_output)\n",
    "\n",
    "    gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)\n",
    "    gradients_of_discriminator = disc_tape.gradient(disc_loss, discriminator.trainable_variables)\n",
    "\n",
    "    generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))\n",
    "    discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2M7LmLtGEMQJ"
   },
   "outputs": [],
   "source": [
    "def train(dataset, epochs):\n",
    "  for epoch in range(epochs):\n",
    "    start = time.time()\n",
    "\n",
    "    for image_batch in dataset:\n",
    "      train_step(image_batch)\n",
    "\n",
    "    # Produce images for the GIF as we go\n",
    "    display.clear_output(wait=True)\n",
    "    generate_and_save_images(generator,\n",
    "                             epoch + 1,\n",
    "                             seed)\n",
    "\n",
    "    # Save the model every 15 epochs\n",
    "    if (epoch + 1) % 15 == 0:\n",
    "      checkpoint.save(file_prefix = checkpoint_prefix)\n",
    "\n",
    "    print ('Time for epoch {} is {} sec'.format(epoch + 1, time.time()-start))\n",
    "\n",
    "  # Generate after the final epoch\n",
    "  display.clear_output(wait=True)\n",
    "  generate_and_save_images(generator,\n",
    "                           epochs,\n",
    "                           seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2aFF7Hk3XdeW"
   },
   "source": [
    "**Generate and save images**\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RmdVsmvhPxyy"
   },
   "outputs": [],
   "source": [
    "def generate_and_save_images(model, epoch, test_input):\n",
    "  # Notice `training` is set to False.\n",
    "  # This is so all layers run in inference mode (batchnorm).\n",
    "  predictions = model(test_input, training=False)\n",
    "\n",
    "  fig = plt.figure(figsize=(4,4))\n",
    "\n",
    "  for i in range(predictions.shape[0]):\n",
    "      plt.subplot(4, 4, i+1)\n",
    "      plt.imshow(predictions[i, :, :, 0] * 127.5 + 127.5, cmap='gray')\n",
    "      plt.axis('off')\n",
    "\n",
    "  plt.savefig('image_at_epoch_{:04d}.png'.format(epoch))\n",
    "  plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dZrd4CdjR-Fp"
   },
   "source": [
    "## Train the model\n",
    "Call the `train()` method defined above to train the generator and discriminator simultaneously. Note, training GANs can be tricky. It's important that the generator and discriminator do not overpower each other (e.g., that they train at a similar rate).\n",
    "\n",
    "At the beginning of the training, the generated images look like random noise. As training progresses, the generated digits will look increasingly real. After about 50 epochs, they resemble MNIST digits. This may take about one minute / epoch with the default settings on Colab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 302
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 199,
     "status": "ok",
     "timestamp": 1554225743725,
     "user_tz": 420
    },
    "id": "Ly3UN0SLLY2l",
    "outputId": "ee38220a-b9e4-4efa-f9a5-646667320c8d"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQIAAAD7CAYAAACBpZo1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnWmAVNWZhp+qgm6abqABI2ET2WQxDCggSxSJgIo6iHE0QRaJG6AS1KC4hDFRB3dBHUETJRpRJ+iAwiRqkEUEFERFXFFAlH2RnW7odX5U3nOrm6bXWm413/OHpru6+px7b33n/ZbznUBhYSGGYRzfBBM9AMMwEo8ZAsMwzBAYhmGGwDAMzBAYhoEZAsMwMENgGAZmCAzDAGrE848FAoGYVC8Fg0XtWUFBQSz+DIWFhYFyjqfwX6+PyThiTXnnGQqFCqH81zsQOPptE3mNyjtPqNo91bz1b+R7xGv+Zc3VFIFhGPFVBLFCVrW4MkgUyaoEKkpFlVc0r0vkvY6VAoykKmPX70oRaOzxGHd5qRaGQBdW/+bn5ydyOEYcOPnkkwHIzc1l06ZNgP8NcM2aNQFo3rw5AHv27CErKwvwjEJubm5CDIQ/llDDMBJK0iuCUChEjRrhaRw5ciTBo0kckp26Frm5uYkcTszQPHfu3AmE77nflYBITU0FoHv37gBs27aNzZs3u68hcffNFIFhGMmrCC699FIAevXqxeTJkwGcdT2eaNOmDQD/+Mc/ADjhhBMA6N27N19//XXCxlUVtOrLp+7UqZNTexs2bADg0KFDgL8CbsdCSmDQoEEADB8+HIDp06fzzTffAJCXlwckLs5hisAwjORRBMoIvPXWWwD069cPCK8MS5cuBWDOnDnA8ZM1CAQC/PSnPwWgZcuWgBcjmD17Nh06dEjY2KqC7vXMmTMBGDhwIF999RXg3feDBw8mZnAVJBAI0K5dOwCuv/56AJcpWLp0qYsNJPqZTRpDMG3aNAAGDBgAeBLqlVdeYeXKlQD8/Oc/B6BLly4MGzYMgOXLl7vXASxbtix+g44Rks6hUMg9SMWr9urVqxf3cUWLtLQ0AM4991wAUlJSyM7OBmD37t0JG1dlad26NQCZmZkAvPvuu0A4QJhoAyDMNTAMIzkUQa1atbjiiisATwl8//33AIwdO9YFjFq0aAHAHXfcQYMGDQDo2rUrAKNHjwbg3nvv5Z577onf4KNEKBRyQafatWsDYQmtVFRxRbBmzZr4DjCKnHjiiUBYCUD4nl9++eXu62SisLDQKVapt61bt7qf+QVTBIZhJIciuOiii9xquG/fPgB+9rOfAZCTk+Net2TJEgCaNWtG+/btAbjuuusA+PWvfw3Arbfe6tJSEydOjMPoq4ZW+oyMDOrXrw94KbMOHTo4pVScjRs3xmeAMUCxAc39yJEjST0fxTX++c9/AkUVXUViBMFg8Kgy+mipCl8bAknESZMmuQugunJFXksiPz+fL774AoBx48YB8PrrrwPhrMOtt94KhHPtADfccIOT0n6Sa+Dl0hs3buyMoYzggAED6NmzZ5HX6wF577334jjK6NKnTx/AMwSrVq1K5HCqjGo7mjRpAnj3KCMjg7179wKlP3cZGRkAnHXWWezYsQPAPd+5ublRCTiaa2AYhj8VgVb/K6+8EggHyhRgGTVqFFDxlVu1Bjk5OS7XrkDbqFGjGD9+PJD4fK7QaqiUU0ZGBgMHDgS8FGrTpk3ddVCNutyGtWvXxnW80UTzE3LrkpFgMMg555wDeEpHz/eSJUtcKnH//v1A+D7q3tepUwfwUud9+/Z1r//Tn/4EwMqVKzlw4EDVx1nldzAMI+nxpSJQ2kj/Llu2zBUGrVixolLv2ahRI/e1Vs/PPvsMgMmTJ/tGCQgFlC644AIAOnbsyCmnnALgqglTUlJcrETKQCvLd999F9fxRou0tDTnE+ueJPMekmAw6FZ2zUt069bNpcEjfX6hwirFsjIzM917bN++HfD2XFR5nFF5F8MwkhpfKgLtNHvppZeAcDcaZQsqSigUAry69dq1azv/WautUpJ+QP7h+eefD8Af//hHILw7TbENrf61atVy81N2Qb9/zz33uIxJeSLTiUbjHzdunJuDxt2lSxdXlJMsaA6NGjVyMYLiZd+1atVyyi2yl4bU4F133QV4CrBGjRpu96V2lkZr96UvDYEe2F27dgHhC6Z6AT34ZeVRFZB58803AYqk2f7yl78A/jIAolmzZkB4iyp4D8WhQ4dcUCjy5ks+ykjo9VdccQXnnXceANdccw0QDk75xSgUN3hXX301EHaBdG/1wVmyZAkffPABACNHjgS8xiQFBQXuvQ4fPgyE55aw7bz/eu5U89GvXz/nlmqrsYzaQw89dNTeiRo1anDLLbcAXg1MZIXlU089BUR/+7W5BoZhEIin5azsuQYNGjRwRRlaxWVJA4FAkepCfe+2224D4P7773ffA8jOznYBl0p04S1XH/yqnN/w9ttvA952W60ib7zxBj/5yU+A8KoJ4RVTq0Xx3vngqSatnqtXr2bw4MEAbjdfScRjnrNmzQK8Zh0a6+eff+5kcpcuXYDwiqifKxhaUnu6yGejf//+QOnBtIqca1CeuaakpHDaaacBcMkllwBw0kknOXWwZ88eACZMmADAli1b3DOoYrE+ffrw3HPPAZ46FDk5OdSqVUtjL+/Q9Xo718AwjNLxZYygOPv27XO+lyy8LGIoFHIrpaxlv379XBlx8dNlJkyY4Nv2VsFg0K0oGq+CpG+99RZDhw4FID09HfB8x8jXa+Xcu3evCygp2NSxY0dX5rpu3bqYzqU06tev74qGdF81/p07d7qeEZ9++ikQLs1VIxI1+dC/derUcfdYz8GhQ4dcu/Mvv/yyyPvHklAo5OYlpbNmzRoXE/j4448B75msX78+nTp1ArxYSWnNZDZu3BizeSSFIcjPz3cGQB8CPfB16tRx1Xe9evUCwtFW5W71oVe+9umnn47fwCtI8+bN3bjl7kR+YBs3bgx4MjLSDVCgTBtbRo8e7X7+7LPPAmHZqaCbeh3GM2Cqv/n000+7oK/Grfnm5ua6XPqMGTOAsLug+61qUMlnbT0Hz43asGGDu0bxPOsiPz/fzUtnF7Ro0YIzzzwT8GoEtGClpqa6eySX5+OPP3YBXd1vvUa1NLHAXAPDMJJDEYC3ckjaKu+8efNmZ0EVhGnSpImzololVJPg537/PXv2PKouQIHBX/ziF04tRMrDyLZlkb+XmZnpZOdZZ50FhCvbFCh97LHHAC9tFyuCwaDr1af7s2PHDqZMmQJ49R0KXtapU8etmHJtDhw44IKDmqfcnchg8auvvgrAM88841yCeFeMSoHqPqSkpLiUbnEKCwvd+KR0e/To4eave6vXvPbaazEbtykCwzCSI31YFrK+2oN/xhlnuJ8p2KaWZUqlVYZYp9VSUlJYtGgR4M0hsrhKfm/dunWB8GpbfNXQ3oNt27a54JkKcyJjCuqtLz88kmjO89xzzz2qu/Q555zj9owUf/5CoZBb9fVvIBCgW7duALzwwguAFxsoKChwO0effPLJIn+nLKKdPgTvWTz99NMBuPbaa7nwwgsBb6+BFOyGDRucWlAjndTUVKcqlBrV9Rs2bFilg4WWPjQMo0ySJkZQGrKgkaWYigU8/PDDgFeu7GdycnIYMmQI4EWItapnZGQcVV4dDAaPOhJeK0zz5s2d7xx5is7ChQsBr717rBk2bJgbd2RhU2krm7JASgEOHDiQESNGAF6Rjeb0zDPP8PjjjwOJL5sGLwal+7d8+fKjGsuKYDDo0ozPP/88EH6GldlRH4YtW7bEcshANTEEkmPqEtuuXTvWr18PwLx58xI2rsogV+Y3v/kN4D0gWVlZbl9BZHWZgmgKtml76sSJE5k/fz7gpaZycnLiVkOhh79169ZFUr0QdhfUX1KGbuzYsUD4Q6/XRQbc9CFX70J1NU6GzUjHMlD5+fksXrwYwLUgy8vLc1WRxStmY4m5BoZhVA9FoPSYik3y8/OddFQK7YcffgBKb3rqB7RiqxmHzmDYvHmza+aqVFNmZqbbkfj+++8D8NFHHwFeujVRSBHUrVvXrewKds6aNavEvRHF0bXIz89385MSkPJJdnRNlA794IMP4qoEhCkCwzCqhyLQKiEVsH//fhesUdpQzU+XLl3qi6DSsdDY1EZt9erViRxOpdFqfuedd7oSZ+3CC4VCx1QChYWFruxZcYTx48fz7bffFnnf6kLxFufaIxNvkt4QBINBt49AgaNFixa5rsXKoUceFupnQyCSYYzlYe7cuUfVzNesWdMdWKtMjwzeli1bqs3cy4MyKaqrUEVkvDHXwDCM5K8sbNOmjdu6qaYesQq2xKNhhx+weR5NrOYqd0nKVT0Jo41VFhqGUSZJGyNQrfwll1zimnMmIu1iGFVBSqC01nHxwBSBYRjJFyNQTb1KU3fs2BG3KLP5zkU5XuYJsZur9mEoLRqrZ7msuSadIUgk9gEpyvEyT6j+czXXwDCM+CoCwzD8iSkCwzDMEBiGYYbAMAzMEBiGgRkCwzAwQ2AYBmYIDMPADIFhGMR592F1L9MUx8s8g8Fg4b9eH9sBxYh4lxgXb88Wz+tW1lyTdhuykXiS1QAkCl2v0jo3JwpzDQxfEAgEfPkBOV4wQ2AYhrkGhj8wNyOxmCIwDMMUQTKhbjY6JDQUCrnDQHS4i+Ff1FVLh9IeOXIkkcMpgikCwzBMESQLtWrVcse4dezY0X1fffD79esHeIe9Gv6hVatWgHdA7UMPPQTA/fffn7AxFcd6FgINGjSgdu3aQPjILSj5jL1EFhS1aNGCdevWAZ6LEIlcAx2Zddppp1X6nEC/FE5FNvaMxXMar4IinbupE493794NhM/lPHToEAATJkwAYPDgwdStWxeAAQMGAPD5558D3vmIlcF6FhqGUSZJ6xqorXnkSqGz5tu3b0/fvn0B6NWrFwCbNm0C4C9/+Ytb9Tt16gRA27Zt+fvf/17i+yY6raUim1mzZpWoBDS+GjXCt1Jz+u1vf8uUKVPiNMrYoPkGAoGkDYZmZGRw4oknAt69UrDwoosuYtKkSYAXAI5k1apVAOzZsweA5s2bOwURbUwRGIaRfDEC+U/PPPMMAO3atSMjI6PIzzIzM6lZsybgWeHc3FwgfLTUjz/+CMA333wDhIM2CsRp5SnpuiTCdx4/fjwADz74oFMrShnOnDmTtLQ0ILy6gLeyZGVl0adPH8BbWcqLX2IEitv06NGDDz74AIju0WDxiBH8+c9/5uqrrwbg+++/B+D8888HYOPGjbRs2RKAyy+/HAjHAzTvrKwswDsWbevWre6g34oqpGpzwEl6ejrgPdS6gHl5ee6DPWvWLACWLl1K165dAfjVr34FQOPGjYGw3FT+duXKlQCMGTOGL774AijdFYjnB6Rhw4YA7Ny5U+/pznaUu7Nq1SrnEtxzzz0A/O53vwPCLs7ChQsBL+hU3nvtF0MwatQoAG688UYeeOABAObNmwdASkoKEM7FyzBW9OzLWBoCfZh37tzpxnrppZcCMGfOnIq8FU2bNgVg9erVbv5XXnklUP5aBAsWGoZRJkkTLJw/fz4ArVu3BuDAgQMAdOjQwQX/InnttdcAaNasGQCXXXaZ+5kCLsrnfvnllwkPCgoFBzdu3Fjk/5s2beLUU08FvGBTIBBwKanu3bsXeT14c1dF286dO30zz9KQ+pMKqF27tquT0P0//fTTAXj++eed8pH754c5PvzwwwB8+OGHDBo0CPDuW0XZvn07AGlpaU4NSnFEqzrRFIFhGMmhCFJSUmjXrh3gWcD27dsDXrFGcbQann322YCXFgRYs2YN4PmblS28iQWdO3cGvFSo/N6ePXu6FUVzue2227jlllsAXMBU16egoMAFTOWbvvDCCy4A5Wfq168PePflyJEjLtCmSkrFiubPn+/m5AclIKRYbr755korAXHSSScB4fiWngsp4mhhisAwDH8rAvm7V111lSuv/MMf/gAcWwlAuLhGMQIVc4j8/HyeeOIJAA4fPhztIVeZJ598EvCUwJtvvgmEy1LlFz7yyCMADB8+3EWkteooE9KiRQv3sx49egBhfzoZyMzMBOCzzz4DwuXSyvrMmDED8PZUKC7gF3SPunTpUuX3UkHV73//eyD8XCvlHe0CK18aAknfxx57DIArrrjCfW/btm2AZyQi5aBe8/DDDztXItIlgHBudvHixTEcfeUZMGCA+9DqIdD/J02a5IyaPhTLli3jnXfeAeD9998Hwi4EhCWpZOSnn34K+Gvba2moHkQp4PT0dFcxqTp9v1YaynjVqlUL8FzUiqBn+5JLLgFg6NChQPhZnjZtWjSGeRTmGhiG4b+Colq1ajF37lwAVxkXCARc4EjWUtI5GAy6dKCkcCgUciuqLLNcixEjRvDKK69UavyxKrRp1KgREJa92iOh8WvceXl5LFiwAIAXX3wRCLsNksZSCS+99BIQlqbaiXjmmWcClDtQmOiCorFjxwLw+OOPu+/9+te/Bry0cDQCvNEuKKpRo4a7xgrUfvTRR06l6V6qCOzCCy/k22+/BTwXZ8uWLS44uGTJEsALnu7bt8+5TRXFCooMwygT38QItNK3atWKtm3bAp5/H5kK0/e0+oMXoJHFPXjwoKtJV0DwT3/6E0Cl1UAs0Jy7desGhC2+fGDtIZAyOHToELNnzwa8Uuq8vDwaNGgAwB133AF46cdAIMDIkSOB8isBPxAKhZxPrOtz4MABN3c/pXqLEwqF2LVrF+CpvKZNm7qdrYp5qHw8Es3ryy+/dM+xYiX6vxqcxALfGALx3XffOZdAAaLatWvTokULwKug04f/hBNOcBdMD3zLli1Zv349ADfddBOA27TiJ/Sgq3PN8uXL3bwUDJMLlJOT43LHcueCwaCrwjvvvPMAz0CuW7eO1atXx2MaUUFyecCAAc6Y6cMxffp032UHSiI3N5f+/fsD8Nxzz7nvaWErXg1Ys2ZN9yHXnpI6deq4TUa6z1OnTgW8QGksMNfAMAz/KAJZv+zsbJcjLm//Pa2KaulUr149VqxYAeC2F/sRKQLJyfz8fPd1eSgoKHA77yJdJSBmaaZoo2twwgknADBkyBA3F7mByeLaFBQUuKrVgQMHAuEKUaVxda8U3M7Pzy+i7iC8s1RqQr/35z//OeZjN0VgGIZ/FEFV0IqhVGEgEHC7Ff1Ufx5JIBBwY6tKU0rFRxRc1LWYPHlyFUcYH3QNVCg2evRoLr74YsCb2969exMzuEqge1nRMSsesnTpUte4RGr2rLPOAjzFGwtMERiGUT0UgVYVZQoaNGjgVhg/U1W1UqdOHW6//XbASzOqtVmykpWVxfTp0wGvsGjmzJmJHFLcUS8KpczPOeccILZxH99VFlaF6667Dgj3IOzQoQMAO3bsiNr7V6XiTkExBYDS09NdgOjgwYNAWB5KIipgpkDoSSed5FKKqj785S9/6XLSMoKS1VXJtye6slDNR+TetW/fPiYbxOJ1rkFlUfdipR1r1apV6cXDKgsNwyiTauEaaLVV8VBKSoprzxVNRVAZNDY1DlGxzIgRIzjttNMAaNKkCRBe/dWFWL8XuctSikABqYKCApYtWwZ4lYV+rrwrL2orp5179evXL3XbeXWleBXt9ddfz1NPPRWbvxWTdzUMI6moFjEC1XWracOPP/7oYgTR3INfFd9ZqU3Vm//v//6vUy3FeyaUREFBgWvSqtLhefPmud2Gaukez115sbqf2pHXpk0bIHzCj06qiiZ+jxG8++67gLcLd8eOHe5ZryhlzbVauAZ6+FVN+OCDD/quCYf2DKjX3h/+8AcGDx4M4LoTh0IhV2f+1VdfAfDoo48C4e23kS4BFK1FqE7IoGs7roJmxxvaKKc6grp167o9GdFuzGKugWEY1cM1iGxIAuGtx4k8Rtuvx7+Xl0TP8+c//zmA22L9f//3f0l9LHplUR2BguB33XWX24egY9TLqwwsfWgYRplUC0UQr6PME71SxotEz7P4MeLaqx9t/K4IhHZmLl261H19/fXXA/C3v/2tXO9hisAwjDKpFoogXiR6pYwXNs+j8cNcGzVq5Dp1bd68GQif71GelHG1ORbdD9gHpCjHyzzBP3NVQFwUFBSUyxU218AwjDKJqyIwDMOfmCIwDMMMgWEYZggMw8AMgWEYmCEwDAMzBIZhYIbAMAzMEBiGgRkCwzCIc6uyUChUCOH66MjuvGWh15ZFSe9V/HerUklZ3WvwtZ07Pz8/Keep8Uf2gNSGnMiNOXomCgoKyr3XIBgMFkLlnp/iHakT0WnaVz0LIy9ARS5oNMqgrZS6bJK9FXpkL8fiPSoiqcyzUMUFpMrvEWuqRfNSUdJK4OeLb8SGmjVrumch2k0+qysWIzAMI/kVQSAQcE0edXbAgQMHEjkkI8Hk5OQkhRz3E6YIDMNIfkUA3rl/Q4cOBaBLly5kZWUlcki+pVu3boB3mtC+ffsSOZwqEwwGj4rG+y3oqfFJsUqlHDlyxDeKxRSBYRjJrwgKCwvp378/4J2V17FjR1auXJnIYfkKHZN1+PBh1/NOR8KpPfbBgwcTM7hKcuaZZwIwd+5chg8fDoQPQvEz559/PgBXXnklEFYzbdu2BaBly5YApKamlvoe77zzTpH30jF4VSXpDQF4ffAlwX744YdEDqcISmPFS67q7wUCAXcC1Nq1a4GijS93794NhI2Dn0hJSaF27doAZGdnA0UPstWx8fPnz3ev1/z8ioLZOp2oU6dO7mdpaWlA+Yvm+vXrB8Df//53ACZNmsTixYurPEZzDQzDqB6KQMeFt27dGvBWu0QTuSpHBoiiRXp6Ohs2bAA8ia+/s3TpUl5++WXAO0MQPCnZvn17IDEFN5HXpXPnzgDceOONAPziF78gMzMT8JTMyJEj3bHoOiFYv19QUMDXX38dv8FXArlmp5xyCuDJ/+zsbKcUIyshdXK21Jr+PXDgAFu3bgXg3/7t3wCYNm2aUxhVUZ2mCAzDqB6KICMjA4juahsNCgsLnTUvrw9YHPn16enpdOnSBYDf/e53QDhgpJVR7N+/H4DnnnuOQ4cOAbB3714gvLL06NGjyOviia7BsGHDeOCBBwCoW7cu4K2SgUDApX6V4szPz3cqQfEgKZlvvvkmTqOvOsX3PyxcuJDRo0cDnqotL5999hkArVq1on79+gD8+OOPlR5btTAEJ510EuB9aCr7oYslxfPFkfJY0r1hw4YuA/Kf//mfAO4DEAgE3Hvk5uYC8Nprr3HvvfcCOMmoD9Ypp5zCfffdB0C9evUA+OSTT9ixY0cMZlc+5AY8/fTTLkimOemDvX37djduHQGel5fn7q0CiPq/cvN+RmNUoFOMHTu2wgZAaPGrVasWjRo1AqpmCMw1MAyjeigCBVe0uoRCIbdq+pVgMEjjxo0BuOGGGwD4j//4D6duisvILVu2cOqppwKe1C8JrZgvv/yycyV0fW655ZZoT6NcKFj2xhtvAOGgluoWFixYAMBdd90FwIYNG5xLE4mCnAoMDhgwAIAvv/wyhiOPDsVdVgV4v//++0q/p64pwJo1ayr9PsIUgWEYya8IQqGQqyJUZWFKSorvCmWKk5+f71aExx57DICsrCzGjRsHeD6wVsonnnii1PdTXERBuN69e7t00u233w6QsDTbqFGjAC8W8sknn/Db3/4W8IJe5a2Qk8oRn3zySbSGGTMU/FQc5Pe//32l30v3WUHT3NzcqFQXmiIwDCO+pyHHosddRkYGd999N4CLnubk5Lio+65du9zrIBxNV/Q6MipdHvzYs1ArxMyZMwG49NJL3fdfeeUVAK644ooKvWc055mZmcmqVasAbxXr27cvK1asqNCYlGVQsZgyLr1792b58uUVei9R3nlC1e6p7pGeRcVqXnjhhQq/10UXXQTAnDlzgPDuUaUPS6OsuSatIdDFbdiwIQ8++CDgbUOuWbOmCxYqUKbX16xZ0wWqTjvtNAA2btxY6t/yc1PPM844A4D3338foEiLLtW4V5RoGoKJEyfyxz/+EfDSnhkZGRUK5qampjJv3jzA22wkOdykSRN27txZ7veKJF6GQEZL7uq2bduA8NgrQsOGDd2+gg4dOgCwc+dOfvrTnwKlN2Epa67mGhiGkbzBQlm/PXv2cO655wJFt3BqxVDKKj09HQgXrDRt2rTIa1JTU49ZlRgIBHxXoKTx1KlTxwUCizfnKGs7a7zo16+fG5uud1mumFTNI488AsDo0aOPmo/eI3JHpV8p/vxoX0h6erpLlUa+Ri6u5ty9e3cgrHilfuQOzJ49OyrNTUwRGIaRvIpANG7c+ChfKz8/3xXfqDeBLG5aWpor21XxTWFh4TEPXCksLIxa84eqEAgEXBGJVsyuXbvSrl07wPM/e/bsCfinXVdk2atW77S0tGO2ksvIyGDdunWAt3IGg8Gj7ovuXaNGjZzP7Xe0+itmsGHDBvcs6hnbuXOnKxBSXEQ9COrVq8d7770HwLJlywD47//+76iMLekNwd13331UFd6tt97qqreEPuiRWz/14fHLh6Y0CgsLXYBNBkG9GsFrdqEPkV+YMWMGAwcOBDypu2jRIq699lrAq5IcMmQIEK6bkBsnCgoKjtmVeNy4ca52QtuW/XY/r7rqKsDbWKTgXr169YqcxQGwfv16ZsyYAeAyK9q+fOqpp7pro9qTyu5VKI65BoZhJK8iaNiwIVA0R64Vc+HChUe1CJMiCIVCvu12W14uu+wyAPr37++Uj9+UgIhso6V70r17d1dbUNKuzOKrf+RrdM+UFm7QoAEXXHAB4PXzW7NmjW/2mvTr18+t3qqBOPvss4GwayAXSeMtLCx010np7WHDhgHh1LdUrHabRgtTBIZhJJ8i0MquQMrhw4fdKvHdd98BYb+zeMqm+EqSzKgxSSAQ8H3jzt27dzNlyhQgHLuBsCorfkJwJAqclfQarZYKuO3fv589e/YAuLTw4cOHE35dVEX50EMPuYK13r17A16F4bFQLEVdmTXX3Nxc1wA12oWApggMw0g+RaCIq5pv5uTkOOs/ZswYILzP2y8nyMTZm+1DAAAPoUlEQVSCyGaks2bNSuBIyqawsJA777wTCK+OEC4Q0v6QkgqFVNyldGPk6cZCyu7zzz93ew3kP/vhlKuJEycC4aj+iBEjAJxyKY3U1FSXLVBhkRTSWWedFTNFmzSGQPJw7NixgPcAbdy40QXPEi0H48Xnn38OwMknn8xbb72V4NGUH6UKH3jgAaZOnQrAhRdeCMBNN90EhFNq+iCrxVdGRkaRlm3guQi9evVy27m1HT0vL++YdSGxRq3iBg8eDMDUqVOP2R+ydu3abs/A7NmzgfBCJwOoD72ChZXdXFUezDUwDCN5dh/++7//OwCvv/663st9X6e+xBq/bEOOLCySq7R9+/aovX8856kdktdccw0A5513nlMCqsQrKChwiuDkk08GPPcoEAi4dmcvvvgiAJs3b3YBOh3yWlKjmljsPtTfbdasGQCrVq1yilXpQKkhVU4WR2O+7rrrAHj11Vc13vIO9yhs96FhGGWSFIqgRo0aroeAYgNz584FYNCgQVEaXdkkWhGoLbmCTgUFBW5FjeZ9TOQ8a9So4eapuRUUFDi/uW/fvgD85je/AcJ7TV577TXAa2S6fv161q9fD3iqoqQCo4KCgqgrAhVK6SSiI0eOOPVanh2hCxYscAVS0TynI6kbk+gC/vKXv3TySNtPJR/jWR2YaEOgJiTacLJ27VqXPYkmiZ5nCX+nSGUo4A5KTUtLc0HIyG3O5XmuY+EaaJzKiowZM8bVFBQnOzubpUuXAp7rG6tem+YaGIZRJr5WBNpePHXqVLca9unTB0hMqjCRK2UgEHCNOrQtdeTIkU6KRhO/KYJYEa9WZVKvUiyJ2AdhisAwjDLxXUFRKBRyOwt/8pOfAOE97U899RSACwIdL6hwpm3btrRo0QLwgmLJcMqPETu/P5qYIjAMw38xglAo5DrwFD/TMNEk2nfWdVHkPFbHwCd6nvEiXjECP+Cr9GEwGCyE0j/YkY0p/IZ9QIpyvMwTqv9czTUwDCO+isAwDH9iisAwDDMEhmGYITAMAzMEhmFghsAwDMwQGIaBGQLDMDBDYBgGZggMwyDO25CTtV5b7afK2+MumeYZ2f9fW57z8/Or3Twj0aatvLw822vwL3zXj8CPVOcy7JJOGq7uHC/zrAjmGhiGYYbAMAxzDYxqSvHj1kuKhRgedkUMwzBFcDxRo0YN2rRpA8A333wDJH/gLBgMuhOE6tevD4TPFFTj282bNwO4E5MPHz6c9HOOBaYIDMMwRQCQkpLi/EatFmqcWh1o1aoVAB9++CH79+8HoF27doC/56lmrf/zP//DeeedB4SPOIOiMQAdHCKCwaC7jwsXLgTCx+YZx8Z3hiA1NZVzzjkHgA8++ADwDv2MBoFAwB1H3aFDBwA2bdrEDz/8ACSfVNZcdHbe9ddfD0CdOnXcmYA6jjslJYUtW7YA/jUA9erVc6c36Qj0ihIMBp2h0HHxOlugOteEVAVzDQzDSLwiKH5k9MUXX8wNN9wAwOTJkwGYM2fOUfKvouj8uW7dutG1a1cAlixZAsCGDRt8owQU8LrmmmsA72Snt99+242xR48eAMyePdsdIV6cwsJCtwpGHp0+ZsyY2A2+EqSkpABw/vnnAzBlypSjlEBBQcFRc4koE2bTpk0ApKenA1C3bl2nKq699lrgaPchnsjFadq0KQAZGRkAbN261bk6Okb9kksuca7c6aefDuBOTB4xYkRU1XEkpggMw0j8SUdSBHXq1AHgySef5NxzzwVg586dANx///0u6KOTZA8ePAiEfd3icwgEAs7qKt7Qv39/ALKysrjvvvsAOHDgQIXGH6uDP3QNOnfuzJw5cwDv3EepgK1bt7rVPzMzE/BWmmJjBMJq58cffwTgoosuAsKrp2IKhw4dOuZ44nHASe3atQG48MILARg/fjwQjnnovkydOhWAxYsXu9cr3tG+fXsA5s+fz4cffgh49zMvL69cCi+WB5xIsUyaNImxY8cCnvqJDHTq6+IFUCWxfft2FzRdvXo1UP6Yh+83HWkiuhDdunWjQYMGgCehLrjgAieJPv74Y8B7kAKBgPvdU089FYDbbrvNPfB6Dx0PNnz48AobgGhQ0g3XB1kS8KWXXnLysXj1W6tWrUp9WLKzswHvA7Jx40YaNWoEwMCBA4HwgyhpnUhq1qzpgpqS7vrgjBs3jnnz5gGe0U9NTXVZDkl+Gcy9e/eSl5cH+CsQeNVVVwFw8803V/maa14NGzZk+PDhQPgZj/xZVTHXwDCMxCsCoWDOli1bnCzWUeiPP/64W8Ul+bRi1qtXz8nK6667DgivLu+++y4A77//PgBvvPEGEA4MJgLJwhNPPBGAjh070qlTJ8BL/Unyw9GWPi8vj3379gG4f8ePH+9SrNu2bTvqb6pmIBI/BEVzc3NZt24d4LmEOuL9vffeO+pw10Ag4IKFGzduBEp3bRKJ5jNy5EggfL2Lq15RUFDAd999B8Do0aMBWLBggbtHCnDPnj0bgH79+jF06FAAFi1aBMA//vGPqNxTUwSGYSReEWhll4/46KOPsnbtWgC3apTkB0lB7Nmzx6UZ5XO3aNGCF198EfCCKlpJEoVWA8UAhgwZ4vz55s2bA+GVUiufrouU0IQJE3j55ZeB8h2HXqNGDaZNmwZ4aiQnJ8fFEhKN7ovuu65BZmamUzy6Bunp6e7eKm7gRwKBAHXr1gW8mE1ubq6bo1ZuPd8XX3yx+7ok9CzcfPPNQLgyVLEvPd9t2rRxQeGqYIrAMIzEK4LIwhAI14ar/LU8EdHCwkKXZrz99tsB6N27t/u5VtRER5Q1JxUIbdmyxRWOSC1s3ryZNWvWAGHfD7zoeEn+fmk8++yzDBkypMj3li9fntDCmkh2794NePM66aSTAJgxYwZ33HEH4CmCQCDArl273Nd+RmqtYcOGQNHCLsWrLrvsMsCL9ZTF119/DcCgQYP461//CuAyQg899BBXX311lcedcEOgD2hWVtZR36soqiybMmWKMw6qpItVRVZ5kSzUOObMmUPfvn0BT7qvX7+exx57DIBPP/0UqLgUVoXm4MGDnZHVw3nffff5IlgIXrBPATFdgx49evD6668D3lbp6dOnu23Efkep4JYtWwLhVKme7XfeeQeofKBz0aJFzphceumlQNioqE4h8jNUUcw1MAwj8YpALkFJRLaXKg1JyOeeew4I7yrUqlJRSR1rtMKvXr2amTNnArgikddff90VzJR2XUpDQbWsrCy3+/C9994DPGnqBzQ/jWnw4MFA+F4rjao9FZ07d+arr74C4M477wS8NPDatWt9o3LAW5WlxnJzc91eiAULFgCeOgwEAhVSv4WFha4QSxWZKSkpTgWaIjAMo0okXBGURmRZrixn8eKM+vXrM336dMALEm7evNl9LxHlxOUhKyvLqRb5latWraq0EhBSR4sWLXJtyYoXYfkBjem//uu/AK80umXLli7Wo3uclpbGz372MwCeeeYZAL799lsgvENVvneiU4uFhYWuSEzKLDs726V9FfepioLR7+o5idxXU5U4mK8NQaSE0kOsSV988cUAjBo1yu0x0If+7bffdsHCqn6wYokCYMqpt2rVylXYVTRgqqBbnz593Htpk5IMzSmnnMLKlSurPvAo8tlnnwHw6quvAjB06FAaN24MeBvLUlNTndSWDNa23UcffZRPPvkEgLvvvhsIZ2SUx493tkhj0fN65MgR/va3vwFVN1ShUIhBgwYB3l6brKwsd5+rUivjnyXCMIyE4WtFIAoLC13+W7lX5VP/+te/0rNnTwDeeustIFy1N3HiRAAXMPMjWq20elx++eX885//BMpXPVi7dm2XQ1Ztu9JWoVDIrUCqPOvcuTMfffRRkb+daLTCa5t4enq6CyBqu/iuXbucItBeje7duwPQq1cvd//VwCMlJcXt/lOdfryQYhULFy50wcLyolSqVn01bZkwYYLbhalnJicnp0pBQmGKwDCM5FAEZaEdeFr9mzVr5laaZOD5558HYNq0aW5lfPPNN4GigSWtiooDPPHEE7Rt27bIz7TS5+XlOT9ZimDNmjW+UQJC41fDkX379jmfesWKFUBR3/qLL74AvN13derU4YknngBwO/NCoVCRnZzxZP78+QBOec2ZM6fUax4ZEAXo0qWLa8iiakvFAAKBgHseVGl50003uWrVqmCKwDCM6qEIhHZyNW/evNx13H5gx44dANx6662uJLr4Dkzwov+Kjjdr1sytKIqhaPWJ3FOgbIrSV35CBTJa6d555x2nCEpruR6pfFTXr2uxb98+XnrppZiNuTQ0LmW1+vbtS5MmTQDv1CW9JjU19ag29GeccYbLABVP9xYWFrr27Pfeey+Au1ZVJeE9C6OJAknbtm1zrkE0c8vx6OWnB0jtxZSPPnDggNvi2rlzZyAsHdUIo/hDEwgEXFXluHHjAHj55ZfLu5Er5vOUFN66dSvg3aeuXbu6MyYiKe7qKcX46KOPuj0b2tzTqVMnt6mpNGLZs1C0b9/edeWWW6PGO9dcc43bli7XriRk1FesWOEMh+YXrZ6F5hoYhlG9XAN1sw0EAk6OJcuuNTF37lzAq6CTyjly5Ihzd7SapqWlldjJGMKryLJlywBcBaOfAoVK7ykQptW8pFRbx44dXUNTraaRK6gUhFKK5VED8SIzM9OdUVFSALs8DWnvueceAB555JGYFciZIjAMo3rFCBQUy8jIcD5zNOcXD99ZSNGoph68VVDtsSPjApHBM4B3333X7Vmv6A7MeMxTPr4CaIoRNG3a1N1HFYidffbZx1w5s7OzXZOOWJ1TAVWbqwK/rVu3BrzA6I4dO1wxkO7pV199xZQpUwBv/tF4hsuaa7UyBJKI6enpLpIcTeJpCES3bt2AcAWlHiQZhEAg4CLr2qOgXPq3335b6W5E8Zyn8u5nn302EM4UKGpe0odfz+usWbMA+NWvfhXzef5rLJWeq9wZ9ZB84YUXAM8NjAcWLDQMo0yqlSJQgG337t2u7100SYQiiHhPJ6clgbOzs2MSPErEPBUAnTx5smtSoorIG2+8kcWLFwPRPcw0XorAD5giMAyjTKqFIjjzzDMB3EGpY8aM4dlnn43630mkIognNs+jqe5zNUVgGEb1UASKLqtIZfr06TEpnrGVsijHyzyh+s+1WhiCeGEfkKIcL/OE6j9Xcw0Mw4ivIjAMw5+YIjAMwwyBYRhmCAzDwAyBYRiYITAMAzMEhmFghsAwDMwQGIaBGQLDMDBDYBgGZggMw8AMgWEYmCEwDAMzBIZhYIbAMAzMEBiGgRkCwzAwQ2AYBmYIDMPADIFhGJghMAwDMwSGYWCGwDAM4P8BEyxrQvKPdAgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x288 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 40s, sys: 30.8 s, total: 3min 11s\n",
      "Wall time: 3min 51s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "train(train_dataset, EPOCHS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rfM4YcPVPkNO"
   },
   "source": [
    "Restore the latest checkpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 702406,
     "status": "ok",
     "timestamp": 1554225743900,
     "user_tz": 420
    },
    "id": "XhXsd0srPo8c",
    "outputId": "ab3fb7e2-c445-4d1a-bf7f-7761891f0058"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.training.tracking.util.CheckpointLoadStatus at 0x7f60b433d5f8>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "checkpoint.restore(tf.train.latest_checkpoint(checkpoint_dir))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "P4M_vIbUi7c0"
   },
   "source": [
    "## Create a GIF\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WfO5wCdclHGL"
   },
   "outputs": [],
   "source": [
    "# Display a single image using the epoch number\n",
    "def display_image(epoch_no):\n",
    "  return PIL.Image.open('image_at_epoch_{:04d}.png'.format(epoch_no))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 305
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 702651,
     "status": "ok",
     "timestamp": 1554225744157,
     "user_tz": 420
    },
    "id": "5x3q9_Oe5q0A",
    "outputId": "b37b81c4-3648-44b9-ac25-d8124600ad23"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASAAAAEgCAYAAAAUg66AAAAxaUlEQVR4nO2deWBV1bX/PzeXzAkEsCKzTDJYCgjIoCIVUFGLWJ+2yCB1AlSKWhSH8rNVH86CWkGtUK2or+gDhdeqRQaRoCAqoqIoIAqEIchMApnu74/Ttc/NJYSb5N67z81dn38Iyc3J3mefs/Z3r73W2r5AIBBAURTFAkm2G6AoSuKiBkhRFGuoAVIUxRpqgBRFsYYaIEVRrKEGSFEUa6gBUhTFGmqAFEWxhhogRVGsoQZIURRrqAFSFMUaaoAURbGGGiBFUayhBkhRFGuoAVIUxRpqgBRFsYYaIEVRrKEGSFEUa6gBUhTFGmqAFEWxhhogRVGsoQZIURRrqAFSFMUaaoAURbGGGiBFUayhBkhRFGuoAVIUxRpqgBRFsYYaIEVRrKEGSFEUa6gBUhTFGmqAFEWxhhogRVGsoQZIURRr1LHdgIrw+XxRuW5SUnl7W1ZWFpW/EwgEwvqctCfcz3uNcNvt9/uB8O93ReNv8x7FYjylz/Jv8DVi1Xcb99iTBihayA0ONUS2iFfDU1WqaugjeV+CxzpaE45Qk3bL74oBknZHu822SSgDJIMq/5aWltpsjhIDTj31VACKi4vZunUr4G3Dn5ycDEDz5s0B2Lt3LwUFBYBrjIqLi2uNYfKGFFAUJSFJGAXk9/upU8fp7tGjRy23xh4i8eVeFBcX22xO1JB+5ufnA86Ye1n5CKmpqQD07NkTgB07drBt2zbzNdSuMVMFpCiKNWq9Arr88ssB6NOnD1OnTgUwM0oi0bZtWwD+9a9/AXDSSScB0LdvX7755htr7aoJonLEb9K5c2ejbjdv3gzA4cOHAe87c0X5DBkyBICRI0cCMGvWLL799lsASkpKAG/7sKqKKiBFUaxR6xSQ7HC98847AAwYMABwZsLc3FwA5s+fDyTOLpjP5+OUU04BoFWrVoDrA5o3bx4dO3a01raaIGM9Z84cAAYPHszXX38NuON+6NAhO42rAj6fj/bt2wNw4403Apidr9zcXOP7qY3Pa60zQDNmzABg0KBBgCtXX3vtNVavXg3AWWedBUDXrl0ZMWIEACtXrjSfA1ixYkXsGh0lZIni9/vNQxwa5FevXr2YtytSpKenA3D++ecDkJKSQmFhIQB79uyx1q7q0KZNGwBycnIAeP/99wHH8VwbDY+gSzBFUaxRqxRQWloaV111FeAqnx9++AGA8ePHG0dky5YtAbjrrrto0KABAN27dwdg7NixANx///3cd999sWt8hPD7/cahmZGRAThLFdnWDVVA69evj20DI8jJJ58MOMoHnDG/8sorzdfxQiAQMOpclOr27dvNz2ozqoAURbFGrVJAl1xyiZn99+/fD8DPf/5zAIqKisznli9fDkCzZs3o0KEDADfccAMAv/3tbwG4/fbbzfbu5MmTY9D6miHKJisri/r16wPu1nPHjh2NMgxly5YtsWlgFBDfj/T96NGjcdsf8Vn9+9//Bsqr16r4gJKSko5JNfKyiqoVBkik+JQpU8zNl7wf2U2oiNLSUr766isAJkyYAMCbb74JOLtot99+O+DEygDcdNNNZsnitUEVY9m4cWNjhMX4Dho0iN69e5f7vDycH3zwQQxbGVn69esHuAZozZo1FltTMyQuq0mTJoA7PllZWezbtw+o/JnLysoC4JxzzmHXrl0A5tkuLi72rCNbl2CKolgjrhWQqJ2rr74acByw4rwbM2YMUHWlIrFCRUVFJlZGHLhjxoxh4sSJgHdiMmT2l+3brKwsBg8eDLihCE2bNjX3QfKIZHm2YcOGWDY3okj/BFk+xxtJSUmcd955gKvq5Nlevny52ZI/cOAA4IyhjHt2djbghp/079/ffP75558HYPXq1Rw8eDAWXakyqoAURbFGXCsg2X6Vf1esWGECCletWlWtazZq1Mh8LWrhiy++AGDq1KmeUT6COCsvuugiADp16sRpp50GYKKfU1JSjC9MlJDMpt9//31M2xsp0tPTjd9DxiRec/ySkpKMkpE+CT169DChJME+HUGCMcVPmZOTY66xc+dOwM2H8yKqgBRFsUZcKyDJfH7llVcAp/qd7H5VFalbLHlFGRkZxj8i6kK29r2A+AAuvPBCAP785z8DTsa0+K5E7aSlpZn+yW6Z/P59991ndgDD2W2xjbR/woQJpg/S7q5du5qAvnhA2t+oUSPjAwpNjUlLSzMqNbiOlSjfe+65B3DVbp06dUwlAKly4OVKAHFtgORF2b17N+AMlsT7yAt3olgIcfa9/fbbAOW2q//2t78B3jI8QrNmzQCnXAO4D+Thw4eNwzH4wROpLsZJPn/VVVdxwQUXAHDdddcBjuPTK8Yo1NBee+21gLPUlLGVl3b58uV89NFHAIwePRpwC5KVlZWZax05cgRw+majf/LMSbzWgAEDzNJfSm6IIX3kkUeOyWurU6cOt912G+DGrwVHgz/zzDOAtw2PoEswRVGs4QvYnuIqoLrH8jRo0MAEdIlqkdnD5/OVi4aW791xxx0APPjgg+X+dmFhoXHmRetUh5ocP/Tuu+8CbtkJmTnfeustfvaznwGOSgBHIcgMGXr8C7gqUdTC2rVrGTp0KIDJLq+IWPRz7ty5gFuoS9r65ZdfmiVJ165dAUcFyM/FyV5RGd7gZ2PgwIFA5Y7aSPYzJSWFbt26AXDZZZcB0KJFC6OG9u7dC8CkSZMAyMvLM8+fBJj269ePmTNnAq4SFoqKikhLS6tSuwUrajDmf1FRFOU/xLUPKJT9+/eb9bXMaGLV/X6/UQYyQwwYMMCkW4QeCDdp0iTPrqGTkpLMLCrtFef7O++8w/DhwwHIzMwEXP9A8OdFKezbt884K8WR2alTJ5MSsHHjxqj2pTLq169vgg1DD/3Lz883NZs+//xzwEljkAJkUuBL/s3OzjZjLM/B4cOHzbE969atK3f9aOH3+02fRNWtX7/e+Hw+/fRTwH0e69evT+fOnQHXD1ZZAbktW7ZY99tVhVplgEpLS43hkZdPXrTs7GwTLdynTx/A2UGQ+AsxNhJz8eyzz8as3VWlefPmpt2yrAw2FI0bNwZcyR68NBAHrCQ9jh071vz8hRdeAByJL85cqSUdS0e8/M1nn33WbCZIu6W/xcXFJh5m9uzZgLMsk/GW6HVZqkgJFnCXq5s3bzb3KFZnxZWWlpo+ydlfLVu25OyzzwbcGB+ZJFNTU834yLLy008/NZsEMtbyGYmDixd0CaYoijVqlQICd6aUJYTEjWzbts3MGuLga9KkiZk5ZFaUmCIvn73Uu3fvY+J6xOH8y1/+0qijYCkeXJ41+PdycnKMxD/nnHMAJxpXHPBPPPEE4G5/R4ukpCRTD1nGZ9euXUybNg1w47PEKZ6dnW1UgiwhDx48aJzO0k9ZVgZvQrz++usAPPfcc2bpFcsId1HbMgYpKSkmLCKUQCBg2iaqvlevXqbvMq7ymTfeeCN6DY8CqoAURbFGrdqGPxEy40gNnDPPPNP8TJy4UppVtqSrQ7S3p1NSUli6dCng9iE4KFP8GnXr1gUcdRE6U0pu2I4dO4xTVgL6gtsl51OJnyWYSPbz/PPPP+a0kvPOO8/k9IX+Lb/fb1SO/Ovz+ejRowcAL730EuD6fsrKykwlg6effrrc3zkRkR5PeQ7POOMMAK6//nouvvhiwM0FE7W+efNmo46keF5qaqpRURJeIPduxIgR1XZC6za8oigJRa3zAVWGzBrBYevi63n00UcBN63DyxQVFTFs2DDA3fUQFZOVlXVMGkpSUpKZ3WS3R2bV5s2bG99I8MmbS5YsAdxjiqLNiBEjTLuDAyIrm5VlV1O20gcPHsyoUaMAN0BP+vTcc8/x5JNPAvbTS+SZk7FbuXLlcdVTUlKS2a5/8cUXAef5lV1KqYGUl5cXzSZHjYQyQCJ95eSB9u3bs2nTJgAWLlxorV3VQZaMv/vd7wD34SwoKDB5X8ERseKcFSeulGqYPHkyixYtAtxt3qKiopjFQMmL16ZNm3IhE+Asy6R+txjY8ePHA46xkc8FO3PFuEhtaDklw+tJqscziqWlpSxbtgzAlFotKSkxEdyh0f3xhi7BFEWxRkIpINlmliC10tJSI9FlK/rHH38EKi9m7wVEoUgRLjnDbNu2baZIv2zb5uTkmAz5Dz/8EIBPPvkEcMMWbCEKqG7dukbJiBN97ty5FeauhSL3orS01PRPlI8ovXhG7oeEFHz00Udxr3wEVUCKolgjoRSQzIqieg4cOGAcgbL9LkXtc3NzrTsrK0PaJuVi165da7M51UbUy913321SQSQz3O/3H1f5BAIBkx4ifqKJEyfy3XfflbtubSD0qB7JX6wNJIwBSkpKMnle4pBcunSpOQVDYmDEQe3z+TxtgIR4aGM4LFiw4Ji8puTkZM466yzA3bkUQ5uXl1dr+n4iZFdQYqIkers2oEswRVGskTCR0G3btjVlDKSYV7QcebEo1OUFtJ/liVY/ZUkqKl1qPkcajYRWFCWhqPU+IMlluuyyy0zR9dqyhakkBqJ8KiuPG6+oAlIUxRq11gckOU8Swr9r166YrXFt+wxihfazPNHqp+TISWhBtJ5jG6ag1hogm9h+YGOF9rM8idLPSKJLMEVRrOFJBaQoSmKgCkhRFGuoAVIUxRpqgBRFsYYaIEVRrKEGSFEUa6gBUhTFGmqAFEWxhhogRVGs4cls+EQJaU+UfkpeXrzGvMZyPEOvEct7pqkYiqIkFJ5UQErtIl6Vjw3kXsW7Og4XVUBKXOHz+RLm5UwE1AApimINXYIpcYUu52oXqoAURbGGKqBaiJTwzM7ONv+XU0TlVFjFm0gJ4QMHDgBw9OhRm82JOqqAFEWxhiqgWkZaWpo5775Tp07m+3KY3YABAwD48ccfY9425fi0bt0agE8++QSARx55BIAHH3zQWptigSdLsnp1m7VBgwZkZGQAztnk4J5UEIzNSOiWLVuyceNGwF2KBSNLMDlfvFu3bhX2IRy8EvEdfGpENB7nWPRz+/btAJxyyikA7NmzB4Du3btz+PBhACZNmgTA0KFDqVu3LgCDBg0C4MsvvwSgtLS02m3QSGhFURKKWr8EqygPKTU1FYAOHTrQv39/APr06QPA1q1bAfjb3/5mVE7nzp0BaNeuHf/85z8rvK5tISmz79y5cytUPtK+OnWcIZc+/f73v2fatGmxaWSUkP76fL64dLJnZWVx8sknA+44iRP6kksuYcqUKYC7qRDMmjVrANi7dy8AzZs3N4opHlAFpCiKNWqtD0jWyM899xwA7du3Jysrq9zPcnJySE5OBtyZp7i4GHDO4f7pp58A+PbbbwHHISgOXplpK7p9NnwjEydOBODhhx826ky23ufMmUN6ejrgzKjgzqYFBQX069cPcGfTcPGKD0j8cr169eKjjz4CInuOerT7+de//pVrr70WgB9++AGACy+8EIAtW7bQqlUrAK688krA8fdInwsKCgD3/Pjt27fz7rvvAlUPudCTUf9DTR7YzMxMwH2ZZPBKSkqMQZk7dy4Aubm5dO/eHYDf/OY3ADRu3BhwZL3EYKxevRqAcePG8dVXXwGVD1YsX8yGDRsCkJ+fb65ZVFQEuMvKNWvWmKXXfffdB8Af/vAHwFlKLlmyBHAdmuG23ysGaMyYMQDcfPPNPPTQQwAsXLgQgJSUFMCJpxGDLPcnXKLVTzEi+fn5pp2XX345APPnz6/StZo2bQrA2rVrTd+vvvpqIPxYInVCK4qSUNQ6J/SiRYsAaNOmDQAHDx4EoGPHjsapHMwbb7wBQLNmzQC44oorzM/EmScxGevWrbPubBZktt2yZUu5/2/dupXTTz8dcB2ZPp/PbO/27Nmz3OfB7btE4ebn53umn5UhaldUT0ZGholzkvE/44wzAHjxxReN0pNltu0+PvroowB8/PHHDBkyBHDHrKrs3LkTgPT0dKN8RWF5OZpaFZCiKNaoVQooJSWF9u3bA67V79ChA+AGeoUis/+5554LuNvrAOvXrwdcf0J1A/aiQZcuXQA3pED8Gr179zazqPTljjvu4LbbbgMwjni5P2VlZcYRL/6Hl156yTg3vUz9+vUBd1yOHj1qnLgS+S2+wEWLFpk+2VY+gqizW2+9tdrKR2jRogXg+C7lmRD172VUASmKYo1aoYDEn3HNNdeYUPQ//elPwPGVDzhBeeIDkkAwobS0lKeeegqAI0eORLrJNebpp58GXOXz9ttvA04Iv6z9H3vsMQBGjhxpdllkppWdvZYtW5qf9erVC3D8JfFATk4OAF988QXgpJXILubs2bMBN+dN/D5eQMana9euNb6WBGH+8Y9/BJxnWsJG4iEoM64NkCwxnnjiCQCuuuoq870dO3YArnEKlt3ymUcffdQs2YKXXuDEVyxbtiyKra8+gwYNMsZCHkD5/5QpU4wxlZdxxYoVvPfeewB8+OGHgLNUA0f+i2T//PPPAW87LYOReC4JpcjMzDQR3pJL5cWXUAxmWloa4LoBqoI815dddhkAw4cPB5zneMaMGZFoZkzQJZiiKNaI20DEtLQ0FixYAGAieX0+n3FIyjVkiZKUlGS21WXJ4ff7jYKQ2UiWcKNGjeK1116rVvujFbjWqFEjwFleSA6btF/aXVJSwuLFiwF4+eWXAWd5JksQUUWvvPIK4CwDJDP+7LPPBgjbAW07EHH8+PEAPPnkk+Z7v/3tbwE3vCISGweR7GedOnXM/RXn/yeffGIUqYyjBI5efPHFfPfdd4C7jMzLyzNO5+XLlwOuQ37//v1maVpVNBBRUZSEIu58QDLLtG7dmnbt2gGu/yZ4S1m+J2oHXOefzDKHDh0yOUPiaH7++ecBqq1+ooH0uUePHoAzy4mPQ3K8RAkdPnyYefPmAW7KSUlJCQ0aNADgrrvuAtxtfJ/Px+jRo4HwlY8X8Pv9xu8h9+fgwYOm714KmQjG7/eze/duwFW0TZs2NVUWxJ8lKTbBSJ/WrVtnnmHxg8n/pbBZvBB3Bkj4/vvvzdJLHI8ZGRm0bNkScCN+xeicdNJJZrDkRWvVqhWbNm0C4JZbbgEwyYxeQl4wqZa3cuVK0y9xsspSs6ioyMR/iKROSkoyUcMXXHAB4BrmjRs3snbt2lh0IyLI0mTQoEHGiMqLOWvWLE/tdlVEcXExAwcOBGDmzJnmezKZhkYvJycnG+Mi+X7Z2dkm+VTGePr06YDrfI8XdAmmKIo14k4BicUvLCw0MR7h1jcWFSDlK+vVq8eqVasATJkNLyIKSKR7aWmp+TocysrKTCZ48JIUiJstW7kHJ510EgDDhg0zfZHldjwsIcvKykyE/eDBgwEnml1CIWScZMOktLS0nJIFp8qBqCf5vb/+9a8x6kFkUQWkKIo14k4B1QSZIWXL3efzmex5D0YjAE4bpW01KTgu/i9xWsu9mDp1ag1bGBvkHkiA6dixY7n00ksBt2/79u2z0raqIuNY1faKrys3N9cULBPlfs455wCuuo8XVAEpimKNhFJAMovKzleDBg3MjOplaqrOsrOzufPOOwF3u15KuMYrBQUFzJo1C3ADEufMmWOzSTFF6kBJ2Ml5550HxI9PT4jbSOiacMMNNwBOjeeOHTsCsGvXrohdvyaRs/I9cS5mZmYa5+OhQ4cAR4qLHBdHrDjYW7RoYbbmJVr617/+tYkrEeMry5eaxMvYjoSWomOyjO7QoUNUEodt97My5DQM2b5PS0ur9oSlkdCKoiQUCbUEkxlKgg5TUlJMGdJIKqDqIG2TgmESZDdq1Ci6desGQJMmTQBH7cipFvJ7wVn/ooDE2VlWVsaKFSsANxLaq5HCVUHK50o2ef369Sstv1IbCY34v/HGG3nmmWdsNqlKqAJSFMUaCeUDktwbKdj0008/GR9QJGvg1MRnICECkhP0v//7v0alhdYsqoiysjJTfF9SLBYuXGiy3+VoIq9liVcHyRJv27Yt4JwKKifbRhLb/ayM999/H3ArQuzatcs851XFhilIqCWYvHQS/fzwww97rviW5HRJLeM//elPDB06FMCcduH3+00u0Ndffw3A448/DjhlKIKXXlA+lqg2IROJlKYQh2wiIcnTEgdUt25dky/nxWJsoegSTFEUayTUEiy4EBk4JTii0X0vS/ZIYrufZ511FoApNfJ///d/CTeeEgckGyv33HOPyRObNGkSEL4S0m14RVESioRSQOLElS5Hq+tenjEjie1+SvF9aYfUy4k0tvsZDlIlIDc313x94403AvCPf/wjrGuoAlIUJaFIKAUUK+JhxowE2s/yeKGfjRo1MlVBt23bBjhn44UTdmHDFKgBigLx9MDWBO1nebzST9lkEcrKysLqgy7BFEVJKDypgBRFSQxUASmKYg01QIqiWEMNkKIo1lADpCiKNdQAKYpiDTVAiqJYQw2QoijWUAOkKIo1PFkRUULJy8rKyhVbPxHhhsJXdK3Q361JfGa8he5XFakqEO5JrV7rp7Q/uMSt5EoF50xJu8MtXxtabaEqhB4uYOPQAC3J+h+Cb35VbkokbqAGhp+YeD9RI7hUbWVGo6rPQiQmrUR7/jxpgKJNRTNfog284lQTlGchHuon10bUB6QoijUSRgH5fD5TP1eOvjl48KDNJimWKSoqStilj1dQBaQoijUSRgGBeyzx8OHDAejatSsFBQU2m+RZevToAbiH/+3fv99mc2pMUlLSMTtMXnKmS9tEnYsiO3r0aK1WZ6qAFEWxRsIooEAgwMCBAwH3KN9OnTqxevVqm83yFHKi5pEjR0wslpwcKyctHDp0yE7jqsnZZ58NwIIFCxg5ciTgnB/mVS688EIArr76asBRbu3atQOgVatWAKSmplZ6jffee6/ctcKN17JBwhggcI9xEbn7448/2mxOOWQ7OFbLAvl7Pp/PHNi4YcMGoHxN4T179gCOUfISKSkpZGRkAFBYWAhQ7pjt7OxsABYtWmQ+L/3zIrJBIocJdu7c2fwsPT0dCD+gc8CAAQD885//BGDKlCksW7YsYm2NJLoEUxTFGgmlgPLy8gBo06YN4M7utglWIcHOx0iRmZnJ5s2bAXcpJX8nNzeXV199FXCPOAZXtnfo0AGwE6gXfF+6dOkCwM033wzAL3/5S3JycgBXuY0ePZqtW7cC8PzzzwPucdxlZWV88803MWt7VZHl72mnnQa4y6zCwkKjioOjtouKigBXmcq/Bw8eZPv27QD84he/AGDGjBlGUXnJ8Q6qgBRFsUhCKaCsrCwgsuoiEgQCATODVTdxU/w2mZmZdO3aFYA//OEPgOOMFCUgHDhwAICZM2dy+PBhAPbt2wc4s2mvXr3KfS6WyD0YMWIEDz30EAB169YFXGXg8/lMCIWECpSWlhpVJP4+UW7ffvttbBpfQ0Jz05YsWcLYsWMBV8GHyxdffAFA69atqV+/PgA//fRTpJoaERLKALVo0QJwX1avZWnDsRG5wcsQWSI1bNjQ7Oj9v//3/wDMi+fz+cw1iouLAXjjjTe4//77AYw8lxf6tNNO44EHHgCgXr16AHz22Wfs2rUr8p0LE1luPfvss8YBK30Sg7Jz507T7pkzZ5qfydiKY1r+L/E1XkXaJ85zYfz48VU2PIJMuGlpaTRq1AjwngHSJZiiKNZIKAUkjjuZTf1+v1EJXiUpKYnGjRsDcNNNNwHwX//1X0bNhUr2vLw8Tj/9dMBdUlWEKIRXX33VLNnk/tx2222R7USYiCP2rbfeAhyHqcQdLV68GIB77rkHgM2bN5ulYzDiPBeH86BBgwBYt25dFFtec0LdArJp8MMPP1T7mnI/AdavX1/t60QTVUCKolgjYRSQ3+83Uc8SCZ2SkuK5ALtQSktLzSz4xBNPAFBQUMCECRMA18chyuCpp56q9Hri9xLnbt++fc3W7J133glgbbt6zJgxgOvr+uyzz/j9738PuA7VcKN6RdUJn332WYRaGR3EoS4+rj/+8Y/VvpaMsTjii4uLPRsNrQpIURRr+AIeTLWNxu5UVlYW9957L4DZESgqKjK7SLt37zafA2d3SHZjgndZwsGLNaHlb82ZMweAyy+/3Hz/tddeA+Cqq66q0jUj2c+cnBzWrFkDuDN3//79WbVqVZXaJLtmEmQqO4h9+/Zl5cqVVbqWEIvxlN+V51D8cC+99FKVr3XJJZcAMH/+fMCpZCDb8JVhwxTUegMk12rYsCEPP/ww4JbjSE5ONk5occDK55OTk40DtFu3bgBs2bKl0r/l5WLtZ555JgAffvghQLlSpJKHVFUi+WJOnjyZP//5z4AbPpCVlVWlTYLU1FQWLlwIuEmoMhZNmjQhPz8/7GsFEwsDJIZSXAI7duwAnHZXhYYNG5q8r44dOwKQn5/PKaecAlTeFxumQJdgiqJYo9Y7ocWq7927l/PPPx8oX85AZkjZ+s3MzAScQLemTZuW+0xqaupxo6h9Pp/nAhulPdnZ2cbBHFqU60SlHWLFgAEDTNvkfp9oySsq7rHHHgNg7Nixx/RHrhGc4e9FQp8dydnLzMw04QbBnxE3gvS3Z8+egKPuRenJsmvevHmeLWqmCkhRFGvUegUkNG7c+Jj1dGlpqQnak9pAMsukp6eb9AYJ2gsEAsc9KDEQCHhiq9Pn85kANFEI3bt3p3379oDrY+jduzfgnezo4BQBUSvp6enHLZmblZXFxo0bAVctJCUlHTMuMnaNGjUyfhUvI2pHfEKbN282z6E8X/n5+SawUHxeUgOoXr16fPDBBwCsWLECgL/85S+xaXw1SBgDdO+99x4TNXz77bebiFNBDExwGQR5ab3yslZGIBAwjlsxRFILG9xCV/LyeoXZs2czePBgwF1WLF26lOuvvx5wo7qHDRsGOHFPslwWysrKjnvKxYQJE0zsk5Tv8NJ4XnPNNYCbcCpO43r16pU7xw5g06ZNzJ49G8DsEkoZj9NPP93cF4kbq24uWSzQJZiiKNao9QqoYcOGQPkYF1EIS5YsOaYUqiggv9/vydMTqsIVV1wBwMCBA43S85ryEYJLhsqY9OzZ08QGVVQlIFTtBH9GxkzCKxo0aMBFF10EuDWT169f74lcwAEDBhi1IvFL5557LuAswWQZKm0NBALmHkmIyIgRIwAnfEQUu1Q+8DKqgBRFsUatVUCiZMRJd+TIETMrfv/994DjVwjd/gydOeMZKUjm8/k8XZAdnJl/2rRpgOObA0eFyvhUFOIgTtmKPiMKQZy5Bw4cYO/evQAmvOLIkSNW74tEfD/yyCMmyLVv376AGxF9PMRPJid8SD+Li4tNYXuvbr0HowpIURRr1FoFJLsIUlS9qKjIzHbjxo0DnFor8TBLVJfgIvNz58612JITEwgEuPvuuwFHEYATWCj5exUFGEpQqGzbJycnH7NjJEr2yy+/NLlg4iOxfSru5MmTAWeXatSoUQBGpVVGamqq2f2SgERRg+ecc05cqfdaZ4BEho8fPx5wH9wtW7YYp6zXlyOR4ssvvwTg1FNP5Z133rHcmvCRLfeHHnqI6dOnA3DxxRcDcMsttwDO9rQYEClnmpWVVa40LbhLsT59+piyJlKWpaSk5LhxXdFEyuEOHToUgOnTpx+39nZGRobJ6Zo3bx7gTK5idMXYiBO6ugm3ttAlmKIo1qh12fC/+tWvAHjzzTfLXetXv/qVOSky2nilHEdwQKIsSXfu3Bmx68eyn5Kxf9111wFwwQUXGOUj0cNlZWVGAZ166qmAuwz1+XymrOvLL78MwLZt24zzd//+/UDFJ8BGup/yN5s1awbAmjVrjDqXbXVRfhLlHYq094YbbgDg9ddfr1JbK0Kz4RVFSShqlQKqU6eOqeEjvp8FCxYAMGTIkMg0LgxsKyA5XkccmmVlZUZBRHK4bfazTp06pp/St7KyMuMb6d+/PwC/+93vACcX8I033gDcAvWbNm1i06ZNgKuiKgpMDDcQNdx+SnClnFx69OhR87vhVCdYvHixCaqM5Bl3WpDsP1T1gZXP//rXvzZSVMowiEyPZTSzbQMkxcckGXHDhg1mNzCS2O5nRX8nOJIdHCcuOImt4twOLvcRTh8i3U/5nOzwjRs3zsQEhVJYWEhubi7guheiVcdcl2CKoiQUtUIBSZmN6dOnm9m/X79+gJ0td5vKwOfzmQJdUqJh9OjRRvZHEq8poGgRi36KUhd1ZiNHTRWQoigJRdwGIvr9fpPp/rOf/Qxwaso888wzAMa5mChIwF27du1o2bIl4DpbvX4qqBI9v47XUQWkKIo14tYH5Pf7TcW/0DPfbWPbNyL3RXaCIrlVG4ztfsYK7Wf08KQBCi2dWhHBBam8hj6w5dF+xgfqhFYUJaHwpAJSFCUxUAWkKIo11AApimINNUCKolhDDZCiKNZQA6QoijXUACmKYg01QIqiWEMNkKIo1lADpCiKNTxZjiNec2qk3ZGuIewFgs/Pklw9KZ4V7u/GG5LMK+V9T0S89lOwkRThSQMUr9TmrJbgvsWyvrZNEqWfNtElmKIo1lADpCiKNXQJpiQ0oX6binxdSvTQO6woijVUASnHUKdOHdq2bQvAt99+C8S/QzYpKcmcOlq/fn3AOXddDjTYtm0bAD/88APgFImP9z7HA6qAFEWxhiqgKpCSkmL8AjI7SkH82kDr1q0B+Pjjjzlw4AAA7du3B7zdTynC/z//8z9ccMEFgHMUM5T38YTGLSUlJZlxXLJkCeAc763Ejrg1QKmpqZx33nkAfPTRRwDs3bs3Ytf3+XycdNJJAHTs2BGArVu38uOPPwLxtySRvsj54jfeeCMA2dnZ5sz0bt26AY6hzcvLA7xreOrVq2dOez311FOrdY2kpCRjoHbu3Am453PV5pguL6FLMEVRrBE3CkhmKnEkXnrppdx0000ATJ06FYD58+eHnR5wPOSM7h49etC9e3cAli9fDsDmzZs9o3zEkXrdddcB7kmw7777rmljr169AJg3bx716tWr8DqBQMDM+snJyeZ748aNi17jq0FKSgoAF154IQDTpk07RvmUlZUd05fgdIqtW7cCkJmZCUDdunWNirr++uuB8NNLIo0sI5s2bQpAVlYWANu3bzfLyV/84hcAXHbZZWa5fMYZZwCQm5sLwKhRoyK6Eog2qoAURbGGJ4/lqSipT76XnZ0NwNNPP835558PQH5+PgAPPvigcSYWFxcDcOjQIcDxZYR21efzmZlG/EkDBw4EoKCggAceeACAgwcPVqn90TrITj7fpUsX5s+fD2C2kUX1bN++3aidnJwcwJ1dK2rj8uXL+emnnwC45JJLAEctiM/o8OHDx21PLA7sy8jIAODiiy8GYOLEiYDj05JxmT59OgDLli0znxd/VocOHQBYtGgRH3/8MeCOZ0lJSViKNlr9FHU2ZcoUxo8fD7hKL/ha8nU419+5c6dxxK9duxYIv/2ajFoJcnNkEHr06EGDBg0AV65edNFFRn5++umngPsA+3w+87unn346AHfccYd50eQacozxyJEjq2x4IkFFD5sYEJHbr7zyipHqodG6rVu3rvRBLSwsBNwXc8uWLTRq1AiAwYMHA85LIEsYmyQnJxtnuSyR5KWdMGECCxcuBNzJJjU11ezaydJKDPW+fftMVrtX5txrrrkGgFtvvbXG91v61LBhQ0aOHAk4z3fwz7yILsEURbFG3CggQZyEeXl5ZvnxzDPPAPDkk08a1SLSWhRCvXr1jHy/4YYbAGc2ff/99wH48MMPAXjrrbcAx+FsA5HgJ598MgCdOnWic+fOgLuFLksrOHZ2KykpYf/+/QDm34kTJ5pQhR07dhzzNyXmJxgvONuLi4vZuHEj4C69161bB8AHH3xg1Krg8/mME3rLli1A5UtIW0hfRo8eDTj3OlThC2VlZXz//fcAjB07FoDFixeb8ZFNk3nz5gEwYMAAhg8fDsDSpUsB+Ne//uWJ8awIVUCKolgjbhSQKBnxATz++ONs2LABwMySFa11RTHt3bvXbNeLT6Vly5a8/PLLgOuwk5nTFjIDio9n2LBhxl/TvHlzwFEGMtPLfRHlN2nSJF599VWAYxRCRdSpU4cZM2YArvoqKioyviLbyLjIuMs9yMnJMQpP7kFmZqYZW/ELeQ2fz0fdunUB1x9XXFxs+idKRZ7tSy+91HxdEfIc3HrrrYATxS5+TXm227ZtazYavIYqIEVRrBE3Cii0Pu+SJUtMmkA4Xv5AIGC26++8804A+vbta34uCsL2joH0SQIL8/LyTNCZqKNt27axfv16wFnfg7vbU5E/pzJeeOEFhg0bVu57K1eutBaQF8qePXsAt18tWrQAYPbs2dx1112Aq4B8Ph+7d+82X3sVUaYNGzYEygeDii/yiiuuAFw/3on45ptvABgyZAh///vfAczu5iOPPMK1114bodZHlrgxQGIYCgoKjvleVZFI2GnTphmjJJG/tqNIRYJLO+bPn0///v0Bd4m0adMmnnjiCQA+//xzoOpLDokoHzp0qDHu8mI88MADnnFaihNZnK1yD3r16sWbb74JuCVDZs2aZcppeBkJp2jVqhXghBvIc/3ee+8B1XeeL1261Bixyy+/HHCMmcQZBb8/XkCXYIqiWCNuFFBlR6MEl9GsDJHqM2fOBJwsd5lFq7p0iTaiaNauXcucOXMATIDZm2++aQLtwj0yJhRx1hYUFJhs+A8++ABwlwFeQPonbRo6dCjgjLWEI0jOW5cuXfj6668BuPvuuwE3nGLDhg2eUXWiQkR5FhcXmzy1xYsXA64S9vl8VVL6gUDABG9K9HhKSopRvKqAFEVR/kPcKKDKCE5fkNkiNLCrfv36zJo1C3Cdz9u2bTPfs5F2EQ4FBQVGpYnvYM2aNdVWPoKowaVLl5ryq6HBm15A2vTf//3fgJtC0qpVK+PLkzFOT0/n5z//OQDPPfccAN999x3gVEwQ/4rNLfpAIGACS0WFFhYWmtAJ8enVRK3J78ozEpzzaNvHGUqtMEDBclVeHrnhl156KQBjxowxOWBibN59913jhK7pCx1NxLEqMTGtW7c2EcFVdcSLM7dfv37mWpK8KgbutNNOY/Xq1TVveAT54osvAHj99dcBGD58OI0bNwbchOPU1FSzrJElh5SwePzxx/nss88AuPfeewFnh1FicWK5+yntkGf16NGj/OMf/wBqbhz9fj9DhgwB3DzIgoICM8a249xC8c5UpyhKwlErFJAQCARM/IrET0hMxN///nd69+4NwDvvvAM4UcaTJ08GMI5YLyKzs8yYV155Jf/+97+B8KKdMzIyTByI5B/JFrDf7zezrkTLdunShU8++aTc37aNKBopl5KZmWkc01I2Zffu3UYBSS5dz549AejTp48ZfynelZKSYjLSJZcqFog6F5YsWWKc0OEi4QiicqRQ26RJk0xFAHleioqKPOd8FlQBKYpijVqlgE6EZISL2mnWrJmZWeOBF198EYAZM2YYJfD2228D5Z2WogLEz/PUU0/Rrl27cj8TZVNSUmL8IKKA1q9f7xnlI0j7pdDY/v37jd9k1apVQHn/yVdffQW4GeHZ2dk89dRTACZb3O/3l6ssECsWLVoEYFTm/PnzK73fwU52gK5du5oibBIZLj4en89nngWJCr/llltMZL3XUAWkKIo1EkoBCZJd3Lx587BzbbzArl27ALj99ttN6khoRQBwd7Nkt6dZs2ZmFhUfmcy4wTlfsjsoW8FeQoLrZHZ/7733jAKq7OigYKUnuVdyL/bv388rr7wStTafqE2yQ9u/f3+aNGkCuCe0ymdSU1OPOUrpzDPPNLuZoSETgUDAHDF0//33A5j75EXipiZ0JBEH5Y4dO8wSLJKxIbGolSwPr5RRlZiSgwcPmnIPXbp0ARyZLkWwQh9Yn89nosAnTJgAwKuvvhp2gm841KSfsuzYvn074I5T9+7dzRltwYQuqWWr/vHHHzc5dZL42blzZ5PsWhnR7meHDh3MCS+ydJRie9ddd50pzSLL54qQiWTVqlXGYEnfvFwTWpdgiqJYIyEVkAT2tWjRwpwtFcks6lgoA1EycoKpqLqjR4+aZaWoh/T09ApPxgBn5hRHtpTlCDcTOxb9FGXwl7/8BXDVS2Zm5jHRwp06dTKF6kVBBKsGUUyyNS9O2hMR7X727t3bnOZS0aZIOIcM3HfffQA89thj1Q6qVQWkKEpCkZAKSJytWVlZRklE8jbEQhkI4ryUnCdwZ3056iXY7xPslAV4//33Td2YqlYEiEU/xYcjzlnxATVt2tSMowSWnnvuucf9W4WFhaZAl1fOeQtGNhPatGkDuM72Xbt2mSBCGc+vv/6aadOmAW7fI/H82jAFCWmARIpnZmaanZFIEksDJPTo0QNwIr7lIRZD5PP5zE6R5JBJLMx3331X7eqHseynxM6ce+65gLPzJTtBFV1f2jZ37lwAfvOb33i6n7JklPrcL730EgALFiyo9jWrii7BFEVJKBJSAUmphj179pi6wpHEhgIKvqYsW2SpUVhYGJVsfxv9FMf61KlTTXEyieC++eabWbZsGUBEa1rbHM9YogpIUZSEIqEU0Nlnnw1gtjzHjRvHCy+8EPG/ozNmebSf8YEqIEVREoqEUkByXakBM2vWrKhYfZ0xy6P9jA90G/4/JMpAaj/jA+1n9NAlmKIo1vCkAlIUJTFQBaQoijXUACmKYg01QIqiWEMNkKIo1lADpCiKNdQAKYpiDTVAiqJYQw2QoijWUAOkKIo11AApimINNUCKolhDDZCiKNZQA6QoijXUACmKYg01QIqiWEMNkKIo1lADpCiKNdQAKYpiDTVAiqJYQw2QoijWUAOkKIo11AApimINNUCKolhDDZCiKNZQA6QoijXUACmKYg01QIqiWEMNkKIo1lADpCiKNdQAKYpiDTVAiqJYQw2QoijWUAOkKIo11AApimINNUCKoljj/wNL6RUscqGeJgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.PngImagePlugin.PngImageFile image mode=RGBA size=288x288 at 0x7F60B433D550>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display_image(EPOCHS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "NywiH3nL8guF"
   },
   "source": [
    "Use `imageio` to create an animated gif using the images saved during training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IGKQgENQ8lEI"
   },
   "outputs": [],
   "source": [
    "anim_file = 'dcgan.gif'\n",
    "\n",
    "with imageio.get_writer(anim_file, mode='I') as writer:\n",
    "  filenames = glob.glob('image*.png')\n",
    "  filenames = sorted(filenames)\n",
    "  last = -1\n",
    "  for i,filename in enumerate(filenames):\n",
    "    frame = 2*(i**0.5)\n",
    "    if round(frame) > round(last):\n",
    "      last = frame\n",
    "    else:\n",
    "      continue\n",
    "    image = imageio.imread(filename)\n",
    "    writer.append_data(image)\n",
    "  image = imageio.imread(filename)\n",
    "  writer.append_data(image)\n",
    "\n",
    "import IPython\n",
    "if IPython.version_info > (6,2,0,''):\n",
    "  display.Image(filename=anim_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cGhC3-fMWSwl"
   },
   "source": [
    "If you're working in Colab you can download the animation with the code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "uV0yiKpzNP1b"
   },
   "outputs": [],
   "source": [
    "try:\n",
    "  from google.colab import files\n",
    "except ImportError:\n",
    "  pass\n",
    "else:\n",
    "  files.download(anim_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "k6qC-SbjK0yW"
   },
   "source": [
    "## Next steps\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xjjkT9KAK6H7"
   },
   "source": [
    "This tutorial has shown the complete code necessary to write and train a GAN. As a next step, you might like to experiment with a different dataset, for example the Large-scale Celeb Faces Attributes (CelebA) dataset [available on Kaggle](https://www.kaggle.com/jessicali9530/celeba-dataset/home). To learn more about GANs we recommend the [NIPS 2016 Tutorial: Generative Adversarial Networks](https://arxiv.org/abs/1701.00160).\n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "dcgan.ipynb",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}