imageVAE.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TUIlmenauAMS/Videocoding/blob/main/imageVAE.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#A Variational Autoencoder for Images\n",
        "\n",
        "This demonstrates the principle of a neural network image coder. The latent space with the \"reparameterization trick\" represents the coded domain including quantization noise.\n",
        "\n",
        "This script defines a VAE with a simple feed-forward neural network architecture. It includes functions for training and testing the VAE on the MNIST dataset. The model is trained for 20 epochs, but you can adjust the num_epochs variable to train for a different number of epochs.\n",
        "\n",
        "Key components:\n",
        "\n",
        "- **Encoder**: Maps input to a latent space.\n",
        "- **Reparameterization Trick**: Samples from the latent space.\n",
        "- **Decoder**: Maps samples from the latent space back to the input space.\n",
        "- **Loss Function**: Combines reconstruction loss and KL divergence."
      ],
      "metadata": {
        "id": "8jvBuiOyyDRB"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2pa2sM1pBT3K",
        "outputId": "20b4415f-9ac6-48da-e80d-6536f0f425ca"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting torch==2.0.1\n",
            "  Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m619.9/619.9 MB\u001b[0m \u001b[31m1.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1) (3.15.4)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1) (4.12.2)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1) (1.12.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1) (3.3)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1) (3.1.4)\n",
            "Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch==2.0.1)\n",
            "  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.0/21.0 MB\u001b[0m \u001b[31m22.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-cuda-runtime-cu11==11.7.99 (from torch==2.0.1)\n",
            "  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m849.3/849.3 kB\u001b[0m \u001b[31m30.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-cuda-cupti-cu11==11.7.101 (from torch==2.0.1)\n",
            "  Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.8/11.8 MB\u001b[0m \u001b[31m47.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-cudnn-cu11==8.5.0.96 (from torch==2.0.1)\n",
            "  Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m557.1/557.1 MB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-cublas-cu11==11.10.3.66 (from torch==2.0.1)\n",
            "  Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m317.1/317.1 MB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-cufft-cu11==10.9.0.58 (from torch==2.0.1)\n",
            "  Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux1_x86_64.whl (168.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m168.4/168.4 MB\u001b[0m \u001b[31m7.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-curand-cu11==10.2.10.91 (from torch==2.0.1)\n",
            "  Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m54.6/54.6 MB\u001b[0m \u001b[31m9.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-cusolver-cu11==11.4.0.1 (from torch==2.0.1)\n",
            "  Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m102.6/102.6 MB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-cusparse-cu11==11.7.4.91 (from torch==2.0.1)\n",
            "  Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m173.2/173.2 MB\u001b[0m \u001b[31m2.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-nccl-cu11==2.14.3 (from torch==2.0.1)\n",
            "  Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m177.1/177.1 MB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting nvidia-nvtx-cu11==11.7.91 (from torch==2.0.1)\n",
            "  Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m98.6/98.6 kB\u001b[0m \u001b[31m12.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting triton==2.0.0 (from torch==2.0.1)\n",
            "  Downloading triton-2.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.3/63.3 MB\u001b[0m \u001b[31m9.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1) (67.7.2)\n",
            "Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1) (0.43.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.1) (3.27.9)\n",
            "Collecting lit (from triton==2.0.0->torch==2.0.1)\n",
            "  Downloading lit-18.1.8-py3-none-any.whl (96 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m96.4/96.4 kB\u001b[0m \u001b[31m12.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch==2.0.1) (2.1.5)\n",
            "Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch==2.0.1) (1.3.0)\n",
            "Installing collected packages: lit, nvidia-nvtx-cu11, nvidia-nccl-cu11, nvidia-cusparse-cu11, nvidia-curand-cu11, nvidia-cufft-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cuda-cupti-cu11, nvidia-cublas-cu11, nvidia-cusolver-cu11, nvidia-cudnn-cu11, triton, torch\n",
            "  Attempting uninstall: triton\n",
            "    Found existing installation: triton 2.3.0\n",
            "    Uninstalling triton-2.3.0:\n",
            "      Successfully uninstalled triton-2.3.0\n",
            "  Attempting uninstall: torch\n",
            "    Found existing installation: torch 2.3.0+cu121\n",
            "    Uninstalling torch-2.3.0+cu121:\n",
            "      Successfully uninstalled torch-2.3.0+cu121\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "torchaudio 2.3.0+cu121 requires torch==2.3.0, but you have torch 2.0.1 which is incompatible.\n",
            "torchtext 0.18.0 requires torch>=2.3.0, but you have torch 2.0.1 which is incompatible.\n",
            "torchvision 0.18.0+cu121 requires torch==2.3.0, but you have torch 2.0.1 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed lit-18.1.8 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 torch-2.0.1 triton-2.0.0\n",
            "Collecting torchvision==0.15.2\n",
            "  Downloading torchvision-0.15.2-cp310-cp310-manylinux1_x86_64.whl (6.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.0/6.0 MB\u001b[0m \u001b[31m41.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from torchvision==0.15.2) (1.25.2)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from torchvision==0.15.2) (2.31.0)\n",
            "Requirement already satisfied: torch==2.0.1 in /usr/local/lib/python3.10/dist-packages (from torchvision==0.15.2) (2.0.1)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision==0.15.2) (9.4.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (3.15.4)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (4.12.2)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (1.12.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (3.3)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (3.1.4)\n",
            "Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (11.7.99)\n",
            "Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (11.7.99)\n",
            "Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (11.7.101)\n",
            "Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (8.5.0.96)\n",
            "Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (11.10.3.66)\n",
            "Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (10.9.0.58)\n",
            "Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (10.2.10.91)\n",
            "Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (11.4.0.1)\n",
            "Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (11.7.4.91)\n",
            "Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (2.14.3)\n",
            "Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (11.7.91)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->torchvision==0.15.2) (2.0.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1->torchvision==0.15.2) (67.7.2)\n",
            "Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1->torchvision==0.15.2) (0.43.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.1->torchvision==0.15.2) (3.27.9)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.1->torchvision==0.15.2) (18.1.8)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision==0.15.2) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision==0.15.2) (3.7)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision==0.15.2) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision==0.15.2) (2024.6.2)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch==2.0.1->torchvision==0.15.2) (2.1.5)\n",
            "Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch==2.0.1->torchvision==0.15.2) (1.3.0)\n",
            "Installing collected packages: torchvision\n",
            "  Attempting uninstall: torchvision\n",
            "    Found existing installation: torchvision 0.18.0+cu121\n",
            "    Uninstalling torchvision-0.18.0+cu121:\n",
            "      Successfully uninstalled torchvision-0.18.0+cu121\n",
            "Successfully installed torchvision-0.15.2\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz to ./data/MNIST/raw/train-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 9912422/9912422 [00:00<00:00, 137009185.12it/s]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ./data/MNIST/raw/train-images-idx3-ubyte.gz to ./data/MNIST/raw\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz to ./data/MNIST/raw/train-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 28881/28881 [00:00<00:00, 40405501.61it/s]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ./data/MNIST/raw/train-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n",
            "100%|██████████| 1648877/1648877 [00:00<00:00, 27565103.04it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ./data/MNIST/raw/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 4542/4542 [00:00<00:00, 4101297.90it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
            "\n",
            "Train Epoch: 1 [0/60000 (0%)]\tLoss: 542.691467\n",
            "Train Epoch: 1 [12800/60000 (21%)]\tLoss: -18996.720703\n",
            "Train Epoch: 1 [25600/60000 (43%)]\tLoss: -21186.136719\n",
            "Train Epoch: 1 [38400/60000 (64%)]\tLoss: -24723.417969\n",
            "Train Epoch: 1 [51200/60000 (85%)]\tLoss: -27693.357422\n",
            "====> Epoch: 1 Average loss: -22051.1017\n",
            "====> Test set loss: -26040.3456\n",
            "Train Epoch: 2 [0/60000 (0%)]\tLoss: -26168.832031\n",
            "Train Epoch: 2 [12800/60000 (21%)]\tLoss: -28875.382812\n",
            "Train Epoch: 2 [25600/60000 (43%)]\tLoss: -33095.562500\n",
            "Train Epoch: 2 [38400/60000 (64%)]\tLoss: -34187.601562\n",
            "Train Epoch: 2 [51200/60000 (85%)]\tLoss: -36657.523438\n",
            "====> Epoch: 2 Average loss: -33063.9536\n",
            "====> Test set loss: -38151.4614\n",
            "Train Epoch: 3 [0/60000 (0%)]\tLoss: -37910.722656\n",
            "Train Epoch: 3 [12800/60000 (21%)]\tLoss: -37934.847656\n",
            "Train Epoch: 3 [25600/60000 (43%)]\tLoss: -39698.097656\n",
            "Train Epoch: 3 [38400/60000 (64%)]\tLoss: -41424.628906\n",
            "Train Epoch: 3 [51200/60000 (85%)]\tLoss: -42365.988281\n",
            "====> Epoch: 3 Average loss: -41105.9436\n",
            "====> Test set loss: -43200.5000\n",
            "Train Epoch: 4 [0/60000 (0%)]\tLoss: -44384.519531\n",
            "Train Epoch: 4 [12800/60000 (21%)]\tLoss: -46304.230469\n",
            "Train Epoch: 4 [25600/60000 (43%)]\tLoss: -45870.546875\n",
            "Train Epoch: 4 [38400/60000 (64%)]\tLoss: -46981.253906\n",
            "Train Epoch: 4 [51200/60000 (85%)]\tLoss: -46741.128906\n",
            "====> Epoch: 4 Average loss: -45910.6798\n",
            "====> Test set loss: -47485.8476\n",
            "Train Epoch: 5 [0/60000 (0%)]\tLoss: -47876.851562\n",
            "Train Epoch: 5 [12800/60000 (21%)]\tLoss: -48305.933594\n",
            "Train Epoch: 5 [25600/60000 (43%)]\tLoss: -50390.769531\n",
            "Train Epoch: 5 [38400/60000 (64%)]\tLoss: -50934.367188\n",
            "Train Epoch: 5 [51200/60000 (85%)]\tLoss: -49558.980469\n",
            "====> Epoch: 5 Average loss: -49301.9981\n",
            "====> Test set loss: -49993.5772\n",
            "Train Epoch: 6 [0/60000 (0%)]\tLoss: -47524.542969\n",
            "Train Epoch: 6 [12800/60000 (21%)]\tLoss: -53362.605469\n",
            "Train Epoch: 6 [25600/60000 (43%)]\tLoss: -51017.394531\n",
            "Train Epoch: 6 [38400/60000 (64%)]\tLoss: -49030.273438\n",
            "Train Epoch: 6 [51200/60000 (85%)]\tLoss: -53177.843750\n",
            "====> Epoch: 6 Average loss: -51127.9193\n",
            "====> Test set loss: -51165.6661\n",
            "Train Epoch: 7 [0/60000 (0%)]\tLoss: -51568.488281\n",
            "Train Epoch: 7 [12800/60000 (21%)]\tLoss: -51273.378906\n",
            "Train Epoch: 7 [25600/60000 (43%)]\tLoss: -52583.261719\n",
            "Train Epoch: 7 [38400/60000 (64%)]\tLoss: -55271.773438\n",
            "Train Epoch: 7 [51200/60000 (85%)]\tLoss: -54480.550781\n",
            "====> Epoch: 7 Average loss: -52647.7640\n",
            "====> Test set loss: -54821.1113\n",
            "Train Epoch: 8 [0/60000 (0%)]\tLoss: -54656.347656\n",
            "Train Epoch: 8 [12800/60000 (21%)]\tLoss: -52491.050781\n",
            "Train Epoch: 8 [25600/60000 (43%)]\tLoss: -53232.789062\n",
            "Train Epoch: 8 [38400/60000 (64%)]\tLoss: -54322.023438\n",
            "Train Epoch: 8 [51200/60000 (85%)]\tLoss: -54305.679688\n",
            "====> Epoch: 8 Average loss: -53911.2584\n",
            "====> Test set loss: -54069.9306\n",
            "Train Epoch: 9 [0/60000 (0%)]\tLoss: -55505.562500\n",
            "Train Epoch: 9 [12800/60000 (21%)]\tLoss: -54217.746094\n",
            "Train Epoch: 9 [25600/60000 (43%)]\tLoss: -55610.886719\n",
            "Train Epoch: 9 [38400/60000 (64%)]\tLoss: -55724.296875\n",
            "Train Epoch: 9 [51200/60000 (85%)]\tLoss: -54350.355469\n",
            "====> Epoch: 9 Average loss: -54685.1960\n",
            "====> Test set loss: -54746.7471\n",
            "Train Epoch: 10 [0/60000 (0%)]\tLoss: -55120.003906\n",
            "Train Epoch: 10 [12800/60000 (21%)]\tLoss: -54217.878906\n",
            "Train Epoch: 10 [25600/60000 (43%)]\tLoss: -57042.652344\n",
            "Train Epoch: 10 [38400/60000 (64%)]\tLoss: -54880.800781\n",
            "Train Epoch: 10 [51200/60000 (85%)]\tLoss: -54627.585938\n",
            "====> Epoch: 10 Average loss: -55384.6139\n",
            "====> Test set loss: -55570.6872\n"
          ]
        }
      ],
      "source": [
        "#ChatGPT:\n",
        "#Write a Python program which implements a variational autoencoder for images, with Pytorch.\n",
        "#Can you also make a \"requirements\" file?\n",
        "!pip install torch==2.0.1\n",
        "!pip install torchvision==0.15.2\n",
        "\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import torch.optim as optim\n",
        "from torchvision import datasets, transforms\n",
        "from torch.utils.data import DataLoader\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Hyperparameters\n",
        "batch_size = 128\n",
        "learning_rate = 0.001\n",
        "num_epochs = 10 #20\n",
        "latent_dim = 20  # Dimension of the latent space\n",
        "\n",
        "# Data loading and preprocessing\n",
        "transform = transforms.Compose([\n",
        "    transforms.ToTensor(),  # Convert images to PyTorch tensors\n",
        "    transforms.Normalize((0.5,), (0.5,))  # Normalize the images to [-1, 1]\n",
        "])\n",
        "\n",
        "train_dataset = datasets.MNIST(root='./data', train=True, transform=transform, download=True)\n",
        "train_loader = DataLoader(dataset=train_dataset, batch_size=batch_size, shuffle=True)\n",
        "\n",
        "test_dataset = datasets.MNIST(root='./data', train=False, transform=transform)\n",
        "test_loader = DataLoader(dataset=test_dataset, batch_size=batch_size, shuffle=False)\n",
        "\n",
        "# Define the VAE model\n",
        "class VAE(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(VAE, self).__init__()\n",
        "        # Encoder layers\n",
        "        self.fc1 = nn.Linear(28*28, 400)\n",
        "        self.fc21 = nn.Linear(400, latent_dim)  # Mean of the latent variables\n",
        "        self.fc22 = nn.Linear(400, latent_dim)  # Log variance of the latent variables\n",
        "        # Decoder layers\n",
        "        self.fc3 = nn.Linear(latent_dim, 400)\n",
        "        self.fc4 = nn.Linear(400, 28*28)\n",
        "\n",
        "    def encode(self, x):\n",
        "        h1 = F.relu(self.fc1(x))  # First layer with ReLU activation\n",
        "        return self.fc21(h1), self.fc22(h1)  # Return mean and log variance\n",
        "\n",
        "    def reparameterize(self, mu, logvar):\n",
        "        std = torch.exp(0.5*logvar)  # Convert log variance to standard deviation\n",
        "        eps = torch.randn_like(std)  # Sample from standard normal distribution\n",
        "        return mu + eps*std  # Reparameterize to obtain the latent variable z\n",
        "\n",
        "    def decode(self, z):\n",
        "        h3 = F.relu(self.fc3(z))  # Decode first layer with ReLU activation\n",
        "        return torch.sigmoid(self.fc4(h3))  # Decode second layer with sigmoid activation to get output\n",
        "\n",
        "    def forward(self, x):\n",
        "        mu, logvar = self.encode(x.view(-1, 28*28))  # Flatten the input image and encode\n",
        "        z = self.reparameterize(mu, logvar)  # Reparameterize to get z\n",
        "        return self.decode(z), mu, logvar  # Decode z and return output with mu and logvar\n",
        "\n",
        "# Loss function combining reconstruction loss and KL divergence\n",
        "def loss_function(recon_x, x, mu, logvar):\n",
        "    BCE = F.binary_cross_entropy(recon_x, x.view(-1, 28*28), reduction='sum')  # Reconstruction loss\n",
        "    KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())  # KL divergence\n",
        "    return BCE + KLD  # Total loss\n",
        "\n",
        "# Training the model\n",
        "model = VAE()\n",
        "optimizer = optim.Adam(model.parameters(), lr=learning_rate)\n",
        "\n",
        "def train(epoch):\n",
        "    model.train()\n",
        "    train_loss = 0\n",
        "    for batch_idx, (data, _) in enumerate(train_loader):\n",
        "        optimizer.zero_grad()  # Clear gradients\n",
        "        recon_batch, mu, logvar = model(data)  # Forward pass\n",
        "        loss = loss_function(recon_batch, data, mu, logvar)  # Compute loss\n",
        "        loss.backward()  # Backward pass\n",
        "        train_loss += loss.item()\n",
        "        optimizer.step()  # Update weights\n",
        "        if batch_idx % 100 == 0:\n",
        "            print(f'Train Epoch: {epoch} [{batch_idx*len(data)}/{len(train_loader.dataset)} ({100.*batch_idx/len(train_loader):.0f}%)]\\tLoss: {loss.item()/len(data):.6f}')\n",
        "    print(f'====> Epoch: {epoch} Average loss: {train_loss/len(train_loader.dataset):.4f}')\n",
        "\n",
        "def test(epoch):\n",
        "    model.eval()\n",
        "    test_loss = 0\n",
        "    with torch.no_grad():\n",
        "        for data, _ in test_loader:\n",
        "            recon, mu, logvar = model(data)\n",
        "            test_loss += loss_function(recon, data, mu, logvar).item()\n",
        "    test_loss /= len(test_loader.dataset)\n",
        "    print(f'====> Test set loss: {test_loss:.4f}')\n",
        "\n",
        "for epoch in range(1, num_epochs + 1):\n",
        "    train(epoch)\n",
        "    test(epoch)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Explanation:\n",
        "1. **Hyperparameters**: Defined at the beginning for easy adjustment.\n",
        "2. **Data Loading**: MNIST dataset is loaded and preprocessed.\n",
        "3. **VAE Model**:\n",
        "   - `__init__()`: Initializes encoder and decoder layers.\n",
        "   - `encode()`: Encodes the input into mean and log variance.\n",
        "   - `reparameterize()`: Reparameterization trick to sample from latent space.\n",
        "   - `decode()`: Decodes the latent variable back to the input space.\n",
        "   - `forward()`: Combines encode, reparameterize, and decode steps.\n",
        "4. **Loss Function**: Combines reconstruction loss (BCE) and KL divergence (KLD).\n",
        "5. **Training and Testing**:\n",
        "   - `train()`: Trains the model for one epoch.\n",
        "   - `test()`: Evaluates the model on the test dataset.\n",
        "6. **Additional Function**:\n",
        "   - `display_and_test_random_image()`:\n",
        "     - Displays an original image.\n",
        "     - Encodes it and estimates the resulting bitrate.\n",
        "     - Decodes it back to an image.\n",
        "     - Displays and saves the decoded image.\n",
        "\n",
        "This version includes detailed comments to help you understand each step of the process."
      ],
      "metadata": {
        "id": "J4K5luBxcqey"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Viewing the Result"
      ],
      "metadata": {
        "id": "vQ4w2rcuS_al"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#ChatGPT:\n",
        "#Can you also append a test, where a random image is displayed and encoded, the entropy of the encoded latent space is\n",
        "#computed and displayed, and then the decoded image is stored and displayed?\n",
        "\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Additional function to display a random image, encode it, compute the bitrate, decode it, and store/display the result\n",
        "def display_and_test_random_image():\n",
        "    model.eval()\n",
        "    with torch.no_grad():\n",
        "        data, _ = next(iter(test_loader))  # Get a batch of test data\n",
        "        data = data[np.random.randint(0,9)].unsqueeze(0)  # Take the first image in the batch and add batch dimension\n",
        "        plt.figure(figsize=(9, 3))\n",
        "\n",
        "        # Original Image\n",
        "        plt.subplot(1, 3, 1)\n",
        "        plt.title(\"Original Image\")\n",
        "        plt.imshow(data[0].view(28, 28).cpu(), cmap='gray')\n",
        "        plt.axis('off')\n",
        "\n",
        "        # Encode and compute bitrate\n",
        "        mu, logvar = model.encode(data.view(-1, 28*28))\n",
        "        std = torch.exp(0.5*logvar)\n",
        "        var = std.pow(2)\n",
        "        # Estimation of the resulting bitrate\n",
        "        bitrate = torch.sum(0.5 * torch.log2(2 * np.pi * np.e * var)).item()\n",
        "\n",
        "        print(f\"Estimated bitrate of the encoded latent space: {bitrate:.4f} bits\")\n",
        "\n",
        "        # Decode the latent representation\n",
        "        z = model.reparameterize(mu, logvar)\n",
        "        decoded_img = model.decode(z).view(28, 28)\n",
        "\n",
        "        # Decoded Image\n",
        "        \"\"\"\n",
        "        plt.subplot(1, 3, 2)\n",
        "        plt.title(\"Decoded Image\")\n",
        "        plt.imshow(decoded_img.cpu(), cmap='gray')\n",
        "        plt.axis('off')\n",
        "        \"\"\"\n",
        "\n",
        "        # Save decoded image\n",
        "        plt.subplot(1, 3, 3)\n",
        "        plt.title(\"Saved Decoded Image\")\n",
        "        plt.imshow(decoded_img.cpu(), cmap='gray')\n",
        "        plt.axis('off')\n",
        "        plt.savefig(\"decoded_image.png\")\n",
        "\n",
        "        plt.show()\n",
        "\n",
        "# Call the test function to display and test a random image\n",
        "display_and_test_random_image()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281
        },
        "id": "SlJ650T8Nr_b",
        "outputId": "c48b7611-7c0f-4f3d-b9a5-3a83c24ec748"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Estimated bitrate of the encoded latent space: -15.9226 bits\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 900x300 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAD3CAYAAADmMWljAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAfw0lEQVR4nO3deXzNV/7H8ffNKrIRBA1NbG1qmdFKtGIJikxtZShjK7FFq1ozlgfjZ2uR0lE8tIpW0VJDFNWFajumammrSmjTqhBa2rEn9i05vz88cus2y7kiRPX1/Itzzvfcz/e68X3fc7/3xGGMMQIAAACQJ4+iLgAAAAC43RGaAQAAAAtCMwAAAGBBaAYAAAAsCM0AAACABaEZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJrdMG7cODkcjgIdu2DBAjkcDu3fv79wi7rG/v375XA4tGDBgpv2GAAA3C4aN26sxo0bF3UZN12vXr0UERFRqHNGRESoV69ehTrnH8UdHZq//fZbde/eXWFhYfL19dVdd92lbt266dtvvy3q0orEf//7XzkcDi1fvryoSwEA3AZ27dqljh07Kjw8XMWKFVNYWJiaN2+umTNnFnVphSIiIkIOh0MOh0MeHh4qUaKEatWqpf79++uLL74o6vJuaw6HQ0899VRRl3FbuWND84oVK/TAAw/ok08+UXx8vGbNmqU+ffpo/fr1euCBB7Ry5Uq35/q///s/nT9/vkB19OjRQ+fPn1d4eHiBjgcA4GbYvHmzoqKilJycrH79+umll15S37595eHhoRkzZhR1eYWmdu3aevPNN/XGG28oMTFRTZo00bvvvquHHnpI//jHP4q6PPyOeBV1ATfD3r171aNHD1WuXFkbNmxQmTJlnH3PPPOMGjZsqB49emjnzp2qXLlynvOcPXtW/v7+8vLykpdXwZ4qT09PeXp6FuhYAABulokTJyo4OFhbt25ViRIlXPqOHDlSNEXdBGFhYerevbtL2+TJk9W1a1dNmzZN1apV0xNPPFFE1eH35I5caX7hhRd07tw5zZ071yUwS1Lp0qU1Z84cnT17VlOmTHG2Z9+3nJKSoq5du6pkyZJq0KCBS9+1zp8/r6efflqlS5dWYGCg2rZtq0OHDsnhcGjcuHHOcbnd0xwREaHWrVtr48aNqlu3rooVK6bKlSvrjTfecHmMEydOaOjQoapVq5YCAgIUFBSkRx55RMnJyYX0TP16bj/88IO6d++u4OBglSlTRqNHj5YxRj/99JMeffRRBQUFqVy5cpo6darL8ZcuXdKYMWNUp04dBQcHy9/fXw0bNtT69etzPNbx48fVo0cPBQUFqUSJEurZs6eSk5NzvR/7+++/V8eOHRUSEqJixYopKipKq1evLrTzBoA/ur1796pGjRo5ArMkhYaGuvx9/vz5atq0qUJDQ+Xr66vq1avrlVdecRnTunXrPBei6tWrp6ioKJe2RYsWqU6dOvLz81NISIj+9re/6aeffspx7Ny5c1WlShX5+fmpbt26+uyzz67zTHPy8/PTm2++qZCQEE2cOFHGGGdfVlaWpk+frho1aqhYsWIqW7asEhISdPLkyRzzrFmzRrGxsQoMDFRQUJCio6P11ltvuYxJSkpynmfp0qXVvXt3HTp0KMdcq1atUs2aNVWsWDHVrFkzz0/E3a3PGKMJEyaoQoUKKl68uJo0aXJDt6dm3+K5bNkyjR8/XmFhYQoMDFTHjh2VkZGhixcvavDgwQoNDVVAQIDi4+N18eJFlznceR1ln+O4ceN01113OWtPSUnJ9X7s9PR0DR48WBUrVpSvr6+qVq2qyZMnKysrq8Dnmpc7cqX53XffVUREhBo2bJhrf6NGjRQREaH3338/R99jjz2matWqadKkSS4/RL/Vq1cvLVu2TD169NBDDz2kTz/9VK1atXK7xtTUVHXs2FF9+vRRz5499frrr6tXr16qU6eOatSoIUnat2+fVq1apccee0yVKlXS4cOHNWfOHMXGxiolJUV33XWX249n07lzZ9133316/vnn9f7772vChAkKCQnRnDlz1LRpU02ePFmLFy/W0KFDFR0drUaNGkmSTp06pddee01dunRRv379dPr0ac2bN09xcXH68ssvVbt2bUlXfwDatGmjL7/8Uk888YQiIyP1zjvvqGfPnjlq+fbbb1W/fn2FhYVpxIgR8vf317Jly9SuXTu9/fbbat++faGdNwD8UYWHh2vLli365ptvVLNmzXzHvvLKK6pRo4batm0rLy8vvfvuu3ryySeVlZWlgQMHSrp6HXn88ce1detWRUdHO489cOCAPv/8c73wwgvOtokTJ2r06NHq1KmT+vbtq6NHj2rmzJlq1KiRtm/f7gzy8+bNU0JCgmJiYjR48GDt27dPbdu2VUhIiCpWrHhD5x8QEKD27dtr3rx5SklJcV57ExIStGDBAsXHx+vpp59WWlqaXnrpJW3fvl2bNm2St7e3pKuLYr1791aNGjU0cuRIlShRQtu3b9fatWvVtWtX55j4+HhFR0crMTFRhw8f1owZM7Rp0yaX81y3bp06dOig6tWrKzExUcePH1d8fLwqVKiQo2536xszZowmTJigli1bqmXLlvr666/VokULXbp06Yaet8TERPn5+WnEiBFKTU3VzJkz5e3tLQ8PD508eVLjxo3T559/rgULFqhSpUoaM2aM81h3XkeSNHLkSE2ZMkVt2rRRXFyckpOTFRcXpwsXLrjUcu7cOcXGxurQoUNKSEjQ3Xffrc2bN2vkyJH65ZdfNH369Bs61xzMHSY9Pd1IMo8++mi+49q2bWskmVOnThljjBk7dqyRZLp06ZJjbHZftm3bthlJZvDgwS7jevXqZSSZsWPHOtvmz59vJJm0tDRnW3h4uJFkNmzY4Gw7cuSI8fX1NUOGDHG2XbhwwWRmZro8RlpamvH19TXPPvusS5skM3/+/HzPef369UaSSUpKynFu/fv3d7ZduXLFVKhQwTgcDvP8888720+ePGn8/PxMz549XcZevHjR5XFOnjxpypYta3r37u1se/vtt40kM336dGdbZmamadq0aY7aH374YVOrVi1z4cIFZ1tWVpaJiYkx1apVy/ccAQDuWbdunfH09DSenp6mXr16Zvjw4ebDDz80ly5dyjH23LlzOdri4uJM5cqVnX/PyMjIcR0zxpgpU6YYh8NhDhw4YIwxZv/+/cbT09NMnDjRZdyuXbuMl5eXs/3SpUsmNDTU1K5d2+U6M3fuXCPJxMbGWs8xPDzctGrVKs/+adOmGUnmnXfeMcYY89lnnxlJZvHixS7j1q5d69Kenp5uAgMDzYMPPmjOnz/vMjYrK8ul/po1a7qMee+994wkM2bMGGdb7dq1Tfny5U16erqzbd26dUaSCQ8Pd7a5W9+RI0eMj4+PadWqlbMeY4z55z//aSS5XMfzIskMHDjQ+ffsDFGzZk2X10iXLl2Mw+EwjzzyiMvx9erVc6ndGPdeR//73/+Ml5eXadeuncu4cePG5aj9ueeeM/7+/uaHH35wGTtixAjj6elpfvzxR+t5Xo877vaM06dPS5ICAwPzHZfdf+rUKZf2AQMGWB9j7dq1kqQnn3zSpX3QoEFu11m9enWXlfAyZcro3nvv1b59+5xtvr6+8vC4+k+UmZmp48ePKyAgQPfee6++/vprtx/LHX379nX+2dPTU1FRUTLGqE+fPs72EiVK5KjR09NTPj4+kq6uJp84cUJXrlxRVFSUS41r166Vt7e3+vXr52zz8PBweWcpXb0l5T//+Y86deqk06dP69ixYzp27JiOHz+uuLg47dmzJ9ePtQAA16d58+basmWL2rZtq+TkZE2ZMkVxcXEKCwvLcTucn5+f888ZGRk6duyYYmNjtW/fPmVkZEiS8xbCZcuWuXxSu3TpUj300EO6++67JV39on5WVpY6derk/D/+2LFjKleunKpVq+a8ve+rr77SkSNHNGDAAOd1Rrr6SW9wcHChPAcBAQGSfs0OSUlJCg4OVvPmzV1qq1OnjgICApy1ffTRRzp9+rRGjBihYsWKucyZfTtndv1PPvmky5hWrVopMjLS+Wn3L7/8oh07dqhnz54u59W8eXNVr17dZW536/v444916dIlDRo0yOX20sGDB9/wc/b44487V7Ml6cEHH5QxRr1793YZ9+CDD+qnn37SlStXnG3uvI4++eQTXblyxa2MlZSUpIYNG6pkyZIuz0ezZs2UmZmpDRs23PD5XuuOuz0jOwxn/wDkJa9wXalSJetjHDhwQB4eHjnGVq1a1e06s//zuFbJkiVd7knKysrSjBkzNGvWLKWlpSkzM9PZV6pUKbcfqyD1BAcHq1ixYipdunSO9uPHj7u0LVy4UFOnTtX333+vy5cvO9uvfX4OHDig8uXLq3jx4i7H/vY5S01NlTFGo0eP1ujRo3Ot9ciRIwoLC3P/5AAAuYqOjtaKFSt06dIlJScna+XKlZo2bZo6duyoHTt2OEPbpk2bNHbsWG3ZskXnzp1zmSMjI8MZ9jp37qxVq1Zpy5YtiomJ0d69e7Vt2zaXj8n37NkjY4yqVauWa03ZgezAgQOSlGOct7d3vl/ivx5nzpyR9GsW2LNnjzIyMnLc050t+wuSe/fulaR8b2vJrv/ee+/N0RcZGamNGze6jMvt+fjtIpm79eU1Z5kyZVSyZMk8a3ZHbnlBUo7bZYKDg5WVlaWMjAxnZnHndZRd+2/zQUhISI7a9+zZo507d+b4/lq2wv5C6x0XmoODg1W+fHnt3Lkz33E7d+5UWFiYgoKCXNqvfRd0M+W1o8a1784nTZqk0aNHq3fv3nruuecUEhIiDw8PDR48uNBvcM+tHndqXLRokXr16qV27dpp2LBhCg0NlaenpxITE53/qVyP7PMaOnSo4uLich1zPW9OAAB2Pj4+io6OVnR0tO655x7Fx8crKSlJY8eO1d69e/Xwww8rMjJSL774oipWrCgfHx998MEHmjZtmsv1qE2bNipevLiWLVummJgYLVu2TB4eHnrsscecY7KysuRwOLRmzZpcrzPZq7+3wjfffCPp1+tKVlaWQkNDtXjx4lzH5xXObpXbob68soEtM1zP68hdWVlZat68uYYPH55r/z333HPdc+bnjgvN0tVv8L766qvauHGjcweMa3322Wfav3+/EhISCjR/eHi4srKylJaW5vIuLjU1tcA152b58uVq0qSJ5s2b59Kenp6eYwW4qCxfvlyVK1fWihUrXD4CGjt2rMu48PBwrV+/XufOnXNZbf7tc5a9euDt7a1mzZrdxMoBALnJ3uXil19+kXT1y/UXL17U6tWrXVYZc9slyd/fX61bt1ZSUpJefPFFLV26VA0bNnT54nqVKlVkjFGlSpXyDTXZv99gz549atq0qbP98uXLSktL05///OcbOs8zZ85o5cqVqlixou677z5nbR9//LHq16+f7yJalSpVJF0N3Xkt5GTXv3v3bpf6s9uy+689z9/avXt3jsd1p75r57x2Vf7o0aO57gJyK7j7OsquPTU11eUT6+PHj+eovUqVKjpz5swtywt33D3NkjRs2DD5+fkpISEhx60EJ06c0IABA1S8eHENGzasQPNnr4DOmjXLpb2wf4OSp6dnjh08kpKSbqt7erPfWV5b5xdffKEtW7a4jIuLi9Ply5f16quvOtuysrL08ssvu4wLDQ1V48aNNWfOHOd/2Nc6evRoYZYPAH9Y69evz3WXqA8++EDSr7cV5Pb/fEZGhubPn5/rvJ07d9bPP/+s1157TcnJyercubNL/1//+ld5enpq/PjxOR7fGOO8bkdFRalMmTKaPXu2y44PCxYsUHp6+nWeravz58+rR48eOnHihEaNGuVc9OnUqZMyMzP13HPP5TjmypUrzsdt0aKFAgMDlZiYmGNHh+xzioqKUmhoqGbPnu2y9dqaNWv03XffOXfcKl++vGrXrq2FCxc67+uVrt43nZKS4jK3u/U1a9ZM3t7emjlzpstzXOi7SVwHd19HDz/8sLy8vHJsRffSSy/lmLNTp07asmWLPvzwwxx96enpLvdTF4Y7cqW5WrVqWrhwobp166ZatWqpT58+qlSpkvbv36958+bp2LFjWrJkifOd4vWqU6eOOnTooOnTp+v48ePOLed++OEHScqxp3NBtW7dWs8++6zi4+MVExOjXbt2afHixYV2L1dhaN26tVasWKH27durVatWSktL0+zZs1W9enXnvWKS1K5dO9WtW1dDhgxRamqqIiMjtXr1ap04cUKS63P28ssvq0GDBqpVq5b69eunypUr6/Dhw9qyZYsOHjxYqPtUA8Af1aBBg3Tu3Dm1b99ekZGRunTpkjZv3qylS5cqIiJC8fHxkq4GRB8fH7Vp00YJCQk6c+aMXn31VYWGhua6uNGyZUsFBgZq6NCh8vT0VIcOHVz6q1SpogkTJmjkyJHav3+/2rVrp8DAQKWlpWnlypXq37+/hg4dKm9vb02YMEEJCQlq2rSpOnfurLS0NM2fP/+6roOHDh3SokWLJF1dXU5JSVFSUpL+97//aciQIS6fOsfGxiohIUGJiYnasWOHWrRoIW9vb+3Zs0dJSUmaMWOGOnbsqKCgIE2bNk19+/ZVdHS08/c7JCcn69y5c1q4cKG8vb01efJkxcfHKzY2Vl26dHFuORcREaG///3vzsdNTExUq1at1KBBA/Xu3VsnTpzQzJkzVaNGDZdrqbv1lSlTRkOHDlViYqJat26tli1bavv27VqzZk2RfVLt7uuobNmyeuaZZzR16lS1bdtWf/nLX5ScnOys/dq8MGzYMK1evVqtW7d2btt79uxZ7dq1S8uXL9f+/fsL93wLdS+O28zOnTtNly5dTPny5Y23t7cpV66c6dKli9m1a1eOsdlbrx09ejTPvmudPXvWDBw40ISEhJiAgADTrl07s3v3biPJZZu2vLacy20LnNjYWJctdC5cuGCGDBliypcvb/z8/Ez9+vXNli1bcowrjC3nfnvePXv2NP7+/rnWWKNGDeffs7KyzKRJk0x4eLjx9fU1999/v3nvvfdMz549c2w1c/ToUdO1a1cTGBhogoODTa9evcymTZuMJPPvf//bZezevXvN448/bsqVK2e8vb1NWFiYad26tVm+fHm+5wgAcM+aNWtM7969TWRkpAkICDA+Pj6matWqZtCgQebw4cMuY1evXm3+9Kc/mWLFipmIiAgzefJk8/rrr+e4vmXr1q2bkWSaNWuW5+O//fbbpkGDBsbf39/4+/ubyMhIM3DgQLN7926XcbNmzTKVKlUyvr6+JioqymzYsCHHdTAv2Vu8SjIOh8MEBQWZGjVqmH79+pkvvvgiz+Pmzp1r6tSpY/z8/ExgYKCpVauWGT58uPn5559zPC8xMTHGz8/PBAUFmbp165olS5a4jFm6dKm5//77ja+vrwkJCTHdunUzBw8ezPX5uO+++4yvr6+pXr26WbFiRa7XUnfry8zMNOPHj3dmiMaNG5tvvvnGhIeH39CWc9dmCGN+zTlbt251ac8tX7j7Orpy5YoZPXq0KVeunPHz8zNNmzY13333nSlVqpQZMGCAy+OcPn3ajBw50lStWtX4+PiY0qVLm5iYGPOvf/0r1+0Tb4TDmHx+gweuy44dO3T//fdr0aJF6tatW1GX87uwatUqtW/fXhs3blT9+vWLuhwAAHAbSk9PV8mSJTVhwgSNGjWqSGq4I+9pvhXOnz+fo2369Ony8PBw/rY8uPrtc5aZmamZM2cqKChIDzzwQBFVBQAAbid5ZSxJaty48a0t5hp35D3Nt8KUKVO0bds2NWnSRF5eXlqzZo3WrFmj/v373/Cv9rxTDRo0SOfPn1e9evV08eJFrVixQps3b9akSZNu2VZ/AADg9rZ06VItWLBALVu2VEBAgDZu3KglS5aoRYsWRfqpNLdnFNBHH32k8ePHKyUlRWfOnNHdd9+tHj16aNSoUfLy4r1Ibt566y1NnTpVqampunDhgqpWraonnnhCTz31VFGXBgAAbhNff/21hg8frh07dujUqVMqW7asOnTooAkTJtzSfbx/i9AMAAAAWHBPMwAAAGBBaAYAAAAsCM0AAACAhdvfWCus33IH/JHwlQEAtxrXa+D6uXO9ZqUZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwIDQDAAAAFoRmAAAAwILQDAAAAFgQmgEAAAALQjMAAABgQWgGAAAALAjNAAAAgAWhGQAAALAgNAMAAAAWhGYAAADAgtAMAAAAWBCaAQAAAAtCMwAAAGBBaAYAAAAsCM0AAACABaEZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwIDQDAAAAFoRmAAAAwMKrqAtA0WrTpk2efatXr86z76mnnsp33tmzZ+fZl5mZaS8MAADgNsJKMwAAAGBBaAYAAAAsCM0AAACABaEZAAAAsCA0AwAAABaEZgAAAMDCYYwxbg10OG52LbgJSpUqlW//jh078uyrUKFCgR+3ePHiefadP3++wPP+3rj54wUAhYbr9Z3p4sWLefb5+Pjk2VeiRIl8583IyChoSXcUd67XrDQDAAAAFoRmAAAAwILQDAAAAFgQmgEAAAALQjMAAABgQWgGAAAALLyKugDcXI0aNcq3v6Dbyi1ZsiTf/gsXLhRoXgAAkFN+28rl59SpU4VcyR8XK80AAACABaEZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJoBAAAAC0IzAAAAYME+zXcAX1/fPPtGjRp1Ux7zzTffzLffGHNTHhcAgDvR5cuXb8q8XI8LDyvNAAAAgAWhGQAAALAgNAMAAAAWhGYAAADAgtAMAAAAWBCaAQAAAAuHcXMvEofDcbNrQQFFRUXl2bd169YCz3vlypU8+7y9vQs87x8JW/0AuNW4Xv8+3azrBa8H97jz/LPSDAAAAFgQmgEAAAALQjMAAABgQWgGAAAALAjNAAAAgAWhGQAAALAgNAMAAAAWXkVdAG5chw4dbsq869atuynzAgCAwsNezLcGK80AAACABaEZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJoBAAAAC7acuwM0atSowMdeunQpz75Ro0YVeF4AAOAqJSWlqEvADWClGQAAALAgNAMAAAAWhGYAAADAgtAMAAAAWBCaAQAAAAtCMwAAAGDhMMYYtwY6HDe7FuQjJiYmz75NmzYVeN6TJ0/m2RcSElLgeXGVmz9eAFBouF7fvm7WNYF/8xvnzr8NK80AAACABaEZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJoBAAAAC0IzAAAAYOFV1AXAPdHR0Tdl3ldeeeWmzAsAAAoPezEXPVaaAQAAAAtCMwAAAGBBaAYAAAAsCM0AAACABaEZAAAAsCA0AwAAABZsOfc7ERUVVaDj0tPT8+1nyzkAAAA7VpoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwIDQDAAAAFmw5d5to0KBBvv1du3Yt0LwZGRn59h88eLBA8wIAgJyeeeaZAh23atWqwi0EhY6VZgAAAMCC0AwAAABYEJoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwYJ/m20SpUqXy7ffwKNj7m48++qhAxwEAgOs3ffr0Ah3Xvn37wi0EhY6VZgAAAMCC0AwAAABYEJoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFW87dJjp27FjgY9PT0/PsmzNnToHnBQAArvz9/Yu6BBQRVpoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwIDQDAAAAFmw5dwtVqFAhz76uXbsWeN6DBw/m2ffVV18VeF4AAODqzJkzBT72008/LcRKcKux0gwAAABYEJoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwIDQDAAAAFuzTfAvFxMTk2efhUfD3L6tWrSrwsQAA4NZo3LhxUZeAG8BKMwAAAGBBaAYAAAAsCM0AAACABaEZAAAAsCA0AwAAABaEZgAAAMCCLeduoVKlShX42GPHjuXZN2PGjALPCwAAADtWmgEAAAALQjMAAABgQWgGAAAALAjNAAAAgAWhGQAAALAgNAMAAAAWbDl3C8XFxRX42B9//DHPvoyMjALPCwAAXFWuXLnAx2ZlZRViJbidsNIMAAAAWBCaAQAAAAtCMwAAAGBBaAYAAAAsCM0AAACABaEZAAAAsCA0AwAAABbs01zIvL298+yrUqVKgee9cOFCnn2XL18u8LwAAMDVnj17CnxsUFBQIVaC2wkrzQAAAIAFoRkAAACwIDQDAAAAFoRmAAAAwILQDAAAAFgQmgEAAAALtpwrZFlZWXn2ffXVV3n21axZM995U1NTC1wTAABw38WLF/Ps8/Pzy/fYs2fPFnY5uE2w0gwAAABYEJoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwIDQDAAAAFuzTXMgyMzPz7Bs1alSefcaYfOfdtm1bgWsCAADuK168eFGXgNsQK80AAACABaEZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJoBAAAAC4ex7XWWPdDhuNm1AHccN3+8AKDQcL0Grp8712tWmgEAAAALQjMAAABgQWgGAAAALAjNAAAAgAWhGQAAALAgNAMAAAAWhGYAAADAgtAMAAAAWBCaAQAAAAtCMwAAAGBBaAYAAAAsCM0AAACABaEZAAAAsCA0AwAAABaEZgAAAMCC0AwAAABYEJoBAAAAC0IzAAAAYEFoBgAAACwIzQAAAIAFoRkAAACwcBhjTFEXAQAAANzOWGkGAAAALAjNAAAAgAWhGQAAALAgNAMAAAAWhGYAAADAgtAMAAAAWBCaAQAAAAtCMwAAAGBBaAYAAAAs/h9ultkSnzCuIAAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    }
  ]
}