notebooks/ML/Linear Regression/Linear_Regression.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "55db894b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#help function\n",
    "def set_spines():\n",
    "    ax = plt.gca()  # gca stands for 'get current axis'\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('bottom')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('left')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2e40c482",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import math\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "from random import random\n",
    "#from mpl_toolkits.mplot3d import axes3d\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ab910c2",
   "metadata": {},
   "source": [
    "<h1> Linear Regression </h1>\n",
    "<div class='legend'>\n",
    "<ul style=\"margin-left: 30px\">\n",
    "      <li>1.0  <a href='#simple~linear~regression'>Simple Linear Regression </a> </li> \n",
    "      <li>1.1 <a href='#fitting~reg'> Fitting the regresion line</a> </li>\n",
    "      <li>1.2 <a href='#grad~sim~linear'>Gradient Descent over simple linear regression</a> </li>\n",
    "      <li>1.3 <a href='#rates-iterations'> Different Learning rates and number of Iteration in Gradient Descent</a> </li>\n",
    "      <li>1.4 <a href='#m-linear-r'>Multiple Linear Regression</a> </li>\n",
    "      <li>1.5 <a href='#reg'>Regularization of gradient descent by learning rate and max iterations</a> </li>    \n",
    "      <li>1.6 <a href='#conclusion'>Conclusion</a> </li>\n",
    "</ul> \n",
    "</div>    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a29f072d",
   "metadata": {},
   "source": [
    "<h2 id='intro~linear~regression'>Introduction </h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62a780fc",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d81b9a1",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "5c5c8e60",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "f66e5ad9",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "ac98678f",
   "metadata": {},
   "source": [
    "<h2 id='simple~linear~regression'>1.0 Simple linear regression </h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b0def14",
   "metadata": {},
   "source": [
    "Simple Linear regression has a single explanatory variable. It concerns two-dimensional sample points with one dependent and one independent variable. (Conventionally x and y data point in a Cartesian coordinate system). The adjective simple refers to the fact that the outcome variable is related to a single predictor."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59207bd2",
   "metadata": {},
   "source": [
    "<h2 href='fitting~reg'>1.1 Fitting the regression line with  least-squared </h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6145a977",
   "metadata": {},
   "source": [
    "Consider the model function $y = \\alpha + \\beta x$ which describes a line with slope $\\beta$ and intercept $\\alpha$. <br>\n",
    "example : let $\\alpha=7$ and $\\beta=3$, geometrically $y = 7 + 3 x$ appears to be as below graphic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ddba3760",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADrCAYAAADKbEVrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAkC0lEQVR4nO3de1iVVfr/8fc20jQ7aAmRaOpACAiedtI0qaXiaAcs7UuWzTBpUVnTwSbH8ttvciaDLOermVmMVlhTpOWIpXnW8hjiMSGTSS1BVEzMQyYi6/fHSsZGQFD2fjabz+u6vJDn2ex987C5r5v1rHUvlzEGERHxjnpOByAiUpco6YqIeJGSroiIFynpioh4kZKuiIgXKemKiHiRkq54ncvlesLlcmW7XK7NLpfrfZfLdYHL5WrqcrkWuFyu3J8/NnE6ThFPUNIVr3K5XM2BRwG3MaYdcB4wEBgBLDLGhAGLfv5cxO8o6YoTAoCGLpcrAGgE7AL6AWk/n08DbnMmNBHPcp1hRZqWq0mNGz9+PCNHjqRhw4b07t2bf/7zn1x66aUcOHCg7DFNmjShqKjotK9NTU0lNTUVgKNHj5Kdne2tsEWqw1XRCVW64lVFRUVkZGSwfft2du3axZEjR3j33Xer/PVJSUlkZWWRlZVFw4YNPRipiGco6YpXLVy4kNatW9OsWTPOP/98+vfvz8qVKwkKCqKgoACAgoICAgMDHY5UxDOUdMWrWrZsyerVq/nxxx8xxrBo0SIiIiKIj48nLc0O6aalpdGvXz+HIxXxjACnA5C6JTY2ljvuuINOnToREBBAx44dSUpK4vDhwyQkJDBlyhRatmzJ9OnTnQ5VxCN0I01qLbfbTVZWltNhiJRHN9JERHyBkq6IiBcp6YqIeJGSrojIqYyBqVPho4888vRKuiIiJ337LfTtC4mJ8M47HnkJJV0RkdJSmDgR2rWD5cthwgSYMcMjL6V5uiJSt339NQwZAitWwG9/C2+8AVdd5bGXU6UrInXT8eOQnAzt20NODrz9Nnz6qUcTLqjSFZG6aP16W92uXw8DBsCrr8IVV3jlpVXpikjd8dNP8MwzcM01sGuXnaHw4YdeS7igSldE6orly211u3Ur3HsvjB0LTby/K5QqXRHxb4cOwR//CN26QXExzJ8Pb77pSMIFJV0R8Wfz5tlpYBMn2sT75ZcQF+doSEq6IuJ/9u+3Cxz69IFGjezQwvjx0Lix05Ep6YqIn/nwQ4iIgPfeg5Ej7QyF665zOqoyupEmIv6hoAAeecSuJOvUyQ4tdOjgdFSnUaUrIrWbMfDWWxAZCbNnw4svwhdf+GTCBVW6IlKbbd8OSUmwcCF07QqTJ8PVVzsdVaVU6YpI7XPihL0x1q6drWpfew2WLvX5hAtKuuKAAwcOcMcdd9C2bVsiIiJYtWoV+/fvJy4ujrCwMOLi4igqKnI6TPFVOTm2qn38cejeHbKz4aGHoF7tSGe1I0rxK4899hh9+vRhy5YtbNy4kYiICFJSUujZsye5ubn07NmTlJQUp8MUX3P8ODz/PHTsaFeVvfuuHcNt0cLpyKpFuwGLVx08eJD27duzbds2XK7/bJgaHh7O0qVLCQ4OpqCggBtuuIGvv/660ufSbsB1yNq1MHgwbNoEd94Jr7wCgYFOR1UZ7QYsvmHbtm00a9aMe++9l44dO3Lfffdx5MgR9uzZQ3BwMADBwcHs3bvX4UjFJxw9Cn/+M3TpAoWFMHMmpKf7esKtlJKueFVJSQnr1q3joYceYv369Vx44YXVGkpITU3F7XbjdrspLCz0YKTiuM8/t71ux4yxjWpycqBfP6ejOmdKuuJVISEhhISEEBsbC8Add9zBunXrCAoKoqCgAICCggICK6hkkpKSyMrKIisri2bNmnktbvGigwdh6FB7k+zECVi0CFJT4dJLnY6sRijpilddccUVtGjRomy8dtGiRURGRhIfH09aWhoAaWlp9PODikbOwpw5dhrYG2/AsGF2DLdHD6ejqlFaHCFeN2HCBAYNGkRxcTFt2rThrbfeorS0lISEBKZMmULLli2ZPn2602GKN+3bB088YWckREbCypXw819D/kazF6TW0uwFP2AMTJ9ueyYUFdkGNU8/DQ0aOB3Zuapw9oIqXRFxxq5dduw2IwPcbjt2Gx3tdFQepzFdEfEuY2DKFDuMMG8evPwyrFpVJxIuqNIVEW/atg3uvx8WL7azEyZPhtBQp6PyKlW6IuJ5J07AuHG2ml2zxs5OWLy4ziVcUKUrIp6WnW0XN3zxBdx8M7z+OoSEOB2VY1TpiohnFBfDX/9qG9R88w3885/w8cd1OuGCKl0R8YQ1a2yDms2b4e677dCCVhACqnRFpCb9+CP86U9w7bV23u2sWbbCVcIto0pXRGrG0qVw3312KOGBB+xeZZdc4nRUPkeVroicmx9+sEn2xhvt50uW2JtlSrjlUtIVkbP38cd2kcPkyXZYYdMmuOEGp6PyaUq6IlJ9hYX2Bll8PFx2GaxeDS+9BI0aOR2Zz1PSFZGqMwbeew8iIuDDD2HUKMjKgmuucTqyWkM30kSkavLy7K67n3xi2y5Onmx730q1qNIVkcqVltplu5GRthPY3/8OK1Yo4Z4lVboiUrF//9s2qFm61O7g8I9/QJs2TkdVq6nSFZHTlZTYlovR0bB+vR1KWLhQCbcGqNIVkV/atMk2qMnKsrMTXnsNmjd3Oiq/oUpXRKxjx+D//T/o3Bm+/RY++ABmzlTCrWGqdEXEzrMdMgRycuCee2yDmssuczoqv6RKV7zuxIkTdOzYkVtuuQWA/fv3ExcXR1hYGHFxcRQVFTkcYR1y5Ijdhfe66+DgQZg9G955RwnXg5R0xevGjx9PRERE2ecpKSn07NmT3NxcevbsSUpKioPR1SEnN4IcNw4efNA2G7/pJqej8ntKuuJVeXl5zJ49m/vuu6/sWEZGBomJiQAkJiYyc+ZMh6KrIw4csN3AevWCgAD47DN7s+zii52OrE5Q0hWvevzxxxkzZgz16v3nrbdnzx6Cg4MBCA4OZu/evRV+fWpqKm63G7fbTWFhocfj9TsZGXaRw9tvw5//DBs3QrduTkdVpyjpitd88sknBAYG0rlz57N+jqSkJLKyssjKyqKZGmNX3Z49cOedcNttEBho9ytLSYGGDZ2OrM7R7AXxmhUrVjBr1izmzJnDTz/9xMGDB7nnnnsICgqioKCA4OBgCgoKCAwMdDpU/2GM3bnhscfg8GH4299shXv++U5HVmep0hWvSU5OJi8vjx07dpCenk6PHj149913iY+PJy0tDYC0tDT69evncKR+4rvv7O67v/sdhIfDhg3wv/+rhOswJV1x3IgRI1iwYAFhYWEsWLCAESNGOB1S7VZaCpMmQVQUfP45vPIKLFtm2zGK41zGmMrOV3pSxElut5usrCynw/AtW7famQnLlkFcHKSmQqtWTkdVF7kqOqFKV8QflJTAmDHQvj18+SW8+SbMm6eE64N0I02kttu4EQYPhnXr4PbbYeJE+HkKnvgeVboitdVPP9kbY2435Ofb7XNmzFDC9XGqdEVqo5Ur7djtV19BYqLdzaFpU6ejkipQpStSmxw+bOfcXn+9bVbz6ad2dZkSbq2hSlektpg/H5KS7Pzbhx+GF16Aiy5yOiqpJlW6Ir6uqAjuvRd++1u44AI793bCBCXcWkpJV8SXzZhhG9S88w4884xdVXb99U5HJedAwwsivmj3bnjkEfjoI+jYEebMsR+l1lOlK+JLjLE3xiIj4ZNPIDnZdgRTwvUbqnRFfMWOHfDAA/aG2fXX223Pw8OdjkpqmCpdEaeVlsKrr0K7dnb+7auv2t0clHD9kipdESdt2WIXOaxYYWcnvPEGXHWV01GJB6nSFXHC8eN2nm379nbb87Q0u9BBCdfvqdIV8bZ162DIEDv963/+x865DQpyOirxElW6It5y9Cg8/TR06WKnhM2YAdOmKeHWMap0Rbxh+XJb3W7daleXjR0LTZo4HZU4QJWuiCcdOmQXOXTtCsXFsGCBbTCuhFtnKemKeMrcuXYa2Guv2c5gX34JvXo5HZU4TElXvGrnzp3ceOONREREEBUVxfjx4wHYv38/cXFxhIWFERcXR1FRkcORnoPvv7c9bvv2hQsvtNPBxo2Dxo2djkx8gJKueFVAQABjx47lq6++YvXq1UycOJGcnBxSUlLo2bMnubm59OzZk5SUFKdDrT5j7O4NkZHw3nswciSsXw+//rXTkYkPUdIVrwoODqZTp04AXHTRRURERJCfn09GRgaJiYkAJCYmMnPmTAejPAsFBTBggJ0C1qIFZGXB889DgwZORyY+RklXHLNjxw7Wr19PbGwse/bsIfjnvb2Cg4PZu3dvuV+TmpqK2+3G7XZTWFjozXDLZ4y9MRYRYRc3jBkDq1fbRQ8i5XAZYyo7X+lJkbN1+PBhunfvzsiRI+nfvz+XXnopBw4cKDvfpEmTM47rut1usrKyPBxpJbZvtzs5LFwI3brZBjVhYc7FI77EVdEJVbridcePH2fAgAEMGjSI/v37AxAUFERBQQEABQUFBAYGOhli5U6cgPHj7cyEL76ASZNgyRIlXKkSJV3xKmMMQ4YMISIigmHDhpUdj4+PJy0tDYC0tDT69evnVIiVy8mxc24ffxxuuAGys+HBB6GefpWkajS8IF61fPlyunbtSnR0NPV+TlQvvPACsbGxJCQk8N1339GyZUumT59O0zPscOvV4YXiYjte+7e/2b3Jxo+Hu+8GV4V/RUrdVuEbQ0lXai2vJd2sLLuEd9MmGDjQJlxfHv4QX6AxXZFqO3oUhg+H2FjYtw9mzoT331fClXOihjci5fnsM9tc/N//hvvvt0MLl17qdFTiB1Tpipzq4EF46CF7k6y0FBYtgtRUJVypMUq6IifNmQNRUTbJDhtmx3B79HA6KvEzSroi+/bBPffAzTfDxRfbzSHHjrXNakRqmJKu1F3GwAcf2AY1H3wAf/mL3UonNtbpyMSP6Uaa1E27dtmx21mzwO22Y7fR0U5HJXWAKl2pW4yxPRIiI2H+fHj5ZVi1SglXvEaVrtQd33xjG9QsXgzdu9vkGxrqdFRSx6jSFf934gT8/e+2ms3KgjfesIlXCVccoEpX/NvmzXYJb2Ym3HKL7QgWEuJ0VFKHqdIV/1RcDKNGQadOsG2bXb47a5YSrjhOla74n8xMGDzYtl286y7boKZZM6ejEgFU6Yo/+fFHePJJuxHkgQPw8cd2g0glXPEhqnTFPyxZYhvUbNtmm4qnpMAllzgdlchpVOlK7fbDD3YaWI8edveGpUvtzTIlXPFRqnSl9vrhB7vIYfdu+NOf7I2zRo2cjkqkUkq6UvsUFsKjj9pet9HRkJFhl/KK1AJKulJ7GGOnfj36qO17e+WVdrFD/fpORyZSZRrTFZ8xd+5cwsPDCQ0NJSUl5Zcn8/IgPh4GDbIrydavh+BgJVypdZR0xSecOHGChx9+mE8//ZScnBzef/99cnJy7O4Nb7xhx24XL7bLeVessM3GRWohDS+IT8jMzCQ0NJQ2bdoAMHDgQD6fMoXItWvtfmU9e9odHX4+L1JbVboFe58+fcy+ffvO+skLCwtp5oMT0xVX9XgjrqKiIg4ePMhVV10FxvDj9u00LCrCdd55dunu5ZeXxXLyPXns2DE6dOjg0bjORl3+OZ4Nf4xr7dq184wxfco9aYyp7N856dy587k+hUcorurxRlzTpk0zQ4YMMWbTJmPcbmPAbGzd2pj8/Aq/plGjRh6P62zU5Z/j2fDTuCrMqxpeEJ/QIjCQG5csgbQ0aNqUf911F1vatSPmyiudDk2kRulGmjhv9Wpihw5l0LZtHLr1Voo3bOC57Gzi+/VzOjKRGufRpJuUlOTJpz9riqt6PBbXkSPwxBNw3XW4Dh1izahRdN68mYjrrychIYGoM8xQuPzncV5fU+d+jueorsVV6Y00oNKTImdt0SK4/37Yvh2GDoXkZLv9eTW43W6ysrI8FKDIOXFVdELDC+JdBw7YnRx69YKAADsdbOLEaidckdpKSVe8Z+ZMu8ghLQ1GjICNG6FbN6ejEvGqc0q606dPJyoqinr16p32Z15ycjKhoaGEh4czb968cr9+//79xMXFERYWRlxcHEVFRecSToXuvPNOOnToQIcOHWjVqlWFcztbtWpFdHQ0HTp0wO2FBirPPfcczZs3L4ttzpw55T6u0uWxHvDUU0/Rtm1bYmJiuP322zlw4EC5j6vy9dqzBxIS4PbbITAQvvjCDic0bHjGWM70vRtjePTRRwkNDSUmJoZ169ZV9ds8Jzt37uTGG28kIiKCqKgoxo8ff9pjli5dyiWXXFL28/3rX//qldjO9HNx4pp9/fXXZdehQ4cOXHzxxYwbN+4Xj/HW9Ro8eDCBgYG0a9eu7FhVc1GN/C5WNp/sTBPRcnJyzJYtW0z37t3NmjVryo5nZ2ebmJgY89NPP5lt27aZNm3amJKSktO+/qmnnjLJycnGGGOSk5PN8OHDz2FaXNUMGzbMjBo1qtxzV111lSksLPR4DCf95S9/MS+99FKljykpKTFt2rQx33zzjTl27JiJiYkx2dnZHo1r3rx55vjx48YYY4YPH17hz+WM16u01JipU41p2tSY+vWNGT3amOLiKsdxpu+9c+fOZvbs2aZPnz6mtLTUrFq1ynTp0qXKz38udu3aZdauXWuMMebgwYMmLCzstJ/LkiVLzM033+yVeE51pp+LU9fspJKSEhMUFGR27Njxi+Peul6fffaZWbt2rYmKiio7VpVcVM3fxQrz6jlVuhEREYSHh592PCMjg4EDB9KgQQNat25NaGgomZmZ5T4uMTERgMTERGbOnHku4ZyRMYZp06Zx1113efR1atKpy2Pr16/PwIEDycjI8Ohr9u7dm4AAO4X72muvJS8vr/pP8t13cPPN8PvfQ3g4bNgAzzwD559f5aeoyveekZHB73//e1wuF9deey0HDhygoKCg+vFWU3BwMJ06dQLgoosuIiIigvz8fI+/bk1w6pqdtGjRIn71q1/Z1YcO6NatG02bNv3Fsarkopr6XfTImG5+fj4tWrQo+zwkJKTcN+SePXsIDg4G7Jt47969nginzLJlywgKCiIsLKzc8y6Xi969e9O5c2dSU1M9GstJr776KjExMQwePLjcP2mqei095c0336Rv377lniv3epWWwmuv2YY0n38Or7wCy5ZBRES1X7sq37vT1wdgx44drF+/ntjY2NPOrVq1ivbt29O3b1+ys7O9Es+Z3sdOX7P09PQKCx8nrhdULRfV1HU744q0Xr16sXv37tOOjx49mn4VTF435UxDc7kqnEFRI6oS5/vvv19plbtixQquvPJK9u7dS1xcHG3btqXbOd7oqSyuhx56iGeffRaXy8Wzzz7Lk08+yZtvvvmLx3nqWlbleo0ePZqAgAAGDRpU7nP89/Xq2KgR16Sm2iQbF2cb1LRqddYxVuV7d+K9dqrDhw8zYMAAxo0bx8X/NQOjU6dOfPvttzRu3Jg5c+Zw2223kZub6/GYzvQ+dvKaFRcXM2vWLJKTk08759T1qqqaum5nTLoLFy6s9pOGhISwc+fOss/z8vK4spzlnEFBQRQUFBAcHExBQQGBgYHVfq2qxllSUsKMGTNYu3ZthY85GWNgYCC33347mZmZ55x0q3r97r//fm655ZbTjlf1WtZ0XGlpaXzyyScsWrSowjdW2fVq2pSxzZrR8d57oXFjeOstSEyEc/xFrsr37qnrUxXHjx9nwIABDBo0iP79+592/tQkfNNNNzF06FD27dvn8UUdZ3ofO3nNPv30Uzp16kRQUNBp55y6XlC1XFRT180jwwvx8fGkp6dz7Ngxtm/fTm5uLl26dCn3cWlpaYD9Ja+ocq4JCxcupG3btoSEhJR7/siRIxw6dKjs//Pnz//F3U1POHUc7V//+le5r3fNNdeQm5vL9u3bKS4uJj09nfj4eI/GNXfuXF588UVmzZpFowr2HCu7Xhs2cOKaa+i1aBH7YmPhq6/gD38454QLVfve4+PjmTp1KsYYVq9ezSWXXFL2Z6InGWMYMmQIERERDBs2rNzH7N69u6w6yszMpLS0lMsuu8yjcVXlfezUNYPK/9p04nqdVJVcVGO/i5XdZTvTXcAZM2aY5s2bm/r165vAwEDTu3fvsnPPP/+8adOmjbn66qvNnDlzyo4PGTKkbKbDvn37TI8ePUxoaKjp0aOH+f7778/0kmctMTHRTJo06RfH8vPzTd++fY0xxnzzzTcmJibGxMTEmMjISPP88897LJaT7rnnHtOuXTsTHR1tbr31VrNr167T4jLG3m0OCwszbdq08Upcv/rVr0xISIhp3769ad++vXnggQdOi2tbTo55IzDQFIMpPO88M/2uuzwSS3nf+6RJk8ykSZNM586dTWlpqRk6dKhp06aNadeu3S9m0XjSsmXLDGCio6PLrtPs2bPLYjPGmAkTJpjIyEgTExNjYmNjzYoVKzweV0Xv41PjcuqaHTlyxDRt2tQcOHCg7JgT12vgwIHmiiuuMAEBAaZ58+Zm8uTJFeaic/hdrDCvahmwVN/KlXZV2ZYtdhjh73+H/7ob7A1aBiw+TMuApQYcPmw3hbz+ejh6FObOhbffdiThitRW6qcrVTN/PiQl2fm3Dz9sV5Q1bux0VCK1jipdqVxREQweDL/9LVxwgZ0ONmGCEq7IWVLSlYrNmGEb1EydaleTbdgAv/mN01GJ1GoaXpDT7d4NjzwCH30EHTvCnDn2o4icM1W68h/G2BtjkZHwySeQkgKZmUq4IjVIla5YO3bAAw/YG2bXXw+TJ9tGNSJSo1Tp1nWlpfbGWLt2dv7txIl2NwclXBGPUNKty7Zsga5d7dzbrl0hO9vuV1bPM2+LypqjV6XpvYg/UNKti44fhxdegPbtbeKdOtXeLGvZ0qMvGxcXx+bNm9m0aRNXX311WaepnJwc0tPTyc7OZu7cuQwdOpQTJ054NBYRpyjp1jXr1kGXLjByJPTrBzk58Lvf1UiDmjOpqDl6VZvei/gDJd264uhRePppm3B377ZzcKdNg3Ja7HnDqc3RnW6qLeJNmr1QFyxfbhvUbN1qV5e9/DI0aeKRlzqb5ujlNV2qqIdvampq2W4IhYWFNRW2iNco6fqzQ4dsdTtxot3BYcEC6NXLoy95Ns3Rq9McOikpiaSkJACv7NgsUtM0vOCvPv3U7lP22mvw+OPw5ZceT7hnUlFz9Ko2vRfxB6p0/c3338MTT8A779jNIFesgF//2umoAHjkkUc4duwYcXFxgL2Z9vrrrxMVFUVCQgKRkZEEBAQwceJEzjvvPIejFfEMNTH3F8bAhx/angn799thhZEjoUEDpyPzGDUxFx9W4XQgVbr+oKDALmqYORM6d7ZLedu3dzoqESmHxnRrM2PgzTftMMLcuTBmDKxerYQr4sNU6dZW27bZnRwWLYJu3eAf/4Crr3Y6KhE5A1W6tc2JEzBuHERH27aLkybBkiVKuCK1hCrd2iQnxy5yWL0abroJXn8dTlnJJSK+T5VubVBcDH/7m20mnpsL775rm4wr4YrUOqp0fV1Wlq1uN22CgQNh/HgIDHQ6KhE5S6p0fdXRozB8OMTGwr59kJEB77+vhCtSy6nS9UWffQb33Qf//jfcfz+89BJcconTUYlIDVCl60sOHoSHHoIbbrDb6CxaBKmpSrgifkRJ11fMnm0b1KSmwrBhtkFNjx5ORyUiNUxJ12mFhTBoENxyi61oV66EsWPhlC5cIuI/lHSdYgykp0NkJEyfDs89Z7fSiY11OjIR8SDdSHNCfr4du/34Y7t9zpQpdgt0EfF7qnS9qbTUjtlGRsLChXYYYeVKJVyROkSVrrd8842dBrZ0qZ2d8I9/QGio01GJiJep0vW0EydsRRsdbcdsU1Nh8WIlXJE6SpWuJ23ebHffXbMGbr3VdgRr3tzpqETEQap0PaG42M5G6NQJduywsxQyMpRwRUSVbo3LzLTVbXa2nX87bhxcfrnTUYmIj1ClW1N+/BGefNLuvPvDD7b14rvvKuGW4+WXX8blcrFv376yY8nJyYSGhhIeHs68efMcjE7Es1Tp1oQlS+zMhG3b4MEH4cUX4eKLnY7KJ+3cuZMFCxbQsmXLsmM5OTmkp6eTnZ3Nrl276NWrF1u3btU27OKXVOmeix9+sPuU9egB9erZ6WCTJinhVuKJJ55gzJgxuFz/2aE6IyODgQMH0qBBA1q3bk1oaCiZmZkORiniOUq6Z+vjj+0ihylT4KmnYONG6N7d6ah82qxZs2jevDnt/2u34vz8fFqcsgtGSEgI+fn55T5Hamoqbrcbt9tNYWGhR+MV8QQNL1RXYSE8+qidkRAdbWcluN1OR+UzevXqxe7du087Pnr0aF544QXmz59/2jljzGnHTq2ET5WUlERSUhIAbl13qYWUdKvKGHjvPXjsMdv3dtQoGDEC6td3OjKfsnDhwnKPf/nll2zfvr2sys3Ly6NTp05kZmYSEhLCzp07yx6bl5fHlVde6ZV4RbzNVV6VcYpKT9YZO3faBjWzZ8O119ohhchIp6Oq1Vq1akVWVhaXX3452dnZ3H333WRmZrJr1y569uxJbm7uGW+kud1usrKyvBSxSLWU/6caqnQrd7JBzfDhdjnvuHHwyCOgu+o1KioqioSEBCIjIwkICGDixImauSB+S5VuRXJz7f5kn30GPXva5NumjdNRySlU6YoPq7DS1eyF/1ZSYjeCjImBDRvsUMKCBUq4IlIjNLxwqo0bYcgQWLsW+vWD114D3dARkRqkShfg2DF49lk79WvnTpg2Df71LyVcEalxqnRXrbLV7Vdfwe9+B//3f3DZZU5HJSJ+qu5WuocPw+OPw29+Y/8/Zw5MnaqEKyIeVTcr3QULbM+EHTvg4YchORkuusjpqESkDqhblW5RkR1K6N3briT7/HN49VUlXBHxmrqTdGfOhKgoSEuzy3c3boSuXZ2OSkTqGP8fXtizB/74R5g+Hdq3t93BOnd2OioRqaP8t9I1xt4Yi4iwncBGj7YbRCrhioiD/LPS/e47eOABmDsXrrvOripr29bpqERE/KzSLS21q8iiomDZMnjlFftRCVdEfIT/VLpff233KVu+HOLibIOaVq2cjkpE5Bdqf6VbUgIpKfYm2ebN8NZbMG+eEq6I+KTaXelu2GDn3a5bBwMG2Dm3V1zhdFQiIhWqnZXuTz/ByJFwzTWQnw8ffmj/KeGKiI+rfZXuihW2uv36a/jDH2DsWGja1OmoRESqpPZUuocP2114u3a1le68eXb8VglXRGqR2pF0582Ddu3smO0f/2hvmPXu7XRUIiLV5ttJd/9+O4TQpw80bGjn3I4fD40bOx2ZnKUJEyYQHh5OVFQUw4cPLzuenJxMaGgo4eHhzJs3z8EIRTzLd8d0P/rItl38/nt45hm7s8MFFzgdlZyDJUuWkJGRwaZNm2jQoAF79+4FICcnh/T0dLKzs9m1axe9evVi69at2hFY/JLvVboFBXb61x13QPPmtl/C6NFKuH5g0qRJjBgxggYNGgAQGBgIQEZGBgMHDqRBgwa0bt2a0NBQMjMznQxVxGN8J+kaA2+/DZGRMHu2XfDwxRfQoYPTkUkN2bp1K8uWLSM2Npbu3buzZs0aAPLz82nRokXZ40JCQsjPzy/3OVJTU3G73bjdbgoLC70St0hN8o3hhR077E4OCxbY2QmTJ8PVVzsdlZyFXr16sXv37tOOjx49mpKSEoqKili9ejVr1qwhISGBbdu2YYw57fEul6vc509KSiIpKQkAt9tds8GLeIGzSbe0FCZOhKefBpfL/v/BB6Ge7xTgUj0LFy6s8NykSZPo378/LpeLLl26UK9ePfbt20dISAg7d+4se1xeXh5Xaidm8VPOZbevvrJV7cm5t9nZMHSoEq4fu+2221i8eDFghxqKi4u5/PLLiY+PJz09nWPHjrF9+3Zyc3Pp0qWLw9GKeIb3K93jx+Gll2DUKDv1a+pUuOceW+mKXxs8eDCDBw+mXbt21K9fn7S0NFwuF1FRUSQkJBAZGUlAQAATJ07UzAXxW67yxtNOUenJalu3zi7h3bABEhJsv9ugoBp9Cak73G43WVlZTochUp4Kq0jv/C1/9KjdDLJLF9i9G2bMgA8+UMIVkTrH88MLy5bZ5uJbt9oq96WXoEkTj7+siIgv8lyle+iQXVHWrRsUF9vpYJMnK+GKSJ3mmUp340a49VbIy4PHH4fnn4cLL/TIS4mI1CaeSbqtW9utz6dNg2uv9chLiIjURp5JuhdfbNsxiojIL2glgoiIFynpioh4kZKuiIgXKemKiHiRkq6IiBcp6YqIeJGSroiIFynpioh40ZlaO4r4LJfLNdcY08fpOESqQ0lXRMSLNLwgIuJFSroiIl6kpCsi4kVKuiIiXqSkKyLiRf8fL0VaTvetUc8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,10)\n",
    "y = 7*x + 3\n",
    "plt.plot(x,y,color='r')\n",
    "set_spines()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d038835c",
   "metadata": {},
   "source": [
    "Suppose we observe $n$ data pairs denoted with $(x_i,y_i), i = 1,2,...,n$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c6354037",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADvCAYAAABR/Qd9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAfGUlEQVR4nO3dfVBU59k/8O8RkJImStIILiARBBEQfFuFPk4SFfCladFgfoSM+ZWpptuJT6dtbLR2ppmknRjW2E611rGzT0we2vxGHmmNWDWSiNJaow++kJdCVRohhQUEFWI1iqD37w/cjbrnLLvLedmX72eGiZ6znL04wJXb69z3dUtCCBARkT5GGB0AEVEoYdIlItIRky4RkY6YdImIdMSkS0SkIyZdIiIdMemS7iRJekGSpAZJkv4uSdJ2SZK+IknSQ5IkvS9JUtPt/z5odJxEWmDSJV1JkhQP4AcAzEKIyQDCAJQAWAugRgiRCqDm9t+Jgs5QSVfwgx9qfrS1tbUlJCSMv3jx4hcDAwPiiSee+D/V1dVvTZw4cXV7e3sZANHe3l42ceLENUNda+HChYZ/Pfzgh8KHIo50SVfx8fF48cUXkZiYCJPJhNGjR2P+/Pk4f/48TCYTAMBkMqGrq2vIa124cEHrcIlUx6RLuurp6UFVVRWam5vR3t6Oq1ev4u233/b48202G8xmM8xmM7q7uzWMlEgbTLqkqwMHDiApKQljxoxBREQEioqK8MEHHyA2NhYdHR0AgI6ODsTExMh+vsViwYkTJ3DixAmMGTNGz9CJVMGkS7pKTEzEsWPH8MUXX0AIgZqaGqSnp6OwsBDl5eUAgPLycixevNjgSIm0EW50ABRacnJy8NRTT2H69OkIDw/HtGnTYLFYcOXKFRQXF2Pbtm1ITExEZWWl0aESaUIaorWj25NERjKbzThx4oTRYRDJkZROcKRLRHSHXfV2bKg+g/bea4iLjsLqBWlYMi1etesz6RIR3bar3o6f7vwE1/pvAgDsvdfw052fAIBqiZcP0oiIbttQfcaZcB2u9d/Ehuozqr0HR7pEFPDUKgm0917z6rgvmHSJKKANVRLwJiHHRUfBLpNg46KjVIuX5QUiCmjuSgKOhGzvvQaBLxPyrnq77LVWL0hDVETYXceiIsKwekGaavEy6RJRQHNXEvC2RrtkWjzKirIQHx0FCUB8dBTKirI4e4GIyMFdScBdQlYqOzg+tMKRLhEFNHclAaVa7OioCK/KDmpi0iWigOauJKCUkCUJmk8NU8LyAhEFPKWSgOPYvWWEF/7nQ9nrqDk1TAmTLhEFNbmEvKH6jOZTw5SwvEBEIUePqWFKONIlopCjVHbQctaCA5MuEYUkraeGKWF5gYhIRxzpEpFhtO5d64+YdInIEHr0rvVHLC+Q7np7e/HUU09h0qRJSE9Px9GjR3Hp0iUUFBQgNTUVBQUF6OnpMTpM0pgevWv9EZMu6e6HP/whFi5ciNOnT+Ojjz5Ceno6rFYr8vLy0NTUhLy8PFitVqPDJI3p0bvWHzHpkq4uX76Mv/71r1ixYgUAYOTIkYiOjkZVVRVKS0sBAKWlpdi1a5eBUZIelBYi6LFAwUhMuqSrc+fOYcyYMfjOd76DadOm4bnnnsPVq1dx/vx5mEwmAIDJZEJXV5fBkZLWjFygYCQmXdLVwMAATp06heeffx719fX46le/6lUpwWazwWw2w2w2o7u7W8NISWt69K71R5IQwt15tyeJvNXZ2Ync3Fy0tLQAAA4fPgyr1Yp//vOfqK2thclkQkdHB+bMmYMzZ9w/UDGbzThx4oQOURN5TVI6wZEu6Wrs2LEYN26cM6HW1NQgIyMDhYWFKC8vBwCUl5dj8eLFRoZJpBnO0yXdbd68GcuWLcONGzeQnJyMt956C7du3UJxcTG2bduGxMREVFZWGh0mkSZYXqCAxfIC+TGWF4iI/AHLC0Qkyx/7IvhjTN5i0iUiF2r2RVArUQZLrwaWF4jIhVp9ERyJUo1dd4OlVwOTLhG5UKsvgpqJMlh6NbC8QEQu4qKjVNm40ddEKVeSUCsmo3GkS0Qu1OqL4EtTG6WSxNxJY4KiVwOTLhG5UKsvgi/JW6kkceh0d1D0amB5gYhkqbFxoy+77rorSRi1maSamHSJyCveTgHzNlEGS+1WCcsLROQxNaeAKQn2Prsc6RKRx4aaAqbGIghfShKBhA1vKGCx4Y3+ktbuVUwKURFhdyXkqIiwgHzQpRI2vCGi4VOqq4ZJUlCsFtMDky4ReUyp3npT4V/MgbZaTA9MukRBZFe9HbOtB5G0di9mWw+q+oALUJ6/Gx+iO/v6gg/SiIKEXl24lKaA3fneQHDNOFATR7pEQcLILlyhurOvLzjSJd3dvHkTZrMZ8fHx2LNnDy5duoSnn34aLS0tGD9+PHbs2IEHH3zQ6DADjtFduIJhtZgeONIl3W3atAnp6enOv1utVuTl5aGpqQl5eXmwWq0GRudfvKnR+tJchvTHpEu6amtrw969e/Hcc885j1VVVaG0tBQAUFpail27dhkUnX9xt/pLLhkH+0quYMGkS7r60Y9+hNdffx0jRnz5o3f+/HmYTCYAgMlkQldXl1Hh+RWlGu0ruxtkkzEA1lUDAGu6pJs9e/YgJiYGM2bMQG1trU/XsNlssNlsAIDu7m4Vo/M/SrXY3mv9LsccD8yOrJ0nm2SDYUPHYMGkS7o5cuQIdu/ejX379uH69eu4fPkynn32WcTGxqKjowMmkwkdHR2IiYlRvIbFYoHFYgEwuAw4mCl121KilKSDZUPHYMHyAummrKwMbW1taGlpQUVFBebNm4e3334bhYWFKC8vBwCUl5dj8eLFBkfqH5RqtA/eFyH7eqUHZsGyoWOw4EiXDLd27VoUFxdj27ZtSExMRGVlpdEh+QWlbluAdwsRhppKxtKDvthljAJWKHcZ8yZRzrYelC1TxN/+PLkEzgdww6bYZYxJlwJWKCddb9xb0wW+TKwbqs8oJuQja+fpGWawYWtHolDlbomu0avYQhFrukQhQGmJbrDvR+aPONIlCmFcxaY/jnSJQliw70fmj5h0iUIcu4Ppi+UFIiIdMekSEemISZeISEes6RLphMttCWDSJdIFO32RA8sLRDpgpy9yYNIl0gGX25IDywtEQ1CjFsvltuTAkS6RG+42h/QGl9uSA5MukRtq1WLddfqi0MLyAhGUSwhq1mK53JYAJl0it9O5WIsltbG8QLpqbW3F3LlzkZ6ejszMTGzatAkAcOnSJRQUFCA1NRUFBQXo6enRLSZ3JQRfarG76u2YbT2IpLV7Mdt60Ov6LwU3Jl3SVXh4OH71q1/hH//4B44dO4YtW7agsbERVqsVeXl5aGpqQl5eHqxWq24xuSsheFuLVevBGwUvlhdIVyaTCSaTCQDwwAMPID09HXa7HVVVVaitrQUAlJaWYs6cOVi/fr0uMQ1VQlCqxcrVgd2Nmt3Vc7lEOHRwpEuGaWlpQX19PXJycnD+/HlnMjaZTOjq6tItDl9LCHIjWrnkDbh/8MbRcWhh0iVDXLlyBUuXLsXGjRsxatQojz/PZrPBbDbDbDaju7tblVh8mc6lNKINk+Q3gXX34I1LhEMLywuku/7+fixduhTLli1DUVERACA2NhYdHR0wmUzo6OhATEyM7OdaLBZYLBYAg1uwq8Xb6VxKI9ebQiAqIsxlu3N3o2YuEQ4tHOmSroQQWLFiBdLT07Fq1Srn8cLCQpSXlwMAysvLsXjxYqNC9IjSyNUxSvZm1Kx0LU5LC06SEMLdebcnibz1t7/9DY8++iiysrIwYsTg//Nfe+015OTkoLi4GP/617+QmJiIyspKPPTQQ26vZTabceLECT3CdnHv3F5gcETryyozNa9FfkO+zgQmXQpgRiZdQN0ZB5y9EHSYdCn4GJ10idxQTLqs6RIR6YhJl4hIR0y6REQ6YtIlItIRky4RkY6YdImIdMSkS0SkIyZdIiIdseENhRSu/CKjMelSyHC3FxoTL+mF5QUKGexbS/6AI10KSnJlBPatJX/ApEtBR6mMEH1fBHq+6Hd5PfvWkp5YXqCgo1RGEAJe74VGpDYmXQo6SuWCz6/1e72rA5HaWF6goONuS3Vv90IjUhtHuhR0fNlSnUgvTLrkN/bv34+0tDSkpKTAarX6fB1ftlQn0gu36yG/cPPmTUycOBHvv/8+EhISMHPmTGzfvh0ZGRmKn2M2m/Gz/6riCjPyR9yuh/xbXV0dUlJSkJycjJEjR6KkpARVVVVuP6f3i378dOcnsPdeg8CXU8N21dv1CZrIB25HugsXLhQXLlzw+eLd3d0YM2aMz5+vFcblHT3i6unpweXLl/HII48AAC5evIirV68iMTERwGCC7bx8Hdf/3QNx7TIiwkagr68PI2OTXa4VETYCk8Y+oGm87oTy99EXwRjXyZMnq4UQC+XOaVpe8NfdWhmXd/SIq7KyEtXV1XjjjTcAAH/4wx9QV1eHzZs3uyx2AAYfjJ2xLkHij//kci0JQLP1CU3jdSeUv4++CNK4WF4g/5aQkIDW1lbn39va2hAXFwdAebGDEq4wI3/GebqkO7m+CN+cORMfNZzGjDX/DxfFV9H99hso2/xfANz3RoiKCHMZAXNqGPkzTZOuxWLR8vI+Y1zeUTMupb4IJz67hIhHV+DjN9YA4hbuzyrA1o9uIHGCXXGxQ8T9o1FWlOV3sxdC4fuoplCLi1PGSFezrQdlE2iYJOGmzM9i/O1EKlfT7f/TT/DPxo80jZfIR6zpkn9QKhXIJVzH65UWO0TfF6FhpETaYE2XdKVUKlAa6Toeisn1THhVmxCJNDWskW5lZSUyMzMxYsQIl6kVZWVlSElJQVpaGqqrq2U//9KlSygoKEBqaioKCgrQ09MznHAUPf3005g6dSqmTp2K8ePHY+rUqbKvGz9+PLKysjB16lSYzWZNYrnTK6+8gvj4eGds+/btk32dWstjPbV69WpMmjQJ2dnZePLJJ9Hb2yv7Ol/ul1JfhGdyxskez7u/3e3XLoTAD37wA6SkpCA7OxunTp3y7IscptbWVsydOxfp6enIzMzEpk2bXF5TW1uL0aNHO7+/v/jFL3SJbajvixH37MyZM877MHXqVIwaNQobN2686zV63a/ly5cjJiYGkydPdh7zNBep8rsohHD34VZjY6M4ffq0ePzxx8Xx48edxxsaGkR2dra4fv26OHfunEhOThYDAwMun7969WpRVlYmhBCirKxMrFmzZqi3HLZVq1aJn//857LnHnnkEdHd3a15DA4vv/yy2LBhg9vXDAwMiOTkZPHpp5+Kvr4+kZ2dLRoaGjSNq7q6WvT39wshhFizZo3i98XX+/XOqTbxH2U1YvxP9oj/KKsR75xqkz3+x+Ofuf3aZ8yYIfbu3SsWLlwobt26JY4ePSpmzZrlw1fsvfb2dnHy5EkhhBCXL18WqampLt+XQ4cOiSeeeEKXeO401PfFqHvmMDAwIGJjY0VLS8tdx/W6X3/5y1/EyZMnRWZmpvOYJ7nIy99Fxbw6rPJCenq67PGqqiqUlJQgMjISSUlJSElJQV1dHb7+9a+7vK62thYAUFpaijlz5mD9+vXDCcktIQR27NiBgwcPavYeartzeSwA5/JYdz0Jhmv+/PnOP+fm5uKPf/yjT9dR2nlXqb3ivcePHj065NdeVVWFb3/725AkCbm5uejt7UVHRwdMJpNPMXvKZDI53+OBBx5Aeno67Ha7pt8XtRh1zxxqamowYcIE5+pDvT322GNoaWm565gnuUit30VNHqTZ7XaMGzfO+feEhATY7a7r4c+fP+/8RptMJnR1dWkRjtPhw4cRGxuL1NRU2fOSJGH+/PmYMWMGbDabprE4/Pa3v0V2djaWL18u+08aT++lVt58800sWrRI9py7++WYGjacvgiefO1G3x8AaGlpQX19PXJyclzOHT16FFOmTMGiRYvQ0NCgSzxD/Rwbfc8qKirwzDPPyJ4z4n4BnuUite7bkCPd/Px8dHZ2uhxft24dFi9eLPs5QuaBiCQpzqBQhSdxbt++XfGbDQBHjhxBXFwcurq6UFBQgEmTJuGxxx7TLK7nn38eL730EiRJwksvvYQf//jHePPNN+96nVb30pP7tW7dOoSHh2PZsmWy13Dcr/+u+Qj/+WwRXv5LD5KzZmL1gjS3O+96Oo/Wk6/diJ+1O125cgVLly7Fxo0bMWrUqLvOTZ8+HZ999hnuv/9+7Nu3D0uWLEFTU5PmMQ31c2zkPbtx4wZ2796NsrIyl3NG3S9PqXXfhky6Bw4c8Pqi7pZ03ik2Ntb5z5qOjg7ExMR4/V6exjkwMICdO3fi5MmTiq9xxBgTE4Mnn3wSdXV1w066nt6/7373u/jmN7/pctzTe6l2XOXl5dizZw9qamoUf7Di4uKwq96O9bUdiJiQg+vtZ2EfN9llTu2dvNl515OvXav744n+/n4sXboUy5YtQ1FRkcv5O5PwN77xDaxcuRIXLlzAww8/rGlcQ/0cG3nP3n33XUyfPh2xsbEu54y6X4BnuUit+6ZJeaGwsBAVFRXo6+tDc3MzmpqaMGvWLNnXlZeXAxj8JVcaOavhwIEDmDRpEhISEmTPX716Ff/+97+df37vvffuerqphY6ODuef33nnHdn3mzlzJpqamtDc3IwbN26goqIChYWFHr/Hrno7ZlsPImntXsy2HvTon/f79+/H+vXrsXv3btx3332yr3Hcrw3VZ3D16lVcb67HyDGDNbpr/TcRppSoveiL4MnXXlhYiN///vcQQuDYsWMYPXq0LrVJIQRWrFiB9PR0rFq1SvY1nZ2dztFRXV0dbt26ha997WuaxuXJz7FR9wxw/69NI+6Xgye5aLi/i07unrIN9RRw586dIj4+XowcOVLExMSI+fPnO8+9+uqrIjk5WUycOFHs27fPeXzFihXOmQ4XLlwQ8+bNEykpKWLevHni4sWLQ72lz0pLS8XWrVvvOma328WiRYuEEEJ8+umnIjs7W2RnZ4uMjAzx6quvahaLw7PPPismT54ssrKyxLe+9S3R3t7uEpcQg0+bU1NTRXJysldxvXOqTUz62bvikZ/scX5M+tm7ztkCSiZMmCASEhLElClTxJQpU8T3vvc9l7gc9ytizHgR8bVEEf3o/73rfRzv5e1730vua9+6davYunWrmDFjhrh165ZYuXKlSE5OFpMnT75rFo2WDh8+LACIrKws533au3evMzYhhNi8ebPIyMgQ2dnZIicnRxw5ckTzuJR+ju+My6h7dvXqVfHQQw+J3t5e5zEj7ldJSYkYO3asCA8PF/Hx8eKNN95QzEXD+F1UzKtcBhzElJbcxkdH4cjaeV5fT242wobqM4rv4TivVV8Ef20JSAQ3y4C5Ii2IKdVPvamrOig1qlk6Ix5/OmmX7fTFnXeJXLH3QhBTqp/60m9WaTbCodPd3ASSyAsc6QYxpe5cvvSbdTdq5oiWyHMc6QYxNbciV3PUTBTKONINcmqNQtUcNROFMiZd8ogjcfvbLg1EgYZJlzw23FHz6tWr8ec//xkjR47EhAkT8NZbbyE6OhrAYCvQbdu2ISwsDL/5zW+wYMEClaIm8i+s6dJdfFnB5qmCggL8/e9/x8cff4yJEyc61983NjaioqICDQ0N2L9/P1auXImbN5V3+yUKZEy65KRGZzB35s+fj/DwwX9c5ebmoq2tDYByK1CiYMSkS07uOoOp7c6WkUa3GiTSE2u65KTGCjZfWkbKLUVX6mxms9mcPWK7u7s9jovIXzDpkpPSppHezMX1pWWkNy3zLBYLLBYLAOiyjx2R2lheICelTSPVmour1DLS01agRMGAI11y0nou7ve//3309fWhoKAAwODDtN/97nfIzMxEcXExMjIyEB4eji1btiAsLGyIqxEFJrZ2DFFKm0YGErZ2JD/G1o70JaU2jQACLvESBRrWdEOQnlPDiOhuTLohSM3m5kTkHZYXAowatVg1poYRkW840g0gai3T1XpqGBEp40hXB0qjU29Hre5qsd6Mdtmmkcg4TLoaU5opcOKzS3dt6HjnDAJAPiGqWYvlFjtExmDS1ZjS6HT7/7bi5j1zpK/138QruxvQN3BLNhm7q8UGw7xbolDAmq7GlEah9yZch95r/YolBKVa7NxJYzRtyUhE6mHS1ZjSjIAwhS5aShy77sptNHnodDfn3RIFCJYXNKa0oePSGfF31XQdx78SMQI9X/S7XMeRvOVqsS/8z4ey7815t0T+h0lXY+5mCpgfecjlOACvd93lvFuiwMGGN37I24di986QAAYTdVlRVlA/TGPDG/JjbHgTSLydzsV5t0SBg0k3SHDeLVFg4OwFIiIdMekSEemISZeISEdMuqS7X/7yl5AkCRcuXHAeKysrQ0pKCtLS0lBdXW1gdETa4oM00lVrayvef/99JCYmOo81NjaioqICDQ0NaG9vR35+Ps6ePcvNKSkocaTrpV31dsy2HkTS2r2YbT3I/gZeeuGFF/D6669DumMZdFVVFUpKShAZGYmkpCSkpKSgrq7OwCiJtMORrheG2tCRnb7c2717N+Lj4zFlypS7jtvtduTm5jr/npCQALud/zOj4MSk64WhNnTkDrtAfn4+Ojs7XY6vW7cOr732Gt577z2Xc3KrIiWFhkA2mw02mw0A0N3dPcxoifTHpOsFd03E1drVIdAdOHBA9vgnn3yC5uZm5yi3ra0N06dPR11dHRISEtDa2up8bVtbG+Li4mSvY7FYYLFYAAwuAyYKNKzpekGpgUxcdBR32B1CVlYWurq60NLSgpaWFiQkJODUqVMYO3YsCgsLUVFRgb6+PjQ3N6OpqQmzZs0yOmQiTTDpesHdho7uEjK5l5mZieLiYmRkZGDhwoXYsmULZy5Q0GKXMS+522QyFDt9GYldxsiPscuYWpQay7DTFxF5gklXRez0RURDYU2XiEhHTLpERDpi0iUi0hGTLhGRjkL+QZq7KWCciUBEagvppKvUwObEZ5fwp5P2kO+jQETqC+nyglK/hO3/2+q2sQ0Rka9COukq9UW4qbBKj30UiGi4QjrpKvVFCFNoK8g+CkQ0XCGTdOV2fFBqYPNMzjjFxjZERMMREknX8cDM3nsNAnc/GCsrykJ8dBQkAPHRUSgrysKrS7Jkj/MhGhENV0h0GZttPQi7TD02PjoKR9bOMyAiUgO7jJEfU+wyFhIjXTYYJyJ/ock8XSMXFsi9d1x0lOxIlw/GiEhvqo90leqnemxVrvTecyeN4YMxIvILqifdoXbM1ZLSex863c0HY0TkF1RPukbWT92995Jp8Tiydh6arU/gyNp5TLgG2bx5M9LS0pCZmYk1a9Y4j5eVlSElJQVpaWmorq42MEIibale0zWyfsrarX87dOgQqqqq8PHHHyMyMhJdXV0AgMbGRlRUVKChoQHt7e3Iz8/H2bNnuTklBSXVR7rudszVmpHvTUPbunUr1q5di8jISABATEwMAKCqqgolJSWIjIxEUlISUlJSUFdXZ2SoRJpRPekumRZvWP3UyPemoZ09exaHDx9GTk4OHn/8cRw/fhwAYLfbMW7cOOfrEhISYLdr/+CVyAiaTBkzcoNGbg5prPz8fHR2drocX7duHQYGBtDT04Njx47h+PHjKC4uxrlz5yC3QEdS6H9hs9lgs9kAAN3d3eoGT6SDkO6nS+o7cOCA4rmtW7eiqKgIkiRh1qxZGDFiBC5cuICEhAS0trY6X9fW1oa4uDjZa1gsFlgsFgCDK9KIAk1ArkiTa15D/m/JkiU4ePAggMFSw40bN/Dwww+jsLAQFRUV6OvrQ3NzM5qamjBr1iyDoyXShl+PdOVWlwGQ3e0B4K4O/m758uVYvnw5Jk+ejJEjR6K8vBySJCEzMxPFxcXIyMhAeHg4tmzZwpkLFLT8tuHNvVvpAIMzEb4SMQI9X/S7vJ7Na0IPG96QH1NseOO3I12l1WX3HnNg8xoiCgR+W9P1NolyAQQRBQK/TbpKSTQ6KoILIIgoYPlt0lVaXfZKYSYXQBBRwPLbmq4jiSr15WWSJaJA5LdJF+DqMiIKPn6RdI3caYKISE+GJ9175+NysQMRBTPDH6QZudMEEZHeDE+63KmXiEKJruUF7tRLRKFOt5Eud+olItIx6XKnXiIiHcsLQ+3UyyRLRKFAt5GuUo2WtVsiCiW6JV3u1EtEpGN5YaheCkREoUDXKWOs3RJRqDN8cQSFjg8//BC5ubmYOnUqzGYz6urqnOfKysqQkpKCtLQ0VFdXGxglkbYM771AoWPNmjV4+eWXsWjRIuzbtw9r1qxBbW0tGhsbUVFRgYaGBrS3tyM/Px9nz57l5pQUlDjSJd1IkoTLly8DAD7//HPExcUBAKqqqlBSUoLIyEgkJSUhJSXlrlEwUTDhSJd0s3HjRixYsAAvvvgibt26hQ8++AAAYLfbkZub63xdQkIC7Ha7UWESaYpJl1SVn5+Pzs5Ol+Pr1q1DTU0Nfv3rX2Pp0qXYsWMHVqxYgQMHDkAI4fJ6SZLfwdpms8FmswEAuru71Q2eSAeS3A88kRYkSfocQLQQQkiDWfVzIcQoSZJ+CgBCiLLbr6sG8IoQ4ugQ19svhFioeeBEKmJNl/TUDuDx23+eB6Dp9p93AyiRJClSkqQkAKkAhizqMuFSIGJ5gfT0XQCbJEkKB3AdgAUAhBANkiTtANAIYADAfwohbipfhihwsbxARKQjlheIiHTEpEtEpCMmXSIiHTHpEhHpiEmXiEhHTLpERDpi0iUi0hGTLhGRjv4/twX5VZQMJ7QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y = 7*x + 3 + np.random.uniform(-10,10,size=x.size)\n",
    "plt.scatter(x,y)\n",
    "set_spines()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a12f673",
   "metadata": {},
   "source": [
    "We can describe the underlying relationship between $y_i$ and $x_i$ invloving the error term $\\varepsilon_i$ in linear equation $y_i =  \\alpha + \\beta x_i + \\varepsilon_i $.The  $\\varepsilon_i$  could be intepreted as the noise (bias) of the data or in some sense some type of randomness originated from various reasons.Also it is know as 'error term' or 'residual'.\n",
    "Our target is to find estimated values $\\alpha$ and $\\beta$ which would provide the best fit in some sense for data points.\n",
    "In order to find the 'best' fit we will use the <a href='https://en.wikipedia.org/wiki/Ordinary_least_squares'>least-squared</a> approach.From  mathematical view point the issue involved the minimization of Lost function respect to $\\alpha,\\beta$. \n",
    " $$1)  L = \\sum_i(y_i - \\alpha - \\beta x_i)^2$$.\n",
    "$L$ is called $Lost$ function the expression $(y_i - \\alpha - \\beta x_i)^2$ is called $Cost$ function.This function is appropiative because of It is a diferencianal everywhere and gives us  a measure of diference between dependent value $y_i$ and predicted value of $y'(\\alpha,\\beta)=\\alpha + \\beta x_i $.<br>\n",
    "In order to find $min_{\\alpha,\\beta}L(\\alpha,\\beta,x_i,_yi)$ we will use <mark>Gradient Descent method</mark>.<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "595546ee",
   "metadata": {},
   "source": [
    " $$L = \\sum_i(y_i - w_0- w_1 x_i)^2  $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45c03b34",
   "metadata": {},
   "source": [
    "Therefore, ridge regression puts further constraints on the parameters, $w_j$'s, in the linear model. In this case, what we are doing is that instead of just minimizing the residual sum of squares we also have a penalty term on the $w$'s. This penalty term is $λ$ (a pre-chosen constant) times the squared norm of the $w$ vector. This means that if the $w_j$'s take on large values, the optimization function is penalized. We would prefer to take smaller $w_j$'s, or $w_j$'s that are close to zero to drive the penalty term small.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1af6ca01",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "16c98ef4",
   "metadata": {},
   "source": [
    "<h2 id='grad~sim~linear'>1.2 Gradient Descent over simple linear regression</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d344fd7b",
   "metadata": {},
   "source": [
    "The methods for finding the maximum and minimum (extrema) have been created with the development of mathematical analysis(Calculus). Gradient Descent is a numerical approach to the local minimum of a first-order differentiable function. The idea is to take repeated steps in the opposite direction of the gradient the function in the current point. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dac3515",
   "metadata": {},
   "source": [
    "Our purpose is to find the value of $x$ where the function gets the minimum value.\n",
    "The GD algorithm is difined : \n",
    "  $$\\alpha = \\alpha - \\nabla_{\\alpha}L(\\alpha,\\beta)*h$$\n",
    "  $$\\beta = \\beta - \\nabla_{\\beta}L(\\alpha,\\beta)*h$$ <br>\n",
    "  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b5fb6e7",
   "metadata": {},
   "source": [
    "where $h$ is called <mark>Learning rate</mark> or step size.\n",
    "Applying GD algorithm over our Lost function 1) we  get : <br> <br>\n",
    "$$\\nabla_{\\alpha}(J(a,b)=\\frac{\\partial{L}}{\\partial{{\\alpha}}}=\\frac{\\partial{\\sum_i(y_i - \\alpha - \\beta x_i)^2}}{\\partial{{\\alpha}}}=-\\frac{2}{n}\\sum_{n=1}^{n}{(y_i-a*x_i - b)*x_i} $$ <br>\n",
    "$$\\nabla_{\\beta}(J(a,b)=\\frac{\\partial{L}}{\\partial{{\\beta }}}=\\frac{\\partial{\\sum_i(y_i - \\alpha - \\beta x_i)^2}}{\\partial{{\\beta}}}=- \\frac{2}{n}\\sum_{n=1}^{n}{(y_i-a*x_i - b)} $$ <br> <br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce919dda",
   "metadata": {},
   "source": [
    "applying the above local derivatives to Gradient Descent formula we get\n",
    " $$\\alpha = \\alpha + \\frac{2}{n}\\sum_{n=1}^{n}{(y_i-a*x_i - b)*x_i}*h$$\n",
    "  $$\\beta = \\beta +\\frac{2}{n}\\sum_{n=1}^{n}{(y_i-a*x_i - b)}*h$$ <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cf53f47",
   "metadata": {},
   "source": [
    "The above equations are performed enough times to reach the minimum of Lost function with conditional that we have chosen the appropriate Learning rate $h$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "610b4789",
   "metadata": {},
   "source": [
    "<h2 id='grad~sim~linear'>1.3 Implementation of Gradient Descent for simple linear regression (Python)</h2>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "56dd372f",
   "metadata": {},
   "outputs": [],
   "source": [
    "#implementation cost function\n",
    "def cost(X,Y,alpha,betta):\n",
    "    '''\n",
    "    Calculation of cost (error) function of simple linear regression\n",
    "   \n",
    "    Parrameters : \n",
    "    X(array or number) : X arguments, independent variable\n",
    "    Y (array or number) : Y arguments, actual dependent variable \n",
    "    Returns :\n",
    "      number of cost function\n",
    "  '''\n",
    "   \n",
    "    return np.sum((Y - (alpha - betta*X ))**2 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "536d9766",
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradient_descent(X,Y,C=0.1,E=100):\n",
    "    '''\n",
    "    \n",
    "    Mimumization of cost (error) function of simple linear regression\n",
    "   \n",
    "    Parrameters : \n",
    "    X(array or number) : X arguments, independent variable\n",
    "    Y (array or number) : Y arguments, actual dependent variable \n",
    "    C (number) :  learning rate\n",
    "    I(int) : number of iteration(Epoch)\n",
    "    Returns :\n",
    "      number of cost function\n",
    "   '''\n",
    "   \n",
    "    a =1 \n",
    "    b = 1 \n",
    "    a_args = []\n",
    "    b_args =   []\n",
    "    n = X.size\n",
    "    \n",
    "    for i in range(E):\n",
    "        \n",
    "        a = a + 2/n*(np.sum((Y - a*X - b)*X))*C\n",
    "        b =  b + 2/n*(np.sum((Y - a*X -b)))*C\n",
    "        #print(a,b)\n",
    "        a_args.append(a)\n",
    "        b_args.append(b)\n",
    "    return a,b,a_args,b_args "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f899228",
   "metadata": {},
   "source": [
    "Applying this implemenation over our generated data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d1aa1384",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0xd7c5c64bb0>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD6CAYAAABEUDf/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAaeUlEQVR4nO3dfZAcd33n8feHRZg1D1k7XhtpJSFRZ+RYce5s5gxBuRx+yMk4xHKcMiVS3OkO16lIAQXUoViKq8BXicsK5riQSrgrHfadLjg2KmxkVbAj/ESlQiGZlSU/yJJigZ+0EtZyRiYpFCOJ7/0xvWi06p6d2e6eh57Pq0q1M9298/ttr/Y7v/n27/dtRQRmZlZNr+t2B8zMrDwO8mZmFeYgb2ZWYQ7yZmYV5iBvZlZhDvJmZhVWSJCX9GlJuyU9LekuSW+UdLakByU9m3w9q4i2zMysdco7T17SGPD3wIURcVTSJuB+4ELglYhYL2ktcFZE3Njstc4555xYtGhRrv6YmQ2aHTt2/CgiRtP2vb6gNl4PDEs6BpwJHATWAe9L9m8Evg00DfKLFi1ifHy8oC6ZmQ0GSS9k7cudromICeALwIvAIeDViPgWcF5EHEqOOQScm9G51ZLGJY1PTk7m7Y6ZmTXIHeSTXPsKYDEwD3iTpA+3+v0RsSEiahFRGx1N/bRhZmazVMSF1yuB5yJiMiKOAfcC7wVeljQXIPl6uIC2zMysDUUE+ReB90g6U5KAK4A9wBZgVXLMKuC+AtoyM7M25L7wGhHbJX0deBw4DuwENgBvBjZJuoH6G8H1edsyM7P2FDK7JiI+B3xu2ubXqI/qzcwsw+adE9y2dR8Hjxxl3sgwa5Yv4dqLxwp7/aKmUJqZWZs275xg3b1PcfTYCQAmjhxl3b1PARQW6F3WwMysS27buu8XAX7K0WMnuG3rvsLa8EjezKxNRaVYDh452tb22XCQNzNrQ7MUC9BW8J83MsxESkCfNzJcWH8d5M3M2pCVYrl5y25eO/7ztvLra5YvOeUNA2B4zhBrli8prL/OyZuZtSErlXLk6LG28+vXXjzGrdddxNjIMALGRoa59bqLPLvGzKxbslIsWabeFLLy+FP/yuKRvJlZG9YsX8LwnKFTtg3PGeKsM+ekHj9vZPgXefyJI0cJTqZyNu+cKL2/DvJmZm3ISrF87neWpgb/NcuXdGSqZBana8zM2tQsxZKWkvn013alHlvkVMksDvJmZgXJCv6dmCqZxekaM7OSZeXxi5wqmcUjeTOzkk2N7sssRJbFQd7MrAPKniqZxekaM7MK80jezAZG2bXbe5GDvJkNhE7Ubu9FTteY2UDo5oKkbiokyEsakfR1SXsl7ZH065LOlvSgpGeTr2cV0ZaZ2Wx0onZ7LypqJP8l4G8j4gLgXwJ7gLXAwxFxPvBw8tzMrCuyFh51YkFSN+UO8pLeCvwmcDtARPwsIo4AK4CNyWEbgWvztmVmNlvdXJDUTUWM5N8BTAL/W9JOSV+R9CbgvIg4BJB8PTftmyWtljQuaXxycrKA7piZna4Ttdt7kSIi3wtINWAbsCwitkv6EvAT4BMRMdJw3I8jomlevlarxfj4eK7+mJkNGkk7IqKWtq+IkfwB4EBEbE+efx24BHhZ0tykA3OBwwW0ZWZmbcgd5CPih8BLkqYSW1cAzwBbgFXJtlXAfXnbMjOz9hS1GOoTwJ2S3gD8APhP1N9ANkm6AXgRuL6gtszMrEWFBPmI2AWk5YOuKOL1zcxsdlzWwMx6Qi/WlenFPrXLQd7Muq7IujJFBeaq1Lpx7Roz67qi6spMBeaJI0cJTgbmzTsnutanbnOQN7OuK6quTJGBuSq1bpyuMbOuK+pG17MNzGkpnm7efLtIHsmbWdcVVVdmNkXIslI8l10wWolaNw7yZtZ1RdWVmc2bRVaK59G9k5WodeN0jZn1hCJudD31/e3MrmmW4unWzbeL5CBvZj2t3SmR7QbmquTeszhdY2Y9q8gpkVmqXmfeI3kz61kzTYksYtHTbFI8/SR3PfkiuZ68mTVavPabZEWo4TlDp7wBDM8Z6ssLo0Uou568mVkpsvLiQ1IlVqN2goO8mfWsrHz5iYwMRL+tRu0EB3kzm7XNOydYtv4RFq/9JsvWP1LoBVHInj8/NotFT4PKF17NbFY6VaUxa0pkY9tQrRkxRfJI3sxmpZtVGotaITsIPJI3s1npdpXGKqxG7YTCgrykIWAcmIiID0g6G/gasAh4HvhgRPy4qPbMrHjtrC6t+krRqihyJP9JYA/w1uT5WuDhiFgvaW3y/MYC2zOzAjXLscPpi4XWLF/ivHgfKCQnL2k+8NvAVxo2rwA2Jo83AtcW0ZaZlSMrx37zlt2ppQUA58X7QFEj+T8D/hB4S8O28yLiEEBEHJJ0bto3SloNrAZYuHBhQd0xs3Zl5dKPHD122rapC6zfWXt5alCvwg2wqyL3SF7SB4DDEbFjNt8fERsiohYRtdHR0bzdMbNZKuouTJ0oKmatKyJdswy4RtLzwN3A5ZK+CrwsaS5A8vVwAW2ZWUmyVpeedeac1OOz3hSqcgPsqsgd5CNiXUTMj4hFwErgkYj4MLAFWJUctgq4L29bZlaerLnnn/udpW2V4p1pamXZq2TtVGXOk18PbJJ0A/AicH2JbZlZAZrNPS9iamWnVsnaSS41bGaFmh7I4WQZ4Nu27kt9AxgbGeY7ay/vZDcrxaWGzaxjmpUc6PYq2UHksgZmVristI9XyXaeR/Jm1jFVv59qL/JI3sw6pur3U+1FDvJm1lGuHtlZTteYmVWYg7yZWYU5yJuZVZhz8mYV5UqQBg7yZpXk8gE2xekaswpyJUib4iBvVkEuH2BTnK4x6zFF5NJdPsCmeCRv1kOKuquSywfYFAd5sx5SVC69WSVIGyxO15h1QVZKpshcussHGDjIm3Vcs+mNzqVb0ZyuMeuwZimZ2eTSfc9UayZ3kJe0QNKjkvZI2i3pk8n2syU9KOnZ5OtZ+btr1v+apWTazaUXdaHWqquIdM1x4L9ExOOS3gLskPQg8B+BhyNivaS1wFrgxgLaM+trM6VksnLpaXn8Zp8KmuXjXfJgcOQeyUfEoYh4PHn8j8AeYAxYAWxMDtsIXJu3LbMqmG1KJm3EnvZmAc0v1Hr0P1gKzclLWgRcDGwHzouIQ1B/IwDOzfie1ZLGJY1PTk4W2R2znjSb6Y1ZI/YhKfX4ZhdqXfJgsBQ2u0bSm4F7gE9FxE+U8Z9vuojYAGwAqNVqUVR/zHpZu9Mbs0bmJyIYnjN0StCe6VOBSx4MlkJG8pLmUA/wd0bEvcnmlyXNTfbPBQ4X0ZbZIMoamU99CmjnU0HWa3maZjXlHsmrPmS/HdgTEV9s2LUFWAWsT77el7cts0G1ZvmSU+bWw8kRe7ufCpq9llVPEemaZcC/B56StCvZ9kfUg/smSTcALwLXF9CW2UCaCuJFzIgp8rWs9ymid9LgtVotxsfHu90NM7O+ImlHRNTS9nnFq5lZhTnIm5lVmIO8mVmFOcibmVWYg7yZWYU5yJuZVZiDvJlZhTnIm5lVmG//Z1Yi1223bnOQNytJs3u5OtBbpzhdY1YS1223XuCRvFkB0tIyrttuvcBB3iynrLTMyJlz+PFPj512vOu2Wyc5XWOWU1ZaJoK27+VqVjQHebOcstIvrx491vZdm8yK5nSNWU7zRoaZSAn080aG275rk1nRPJI3y2nN8iVOy1jP8kjeLCffTs96WelBXtJVwJeAIeArEbG+7DbNypK1gtVpGetVpQZ5SUPAXwK/BRwAvidpS0Q8U2a7ZmXwClbrR2WP5C8F9kfEDwAk3Q2sABzkraeljdibrWB1kLdeVXaQHwNeanh+AHh34wGSVgOrARYuXFhyd8xmljVinx7gp3gFq/WysmfXKGVbnPIkYkNE1CKiNjo6WnJ3zGaWNWIfUtp/Z69gtd5W9kj+ALCg4fl84GDJbZqdIutiadb2rJH5iQiG5wyd8gbgqZLW68oO8t8Dzpe0GJgAVgK/X3KbZr+QlXoZf+EV7tkxkXoRNWtx01hDbt5TJa1flBrkI+K4pI8DW6lPobwjInaX2aZZo6zUy13bX+JExGnbb9u6jzXLl5yWg58asXuqpPWb0ufJR8T9wP1lt2OWplnqJet4L26yKvGKV6u0rNTLkJQa6KcuonrEblXh2jVWaVl1ZT707gWuN2MDwSN5q7RmqZfa2892SsYqT5GRm+yGWq0W4+Pj3e6G9amsKZFmVSdpR0TU0vZ5JG+V4LoyZukc5K3vuK6MWesc5K2vuK6MWXsc5K1QZefFm9WVaTYl0mxQOchbYTqRF3ddGbP2eJ68FaZZXnw2Nu+cYNn6R1i89pssW/8Im3dOZI7Mx0aGufW6ixgbGUYNz52Pt0HnkbwVJmuUPZu8eNangt9719gphcXAdWXMmvFI3gqTNcqeTV4861PBo3snPWI3a4NH8laYZtUb29XsU4FH7Gat80jeCnPtxWOFjbKL/FRgNsg8krdCFTXKLvJTgdkgc5C3nuSa7mbFcJC3nuXcu1l+uXLykm6TtFfSk5K+IWmkYd86Sfsl7ZO0PHdPrZLS5sKbWXHyXnh9EPjViPg14B+AdQCSLqR+0+6lwFXAlyUNZb6KDaSpufATR44SnJwL70BvVpxcQT4ivhURx5On24D5yeMVwN0R8VpEPAfsBy7N05ZVT9ErZM3sdEVOofwI8EDyeAx4qWHfgWTbaSStljQuaXxycrLA7livK3KFrJmlmzHIS3pI0tMp/1Y0HHMTcBy4c2pTykul3oIqIjZERC0iaqOjo7P5GaxPeS68WflmnF0TEVc22y9pFfAB4Io4eS/BA8CChsPmAwdn20mrJs+FNytf3tk1VwE3AtdExE8bdm0BVko6Q9Ji4HzgsTxtWfUUuULWzNLlnSf/F8AZwIOSALZFxEcjYrekTcAz1NM4H4uI9Fv32EDIupmI58KblStXkI+If9Fk3y3ALXle36rBN9k26x4XKLPSeaqkWfc4yFvpPFXSrHtcu8aaKuLG3PNGhplICeieKmlWPo/kLVNRZQfWLF/C8JxTq1p4qqRZZ3gkX0FZo+92R+XNcuntjOZdNtisexzkKyZrJsv4C6+ccgPsxhkukB6Ai8yle6qkWXc4yFdM1uj7ru0vcSLitO03b9nNa8d/nhr8m+XSi8jVm1n5nJOvmKxR9vQAP+XI0WOZKZmsXPplF4y6RLBZn3CQr5isGStDSqsZl+3gkaOZZQce3Tvpee9mfcLpmorJKvr1e+8aOyUnP7X9jXNex49/euy015l6s0jLpX/6a7tS2/a8d7Pe4yBfMc1mstTefvZp24G2K0F63rtZ/3CQr6CsmSzNZri0cxHVJYLN+oeDvLU9vdHz3s36h4O8zYrnvZv1B8+uMTOrMAd5M7MKc5A3M6swB3kzsworJMhL+oykkHROw7Z1kvZL2idpeRHtmJlZe3LPrpG0APgt4MWGbRcCK4GlwDzgIUnv9M282+dCYGaWRxFTKP878IfAfQ3bVgB3R8RrwHOS9gOXAt8toL2BMdMNsP0GYGYzyRXkJV0DTETEEzq1ANYYsK3h+YFkW9prrAZWAyxcuDBPdypnphtgN3sDMDODFnLykh6S9HTKvxXATcBn074tZVtqrduI2BARtYiojY6Ottf7imt2046Z3gDMzKCFkXxEXJm2XdJFwGJgahQ/H3hc0qXUR+4LGg6fDxzM3dsB06wQWJF3bTKz6pr17JqIeCoizo2IRRGxiHpgvyQifghsAVZKOkPSYuB84LFCejxAmt0AO6vioytBmlmjUmrXRMRuSZuAZ4DjwMc8s6Z9MxUCcyVIM5uJIuO2cN1Qq9VifHy8293oG55dY2YAknZERC1tn6tQ9jFXgjSzmbisgZlZhTnIm5lVmIO8mVmFOcibmVWYL7x2WNaMGM+UMbMyOMh3UFbBsfEXXuGeHROuQ2NmhXO6poOy6s3ctf0l16Exs1I4yHdQVl2ZExkL0lyHxszycpDvoKy6MkNKK9rpOjRmlp+DfEk275xg2fpHWLz2myxb/wibd05kFhz70LsXZBYiMzPLw0G+BFMXWCeOHCU49ULqrdddxNjIMALGRoa59bqL+JNrL0rd7ouuZpaXC5SVYNn6R1LrwI+NDPOdtZd3oUdmVmXNCpR5JF8C39DDzHpFJebJd3MhUVrbze7oZGbWSX0/ks/Kf2/eOdG1ti+7YNQXUs2sJ/R9kO/mDa2z2n5076QvpJpZT+j7dE0389/N2vYNPcysF+QeyUv6hKR9knZL+nzD9nWS9if7ludtJ0s3b2jtm2mbWa/LFeQlXQasAH4tIpYCX0i2XwisBJYCVwFfljSU+UI5ZC0w6kT+u5ttm5m1Im+65g+A9RHxGkBEHE62rwDuTrY/J2k/cCnw3ZztnWYqJdKN2TXdbNvMrBW5FkNJ2gXcR320/s/AZyLie5L+AtgWEV9NjrsdeCAivp7yGquB1QALFy581wsvvDDr/piZDaJmi6FmHMlLegh4W8qum5LvPwt4D/CvgU2S3gGkVdxKfTeJiA3ABqiveJ2pP2Zm1roZg3xEXJm1T9IfAPdG/ePAY5J+DpwDHAAWNBw6HziYs69d47s2mVm/ypuT3wxcDnxb0juBNwA/ArYAfy3pi8A84HzgsZxtlS4tmAOpd3MC37XJzHpf3iB/B3CHpKeBnwGrklH9bkmbgGeA48DHIuJEk9fpuqxb871xzusyF1s5yJtZr8sV5CPiZ8CHM/bdAtyS5/U7KWv16vRtU1xszMz6Qd+XNShKu0HbC57MrB84yCeygvbI8BwveDKzvuUgn8havXrzNUtdbMzM+lbfFygrykyrVx3UzawfOcg3cOVIM6uagQzyXtxkZoNi4IJ81nx4cErGzKpn4C68dvNOUmZmnTZwQb6bd5IyM+u0Sqdr0nLv80aGmUgJ6F7cZGZVVNmR/FTufeLIUYKTuffLLhj14iYzGxiVDfJZufdH9056cZOZDYzKpmua5d49H97MBkVlR/JZOXbn3s1skFQ2yGfVonHu3cwGSWXTNTPVojEzGwSVDfLgWjRmZpVN15iZWc4gL+lfSdomaZekcUmXNuxbJ2m/pH2SlufvqpmZtStvuubzwH+NiAckXZ08f5+kC4GVwFJgHvCQpHf2+s28zcyqJm+6JoC3Jo9/CTiYPF4B3B0Rr0XEc8B+4NKU7zczsxLlHcl/Ctgq6QvU3zDem2wfA7Y1HHcg2XYaSauB1QALFy7M2R0zM2s0Y5CX9BDwtpRdNwFXAJ+OiHskfRC4HbgSUMrxkfb6EbEB2JC0NSnphRb7nuYc4Ec5vr8s7ld73K/2uF/tqWK/3p61QxGpsbclkl4FRiIiJAl4NSLeKmkdQETcmhy3Fbg5Ir4768Za6894RNTKbGM23K/2uF/tcb/aM2j9ypuTPwj82+Tx5cCzyeMtwEpJZ0haDJwPPJazLTMza1PenPx/Br4k6fXAP5Pk1iNit6RNwDPAceBjnlljZtZ5uYJ8RPw98K6MfbcAt+R5/VnY0OH2WuV+tcf9ao/71Z6B6leunLyZmfU2lzUwM6swB3kzswrrqyAv6XpJuyX9XFJt2r4Za+VIOlvSg5KeTb6eVVI/v5bU89kl6XlJuzKOe17SU1O1f8roy7T2bpY00dC3qzOOuyo5j/slre1Av26TtFfSk5K+IWkk47jSz9dMP7vq/jzZ/6SkS8roR0q7CyQ9KmlP8jfwyZRj3ifp1Ybf72c71Lemv5dunDNJSxrOwy5JP5H0qWnHdOR8SbpD0mFJTzdsaykWFfK3GBF98w/4FWAJ8G2g1rD9QuAJ4AxgMfB9YCjl+z8PrE0erwX+tAN9/m/AZzP2PQ+c08HzdzPwmRmOGUrO3zuANyTn9cKS+/XvgNcnj/806/dS9vlq5WcHrgYeoL7g7z3A9g797uYClySP3wL8Q0rf3gf8Taf+P7X6e+nWOZv2e/0h8PZunC/gN4FLgKcbts0Yi4r6W+yrkXxE7ImIfSm7Wq2VswLYmDzeCFxbSkcTyQKxDwJ3ldlOwS4F9kfEDyLiZ8Dd1M9baSLiWxFxPHm6DZhfZntNtPKzrwD+b9RtA0YkzS27YxFxKCIeTx7/I7CHjFIhPagr56zBFcD3IyLPavpZi4i/A16ZtrmVWFTI32JfBfkmxoCXGp5n1co5LyIOQf2PBji35H79G+DliHg2Y38A35K0I6nh0wkfTz4y35HxEbHVc1mWj1Af9aUp+3y18rN3+/wgaRFwMbA9ZfevS3pC0gOSlnaoSzP9Xrp9zlaSPdDqxvmC1mJRIeet5+4MpSa1ciLivqxvS9lW6tzQFvv5IZqP4pdFxEFJ5wIPStqbvOuX0i/gfwB/TP3c/DH1VNJHpr9EyvfmPpetnC9JN1FfPHdnxssUfr6mdzNl2/SfveP/105pXHozcA/wqYj4ybTdj1NPSfxTcr1lM/XV5mWb6ffStXMm6Q3ANcC6lN3dOl+tKuS89VyQj4grZ/FtB4AFDc/nc7LscaOXJc2NiEPJx8XDs+kjzNxP1VcBX0fGYrHkNQ4mXw9L+gb1j2e5glar50/S/wL+JmVXq+ey0H5JWgV8ALgikoRkymsUfr6maeVnL+X8tELSHOoB/s6IuHf6/sagHxH3S/qypHMiotRiXC38Xrp2zoD3A49HxMvTd3TrfCVaiUWFnLeqpGtarZWzBViVPF4FZH0yKMKVwN6IOJC2U9KbJL1l6jH1i49Ppx1blGl50N/NaO97wPmSFiejoJXUz1uZ/boKuBG4JiJ+mnFMJ85XKz/7FuA/JDNG3kO9KN+hgvtxmuT6zu3Anoj4YsYxb0uOQ/W7tL0O+H8l96uV30tXzlki89N0N85Xg1ZiUTF/i2VfWS7yH/XAdAB4DXgZ2Nqw7ybqV6L3Ae9v2P4Vkpk4wC8DD1MvpPYwcHaJff0/wEenbZsH3J88fgf1q+VPALuppy3KPn9/BTwFPJn8Z5k7vV/J86upz974fof6tZ967nFX8u9/dut8pf3swEenfpfUP0L/ZbL/KRpmeZV8jn6D+kf1JxvO09XT+vbx5Nw8Qf0C9ns70K/U30uPnLMzqQftX2rY1vHzRf1N5hBwLIlfN2TFojL+Fl3WwMyswqqSrjEzsxQO8mZmFeYgb2ZWYQ7yZmYV5iBvZlZhDvJmZhXmIG9mVmH/H6GmjHBmjdHEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(x,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "1f5a1a2b",
   "metadata": {},
   "outputs": [],
   "source": [
    "a,b,a_r,b_r = gradient_descent(x,y,C=0.01,E=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "bf465ed8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0xd7c5cd8c70>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAEtCAYAAADKspE0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAi50lEQVR4nO3deXwV9b3/8dcnCRAgBCSAglEBFdlUlihSubRqsdjaKopetyoupatKqbcP+dW2cu3PtnaztbftTRdtcWvdEXdR9Fa91UBAwUK1QCECsslO9s/940z0JGSZkDM5k5z38/GYx5kzy3c+3yF88s13Zr5j7o6IiMRTVroDEBGRpilJi4jEmJK0iEiMKUmLiMSYkrSISIwpSYuIxJiStIhIjClJC2Z2iZmVmNkeM9toZk+Z2aQUH6Obmf3BzHaZ2SYzmx0ipn+Z2V4ze9TM+iatu9DMXjWzfWa2qI1x3WVmlUHd66ZlbSmzkWMMNLP5ZrbBzNzMBjdY3+y5MbMxZrY4qO9iMxvTYP3Xg/12BuV0S2X8kl5K0hkuSAi3A7cChwJHAr8CzknxoW4GjgWOAk4DvmlmU5uIaRTw38Dng5j2BTHV2R7E/IMUxXabu+clTSemqNw6tcDTwPlNrL+ZJs6NmXUFHgPuBg4B/gg8FizHzD4F3AicAQwGhgJzUxy/pJO7a8rQCegN7AEuaIdjvQecmfT9FuD+Jra9Fbg36fvRQCXQq8F21wCL2hjXXcD32ul85wAODA57boAzg/WWtH4dMDWYvxe4NWndGcCmdP9saUrdpJZ0ZpsI5AKPhN3BzG40sx1NTU3scwgwCEjuRlgGjGriMKOSt3X3f5JI0sPCxhkFMzuyubqb2SUHUWZL52YU8KYHGTjwZoP1Dfc91MwKWhuLxFNOugOQtCoAtrp7ddgd3P0HtL6bIS/43Jm0bCfQq5ntdzZY1tz2bXWDmX0t6ftj7n5Fw43cfR3QJ8XHbunctHQuGq6vm+8FbEtdmJIuStKZbRvQz8xyWpOoD8Ke4DMfKE+a393M9vkNljW3fVv92N1viqjslrR0blo6Fw3X181Hda6knam7I7O9RiIxnBt2BzP7fw3uhKg3NbaPu38AbASSL8idCKxo4jArkrc1s6FAN+AfYeOMQtDd0WTdzezS1pYZ4tysAE4wM0taf0KD9Q33fd/d1YruJJSkM5i77wS+A/yXmZ1rZj3MrIuZnWVmtzWxz61e/06IelMzh/sTcJOZHWJmw4EvkLho15h7gM+a2b+ZWU/gP4GH3X03gJllm1kuib8Es8ws18y6NHXg4La3TzR7MkJw93XN1d3d72kmhlwSv2gAugXf6zR3bhYBNcB1wa16dd0yLyTte7WZjQz6t2+i6fMqHVG6r1xqSv8EXAqUAHuBTcATwMdSfIxuwB+AXcD7wOwG6/cA/5b0/RISdzHsJXELWt+kdTNI3CWRPN3VxHELSfzpX9DE+rtIXJTckzRtjeAcN4zXW3FuxgKLgf3AEmBsg/Wzg/12AXcC3dL9M6UpdZMF/8ginZKZXQaMcvc56Y5F5GAoSYuIxJj6pEVEYkxJWkQkxpSkRURiTElaRCTGUv3EYZuuQk6dOpWnn346VbGIiHQU1tSKWLWkt27dmu4QRERiJVZJWkRE6lOSFhGJMSVpEZEYU5IWEYkxJWkRkRhTkhYRibH4vZll6VLYvRvcE5OISEdQUACjR6e82Hgl6XffhbFj0x2FiEjrnXMOPPpoyouNT5LesAF27oSrr4aLLgKzjyYRkbjr1y+SYuOTpJcFb6WfMQMmTUprKCIicRGfC4d1j4Qfdlh64xARiZH4JemCgvTGISISI5El6VWrVjFmzJgPp/z8fG6//famd9gWvIG+d++oQhIR6XAi65M+7rjjWLp0KQA1NTUcfvjhTJs2rekdduyA7GzIik/jXkQk3dolIy5cuJCjjz6ao446qumNKit1J4eISAPtkqTvv/9+Lr744uY3qqxUK1pEpIHIs2JlZSXz58/nggsuaHR9cXExRUVFPD1/PjW1tVGHIyLSoUSepJ966inGjRvHoYce2uj6mTNnUlJSwtTTTyc7Jz63bYuIxEHkSfq+++5ruasD1CctItKISJP0vn37eO655zjvvPNa3lhJWkTkAJH2L/To0YNtdfc/t0QXDkVEDhCfrKiWtIjIAZSkRURiTElaRCTG4pWk1SctIlJPfLKiWtIiIgeIT5KurlaSFhFpID5JWo+Ei4gcIF5JWi1pEZF64pWkRUSkHiVpEZEYi1eSVneHiEg98UnS7umOQEQkduKTpNXdISJygHglaXV3iIjUE68kLSIi9cQrSaslLSJST7yStIiI1BOfJK27O0REDhCfJK3uDhGRA8QrSYuISD1K0iIiMRavJK3uDhGReuKTpHXhUETkAJEm6R07djB9+nSGDx/OiBEjeO211xrfUAlaRKRROVEWfv311zN16lQefPBBKisr2bdvX+Mb1vVHq7tDRKQe89S2Yj8sbNeuXZx44omsXr0aayn5VlVB164UDRpEyXvvpTIeEZGOoMkkGVl3x+rVq+nfvz9XXnklY8eO5ZprrmHv3r0HbFdcXMzECRMA2NtUS1tEJENFlqSrq6tZsmQJX/7ylyktLaVnz5784Ac/OGC7mTNn8torrwDQs2fPqMIREemQIkvShYWFFBYWMiFoJU+fPp0lS5Y0vrHukRYRaVRkSfqwww7jiCOOYNWqVQAsXLiQkSNHNr5xXb+4LhyKiNQT6d0dd9xxB5deeimVlZUMHTqUO++8s/EN1ZIWEWlUZHd3tMqOHXDIIRQVFlKyfn0q4xER6Qja/+6OVtF90iIijYpXkhYRkXpaTNJm1tPMsoL5YWb2OTPrktIo1JIWEWlUmJb0y0CumR0OLASuBO5KaRRqSYuINCpMkjZ33wecB9zh7tOAJu6lO0gaYElEpFGhkrSZTQQuBZ4IlqX21j11d4iINCpMkr4emAM84u4rzGwo8GJKo1B3h4hIo5ptEZtZNvBZd/9c3TJ3Xw1cl9IolKRFRBrVbEva3WuA8ZFHoe4OEZFGhelbLjWz+cADwIdjjbr7wymLQi1pEZFGhUnSfYFtwOlJyxxIXZLWAEsiIo1qMUm7+5WRR6GWtIhIo8I8cTjMzBaa2fLg+wlmdlNKo1CSFhFpVJhb8H5L4ha8KgB3fxO4KKVR6MKhiEijwiTpHu7+eoNl1SmNQi1pEZFGhUnSW83saIKxos1sOrAxpVEoSYuINCrM3R1fBYqB4Wb2HrCGxCPiqTN6NOzfD5MmpbRYEZGOLkySdnf/pJn1BLLcfbeZDUlpFFlZkJub0iJFRDqDMN0dDwG4+1533x0sezC6kEREpE6TLWkzGw6MAnqb2XlJq/IBNXtFRNpBc90dxwFnA32AzyYt3w18IcKYREQk0GSSdvfHgMfMbKK7v9aOMYmISCBMn/S2yJ84FBGRRkX6xOHgwYM5/vjjGTNmDEVFRQcfpYhIhgpzC14Pd3/d6j+yHfqJwxdffJF+/fq1OjAREYnLE4ciItKoMEn6q8B/89ETh7OAL4cp3Mw488wzGT9+PMXFxY1uU1xcTFFREUVFRWzZsiVk2CIimcG8bsD9ljZMeuKwmc3qFbZhwwYGDRrE5s2bmTJlCnfccQeTJ09ucueioiJKSkpCxSMi0ok0OQRoi33SZtYHuBwYDOTU9U27e4svox00aBAAAwYMYNq0abz++uvNJmkREakvTHfHkyQS9FvA4qSpWXv37mX37t0fzj/77LOMHj364CMVEclAYe7uyHX32a0t+P3332fatGkAVFdXc8kllzB16tTWFiMiktHCJOl5ZvYFYAFQUbfQ3bc3t9PQoUNZtmxZG8MTEclsYZJ0JfAj4Ft8dGHQgaFRBSUiIglhkvRs4Bh33xp1MCIiUl+YC4crgH1RByIiIgcK05KuAZaa2YvU75Nu8RY8ERFpmzBJ+tFgEhGRdtZiknb3P7ZHICIicqAwTxweC3wfGEnSa7PcXXd3iIhELMyFwzuBX5MYnvQ04E/AvCiDEhGRhDBJuru7LyQxGNO/3P1m4PRowxIREQh34bDczLKAd8zsa8B7wIBowxIREQjXkp4F9ACuA8YDlwFXRBiTiIgEmm1Jm1k2cKG7/wewB7iyXaISERGghZa0u9cA463BCw5FRKR9hOmTLgUeM7MHgL11C9394ciiEhERIFyS7gtso/4dHQ4oSYuIRCzME4fqhxYRSZMwTxzmAlcDo6j/xOFVEcYlIiKEuwVvHnAY8CngJaAQaO6N4SIikiJhkvQx7v5tYG8w2NJngOOjDUtERCBckq4KPneY2WigN4m3h4uISMTC3N1RbGaHAN8G5gN5wbyIiEQszN0dvwtmX0IvnxURaVctdneYWYGZ3WFmS8xssZndbmYFYQ9QU1PD2LFjOfvss9sWqYhIBgrTJ30/sBk4H5gObAX+HPYAP//5zxkxYsTBRScikuHCJOm+7n6Lu68Jpu8BfcIUXlZWxhNPPME111zTpiBFRDJVmCT9opldZGZZwXQh8ESYwmfNmsVtt91GVlaYw4iISENhsucXgXuBimC6H5htZrvNbFdTOy1YsIABAwYwfvz4ZgsvLi6mqKiIoqIitmzZ0orQRUQ6P3P3VJb3YWFz5sxh3rx55OTkUF5ezq5duzjvvPO4++67m9y5qKiIkpKSVMYjItIRNDkcdGRJOtmiRYv48Y9/zIIFC5rdWUlaRDJUk0lancUiIjHWLi3psNSSFpEM1WRLusknDs2sb3Mluvv2tkQkIiIta+6x8MUkWsYGHAl8EMz3AdYBQ6IOTkQk0zXZJ+3uQ9x9KPAM8Fl37+fuBcDZ6NVZIiLtIsyFw5Pc/cm6L+7+FPDx6EISEZE6YYYq3WpmNwF3k+j+uIzEi2lFRCRiYVrSFwP9gUeCqX+wTEREIhZmPOntwPVmlufue9ohJhERCYQZT/pjZvY28Hbw/UQz+1XkkYmISKjujp+ReFP4NgB3XwZMjjIoERFJCPVYuLuvb7CoJoJYRESkgTB3d6w3s48BbmZdgeuAv0cbloiIQLiW9JeArwKHA2XAGOArEcYkIiKBMC3p49z90uQFZnYq8Eo0IYmISJ0wLek7Qi4TEZEUa24UvInAx4D+ZjY7aVU+kB11YCIi0nx3R1cgL9imV9LyXcD0KIMSEZGEJpO0u78EvGRmd7n7v9oxJhERCYS5cLjPzH4EjAJy6xa6++mRRSUiIkC4C4f3ACtJDPI/F1gLvBFhTCIiEgiTpAvc/fdAlbu/5O5XAadEHJeIiBCuu6Mq+NxoZp8BNgCF0YUkIiJ1wiTp75lZb+AbJO6Pzge+HmlUIiIChBtPekEwuxM4LdpwREQkWYtJ2syGANcCg5O3d/fPNbdfeXk5kydPpqKigurqaqZPn87cuXPbGq+ISEYJ093xKPB74HGgNmzB3bp144UXXiAvL4+qqiomTZrEWWedxSmn6JqjiEhYYZJ0ubv/orUFmxl5eXkAVFVVUVVVhZm1thgRkYwW5ha8n5vZd81sopmNq5vCFF5TU8OYMWMYMGAAU6ZMYcKECW0MV0Qks4RpSR8PfB44nY+6Ozz43qzs7GyWLl3Kjh07mDZtGsuXL2f06NH1tikuLqa4uBiALVu2tCZ2EZFOz9y9+Q3MVgInuHtliPKaLGzu3Ln07NmTG264ocmdi4qKKCkpCXEYEZFOpcm+4DDdHcuAPq094pYtW9ixYwcA+/fv5/nnn2f48OGtLUZEJKOF6e44FFhpZm8AFXULW7oFb+PGjVxxxRXU1NRQW1vLhRdeyNlnn93GcEVEMkuYJP3dgyn4hBNOoLS09GB2FRGRQJgnDl9qj0BERORAzb0+66/uPsnMdlP/gqAB7u75kUcnIpLhmnszy6Tgs1dT24iISLRavLvDzOaFWSYiIqkX5ha8UclfzCwHGB9NOCIikqzJJG1mc4L+6BPMbJeZ7Q6+vw881m4RiohksCaTtLt/P+iP/pG757t7r2AqcPc57RijiEjGCtPd8S0zu8zMvg1gZkeY2ckRxyUiIoRL0v8FTAQuCb7vCZaJiEjEwjxxOMHdx5lZKYC7f2BmXSOOS0RECNeSrjKzbIIHWsysP614Q4uIiBy8MEn6F8AjwAAz+//AX4FbI41KRESAcGN33GNmi4EzSDwSfq67/z3yyEREJFSfNO6+ElgZcSzSBrW1TlVtLTW1Tq1DrTteG3yS+Kx1xx3cm/kOuCeVEayv4w3e69DCOyMaXd9SGQ13afhiigPXH3iElmIQSbU+PbpwzIDUj6IRKknLwSuvqmHX/ip27q9iV3nwub+anfur2FNRzf7KGvZX1VBelfT54bJaKqprqa6ppbrWqaqppbrGqa6tparGqa6ppao28VmrRCSSVlNGHspvLy9KeblK0m2wt6Kaf23bx7rte1m3fR/v76pg8+4KNu8q//Bzb2VNs2VkGfTomkNul2xyu2TRvUs23btmk9slm97du9A1J4su2UZOVhY52UaXus/sLLKzrNFl2WaYQVbSZ5YBwWfddyNpfVaD78G+id3qv9mn4UvfG77354DtG6n3AWUcUGbzB2ntMfWieola357R3PSmJB1CeVUN77y/hxUbdrJiwy5WbtrFmq372Lqnot523btkc2h+Nwb0ymXkoHw+cVx/+uV1o3f3LuR370LvYMrPzaF39y7k5ebQNTvrgAQjIlJHSboRu8qreGPNdv62Zjt/W72NFRt2UR30J+R1y2HEwF6cMXwARxb0YHBBT44q6MGRBT3o1S1HCVdEUkpJOrB++z6ee/t9nn17E2+s/YCaWqdrdhZjjujDzMlDGX14b0YNyueIQ3qQlaVELCLtI6OT9O7yKp54cyN/KVnPknU7ABh2aB5f+vhQTj2mH+OOPITcLtnpDVJEMlpGJum1W/dS/D+reWTJe+yvquGYAXnceNZwpo46jMH9eqY7PBGRD2VUkn538x5+9vw/eOqtjeRkZXHu2EFcfPKRjDmij/qSRSSWMiJJb91Twc+ff4d7X19H9y7ZfPHjR3PlqYMZ0Cs33aGJiDQrsiS9fv16Lr/8cjZt2kRWVhYzZ87k+uuvj+pwjXJ3Higp45Yn3mZfZQ2XTjiS6884loK8bu0ah4jIwYosSefk5PCTn/yEcePGsXv3bsaPH8+UKVMYOXJkVIesZ9POcr750Ju8/I8tnDykL7dOO55jBuS1y7FFRFIlsiQ9cOBABg4cCECvXr0YMWIE7733Xrsk6Vff3cq195Wyr7KG/zxnFJdNOEq3zYl0UlVVVZSVlVFeXp7uUFqUm5tLYWEhXbp0Cb1Pu/RJr127ltLSUiZMmHDAuuLiYoqLiwHYsmVLm47j7hS/vJofPr2Sof3z+PMXx0Uy4ImIxEdZWRm9evVi8ODBsb4BwN3Ztm0bZWVlDBkyJPR+YcaTbpM9e/Zw/vnnc/vtt5Ofn3/A+pkzZ1JSUkJJSQn9+/c/6OPU1Drfnb+C7z+1krNGD+TRr56qBC2SAcrLyykoKIh1gobE+DIFBQWtbvFH2pKuqqri/PPP59JLL+W8886L7jg1tcz681KeeHMjMycP5capw9W9IZJB4p6g6xxMnJElaXfn6quvZsSIEcyePTuqw1BT63w9SNBzzhrOFz9+dGTHEhFpb5F1d7zyyivMmzePF154gTFjxjBmzBiefPLJlB6jtta58aE3WaAELSKdVGQt6UmTJh3wRo1Uu33hOzywuIzrzzhWCVpEOqXILxxGZf6yDfxi4TtcML6QWZ88Nt3hiEiGW7FiBZ/85CcZNmwYt9xyC9deey1vvPFGm8vtkI+FL39vJ//xwDJOHtyX700b3WEuGohIxGbNgqVLU1vmmDFw++3NblJeXs4FF1zAAw88wNChQxk+fDjjx4/npJNOavPhO1yS3ltRzXX3lXJIj678+rJxdMvRUKIikl7PP/88Y8eOZdSoUQBUVlbyjW98IyVld7gkffP8FazZtpd7rzlFY3CISH0ttHijUlpayrhx4wDYsGEDeXl5nHrqqSkpu0P1ST/11kYeWFzG1047holHF6Q7HBERALp160ZZWRkAc+bMobKyMmVld5gkvXN/Fd+Zv4LRh+dz/Rm6UCgi8XHJJZfw8ssvc9xxx3HiiScyceJEZs2alZKyO0x3x21Pr2TbngrunHESOdkd5neLiGSAwsJCFi9eHEnZHSLbLf7Xdu752zquPHUIow/vne5wRETaTeyTdG2tM/fxtxnYO5fZU4alOxwRkXYV+yS94K2NvFm2k2+ceRw9u3WY3hkRkZSIdZKuqK7hR8+sZPhhvZg29vB0hyMi0u5inaTv+d91rN++nzmfHkG2hh4VkQwU2yRdXlXDrxb9k4lDC5h8bL90hyMikhaxTdIPLC5j654Krj3jGI3NISIZK5ZJuqqmlt8s+ifjjuzDxKF6slBEMlcsk/T8pRt4b8d+vnqaWtEi0nF85zvf4fjjj2fYsGEfvmC7rWKXpN2dP7yyhmGH5nH68AHpDkdEJJRnnnmG0tJSli5dykMPPcSjjz6aknJjl6SXrPuAFRt2cfnEeL+eXUQk2fz585kxYwZVVVX88pe/5Pzzz09JubF7OuSPr/6LXrk5ui9aRFpt7uMreHvDrpSWOXJQPt/97KgWt1u8eDEnnXQSBQUFDB48mJ/97GcpOX6sWtLVNc6Tb23kgvFH6OlCEekwamtrKSsrY8aMGWzdupXx48fz05/+NCVlxyoTbt9XSVatc9kpR6Y7FBHpgMK0eKOwatUqjj02MYRy9+7dOfXUU9m0aVNKyo5NS9rd2bGvkpMH92Vo/7x0hyMiElppaSkVFRXU1NRQUVHBvffey7nnnpuSsmPTkl6ybgcV1bVMLypMdygiIq2ydOlS9u/fz9FHH02/fv34yle+woknnpiSsiNL0ldddRULFixgwIABLF++vMXtH1xcRpYZnz5+YFQhiYhEorS0lHnz5jF69OiUlx1Zd8eMGTN4+umnQ227v7KGBcs2kN89hzxdMBSRDmblypUMHz48krIjS9KTJ0+mb9++obZ9d/MesrONvj26RhWOiEhk1q9fT05ONA3MtDdbi4uLKS4uJtuy2bfrg3SHIyISK2m/u2PmzJmUlJSw+I2/0b9//3SHIyISK2lP0iIibeXu6Q4hlIOJU0laRDq03Nxctm3bFvtE7e5s27aN3NzcVu0XWZ/0xRdfzKJFi9i6dSuFhYXMnTuXq6++OqrDiUiGKiwspKysjC1btqQ7lBbl5uZSWNi6Z0Esxb992lRYUVERJSUlqYpFRKSjaHLIT3V3iIjEmJK0iEiMKUmLiMRYqvuk28TMnnb3qemOQ0QkLmKVpEVEpD51d4iIxJiStIhIjClJi4jEmJK0iEiMKUmLiMSYkrSISIwpSYuIxJiStIhIjClJi4jEmJK0iEiMKUmLiMSYkrSISIwpSYuIxJiStIhIjClJi4jEmJK0iEiMKUmLiMSYkrSISIwpSYuIxJiStIhIjClJi4jEWE4qCzOzp4F+bSiiH7A1ReF0FJlW50yrL6jOmaItdd7q7lMbW2HufvAhpZiZlbh7UbrjaE+ZVudMqy+ozpkiqjqru0NEJMaUpEVEYixuSbo43QGkQabVOdPqC6pzpoikzrHqkxYRkfri1pIWEZEksUjSZjbVzFaZ2btmdmO640kVMzvCzF40s7+b2Qozuz5Y3tfMnjOzd4LPQ5L2mROch1Vm9qn0RX/wzCzbzErNbEHwvVPXF8DM+pjZg2a2Mvj3ntiZ621mXw9+ppeb2X1mltsZ62tmfzCzzWa2PGlZq+tpZuPN7K1g3S/MzEIH4e5pnYBs4J/AUKArsAwYme64UlS3gcC4YL4X8A9gJHAbcGOw/Ebgh8H8yKD+3YAhwXnJTnc9DqLes4F7gQXB905d36AufwSuCea7An06a72Bw4E1QPfg+1+AGZ2xvsBkYBywPGlZq+sJvA5MBAx4CjgrbAxxaEmfDLzr7qvdvRK4HzgnzTGlhLtvdPclwfxu4O8kfsDPIfGfmuDz3GD+HOB+d69w9zXAuyTOT4dhZoXAZ4DfJS3utPUFMLN8Ev+Zfw/g7pXuvoPOXe8coLuZ5QA9gA10wvq6+8vA9gaLW1VPMxsI5Lv7a57I2H9K2qdFcUjShwPrk76XBcs6FTMbDIwF/gYc6u4bIZHIgQHBZp3hXNwOfBOoTVrWmesLib8CtwB3Bt08vzOznnTServ7e8CPgXXARmCnuz9LJ61vI1pbz8OD+YbLQ4lDkm6sb6ZT3XJiZnnAQ8Asd9/V3KaNLOsw58LMzgY2u/visLs0sqzD1DdJDok/iX/t7mOBvST+DG5Kh6530Ad7Dok/6QcBPc3ssuZ2aWRZh6lvKzRVzzbVPw5Jugw4Iul7IYk/nToFM+tCIkHf4+4PB4vfD/4EIvjcHCzv6OfiVOBzZraWRLfV6WZ2N523vnXKgDJ3/1vw/UESSbuz1vuTwBp33+LuVcDDwMfovPVtqLX1LAvmGy4PJQ5J+g3gWDMbYmZdgYuA+WmOKSWCK7i/B/7u7j9NWjUfuCKYvwJ4LGn5RWbWzcyGAMeSuODQIbj7HHcvdPfBJP4dX3D3y+ik9a3j7puA9WZ2XLDoDOBtOm+91wGnmFmP4Gf8DBLXWzprfRtqVT2DLpHdZnZKcL4uT9qnZem+ehpc+fw0iTsf/gl8K93xpLBek0j8WfMmsDSYPg0UAAuBd4LPvkn7fCs4D6toxRXguE3AJ/jo7o5MqO8YoCT4t34UOKQz1xuYC6wElgPzSNzR0OnqC9xHot+9ikSL+OqDqSdQFJyrfwK/JHiQMMykJw5FRGIsDt0dIiLSBCVpEZEYU5IWEYkxJWkRkRhTkhYRiTElaYkNM1tkZpG/F8/MrgtGqrsn6mM1OO7NZnZDiO3WBiOmLQ2mX7RHfBJPKX1buEi6mFmOu1eH3PwrJO5hXRNlTG10mrtn2tu2pRFqSUurmNngoBX622A84WfNrHuw7sOWsJn1Cx4Px8xmmNmjZva4ma0xs6+Z2exgMKL/NbO+SYe4zMxeDcYpPjnYv2cwru8bwT7nJJX7gJk9DjzbSKyzg3KWm9msYNlvSAyINN/Mvt5g+2wz+1FwnDfN7IvB8k+Y2ctm9oiZvW1mvzGzrGDdxUGrd7mZ/TCprKlmtsTMlpnZwqTDjAzO02ozu65N/xiSGdL9RI+mjjUBg4FqYEzw/S/AZcH8IqAomO8HrA3mZ5AYtrEX0B/YCXwpWPczEgNP1e3/22B+MsEYvsCtScfoQ+Lp1J5BuWUkPfGVFOd44K1guzxgBTA2WLcW6NfIPjOBm4L5biSeIBxC4unJchLJPRt4DphOYnChdUGdcoAXSAxB2Z/EaGhDgrL6Bp83A68GZfcDtgFdGoljbRD70mD6err/3TWlb1J3hxyMNe6+NJhfTCJxt+RFT4ypvdvMdgKPB8vfAk5I2u4+SIzja2b5ZtYHOJPEwE11/bm5wJHB/HPu3nC8X0g8kv+Iu+8FMLOHgX8DSpuJ8UzgBDObHnzvTWL8hUoSYzCsDsq6Lyi/Cljk7luC5feQ+OVSA7zsQXdKg/iecPcKoMLMNgOHUn8Yyzrq7hBAfdJycCqS5muA7sF8NR91oeU2s09t0vda6v8cNhynoG6ox/PdfVXyCjObQGJY0MaEfz1R/X2udfdnGhznE83E1VQ5TY230PDc6f+gNEt90pJKa0l0M0CiO+Bg/DuAmU0iMZj8TuAZ4NpgBDHMbGyIcl4Gzg1GausJTAP+p4V9ngG+HAwvi5kNC/aFxBs2hgR90f8O/JXECxw+HvS/ZwMXAy8BrwXLhwTl9G14IJGw9FtcUunHwF/M7PMk+mcPxgdm9iqQD1wVLLuFxBtf3gwS9Vrg7OYKcfclZnYXHw2J+Tt3b66rAxKv/BoMLAmOs4WPXnP0GvAD4HgSvwAecfdaM5sDvEii9fykuz8GYGYzgYeDpL4ZmNJSxRt40cxqgvk33f3yVu4vnYRGwRNpQdDdcYO7N/uLQSQK6u4QEYkxtaRFRGJMLWkRkRhTkhYRiTElaRGRGFOSFhGJMSVpEZEYU5IWEYmx/wPIqH10PD76OgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(a_r,color='r',label=r'$\\alpha $')\n",
    "plt.plot(b_r,label=r'$\\beta$')\n",
    "plt.xlabel(\"number of epoch E\")\n",
    "plt.ylabel(\"etimated parameters\")\n",
    "plt.title('C = 0.01 , E = 1000')\n",
    "set_spines()\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "734357d2",
   "metadata": {},
   "source": [
    "From graphics we can see that the curves of $\\alpha$ tends to $7$ and $\\beta$ tends to $3$.\n",
    "<br> \n",
    "The fit line will be : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "48473221",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADvCAYAAABR/Qd9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAukUlEQVR4nO3deVyVdfr4/9dhFTdcAjyAqAjK6kouOZMmrtmAaT+1rI+NTsynPn3HadFsbHEyA21qtHLsw0wLM30mJiuB3FBRGzUN9wQEUcHgAAKyqMh+7t8fCiNyDnD0LHC4no+Hj/Q+N/e5uIGrN9f9fl9vlaIoCCGEMA8bSwcghBCdiSRdIYQwI0m6QghhRpJ0hRDCjCTpCiGEGUnSFUIIM5KkK8xOpVK9oFKpUlUqVYpKpfpSpVJ1UalUfVQq1W6VSpV567+9LR2nEKYgSVeYlUql8gB+B4QoihIE2AILgBVAkqIovkDSrX8LYXVaS7qK/JE/xvyTm5ub6+npOfDKlSs36urqlFmzZv1/iYmJnw0ZMmRZXl5eJKDk5eVFDhkyZHlr15oxY4bFPx/5I3/0/NFLRrrCrDw8PHj55Zfx8vJCrVbj7OzMtGnTuHz5Mmq1GgC1Wk1hYWGr1youLjZ1uEIYnSRdYValpaXEx8eTlZVFXl4eFRUVfPHFF23++OjoaEJCQggJCaGoqMiEkQphGpJ0hVnt2bOHQYMG4eLigr29PXPmzOGHH37Azc2N/Px8APLz83F1ddX58RERERw7doxjx47h4uJiztCFMApJusKsvLy8OHLkCDdu3EBRFJKSkvD39ycsLIyYmBgAYmJiCA8Pt3CkQpiGnaUDEJ3L2LFjeeyxxxg1ahR2dnaMHDmSiIgIrl+/zrx58/jkk0/w8vJi8+bNlg5VCJNQtdLascUXhbCkkJAQjh07ZukwhNBFpe8FGekKIcRt4k5qeDcxg7yyStx7ObFs+lBmj/Qw2vUl6QohxC1xJzW8+u0ZKmvrAdCUVfLqt2cAjJZ45UGaEELc8m5iRmPCbVBZW8+7iRlGew8Z6QohOjxjlQTyyioNOn43JOkKITq01koChiRktXMX8sqrmh137+VktHgl6QohOrTWSgJtrdGeyinDRtV80oGTvS3Lpg81WrxS0xVCdGgtlQTaUqMtr6zl9bgUHv3LIWrqtTz9wEDcnbugAjx6ORE5J1hmLwghRAP3Xk5odCRe915OLSbkLSdyeWtrGqU3agF40Pc+Ni4cRY8u9qwKCzRZvDLSFUJ0aMumD8XJ3rbJsYaSgL5abHdHO17afLox4QIczS4l6Wzr3e3ulSRdIUSHNnukB5FzgvHo5dSsJKArIdvZqLhWXYf2jvW2xp4apo+UF4QQHd7skR46664Nx95NzEBTVomtjYq6O7PtbYw5NUwfGekKIazaAz59GT3g5pZ7/Xs78Y8lY/DQU3Yw5tQwfWSkK4SwSvVahX/+eIl1iRlU12pZGurLs5MG0+VWvff2qWRg/Klh+kjSFUJYnRRNOSvjUjidU8YEn76sDg/C26V74+u3lx1M1dhGH0m6Qgircb26jvd3nePzH7Lo082BDQtGEDbcHZWORQ/66sCmJklXCNHhKYrCzpQC/vhdGpevVbFwrBfLpvnh3NXe0qE1I0lXCGExxmhUk1NygzfiU9iXUUSAuiebnhzFSK/eJor43knSFUJYxL32rq2p0/LXAxf5ICkTOxsVrz8SwKLxA7Czbd+Tstp3dMIqlZWV8dhjj+Hn54e/vz+HDx+mpKSEqVOn4uvry9SpUyktLbV0mMLE7qV37ZGLV3j4gwO8m5jBZD9X9rw0kSW/GNTuEy5I0hUWsHTpUmbMmEF6ejqnT5/G39+fqKgoQkNDyczMJDQ0lKioKEuHKUzsbnrXXrlezcubT7Mg+ghVtfV8+nQIm54cjdrZ9PNrjUU2phRmdfXqVYYPH87FixebPFEeOnQo+/fvR61Wk5+fz6RJk8jIaHnEIxtTdmwTovbqbFTj0cuJQysmNzmm1SpsPp5D5I50rlfV8cyD3vxusi9ODrbNPr6d0LsxpYx0hVldvHgRFxcXfv3rXzNy5Eh+85vfUFFRweXLl1Gr1QCo1WoKC03feERYVkuNam6XXnCVef97mFe+OcMQ1x5sX/pLXpnh154Tbosk6Qqzqqur48SJEzz77LOcPHmSbt26GVRKiI6OJiQkhJCQEIqKikwYqTC1lhrVANyoqSNyx1ke+eAgF4qu8+5jw/jXb8cxxK2HZQO/R1JeEGZVUFDAuHHjyM7OBuDAgQNERUVx/vx5KS+IRrvTLrMqIRVNWSXzQ/qzYqYfvbs5WDosQ0h5QbQP/fr1o3///o0JNSkpiYCAAMLCwoiJiQEgJiaG8PBwS4YpLERTVskzfz/GM38/RjdHWzb/93jWPjasoyXcFslIV5jdqVOn+M1vfkNNTQ3e3t589tlnaLVa5s2bx88//4yXlxebN2+mT58+LV5HRrrWo7Zey2eHsvjz7kwAfj/Fl8W/GIR9B5gCpofeka4kXdFhSdK1DscvlbBySwrpBdeY4u/KqrBAPHt3tXRY90pv0pUVaUIIiyi7UcPanel8mZyDu3MXop8azbTAfpYOy+Qk6QohdDJGXwRdFEXh2xMa1mw/S3llLREPerM01Jdujq2nI1PFZE6SdIUQzdxrX4Q7r9WQKF16ONKjix0XiioY5dWLNY8G46/uafaYLKnDVqmFEKZzL30RbteQKDVllShA4bVqLhRVMD+kP1//9wNtTrjGjMnSJOkKIZq5m74IuuhKlAAHzxdjY6P3WZNJY7I0KS8IIZpx7+Wksy+CIRs3FpRX6bwGtJ4oddVujRFTeyAjXSFEM23ti6BLXb2WTw9mEfrefr3ntJQo7yxJNNRuH/JzueuY2hNJukKIZlrri6DPqZwywjce4q2taYQM7MNrs/wNTpT6arf70ovuKqb2RsoLQgidDNm4sbyylj8lZvDFj5dw6e7IxidG8XBwP1QqFfd1dzRomldLtVtLbSZpTJJ0hRAGub3eqnbuQqi/GztSCiipqGbR+IG8NG0IPbr8Z0NIQxOltdRu9ZHyghCize6st+aVV/GPI5dwsrch4flfsCossEnCvRv3Uk/uCGSkK4RoM31TwOq1CucLr/Pbfxy/59ViDR/T0Vee6SMNb0SHJQ1vzG/gim16X3Oyt22SkJ3sbTvkgy4jkX66Qoi7V3itiqWxJ/W+bqtSWcVqMXOQ8oIQQq96rcI/k39m3c50qmu1TA904/uMIqrqtI3n3DnCvV1HWy1mDpJ0hbAixuzClaIpZ2VcCqdzypjg05fV4UF4u3TX+R7vJmZY9YwDY5KkK4SVMFYXruvVdby/6xyf/5BFn24OrJ8/gvAR7qhUqsZr6bre7e8N1jXjwJgk6QphJVrqwtWWpKsoCjtTCvjjd2lcvlbFwrFeLJvmh3PX1qeAWfuMA2OSpCvMrr6+npCQEDw8PNi6dSslJSXMnz+f7OxsBg4cyFdffUXv3r0tHWaHcy9duHJKbvBGfAr7MooIUPdk05OjGOll2NfAGlaLmYPMXhBmt2HDBvz9/Rv/HRUVRWhoKJmZmYSGhhIVFWXB6NqXuJMaJkTtZdCKbUyI2kvcSY3ec/XVT1uqq9bUadm47zxT//w9yVklvP5IAAnPTzA44Yq2k5GuMKvc3Fy2bdvGypUref/99wGIj49n//79ACxatIhJkyaxdu1aC0bZPrRUo4Xmv8ovmz7UoLrqkYtXeC0uhfOF15kR2I83wwJQO8uDL1OTpCvM6ve//z3r1q3j2rVrjccuX76MWq0GQK1WU1hYaKnw2hV9NdpVCalU12mbJePIOcFEzgluta565Xo172xP55sTuXj2duLTp0OY7Odmts+rs5OkK8xm69atuLq6Mnr06MaRraGio6OJjo4GoKioyIjRtT/6arFllbXNjjU8MDu0YrLOumrcSQ3rdqaTV16FSnVzudRzkwbz/yb74uRg2+x8YTqSdIXZHDp0iISEBLZv305VVRVXr17lySefxM3Njfz8fNRqNfn5+bi6uuq9RkREBBEREcDNZcDWTF+3LX30Jem4kxpe+eYnqm8taFAUcLCzYYhbD0m4FiAP0oTZREZGkpubS3Z2NrGxsUyePJkvvviCsLAwYmJiAIiJiSE8PNzCkbYP+rpt9dYzhUvXA7MbNXW8FpfSmHAbVNdpZYmuhchIV1jcihUrmDdvHp988gleXl5s3rzZ0iG1C/rmvkLbFiLsTrvMqoRUrlfX6bx+w8jYmKvYROuky5josDpzl7GWEqWmrJJVCansTrvMULceXKmopvh6TbNreNz6OF0JvBN3BzMWvV3GJOmKDqszJ11dauu1fHYoiz/vzgTg91N8WfyLQWz7KV9vYtXXM8GjlxOHVkw2W+xWSG/SlfKCEFbg+KUSVm5JIb3gGlP8XVkVFohn765Ay0t0X/jXKZ3Xk+5gpiNJV4gOrLSihrU704k9moO7cxeinxrNtMB+zc7Tt0TX2vcja48k6QrRASmKwjcnNLyz/SzllbVEPOjN0lBfujka9iNt6Co2ce8k6QrRwZwvvMbKLSn8mFXCKK9erHk0GH91z7u6lnQHMz95kCY6rM72IK2ypp6P9mUS/e+LdHWwY8VMP+aH9MfGRu8zG2E58iBNiI5sX3ohbySkkFNSyZxRHvzhYX/u6+5o6bDEXZCkK0Q7VlBexVtbU9l+poDBLt348plxjB/c19JhiXsgSVeIdqiuXkvM4Uu8vyuDOq3Cy9OGEPHgYBzsZOV+RydJVwgzaety21M5ZazccobUvKtMHOLC6vAgvPp2tUDEwhQk6QphBm3ZNLK8spY/JWbwxY+XcOnuyMYnRvFwcL/GDSGFdZCkK4QZtLRpZPgIdxJO57F661lKKqpZNH4gL00bQo8urW8IKToeSbpCmIG+ZbWaskqe+iSZg+eLGebpzOe/vp8gD2czRyfMSZKuEK0wRuvDlhqSn84pY3V4IE+MHYCtzLm1evIoVIgWNNRiNWWVKPynFtvSrry66GpIDjDKqxdJL03kqfEDJeF2EpJ0hWhBS7VYQ8we6cGrM/0aE6+tjYpnJw7m2+cm4Nqzi9HiFe2flBeEQH8JQV8t1pDWh/VahX8m/8y7uzKo1yr8LtSX5yYNpouOka+wfpJ0RafX0nSue219mKIpZ2VcCqdzypjg05fV4UF4u3Q3XvCiw5HygjCrnJwcHnroIfz9/QkMDGTDhg0AlJSUMHXqVHx9fZk6dSqlpaVmi6mlEoK+zSFban0Yd1LD+MgkBq7YxiMfHuRC4XU2LBjBF0vGSsIVknSFednZ2fHee+9x9uxZjhw5wsaNG0lLSyMqKorQ0FAyMzMJDQ0lKirKbDG1VEKYPdKDyDnBePRyQsXNbWxa2j9sy4lcln/9E/nlVY3H6uq1KAqyyEEAUl4QZqZWq1Gr1QD06NEDf39/NBoN8fHx7N+/H4BFixYxadIk1q5da5aYWish6Nt14c468OIJA1mXmEFNfdPtzqtubXfe0jQz2ZG385CRrrCY7OxsTp48ydixY7l8+XJjMlar1RQWFpotjrstIdw5lWz1trNU12l1nt/SgzdjTUsTHYMkXWER169fZ+7cuaxfv56ePdu+60F0dDQhISGEhIRQVFRklFgMLSGA7jow6P+BaunBm7GmpYmOQcoLwuxqa2uZO3cuCxcuZM6cOQC4ubmRn5+PWq0mPz8fV1dXnR8bERFBREQEcHPnCGPRV0LQR9/qMi03R8mG7DlmjGlpouOQka4wK0VRWLJkCf7+/rz44ouNx8PCwoiJiQEgJiaG8PBwS4XYIq1WITb5Z/Q9E2sYJRsyatY3CpYdea2T7JEmzOrgwYP88pe/JDg4GBubm//Pf+eddxg7dizz5s3j559/xsvLi82bN9OnT58Wr2XuPdLSC66ycksKxy+VMtilG7mllU1quE72tq0mWF3unCd8L9cS7YbeqSqSdEWHZa6ke6Omjg17MvnbwSycnez5w8P+zB3lQfypPKPNOJDZC1ZHkq6wPuZIurvTLrMqIRVNWSXzQ/qzYqYfvbs5mPQ9hVWQ3YCFMISmrJJVCansTrvMULcefP3f4wkZ2HK5Q4i2kKQrxG1q67V8ejCL9XsyAXh1ph+LfzEIe1t55iyMQ5KuELccv1TCyi0ppBdcY4q/K6vCAvHsLRtCCuOSpCs6vdKKGtbuTCf2aA7uzl2Ifmo00wL7WTosYaUk6YpOS1EUvjmh4Z3tZymvrCXiQW+WhvrSzVF+LITpyHeX6JTOF15j5ZYUfswqYZRXL9Y8Goy/uu3LkYW4W5J0RadSWVPPR/syif73Rbo62BE5J5j5If2xkf3JhJlI0hWdxr70Qt5ISCGnpJI5Iz34wyx/7uvuaOmwRCcj82CE1Ssor+K5/zvOrz8/SnWtlvu6ObDlpIbwjw5J+0RhdjLSFVarrl5LzOFLvL8rgzqtwqxgNUlnL1N1q1/C7XuhyZJbYS4y0hVW6VROGeEbD7F6axohA/uw+4WJnMopa0y4DaRvrTA3GekKq1JeWcufEjP44silxvaLmZevceLnUulbK9oFSbrCKiiKQsLpPFZvPcuVimpsbFTUa2/2a8orr+LVb8/Qq6s9pTdqm32s9K0V5iRJV3R4WcUVvB6XwsHzxQzzdEYFFF2vbnJOZW09jnY2Bu/qIISxSU1XdFiKAuv3nGP6+n9zOqeM1eGBbHluAsV3JNwG5ZW1Bu/qIISxyUhXdEgHM4s5V3iN9XsyCRvuzmuz/HHt2QVoeUt1Q/dCE8LYJOmKDqXwWhVrtp0l/lQeKPCPJWP4pa9Lk3OWTR+qc/sbKSOI9kDKC6Ld2LlzJ0OHDsXHx4eoqKgmr9VrFf5x5BKh733PjjMFLA31ZYhbj2YJF+5uS3UhzEW26xHtQn19PUOGDGH37t14enpy//338+WXXxIQEECKppyVcSmcziljgk9fVocH4e3SnZCQEF77a7zsLSbaI9muR7RvycnJ+Pj44O3tDcCCBQvYHPcdqgvw+Q9Z9OnmwIYFIwgb7o7q1gTcshu1TcoIssJMdAQtjnRnzJihFBcX3/XFi4qKcHFp/uufpUlchjFHXKWlpVy9epUBAwYAoCksobRGhaKyoU83B5zsbSm8Vk3VtVKUyqvY29pQXV2Ng5t3s2vZ29rg16+HSeNtSWf+Ot4Na4zr+PHjiYqizND1mknLC+baIttQEpdhzBHX5s2bSUxM5M11H/BGfAr7MoroWX+VmP83g0tXbuh8MJYRNRuvl75pdi0VkBU1y6TxtqQzfx3vhpXGJeUF0b71c/fgVI0bU97/HjsbFeO75BHkUMxIr948/8+TTRIu0Ozft5MVZqI9k6QrzC7upKbJw69HR3qwI6WGYvcHcKitokKrIuGLKMZ/+Feg5d4IssJMdDQmTboRERGmvPxdk7gMY8y44k5qmj38+mjfebo52lJ/OZOs+HdB0dI9eCqbTtfgNVijd7GDfXdnIucEt7vZC53h62hMnS0umTImzGpC1F6dCdQG0DY/HY9biVRXTbf2m1c4n3badMEKcff01nRlcYQwK10JF3QnXLhZWtC32KFXV3uTxSmEqUhNV5jFjZo6NuzJ1Pu6rUpFvY7fuhoeiunqmfC2cUMUwizuaaS7efNmAgMDsbGxaTa1IjIyEh8fH4YOHUpiYqLOjy8pKWHq1Kn4+voydepUSktL7yUcvebPn8+IESMYMWIEAwcOZMSIETrPGzhwIMHBwYwYMYKQkBCTxHK7VatW4eHh0Rjb9u3bdZ7X0vJYU1i2bBl+fn4MGzaMRx99lLKyMp3ntfV+7UotYOr7/+Z//32Rcd596GLX9NvOyd6Wx8f2x8nettnx0O55LX7uiqLwu9/9Dh8fH4YNG8aJEycM/4TvQk5ODg899BD+/v4EBgayYcOGZufs378fZ2fnxq/vW2+9ZZbYWvu6WOKeZWRkNN6HESNG0LNnT9avX9/kHHPdr8WLF+Pq6kpQUFDjsbbmIqP8LCqK0tKfFqWlpSnp6enKxIkTlaNHjzYeT01NVYYNG6ZUVVUpFy9eVLy9vZW6urpmH79s2TIlMjJSURRFiYyMVJYvX97aW96zF198UfnjH/+o87UBAwYoRUVFJo+hwZtvvqm8++67LZ5TV1eneHt7KxcuXFCqq6uVYcOGKampqSaNKzExUamtrVUURVGWL1+u9+vS2v3KKalQlnx+VBnwylZl2vvfK0ezriiKoihbTuQqD0QmKQNf2ao8EJmkbDmRq/P410cvtfi5jx49Wtm2bZsyY8YMRavVKocPH1bGjBljrNvQory8POX48eOKoijK1atXFV9f32Zfl3379imzZs0ySzy3a+3rYql71qCurk5xc3NTsrOzmxw31/36/vvvlePHjyuBgYGNx9qSiwz8WdSbV++pvODv76/zeHx8PAsWLMDR0ZFBgwbh4+NDcnIy48ePb3be/v37AVi0aBGTJk1i7dq19xJSixRF4auvvmLv3r0mew9j07U8Nj4+noCAAJO957Rp0xr/Pm7cOL7++muDPr62XsunB7N4b9c5autvVmuvVdWSW1pJyEDdpQJofvzw4cOtfu7x8fH813/9FyqVinHjxlFWVkZ+fj5qtdqgmA2lVqsb36NHjx74+/uj0WhM+nUxFkvdswZJSUkMHjy4cfWhuT344INkZ2c3OdaWXGSsn0WTPEjTaDT079+/8d+enp5oNM23ur58+XLjF1qtVlNYWGiKcBodOHAANzc3fH19db6uUqmYNm0ao0ePJjo62qSxNPjoo48YNmwYixcv1vkrTVvvpal8+umnzJw5U+druu7XsewSHvngIJE70qnTahunvzRsmWPIludt+dwtfX8AsrOzOXnyJGPHjm322uHDhxk+fDgzZ84kNTXVLPG09n1s6XsWGxvL448/rvM1S9wvaFsuMtZ9a3WkO2XKFAoKCpodX7NmDeHh4To/RtHxQKShSYmptCXOL7/8Uu8XG+DQoUO4u7tTWFjI1KlT8fPz48EHHzRZXM8++yyvv/46KpWK119/nZdeeolPP/20yXmmupdtuV9r1qzBzs6OhQsX6rxGw/36POk0zy95ktU/OWHbvQ+9utrTp5sDJRU1Tc5v2Hm3rfNo2/K5W+J77XbXr19n7ty5rF+/np49ezZ5bdSoUVy6dInu3buzfft2Zs+eTWam/oeJxtLa97El71lNTQ0JCQlERkY2e81S96utjHXfWk26e/bsMfiinp6e5OTkNP47NzcXd3f3Zue5ubk1/lqTn5+Pq6urwe/V1jjr6ur49ttvOX78uN5zGmJ0dXXl0UcfJTk5+Z6Tblvv3zPPPMMjjzzS7Hhb76Wx44qJiWHr1q0kJSXp/cZyd3dny4lc1uzV0GfuKlR2DgBU1dQ32+q8gSE777blczfV/WmL2tpa5s6dy8KFC5kzZ06z129Pwg8//DDPPfccxcXF3HfffSaNq7XvY0vesx07djBq1Cjc3NyavWap+wVty0XGum8mKS+EhYURGxtLdXU1WVlZZGZmMmbMGJ3nxcTEADd/yPWNnI1hz549+Pn54enpqfP1iooKrl271vj3Xbt2NXm6aQr5+fmNf9+yZYvO97v//vvJzMwkKyuLmpoaYmNjCQsLa/N7xJ3UMCFqL4NWbGNC1N42/Xq/c+dO1q5dS0JCAl27dtV5TkVFBaezLvPKN2eorVeoKy2gWpMOQFWdFlt9idqAvght+dzDwsL4+9//jqIoHDlyBGdnZ7PUJhVFYcmSJfj7+/Piiy/qPKegoKBxdJScnIxWq6Vv374mjast38eWumfQ8m+blrhfDdqSi+71Z7FRS0/ZWnsK+O233yoeHh6Kg4OD4urqqkybNq3xtbffflvx9vZWhgwZomzfvr3x+JIlSxpnOhQXFyuTJ09WfHx8lMmTJytXrlxp7S3v2qJFi5RNmzY1OabRaJSZM2cqiqIoFy5cUIYNG6YMGzZMCQgIUN5++22TxdLgySefVIKCgpTg4GDlV7/6lZKXl9csLkW5+bTZ19dX8fb2NiiuLSdyFb/XdigDXtna+MfvtR2NswX0GTx4sOLp6akMHz5cGT58uPLb3/62SVw3quuUP3x5WBmwLE7p//t/Kc4PPK70+uVTTd6n4b0Mfe876frcN23apGzatEkZPXq0otVqleeee07x9vZWgoKCmsyiMaUDBw4ogBIcHNx4n7Zt29YYm6IoyocffqgEBAQow4YNU8aOHascOnTI5HHp+z6+PS5L3bOKigqlT58+SllZWeMxS9yvBQsWKP369VPs7OwUDw8P5W9/+5veXHQPP4t686osA7Zi+pbcevRy4tCKyQZfL+6khtVb07hyq1Z7/8De5JRUUnC1Sud7LJs+1KR9EdprS0AhkNaOnZO++qkhddUGnx/K4q2taWhv+99wiuYqc0d78M1xjc5OX7LzrhDNSe8FK6avfmpIXbXu1pzbP37XNOHCzdkI+9KLZBNIIQwgI10rdq9bkZ/KKWPlljOk5l3Ve05DQxpJskK0jYx0rdjdbkVeXlnLG/EpPPqXQxRfr+YvC0fh7txF57myS4MQhpGRrpUzZBSqKAoJp/NYvfUsJRXVPP3AQF6cOoQeXeypqdPe06hZCHGTJF0BQFZxBa/HpXDwfDHDPJ35/Nf3E+Th3Ph6Q+Jub7s0CNHRyJSxTq6qtp6Pv7/AX/ZfwNHWhuUzhvLE2AHY2hh/WeiyZcv47rvvcHBwYPDgwXz22Wf06tULuNkK9JNPPsHW1pYPPviA6dOnt3o9mTIm2jHZOUI0d+h8MTM3HGD9nkxmBPYj6aWJ9Ohiz4Pr9hm0gq2tpk6dSkpKCj/99BNDhgxpXH+flpZGbGwsqamp7Ny5k+eee476ev27/QrRkUnS7YQKr1WxNPYkC//2I4qi8I8lY/jg8ZH8cOEKr357Bk1ZJQo3t9YxtDNYS6ZNm4ad3c2K1rhx48jNzQX0twIVwhpJTbcTqdcq/DP5Z9btTKe6VsvSUF+enTSYLrd2bHg3MaPJgzIwvDNYW3366afMnz8fuNkyb9y4cY2vWaI9oxDmIkm3k0jRlLMyLoXTOWVM8OnL6vAgvF26NznHGCvY7qZlpK7nCvo6m0VHRzf2iC0qKmpzXEK0F5J0rdz16jre33WOz3/Iok83BzYsGEHYcHedSc29l5POXg2GzMW9m5aRhrTMi4iIICIiAsAs+9gJYWxS07VSiqKw40w+U977ns9+yOLxMV4kvTiJ8BEeekeRy6YP1bk5pLHm4uprGdnWVqBCWAMZ6VqhnJIbvBGfwr6MIgLUPdn05ChGevVu9eNMPRf3+eefp7q6mqlTpwI3H6Z9/PHHBAYGMm/ePAICArCzs2Pjxo3Y2tq2cjUhOiaZp2tFauq0/PXART5IysTORsWL04ayaPwA7Gyb/0ITd1LT4Rc6yDxd0Y5Ja0drd+TiFV6LS+F84XVmBvXjjV8FoHbWXYuNO6lpsqS3YWoY0OESrxAdjSTdDu7K9Wre2Z7ONydy8eztxKdPhzDZr/n+U7cz59QwIURTknQ7KK1W4atjOUTuSOdGTR3/89Bgnn/IFyeH1muhxmxuLoQwjCTdDibupIZ3tp+l8Fo1AINduvHxk+PxdevR5msYY2qYEOLuyJSxDuRfyT/z8ubTjQkXQFNa2WKTcV1MPTVMCKGfjHTNQN9MAUNmEOxKLeAPW1Kov2O2SVWd1uBarLRpFMJyZMqYid05UwBujir1begYOScY+E9CdO3piEt3R1JaGM2qgKyoWSb7HNormTIm2jG9U8Yk6ZqYvm3QbVWqZqNWgF5O9lTXaZvNLggb7s6x7BLyyi2z3Xl7JElXtGPST9dS9M0I0JVwAcoqa5slXIDjl0pZPsNPZy32IT8Xk7ZkFEIYjyRdE9M3I8BWT/8DfRp23dW10eS+9CK9826FEO2LPEgzMX3boOuq6bakIXnr2mjyhX+d0vkxMu9WiPZHRrompm90+vbsYCLnBOPaw7Hx3IF9u7L8LqZz6RtNy7xbIdofGemaga7RaWVNPZmF1yi9UYOzkz0rZvoxP6Q/NjYq3Hs5GfRQTN9oWubdCtH+SNK1gH0ZhbwRn0JOSSVzRnnwh4f9ua/7f0a8upJ0S2TerRAdhyRdMyoor+KtralsP1PAYJdufPnMOMYP7muUaxuaqIUQliFJ1wzq6rX8/fAl3tuVQZ1WYdn0oTzzS28c7KSkLkRnI0nXxE7nlLEy7gwpmqtMHOLC6vAgvPp2bf0DhRBWSZKuiZRX1vKnxAy++PESrj0c+cvCUcwM6qd3fzIhROcgv98amaIoxJ/SEPre9/zfj5d4+oGB7HlxIg8HqyXh3vKnP/0JlUpFcXFx47HIyEh8fHwYOnQoiYmJFoxOCNOSka4RZRVX8HpcCgfPFzPM05nPf30/QR7Olg6rXcnJyWH37t14eXk1HktLSyM2NpbU1FTy8vKYMmUK586dk80phVWSpGsgXe0YZwT14+PvL/CX/RdwtLXhrfBAFo4dgK2NjGzv9MILL7Bu3TrCw8Mbj8XHx7NgwQIcHR0ZNGgQPj4+JCcnM378eAtGKoRpSHnBAA1tGm9vLLP865/45dp9rN+TSZB7T7o52vFmfCoPrtsnDWfukJCQgIeHB8OHD29yXKPR0L9//8Z/e3p6otHIvRPWSUa6BtC1oWNNvZaSGzU8O3Ewn/+Q3el32J0yZQoFBQXNjq9Zs4Z33nmHXbt2NXtNV3tRffXv6OhooqOjASgqKrrHaIUwP0m6BtDbplGrkHA6T3bYBfbs2aPz+JkzZ8jKymoc5ebm5jJq1CiSk5Px9PQkJyen8dzc3Fzc3d11XiciIoKIiAjgZj9dIToaKS8Y4Palurfz6OUkO+y2Ijg4mMLCQrKzs8nOzsbT05MTJ07Qr18/wsLCiI2Npbq6mqysLDIzMxkzZoylQxbCJGSk2wbXqmp5f/c5iiuqm73W0Fjm3cQM2WH3LgUGBjJv3jwCAgKws7Nj48aNMnNBWC3ZrqcFiqKwI6WAP36XSuG1ap4cOwB/dQ827rugc5NJXZ2+IucEd6rygjnJdj2iHdM7dUlGunr8fOUGbySksD+jiAB1Tz5+cjQjvXoD8MTYAc3Ol05fQoi2kJHuHWrqtPz1wEU+SMrEzkbFC1OH8PQDA7GzlfJ3eyMjXdGOyUi3LY5cvMJrcSmcL7zOzKB+vPGrANTOUpMVQhiPJF3gyvVq3tmezjcncvHs7cRnT9/PQ36ulg5LCGGFOnXS1WoVvjqWQ+SOdG7U1PE/Dw3m+Yd8cXKQJ+dCCNPotEn3bP5VVm45w4mfyxgzqA9rZgfh69bD0mEJIaxcp0u6FdV1bEjK5JODWTg72fPEGC/2ZxQy7c//bjYFTGYiCCGMrVMl3V2pBaxKSCWvvIoF9/cn2MOZt7edbdYv4dilEr45run0fRSEEMbXKZJubukNViWksefsZYa69eDrx0cSMrAPE6L26uyX8OWPOdTfMZWuM/ZREEIYn1Un3dp6LZ8ezGL9nkwAXp3px+JfDML+1pxbvQ1s9Mxdlj4KQoh7ZbVJ91h2CSu3pJBx+RpT/N1YFRaAZ++mG0K693LS2S/BVqXSmXilj4IQ4l5ZXdItrahh7c50Yo/m4O7cheinRjMtsB9xJzXM/98jTR6MLZs+VGe/hLmjPZrUdBuOL5s+1BKfkhDCiljNMmBFUfjmhIZ3tp+lvLKWJb8YxNJQX7o52rXYjAZ090uQ2QvtnywDFu2Y3mXAVpF0zxdeY+WWFH7MKmH0gN68PTsIf3XPxtcnRO3VWUbw6OXEoRWTzRmqMCJJuqIds87eC5U19Xy0L5Pof1+kq4MdUXOCmRfSH5s7NoSUBuNCiPbCJEnXHL+a78so5I34FHJKKpkzyoM/POzPfd0ddb63vgdm8mBMCGFuRk+6d9ZPjb2woKC8ire2prL9TAGDXbrx5TPjGD+4b4vvLQ/GhBDthdGbxOraMbdhYcG9qLs15zb0vf0knS1k2fSh7Fj6YGPCbem996UXETknGI9eTqi4WcuVHR2EEJZg9JGuKeqnp3LKWLnlDKl5V5k4xIXV4UF49e3a7LyW3nv2SA9Jsu3Ahx9+yEcffYSdnR2zZs1i3bp1AERGRvLJJ59ga2vLBx98wPTp0y0cqRCmYfSka8z6aXllLX9KzOCLHy/h0t2RjU+M4uHgfqhUuh8MSu22fdu3bx/x8fH89NNPODo6UlhYCEBaWhqxsbGkpqaSl5fHlClTOHfunGxOKayS0csLy6YPxcm+6Q+LofVTRVGIP6Uh9L3v+b8fL7Fo/ECSXprIrGFqvQnXWO8tTGfTpk2sWLECR8ebW9m7ut5sFB8fH8+CBQtwdHRk0KBB+Pj4kJycbMlQhTAZoyfd2SM97ql+mlVcwVOfJLM09hTuvbqQ8PwvWBUWSI8u9iZ/b2Fa586d48CBA4wdO5aJEydy9OhRADQaDf379288z9PTE41GY6kwhTApk0wZu5v6aVVtPR9/f4G/7L+Ao60Nq8MDeWLsAGxt9I9sjfXewnimTJlCQUFBs+Nr1qyhrq6O0tJSjhw5wtGjR5k3bx4XL15E1wIdfb/RREdHEx0dDUBRUZFxgxfCDNrF4oiDmcW8Hp9CVnEFYcPdeW2WP649u1g6LHEX9uzZo/e1TZs2MWfOHFQqFWPGjMHGxobi4mI8PT3JyclpPC83Nxd3d3ed14iIiCAiIgK4uSJNiI7GovuKF16rYmnsSZ785EcUReEfS8bwweMjW024cSc1TIjay6AV25gQtZe4k/KraEcwe/Zs9u7dC9wsNdTU1HDfffcRFhZGbGws1dXVZGVlkZmZyZgxYywcrRCmYZGRbr1W4Z/JP7NuZzrVtVqWhvry7KTBdLnjIZiu1WWASRdfCNNZvHgxixcvJigoCAcHB2JiYlCpVAQGBjJv3jwCAgKws7Nj48aNMnNBWC2zN7xJ0ZSzMi6F0zllTPDpy+rwILxdujc7T19nsC72NpTeqG12vjSv6Xyk4Y1oxyzf8OZaVS3v7z5HzA/Z9OnmwIYFIwgb7q73gYm+1WV3HmsgzWuEEB2ByZOuoijsSCngj9+lUnitmoVjvVg2zQ/nri1PATM0icoCCCFER2DSpPvzlRu8kZDC/owiAtQ9+fjJ0Yz06t2mj9W3uqyXkz3VdVppXiOE6JBMknRr6rT89cBFPkjKxM5GxeuPBLBo/ADsbNs+WULfVjqrwgIB3bs9CCFEe2eSpJuaV867iRnMDOrHG78KQO1s+K/+DUlUX3KVJCuE6IhMNnshveAqfv16tn6iEHdJZi+Idsz8sxcMSbiyCaQQorOw+DJgU+80IYQQ7YlFlwGD6XaaEEKI9sjiSVd26hVCdCZmLS/ITr1CiM7ObCPdhtqtpqwShf/Ubh/yc5HdHoQQnYbZkq7s1CuEEGYsL8hOvUIIYcaRrr4ardRuhRCdidmSruzUK4QQZiwvtNZLQQghOgOzThmT2q0QorOz+OII0XmcOnWKcePGMWLECEJCQkhOTm58LTIyEh8fH4YOHUpiYqIFoxTCtCzee0F0HsuXL+fNN99k5syZbN++neXLl7N//37S0tKIjY0lNTWVvLw8pkyZwrlz52RzSmGVZKQrzEalUnH16lUAysvLcXd3ByA+Pp4FCxbg6OjIoEGD8PHxaTIKFsKayEhXmM369euZPn06L7/8Mlqtlh9++AEAjUbDuHHjGs/z9PREo9FYKkwhTEqSrjCqKVOmUFBQ0Oz4mjVrSEpK4s9//jNz587lq6++YsmSJezZswddjfT17RIdHR1NdHQ0AEVFRcYNXggzaG3nCCGMRqVSlQO9FEVRVDezarmiKD1VKtWrAIqiRN46LxFYpSjK4Vaut1NRlBkmD1wII5KarjCnPGDirb9PBjJv/T0BWKBSqRxVKtUgwBdotagrCVd0RFJeEOb0DLBBpVLZAVVABICiKKkqleorIA2oA/5HUZR6/ZcRouOS8oIQQpiRlBeEEMKMJOkKIYQZSdIVQggzkqQrhBBmJElXCCHMSJKuEEKYkSRdIYQwI0m6QghhRv8/GAIazx5Z8xYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(x,y)  #7 \n",
    "plt.plot(x, 7*x +  3)\n",
    "set_spines()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b360136d",
   "metadata": {},
   "source": [
    "It seems that the equation $y = 7 + 3 x$ describes the data as well as possible."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6740b689",
   "metadata": {},
   "source": [
    "<h2 id='rates-iterations'>1.3  Different Learning rates and number of Iteration in Gradient Descent</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5b2809d",
   "metadata": {},
   "source": [
    "Again let's generate the data with bigger residuals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ac8edbb8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAEoCAYAAAD7ZyLFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAf9klEQVR4nO3df3AU553n8fdjRGScXzKLhEcatFZKREGC2JApgY87QizLOLmsILKtCLxlEexSbShXOaEMJ1/qNsFbjoTzw05cWqeUw9lZZ2uVUJtFLryRNjGRk7KLyLPGPwAbKzbcSSMdkg4RzgFjkPv+kFCEPCPNSN1Pz2g+ryqXRU+r55kW+vD009/naeM4DiIiYsdVfjdARCSTKHRFRCxS6IqIWKTQFRGxSKErImKRQldExKKsaV5XPZmkrNtuu4329na/myESi4n3gnq6kraGhob8boJI0hS6IiIWKXRFRCxS6IqIWKTQFdf19PTwuc99jmXLllFWVsYPfvADAE6fPk1lZSVLly6lsrKS4eHh8e9pbGykuLiYkpISOjo6/Gq6iOcUuuK6rKwsvve97/H6669z6NAhmpubOXbsGE1NTVRUVNDd3U1FRQVNTU0AHDt2jNbWVo4ePUp7ezvbt29nZGTE508h4g2FrrguEAiwatUqAD760Y+ybNkyotEobW1t1NXVAVBXV8f+/fsBaGtro7a2luzsbIqKiiguLqarq8uv5sscs/9wlLVNBylqeIa1TQfZfzjqa3sUuuKpkydPcvjwYVavXs2pU6cIBALAaDAPDAwAEI1GWbJkyfj3BINBotHYvxgtLS2EQiFCoRCDg4PefwBJa/sPR3nwF68RPXMeB4ieOc+Dv3jN1+BV6Ipn3nnnHW6//XYee+wxPvaxj8XdL9aazsbEri2vr68nEokQiUTIzc11ra0yN32n4zjnL145VHX+4gjf6TjuU4sUuuKRixcvcvvtt3PXXXdRXV0NwOLFi+nv7wegv7+fvLw8YLRn29PTM/69vb295Ofn22+0zDl9Z84ntd0Gha64znEc7rnnHpYtW8aOHTvGt1dVVREOhwEIh8Ns3LhxfHtraysXLlzgxIkTdHd3U15e7kvbZW7Jz1mQ1HYbplt7QSRpzz//PE899RQrVqzgxhtvBODb3/42DQ0N1NTUsHfvXgoLC9m3bx8AZWVl1NTUUFpaSlZWFs3NzcybN8/HTyBzxc4NJTz4i9euGGJYMH8eOzeU+NYmM80z0rTgjaSsUChEJBLxuxmS4vYfjvKdjuP0nTlPfs4Cdm4oYdPKAq/fNu6CN+rpisictmllgY2QTZjGdEVELFLoiohYpNAVEbFIY7oikpF8usGm0BWRzHN5evDlUrLL04MBz4NXwwsiknH8nB6s0BWRjOPn9GCFrohkHD+nByt0RSTj7NxQwoL5V041tzU9WDfSRCTjXL5ZpuoFEZEpuFnm5df0YIWuiKQFP8u83KQxXRFJC6n4FIiZUOiKSFpIxadAzISGF0QkLeTnLCAaI2DdLvPyenqwerriiW3btpGXl8fy5cvHt7388susWbOGG2+8kVAodMVj1hsbGykuLqakpISOjg4/miwpzkaZl42nByt0xRNbt26lvb39im27du3im9/8Ji+//DIPPfQQu3btAuDYsWO0trZy9OhR2tvb2b59OyMjI7EOKxls08oCGqtXUJCzAAMU5CygsXqFq71QG+PGGl4QT6xbt46TJ09esc0Yw9mzZwH44x//OP7E37a2Nmpra8nOzqaoqIji4mK6urq46aabbDdbUpzXZV42xo0VumLNY489xoYNG3jggQd4//33eeGFFwCIRqOsWbNmfL9gMEg06t7lnEiibIwba3hBrHniiSd49NFH6enp4dFHH+Wee+4BRh/ZPpkxsZ/r19LSQigUIhQKMTg46Gl7JfPYGDdW6Io14XCY6upqAO68887xG2nBYJCenp7x/Xp7e8eHHiarr68nEokQiUTIzc31vtGSUWyMG2t4QazJz8/nueeeY/369Rw8eJClS5cCUFVVxZYtW9ixYwd9fX10d3dTXl7uc2slU3k9bqzQFU9s3ryZzs5OhoaGCAaD7N69mx//+Mfcf//9XLp0iauvvpqWlhYAysrKqKmpobS0lKysLJqbm5k3b9407yCSnkys8bQJpnxRxE+hUIhIJOJ3M9KCX88Dm6l0a28MsW9KoJ6uyJzn90IxyQao3+31mkJXZI6bquB/08oC13qVsY4DJB2g07U30fdO1YBW6IrMcVMV/LvVq4x3nKvnX5V0gCY7QSHdesYqGROZ46Z6Hphb017jHWf43MWY+081wyvZ55el25KPCl2ROW6qgn+3pr0mu/9UM7ySnaCQbks+KnRF5ripCv7deipuvP1zFsxPeoZXshMU/Hyy70yoZEzSlkrGZm/yeCiMhmKys7CmOg54+wBItz6Dy1QyJiIf5NZTcac7jpfh5+eTfWdCPV1JW+rppr90KvVKknq6IpJa0q3Uyy26kSYivki3Ui+3KHRFxBfpVurlFoWuiPgi3Uq93KLQFRFf2HhKQyrSjTQR8UW6lXq5RaErIr7x+ikNqUjDCyIiFil0xRPbtm0jLy+P5cuXX7H98ccfp6SkhLKyMnbt2jW+vbGxkeLiYkpKSujo6LDdXBFrNLwgnti6dSv33Xcfd9999/i23/zmN7S1tfHqq6+SnZ3NwMAAAMeOHaO1tZWjR4/S19fHLbfcwptvvqnnpMmcpJ6ueGLdunUsXLjwim1PPPEEDQ0NZGdnA5CXlwdAW1sbtbW1ZGdnU1RURHFx8fjj2UXmGoWuWPPmm2/yu9/9jtWrV/PZz36WF198EYBoNMqSJUvG9wsGg0SjUb+aKeIpDS+INZcuXWJ4eJhDhw7x4osvUlNTw9tvv02sRZeMib1eSEtLy/ij2wcHBz1tr4gX1NMVa4LBINXV1RhjKC8v56qrrmJoaIhgMEhPT8/4fr29veTn58c8Rn19PZFIhEgkQm5urq2mi7hGoSvWbNq0iYMHDwKjQw3vvfceixYtoqqqitbWVi5cuMCJEyfo7u6mvLzc59aKeEPDC+KJzZs309nZOd6T3b17N9u2bWPbtm0sX76cD33oQ4TDYYwxlJWVUVNTQ2lpKVlZWTQ3N6tyQeYsLWIuaUuLmEsKi7uIuYYXREQs0vCCSAqYw4+tkUkUuiI+y9TH1mQqDS+I+CxTH1uTqRS6Ij7L1MfWZCqFrojPMvWxNZlKoSvis0x9bE2m0o00EUviVSi4+dgaVUGkPk2OkLSVTpMjJlcowGhvtrF6hWuhaOM9JGGaHCHiJxsVCqqCSA8KXRELbFQoqAoiPSh0RSywUaGgKoj0oNAVscBGhYKqINKDqhdELHCzQsHP95DZU/WCpK10ql6QjBO3ekE9XRGJSTW/3lDoSkZRkCRGK595RzfSxBPbtm0jLy+P5cuXf+C17373uxhjGBoaGt/W2NhIcXExJSUldHR0eNKmy0ESPXMehz8Hyf7Detz7ZKr59Y5CVzyxdetW2tvbP7C9p6eHX/3qVxQWFo5vO3bsGK2trRw9epT29na2b9/OyMjIB753tmYaJPsPR1nbdJCihmdY23QwI0JaNb/eUeiKJ9atW8fChQs/sP3rX/86jzzyCMb8+T5DW1sbtbW1ZGdnU1RURHFxMV1dXa63aSZBkqm9Y9X8ekehK9Y8/fTTFBQUcMMNN1yxPRqNsmTJkvE/B4NBotHYodbS0kIoFCIUCjE4OJjU+88kSDL1Mls1v95R6IoV586d4+GHH+ahhx76wGuxyhYn9oQnqq+vJxKJEIlEyM3NTaoNMwmSTL3M3rSygMbqFRTkLMAABTkLtHCOS1S9IFa89dZbnDhxYryX29vby6pVq+jq6iIYDNLT0zO+b29vL/n5+a63YSaTB/JzFhCNEbCZcJk9cdlJcY9CV6xYsWIFAwMD43++/vrriUQiLFq0iKqqKrZs2cKOHTvo6+uju7ub8vJyT9qRbJDs3FASc7lEvy+zVfqWvhS64onNmzfT2dnJ0NAQwWCQ3bt3c88998Tct6ysjJqaGkpLS8nKyqK5uZl58+bF3Nc2v6fWxgpXQDW0aUzTgCVtzfVpwPEWJb96/lUMn7v4gf0LchbwfMPNNpso8WkasEi6iVc5MXnbZbZu7mloY3YUuiIpKtkQtXFzT9ODZ08lYyIpKl6I5iyY71sNbabWLbtJPV1JC/FuKHl5/Jn23Nw6VrzKiW9VlQH+3NzL1LplNyl0JeXFu6S9GONmkpvHh+Qvmd081nSVE35czmdy3bJbVL0gKW9t08GYv+hDP93BO72zv6yNd/yZVAO4eaxUpMe8J0zVC5K+4l26Xhx539Pjz+SSea5ffvtdtzwXKHQl5cW7pJ0/z537wG5eMmfC5bemB8+Oqhck5cVbqOa6j13t6fFncrNOq3PJdBS6kvLirXiVc818T48/k96cVueS6ehGmqStuT4NWNJa3Btp6umKiFik0BURsUihKyJikUJXRMQi1elKwrSkn8jsqacrCUn2UeTbtm0jLy+P5cuXj2/buXMnn/rUp/j0pz/Nl770Jc6cOTP+WmNjI8XFxZSUlNDR0eHxpxHxj0JXEpLskn5bt26lvb39im2VlZUcOXKEV199lU9+8pM0NjYCcOzYMVpbWzl69Cjt7e1s376dkZHYC3WLpDuFriQk2TUF1q1bx8KFC6/Yduutt5KVNTqitWbNGnp7ewFoa2ujtraW7OxsioqKKC4upqury8XWi6QOha4kJN7aATNdU+DJJ5/k85//PADRaJQlS5aMvxYMBolGYw9biKQ7ha4kxM01BR5++GGysrK46667AIg1K9KY2BN6WlpaCIVChEIhBgcHk35vEb+pekES4taSfuFwmAMHDvDss8+OB2swGKSnp2d8n97eXvLz82N+f319PfX19cDoNGCRdKPQlYTNdkm/9vZ29uzZw3PPPcc111wzvr2qqootW7awY8cO+vr66O7upry83I0mi6Qcha54YvPmzXR2djI0NEQwGGT37t00NjZy4cIFKisrgdGbaT/60Y8oKyujpqaG0tJSsrKyaG5uZt68edO8g0h60ipjkra0ypikMD2uRzKLZs9JqlLoypzj5hN5Rdym0JU5Z6rZc26GrnrTMhMKXbHO67Cy8URe9aZlpjQ5QqxKduGcmXB79lwsya5FIXKZerpi1XRh5UYPeOeGkit6oeD+E3lt9KZlblLoilXxQulyj9eNy3W3Zs9NJT9nAdEYn8XN3rTMTQpdsSpeWM0zxtWbX7OdPTcdG71pmZs0pitWxVs4ZyTOJJ1UvVzftLKAxuoVFOQswAAFOQtorF6hm2gyLfV0xap4l/7f6TiedpfrXvemZW5S6Ip18cJKl+uSCRS6cgU3a2iTOZaNm18iqUAL3sxxyQTf5IJ/GO1tzmSs0s1jxaMFbySFxV3wRjfS5rBkJyK4WfCvyQMisSl057Bkg8/Ngn9NHhCJTaE7hyUbfG5On7UxFVckHSl057Bkg8/Nh0+6eayp7D8cZW3TQYoanmFt00FX13AQ8YJCdw5LNvjcLPh/+vH/QV/zXzPwD/eNH+u/VwRp3rWVpUuXUllZyfDw8Pj+jY2NFBcXU1JSQkdHR0LvcebcRc8XzxFxm6oX5ji/1nz97W9/y0c+8hHuvvtujhw5AsCuXbtYuHAhDQ0NNDU1MTw8zJ49ezh27BibN2+mq6uLvr4+brnlFt58881pn5P2kWAJi/76+x/YXpCzgOcbbvbkc4kkSI/ryVR+zZpat24dJ0+evGJbW1sbnZ2dANTV1bF+/Xr27NlDW1sbtbW1ZGdnU1RURHFxMV1dXdx0001TvsfFkfdjbtfNOkllCl2ZtUR706dOnSIQCAAQCAQYGBgAIBqNsmbNmvH9gsEg0ej0QwTz58UeHdPNOkllCl2ZlameoHDjtYkdI9YQlzGxr85aWlpoaWkB4JqRP7Fg/jxNHZa0ohtpMivJ1AIvXryY/v5+APr7+8nLywNGe7Y9PT3j+/X29pKfnx/z/err64lEIkQiEQoLrtNKX5J21NOVWUmmFriqqopwOExDQwPhcJiNGzeOb9+yZQs7duygr6+P7u5uysvLE3p/rfQl6UahK7MSb1Hyd375PW4Kv87Q0BDBYJDdu3fT0NBATU0Ne/fupbCwkH379gFQVlZGTU0NpaWlZGVl0dzcPG3lgki6UsmYzIqNhW3i0YI3ksJUMibe0JKMIslR6MqsaVxVJHGqXhARsUihKyJikYYX0oxfaymIiDsUumlkqtlfkNzNLIW3iD8Uumkk3uyvbz19lAuX3o8ZxrGCdKrwVvCKeEtjumkk3uyvM+cvJvVYHj2/TMQ/Ct00kuzqWck+rkdLIop4T6GbRuI9CeLaa+bH3D/Zx/VoSUQR7yl000i8x+l886/Kknosj63nl4nIB+lGWpqZavZXotUImror4h8teCNpSwveSAqLu+CNhhdERCzS8EKSNKlARGZDoZsETSoQkdnS8EISNKlARGZLoZsETSpwx6OPPkpZWRnLly9n8+bNvPvuu5w+fZrKykqWLl1KZWUlw8PDfjdTxBMK3SRoUsHsRaNRfvjDHxKJRDhy5AgjIyO0trbS1NRERUUF3d3dVFRU0NTU5HdTRTyh0E2CJhW449KlS5w/f55Lly5x7tw58vPzaWtro66uDoC6ujr279/vbyNFPKLQTUK8GWG6iZa4goICHnjgAQoLCwkEAnz84x/n1ltv5dSpUwQCAQACgQADAwMxv7+lpYVQKEQoFGJwcNBm00VcockRYtXw8DC33347P/vZz8jJyeHOO+/kjjvu4L777uPMmTPj+1177bXTjutqcoSkME2OkNTw61//mqKiInJzc5k/fz7V1dW88MILLF68mP7+fgD6+/vJy8vzuaUi3lDo+mj/4Shrmw5S1PAMa5sOsv9w1O8mea6wsJBDhw5x7tw5HMfh2WefZdmyZVRVVREOhwEIh8Ns3LjR55aKeEOTI3ySqRMtVq9ezR133MGqVavIyspi5cqV1NfX884771BTU8PevXspLCxk3759fjdVxBMa0/XJ2qaDRGPU9xbkLOD5hpt9aFH60ZiupDCN6aYaTbQQyUwKXZ9oooVIZvJkTFcrcV0p1vnYuaHkijFd0EQLkUzg+pju5BtEMBomtiYRxAt8v/4hmOp8gJ7eMBsa05UUFndM1/XQ9fMGUbyAu/0zBfzLf0R9+YdAN8y8o9CVFBY3dF0fXvDzBlG8pRf/+fc9jEz6x+Xykoxeh+5MzoeGZ0TmLtdvpPl5gyhekE0O3On2d1Oy5+Nybz165jwOf67fzYSJEyKZwPXQ9XMlrnhBNs/E7unb+Icg2fOhhdJF5jbXQ9fPlbjiBdzm1Ut8+4cg2fOh+l2Ruc2TkrFNKwt8GYO8/J6xxkNDf7nQt6qGZM5Hfs6CmDfeVL8rMjdk9DRgv8vb0qVNqUrVC5LCNA04llQcP9VC6SJzW0avMpaq46d+Dc+IiPcyuqer9Q9ExLaMDl09aFJEbMvo4YWpqh1ERLyQ0aELGj/1w5kzZ7j33ns5cuQIxhiefPJJSkpK+PKXv8zJkye5/vrr+fnPf861117rd1NFXJfSwwvp9gyxdGuvX+6//35uu+023njjDV555RWWLVtGU1MTFRUVdHd3U1FRQVNTk9/NFPFEytbpplu9arq11y9nz57lhhtu4O2338ZMmJ5dUlJCZ2cngUCA/v5+1q9fz/HjU5fuqU5XUlj61en6XUObbK/V7/ami7fffpvc3Fy+8pWvsHLlSu69917+9Kc/cerUKQKBAACBQICBgQGfWyrijZQNXT9raGey0leq1vymmkuXLvHSSy/x1a9+lcOHD/PhD384qaGElpYWQqEQoVCIwcFBD1sq4o2UCN1YvUo/a2hn0mtVzW9igsEgwWCQ1atXA3DHHXfw0ksvsXjxYvr7+wHo7+8nLy8v5vfX19cTiUSIRCLk5uZaa7eIW3wP3Xi9ys99Kte3GtqZ9FpV85uY6667jiVLloyP1z777LOUlpZSVVVFOBwGIBwOs3HjRj+bKeIZ30vG4vUqf/PGII3VK3xZGWwmK32p5jdxjz/+OHfddRfvvfcen/jEJ/jJT37C+++/T01NDXv37qWwsJB9+/b53UwRT/hevVDU8EzMNzHAiab/+oHtNqoEVImQHlS9ICksdasXkh0LtVEloJW+RMQrvg8v7NxQErNXGW8s1FaVgGaqiYgXfO/pJturVJWAiKQz33u6kFyvMtmesYhIKkmJ0E2GqgREJJ2lXeiCxltFJH35PqYrIpJJFLoiIhYpdEVELFLoiohYpNAVEbHIavWC1wvViIikOmuhO3kRmctLOAIKXhHJGNaGF/Q4GxERi6Grx9mIiFgMXS1UIyJiMXT1OBsREYuhq4XB5bKRkRFWrlzJF7/4RQBOnz5NZWUlS5cupbKykuHhYZ9bKOId3x/XI5nn+9//PpFIhLNnz3LgwAF27drFwoULaWhooKmpieHhYfbs2TPtcfS4Hklhqfu4Hsksvb29PPPMM9x7773j29ra2qirqwOgrq6O/fv3+9Q6Ee8pdMWqr33tazzyyCNcddWf/+qdOnWKQCAAQCAQYGBgwK/miXhOoSvWHDhwgLy8PD7zmc/M+BgtLS2EQiFCoRCDg4Mutk7EDo3pijUPPvggTz31FFlZWbz77rucPXuW6upqXnzxRTo7OwkEAvT397N+/XqOH59+0ozGdCWFaUxX/NfY2Ehvby8nT56ktbWVm2++mZ/+9KdUVVURDocBCIfDbNy40eeWinhnup6uiCeMMeuBBxzH+aIx5i+AnwOFwP8G7nQc53QCx2h3HOc2Txsq4jKFroiIRRpeEBGxSKErImKRQldExCKFroiIRQpdERGLFLoiIhYpdEVELFLoiohYpNAVEbFIoSsiYpFCV0TEIoWuiIhFCl0REYsUuiIiFil0RUQsUuiKiFik0BURsUihKyJikUJXRMQiha6IiEUKXRERixS6IiIWKXRFRCxS6IqIWKTQFRGxSKErImKRQldExCKFroiIRQpdERGLFLoiIhYpdEVELFLoiohYpNAVEbFIoSsiYpFCV0TEIoWuiIhFCl0REYsUuiIiFil0RUQsUuiKiFik0BURsUihKyJikUJXRMQiha6IiEUKXRERixS6IiIWZU31ojGmHVg0i+MvAoZm8f1eUbuSo3YlR+1Kzlxs15DjOLfFesE4jjPzJk3DGBNxHCfk2RvMkNqVHLUrOWpXcjKtXRpeEBGxSKErImKR16Hb4vHxZ0rtSo7alRy1KzkZ1S5Px3RFRORKGl4QEbFoVqFrjLnTGHPUGPO+MSY06bUHjTF/MMYcN8ZsiPP9C40xvzLGdI/9/9rZtGeKdv7MGPPy2H8njTEvx9nvpDHmtbH9Il60ZdL7fcsYE53Qti/E2e+2sfP4B2NMg4V2fccY84Yx5lVjzL8aY3Li7Of5+Zrus5tRPxx7/VVjzCov2hHjfZcYY35jjHl97Hfg/hj7rDfG/HHCz/dvLbVtyp+LH+fMGFMy4Ty8bIw5a4z52qR9rJwvY8yTxpgBY8yRCdsSyiJXfhcdx5nxf8AyoAToBEITtpcCrwDZQBHwFjAvxvc/AjSMfd0A7JlNexJs8/eAv43z2klgkddtmPB+3wIemGafeWPn7xPAh8bOa6nH7boVyBr7ek+8n4vX5yuRzw58AfglYIA1wO8t/ewCwKqxrz8KvBmjbeuBA7b+PiX6c/HrnE36uf4f4C/9OF/AOmAVcGTCtmmzyK3fxVn1dB3Hed1xnOMxXtoItDqOc8FxnBPAH4DyOPuFx74OA5tm057pGGMMUAP8s5fv47Jy4A+O47ztOM57QCuj580zjuP8u+M4l8b+eAgIevl+U0jks28E/tEZdQjIMcYEvG6Y4zj9juO8NPb1/wNeBwq8fl+X+HLOJqgA3nIc539ZfM9xjuP8Fjg9aXMiWeTK76JXY7oFQM+EP/cS+y/kYsdx+mH0LzGQ51F7LvsvwCnHcbrjvO4A/26M+Q9jTL3HbbnsvrFLvCfjXNIkei69so3RXlEsXp+vRD673+cHY8z1wErg9zFevskY84ox5pfGmDJLTZru5+L3OaslfsfHj/MFiWWRK+dtymnAAMaYXwPXxXjpG47jtMX7thjbPC2TSLCdm5m6l7vWcZw+Y0we8CtjzBtj/yp60i7gCeDvGD03f8fo0Me2yYeI8b2zPpeJnC9jzDeAS8A/xTmM6+drcjNjbJv82a3/XbvizY35CPAvwNccxzk76eWXGL2EfmdsvH4/sNRCs6b7ufh2zowxHwKqgAdjvOzX+UqUK+dt2tB1HOeWZA/K6L8ASyb8OQj0xdjvlDEm4DhO/9jlzcAM3guYvp3GmCygGvjMFMfoG/v/gDHmXxm9nJhViCR6/owxPwYOxHgp0XPparuMMXXAF4EKZ2xAK8YxXD9fkyTy2T05P4kwxsxnNHD/yXGcX0x+fWIIO47zb8aYvzfGLHIcx9N1BhL4ufh2zoDPAy85jnNq8gt+na8xiWSRK+fNq+GFp4FaY0y2MaaI0X+tuuLsVzf2dR0Qr+fshluANxzH6Y31ojHmw8aYj17+mtGbSUdi7euWSeNoX4rzfi8CS40xRWO9hFpGz5uX7boN+G9AleM45+LsY+N8JfLZnwbuHrsjvwb44+XLRC+N3R/YC7zuOM734+xz3dh+GGPKGf19+78etyuRn4sv52xM3KtNP87XBIlkkTu/i7O8C/glRtP/AnAK6Jjw2jcYvdN3HPj8hO3/k7FKB+AvgGeB7rH/L5ztnckp2voPwN9M2pYP/NvY159g9G7kK8BRRi+zvb6L+hTwGvDq2A8vMLldY3/+AqN3x9+y1K4/MDp29fLYfz/y63zF+uzA31z+WTJ6ydc89vprTKii8fgc/WdGLy1fnXCevjCpbfeNnZtXGL0h+Z8stCvmzyVFztk1jIboxydss36+GA39fuDiWH7dEy+LvPhd1Iw0ERGLNCNNRMQiha6IiEUKXRERixS6IiIWKXRFRCxS6IqIWKTQFRGxSKErImLR/wdJ57llF5HDcQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y = 7*x + 120 +np.random.normal(0, 10, size=x.size)  \n",
    "plt.scatter(x,y)\n",
    "set_spines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2403cc22",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0xd7c5c72e20>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEWCAYAAABv+EDhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAmIklEQVR4nO3de5wcVZn/8c83F5gACYHMgGAICUi4hHsCbgBdVFTUFVDAH4IrqCu6iorIT0EgsqKsiAsIuLpREUXwwio3xQUvLOgLUTIkQEJAketwT4CES+559o+qNp1meqZmuqurZvr7fr2K7q7beabS1NN1Tp1TigjMzKz9jCg6ADMzK4YTgJlZm3ICMDNrU04AZmZtygnAzKxNOQGYmbUpJwAzszblBGCFknS0pDmSXpT0hKRfSTqgyWVsKOkSSUslPSnppAwxPSzpJUlXS9q8atmlklam8VamkYOMq7d93TmYffVRxoGS1taUcWzV8gEdGxtenACsMOnJ5gLgbGBLYBLwn8ChTS7qTGAHYFvgDcBnJR1cJ6ZpwH8B/5zG9HIaU7WvRsQmVdOaBmKr3dceDeyrnsdryvh+1bIzyXhsbPgZVXQA1p4kbQp8EfhARPy8atF16dRM70/LeQ54TtK3geOA/+ll3WOA6yLiljTOM4CFksZGxAtNjqsMBnJsbJjxFYAVZSbQAVyVdQNJp0h6vt5UZ5vNgK2B6qqVO4FpdYqZVr1uRPwNWAlMrVrnY5KeldQt6fCs8TdC0qS+/nZJR/ex+RaSnpL0oKTzJW2c7nOgx8aGmSGXANL6yqclzc+w7iRJN0maK+kuSW9vRYyWyQRgUUSszrpBRHwlIsbXm+pstkn6uqRq3hJgbB/rL6mZV73+hSRVJlsAZwCXSto/69/Qi5NrTuTf722liHikr789Iq6os/97gT2BrYA3AtOB89JlAz02NswMuQQAXApkraM8HfhpROwFHMUr63KtOIuBTkl5V0O+mL6Oq5o3DqhXnfNizbrrrR8Rd0TE4ohYHRHXA5cD724gvq/VnMiP7X+T7CLiyYi4JyLWRsSDwGeBI9LFAz02NswMuQSQ1s0+Wz1P0vaS/ie9JP+9pJ0qq7Puy70p8HgLQ7W+/RFYDhyWdQNJn6+5m2W9qbdt0rrtJ4DqxtU9gAV1illQva6k7YANgb/UWT8AZf0bBiu9mq37t0s6JuOu/h7vII6NDTcRMeQmYDIwv+rzb4Ed0vevBX6Xvt8KuBvoAZ4Dphcdu6f1/h1PAp4iSQIbAaOBt5HcGdPMcr4C3AxsBuxEctI7uM6604ClwOuAjYEfAj+uWn4ESdXJCOAtJL+WD+yj7Ki3nORq9ks5H+MDSe6uErANcBPwvcEcG0/Dbyo8gEEFXZUA0v8ZlwHzqqaF6bKTgM+k72cC9wAjio7f03r/lscAc4CXgCeBXwL7NbmMDYFL0hP7U8BJNctfBF5X9flo4JE0pmuAzauW/Z6knnwpSYPpUX2UOzFNEBPqLL+UpIH5xappUZP/9pOAx0huZ30UuAgYm/XYeBrek9IvwZAiaTLwi4jYVdI44L6I2KqX9RaQ/Jp5NP38APAPEfF0SwO2tiTpfcC0iDi16FjMejPk2gBqRcRS4EFJRwIoUanTfAR4Uzp/Z5LbDp8pJFBrOxHxQ5/8rcyG3BWApB+R1Gt2klyyfgH4HfBNkjr/0SR1tl+UtAvwbZJqogA+GxE3FhG3mVnZFJYAJG0D/AB4FbAWmB0RXy8kGDOzNlRkAtgK2Coi7pA0FugGDouIewoJyMyszRQ2FlBEPEFyyxkR8YKkhcCrSe7U6VVnZ2dMnjy5NQGamQ0T3d3diyKiq3Z+KQaDS+/q2Qv4Uy/LjgeOB5g0aRJz5sxpbXBmZkOcpId7m1/4XUCSNgF+BpyY3tGznoiYHREzImJGV9crEpiZmQ1SoQlA0miSk//lsf6QwGZmlrPCEoAkAd8l6bV7Xn/rm5lZcxXZBrA/yVOX7pY0L533+UhGWMxs1apV9PT0sHz58mbH13QdHR1MnDiR0aNHFx2KmVmhdwH9gSaMotjT08PYsWOZPHkyyUVFOUUEixcvpqenhylTphQdjplZ8Y3AjVq+fDkTJkwo9ckfQBITJkwYElcqZtYehnwCAEp/8q8YKnGaWXsYFgnAzKy0Vq+G73wHVq0qOpJXcAIwM8vT9dfDhz8Mv/td0ZG8ghOAmVmeKqMXPPVUsXH0wgmgSRYsWMBBBx3E1KlTOeuss/jEJz7B7bffXnRYZla07u7k9ZnyPYqkFGMBNc2JJ8K8ec3d5557wgUX9LnK8uXLOfLII7nyyivZbrvt2GmnnZg+fTr77LNPc2Mxs6ElwglguPvNb37DXnvtxbRp0wBYuXIln/nMZwqOyswK9/jj66p+Fi0qNpZeDK8E0M8v9bzMnTuXvffeG4DHH3+cTTbZhP3337+QWMysRCr1/yNHlvIKwG0ATbDhhhvS09MDwKmnnsrKlSsLjsjMSqG7G0aMgH33dQIYro4++mhuueUWdtxxR/bYYw9mzpzJiSeeWHRYZla07m7YeWfYdttSJoDhVQVUkIkTJ9JdaegxM4N1DcAHHwxjx5ayDcBXAGZmeXjssaQBePp06OqC558vXW9gJwAzszxUagUqCQBKdxXgBGBmlodKA/Cee65LACVrB3ACMDPLQ3c37LILbLQRdHYm83wFYGY2zFUagKdPTz77CsDMrE1UNwCDE4CZWduobgAG2HxzkJwAzMyGveoGYIBRo2CzzdwGMJzNmjWL3XbbjalTpzJ79uyiwzGzolQ3AFd0dfkKYLi64YYbmDt3LvPmzeNnP/sZV199ddEhmVkRIpJB4CrVPxVOAMPXtddey3HHHceqVau4+OKLOfzww4sOycyK8Nhj8PTTQyIBDKuxgP7tugXc8/jSpu5zl63H8YV3Tut3ve7ubvbZZx8mTJjA5MmTOf/885sah5kNEbUNwBWdnXDrra2Ppw+FXgFIukTS05LmFxlHo9auXUtPTw/HHXccixYtYvr06Zx33nlFh2VmRahtAK7o6koagdeuLSSs3hR9BXApcDHwg2bsLMsv9Tzcd9997LDDDgCMGTOG/fffnyeffLKQWMysYHPmvLIBGJIEsGZNMijc5psXElqtQq8AIuIW4NkiY2iGuXPnsmLFCtasWcOKFSu44oorOOyww4oOy8xarbYHcLUSdgYrfSOwpOMlzZE055kSHbhq8+bNY9myZWy//fbsv//+HHvsseyxxx5Fh2VmrVavARhKOR5Q0VVA/YqI2cBsgBkzZkTB4fRq7ty5XHbZZey6665Fh2JmRarXAAy+Ahiu7r33XnbaaaeiwzCzotVrAIZSJoDSXwEMBY8++mjRIZhZGdRrAIZ1VUAlSgBF3wb6I+CPwI6SeiR9qMh4zMwGra8GYIAxY2Djjd0GUBER7y2yfDOzpumrAbiiZL2Bh0UbQEQp24ZfYajEaWaDUGkAnjGj/jpOAM3V0dHB4sWLS39yjQgWL15MR0dH0aGYWR7mzEkagPu6BbxkCWDINwJPnDiRnp4eytpHoFpHRwcTJ04sOgwzy0NvQ0DX6uyE+eUZ+WbIJ4DRo0czZcqUosMws3ZWaQB+29v6Xq9kVwBDvgrIzKxwlQbgvur/IUkAy5bBSy+1Jq5+OAGYmTWqrx7A1Uo2HIQTgJlZo7I0AEPpegM7AZiZNSpLAzA4AZiZDSv99QCu5gRgZjaMZG0ABrcBmJkNK3PmJK9ZrgA23RRGj/YVgJnZsFAZAjrLQ6Ck5CrACcDMbBjI2gBcUaLOYE4AZmaDVWkAzlL/X9HZ6TYAM7Mhr6en/yGga/kKwMxsGMjaA7iaE4CZ2TAwkAbgiq4ueP55WLUqt7CycgIws+Ht2mvh9a+HBx9s/r67u2HatOwNwLCuL8Dixc2PZ4D6TQCSNpY0In0/VdIhkkbnH5qZWQMi4MtfhsMOg9//Hr74xebvP2sP4Gol6g2c5QrgFqBD0quB3wIfAC7NMygzs4a8/DK8971w+ulw9NHwkY/AZZfBAw80r4zBNADDkEsAioiXgXcDF0XEu4Bd8g3LzGyQHn0UXvc6+OlP4ZxzkhP/rFkwahScfXbzyhlMAzAMvQQgaSZwDPDLdN6Qf5KYmQ1Dt96a3JN///1w3XXw2c8mvW+33ho+/GH4/vfhoYeaU9ZgGoChVOMBZUkAnwJOBa6KiAWStgNuyjcsM7MBuuQSOPBAGDcObrsN3vGO9Zd/7nPJCfvf/7055Q2mARhgwoTktexXAJJGAu+MiEMi4hyAiHggIj7ZkujMzPqzejWceCJ86ENJAvjzn2HnnV+53sSJyTrf+x488khjZUYkg8ANtPoHkqqozTcvfwKIiDXAIP7CbCQdLOk+SfdLOiWvcsysQBGwYkU++3722eRB7F//Onz603D99bDZZvXXPyU9zXzlK42V29OTnMAHkwCgNJ3BstTlz5V0LXAl8PcnGUfEzxspOL26+AbwZqAHuF3StRFxTyP7NbMCrFoFDz+c3GXzt78lr9XvX3wxaZg98kg4/HDYaqvGy1y4EA45JPk1f8kl8IEP9L/NpEnJet/9Lnz+88lVwWAMtgG4oiTjASki+l5B+l4vsyMiPthQwUnD8pkR8db086npjutW0M2YMSPmVMbeHoB/u+Ba7nnyhcGGakNB31/j9anuh4EVGH197mWe0v/8/TUtu7fX2nmQ/JKOgLVre3m/FtZWv9bMC2DkSBg1EkaOWvc6cmRSJdHra7pOJYbVq2H5Mli2HJYtg+VVr8uXr/+njxB0jIExHcnryBGwaHFyeyYk4+J3dSXTBhsM9OAnnagWLkz2O23XpN4/q+XLk2qirbeG17xm4GVD0pD8yMNwwOuSdoWBWrAgOXYDGERul63H8YV3Tht4WYCk7oh4RWH9XgFERIa0OiivBh6t+twDvLZ2JUnHA8cDTJo0aXAl9fSAE8DQFmQ4V2c5mUevbzOtWzlxr1dcH59fMS9NCJUfXbWvA1FJEiNGpK8Cjagzb0Tyfs2apCpmzZrkZL5mTbayKvurXX/0aBgzBjYdB1tuCR0dyecxHbDBhq/cz5TtkgTwzDPwzNPJnTr331+VDDp7367Wo48mVxWbbAK77gobZtimWkcHvGpLeOLx5IpgMAnohRdgo40Hd/KH5NgtXTK4bZuo3wQgaSrwTWDLiNhV0u7AIRHxpQbL7u3/1lf+hoqYDcyG5ApgMAV94WsfG8xmZq1T/Yt+zZrktTKtWZMs22CD5MQxevT6VwaDtXYtvPQSLF36yumFF9b/vGJFcrLcfnvYbjuYMgXGjm2s/IUL4cor4cr/hPnzk3n777+umqi2embZMviXf4ErroCjjoILvjvwO3AqHtgSpk6FzhPgggsGtm0EbHlocpfRRz4xuPJP+wX88By4ZOXgk0gzRESfE3AzsC8wt2re/P62y7DfmcANVZ9PBU7ta5vp06eHmQ1DCxdGnHVWxO67V1JhxMyZEeedF/HIIxE9PREzZkRIEWefHbF2beNlHndcREdHxBNPDGy7Rx5J4rvoosGXfd55yT6efXbw+xgAYE70ck7Nkno2iog/18xb3YTcczuwg6QpkjYAjgKubcJ+zWyo2WmnZNiGO++E++6DL30pqS466aTkymPHHeHee+Gaa+DUU5tzBfT5z8PKlXDuuQPbrtIAPJCHwNQqSW/gLAlgkaTtSatnJB0BPNFowRGxGjgBuAFYCPw0IhY0ul8zG+KmToXTToN585Jk8OUvJ3f73HYbvPOdzStnhx3gmGPgm99MxvTJqrs7aSQfaA/gaiVJAFluA/04SR38TpIeAx4kGRaiYRFxPXB9M/ZlZsPQ1KnJL/W8nHYaXH45fO1r8NWvZttmzpzkGcBjxgy+3JIkgCxXABERBwFdwE4RcUDG7czMym3HHZMG5W98I9vJeLBDQNcqyXhAWU7kPwOIiJcionIv5X/nF5KZWQudfnpyh9F55/W/bqM9gCtKcgVQtwpI0k7ANGBTSe+uWjQO6Mg7MDOzlth5Z3jPe+Dii+Hkk9cN1tabZjQAQ1J9tPHGhSeAvq4AdgT+CRgPvLNq2hv4cO6RmZm1yumnJ8NVnH9+3+vNmdN4A3BFCcYDqnsFEBHXANdImhkRf2xhTGZmrbXrrnDEEXDhhfCZz9QfUK67u/EG4IoSjAeUpQ1gsaTfSpoPIGl3SafnHJeZWWudcUbSA7pez+BmNQBXlOAKIEsC+DZJL91VABFxF0mnLTOz4WP33eFd70qGln7++VcurzQAN1r/XzFEEkBePYHNzMrljDNgyZKkKqhWo0NA1+rsHBIJIJeewGZmpbPXXkmv4/PPTwbBq9bMBmBIrgCWLVs3RHYBsiSAjwP/xbqewCcC/5pnUGZmhZk1K6kCuuii9ec3swEYStEXoN8EEMkzgNfrCRwRD+UemZlZEaZPT4Z6Pu+8pFEY1jUAN6v+H0qRALI8D2A88H5gMjBK6Sh84QfDm9lwNWsWvPa1yRARp5zSvB7A1UowHESWKqDrSU7+dwPdVZOZ2fC0775w8MHwH/+RdBCrPIq2mQlgKFwBAB0RcVLukZiZlcmsWbDffslw0UuWNLcBGIZMArhM0oeBXwArKjMj4tncojIzK9rMmfDmNycPjNl55+Y2AEPyLORRo8rdCAysBM4F/si66p85eQZlZlYKs2YlJ+hbbmluAzAkTzUreDiILAngJOA1ETE5Iqak03Z5B2ZmVrgDDoA3vjF538z6/4qCewNnSQALgOJ6KpiZFemss5LqmkoiaKaCE0CWNoA1wDxJN7F+G4BvAzWz4W+//eC555rzIPpaXV1wxx3N329GWRLA1elkZtae8jj5Q+FtAP0mgIj4fisCMTNrO11dydXFqlUwenTLi++3DUDSDpL+W9I9kh6oTK0IzsxsWKv0BVi8uJDiszQCfw/4JskQ0G8AfgBclmdQZmZtoeDOYFkSwJiI+C2giHg4Is4EcmgONzNrMwWPB5QlASyXNAL4q6QTJL0L2KKRQiUdKWmBpLWSmty7wsxsiBgCVwAnAhsBnwSmA+8Djm2w3PnAu4FbGtyPmdnQVXAC6PMuIEkjgfdExP8HXgQ+0IxCI2Jhuv9m7M7MbGiaMCF5LeMVQESsAaarwDO1pOMlzZE055mCn59pZtZUo0bBZpsV1gaQpSPYXOAaSVcCL1VmRsTP+9pI0m+AV/Wy6LSIuCZrgBExG5gNMGPGjMi6nZnZkFDgcBBZEsDmwGLWv/MngD4TQPoYSTMz60uZE0BENKXe38zMetHVBfffX0jRWZ4J3AF8CJgGdFTmR8QHB1toeivpRSQPmv+lpHkR8dbB7s/MbMjq7ITbbiuk6Cy3gV5GUpf/VuBmYCLwQiOFRsRVETExIjaMiC198jezttXVlTQCR+ubOLMkgNdExBnAS+nAcO8Adss3LDOzNtHVBatXw/PPt7zoLAlgVfr6vKRdgU2ByblFZGbWTgrsDJYlAcyWtBlwBnAtcA9wTq5RmZm1iwLHA8pyF9B30rc3A34WsJlZM5X5CkDSBEkXSbpDUrekCyRNaEVwZmbDXpkTAPBj4GngcOAIYBHwkzyDMjNrG5UqoAISQKaewBFxVtXnL0k6LKd4zMzay0YbJVMBbQBZrgBuknSUpBHp9B7gl3kHZmbWNgoaDiJLAvgIcAWwIp1+DJwk6QVJS/MMzsysLRSUALLcBTS2FYGYmbWtzs7SXgGYmVmeKsNBtJgTgJlZ0UrcBmBmZnnq6oKXX06mFqrbBiBp8742jIhnmx+OmVkbqu4LsO22LSu2r0bgbpInfwmYBDyXvh8PPAJMyTs4M7O2UOkNvGhRSxNA3SqgiJgSEdsBNwDvjIjOiJgA/BP9PA7SzMwGoKDhILK0AewTEddXPkTEr4B/zC8kM7M2U1ACyDIUxCJJpwM/JKkSeh/JQ+LNzKwZChoPKMsVwHtJnt17VTp1pfPMzKwZxo+HUaNa3hcgS0/gZ4FPSdokIl5sQUxmZu1FKqQ3cJbnAewn6R6SJ4EhaQ9J/5l7ZGZm7aSAzmBZqoDOB95KWu8fEXcCr88zKDOzttPZ2fIqoEw9gSPi0ZpZa3KIxcysfRVwBZDlLqBHJe0HhKQNgE8CC/MNy8yszZS0CuijwMeBVwM9wJ7AxxopVNK5ku6VdJekqySNb2R/ZmZDXlcXPPccrFrVsiKzJIAdI+KYiNgyIraIiPcBOzdY7q+BXSNid+AvwKkN7s/MbGir9AV4tnXDrGVJABdlnJdZRNwYEavTj7cBExvZn5nZkFdAb+C+RgOdCewHdEk6qWrROGBkE2P4IPCTPuI4HjgeYNKkSU0s1sysRMqUAIANgE3SdaofC7kUOKK/HUv6DfCqXhadFhHXpOucBqwGLq+3n4iYDcwGmDFjRvRXrpnZkFSmBBARNwM3S7o0Ih4e6I4j4qC+lks6lmRk0TdFhE/sZtbeKm0ALewLkOU20JclnQtMAzoqMyPijYMtVNLBwOeAf4yI1j4Cx8ysjCZMSF5beAWQpRH4cuBekgfA/BvwEHB7g+VeTFKt9GtJ8yR9q8H9mZkNbaNHw2ablaMKqMqEiPiupE9VVQvd3EihEfGaRrY3MxuWWtwZLEsCqPRKeELSO4DH8W2bZmbN1+LxgLIkgC9J2hT4DMn9/+OAT+calZlZO+rqggceaFlxWZ4H8Iv07RLgDfmGY2bWxrq64E9/allx/SYASVOATwCTq9ePiEPyC8vMrA11dSVVQBHJQ2JylqUK6Grgu8B1wNpcozEza2ednbB6NSxZkjwmMmdZEsDyiLgw90jMzNpddW/gkiSAr0v6AnAjsKIyMyLuyC0qM7N2VJ0Adtgh9+KyJIDdgH8G3si6KqBIP5uZWbNUhoNoUV+ALAngXcB2EbEy72DMzNpa5QqgRX0BsgwFcScwPuc4zMysxSOCZrkC2BK4V9LtrN8G4NtAzcyaaaONkqlECeALuUdhZmaJzs7yJIB0ADgzM2uFSmewFujrkZB/iIgDJL1ActfP3xcBERHjco/OzKzdtHBE0L6eCHZA+jq23jpmZtZkXV2wcGFLiur3LiBJl2WZZ2ZmTdDCNoAst4FOq/4gaRQwPZ9wzMzaXFcXvPxyMuWsbgKQdGpa/7+7pKWSXkg/PwVck3tkZmbtqIWdweomgIj497T+/9yIGBcRY9NpQkScmntkZmbtqIWdwbJUAZ0m6X2SzgCQtI2kfXOOy8ysPbVwPKAsCeAbwEzg6PTzi+k8MzNrthZWAWXpCfzaiNhb0lyAiHhO0gY5x2Vm1p5KVgW0StJI0s5gkrrwk8HMzPIxfjyMHFmaBHAhcBWwhaQvA38Azs41KjOzdiW1rC9AlrGALpfUDbyJZBiIwyKioW5qks4CDiW5kngaOC4iHm9kn2Zmw0aLxgPK0gZARNwL3NvEcs+NiMpdRZ8EZgEfbeL+zcyGrhaNB5SlCqjpImJp1ceNWX+wOTOz9taiBJDpCiAPaXvC+4ElwBv6WO944HiASZMmtSY4M7MitagNILcrAEm/kTS/l+lQgIg4LSK2AS4HTqi3n4iYHREzImJGV+X2KDOz4ayrC557DlavzrWY3K4AIuKgjKteAfwSP3nMzCxR+bG7eDFsuWVuxRTSBiBph6qPh9DcBmYzs6GtRZ3BimoD+IqkHUluA30Y3wFkZrZOZTygnG8FLSQBRMThRZRrZjYktOgKoJAqIDMz64MTgJlZm5owIXl1AjAzazOjRyeDwuXcBuAEYGZWRi3oDewEYGZWRk4AZmZtqgXDQTgBmJmVUQuGhHYCMDMro0oCiPwGS3YCMDMro64uWLUKlizJrQgnADOzMqoMB5FjO4ATgJlZGVV6A+fYDuAEYGZWRi0YDsIJwMysjJwAzMzalNsAzMza1MYbw5gxbgMwM2tLOQ8H4QRgZlZWTgBmZm0q5/GAnADMzMoq5/GAnADMzMrKVUBmZm2qqwteegmWLctl904AZmZllXNfACcAM7Oyynk8ICcAM7Oyynk4iEITgKSTJYWkziLjMDMrpeGaACRtA7wZeKSoGMzMSm0YtwGcD3wWyO95Z2ZmQ9n48TBy5PBqA5B0CPBYRNyZYd3jJc2RNOeZHO+HNTMrnREjcu0NPCqXvQKSfgO8qpdFpwGfB96SZT8RMRuYDTBjxgxfLZhZe8mxM1huCSAiDuptvqTdgCnAnZIAJgJ3SNo3Ip7MKx4zsyFpKF4B1BMRdwNbVD5LegiYERH5DXhhZjZUdXXBXXflsmv3AzAzK7OhWAWUVURMLjoGM7PS6uqCZ5+F1athVHNP2b4CMDMrs0pfgMWLm75rJwAzszLLcTwgJwAzszLLcTgIJwAzszLbdls44ggYN67puy68EdjMzPqw/fZw5ZW57NpXAGZmbcoJwMysTTkBmJm1KScAM7M25QRgZtamnADMzNqUE4CZWZtyAjAza1OKGDoP2ZL0DPDwIDfvBMr8zAHH1xjH1xjH17gyx7htRHTVzhxSCaARkuZExIyi46jH8TXG8TXG8TVuKMRYy1VAZmZtygnAzKxNtVMCmF10AP1wfI1xfI1xfI0bCjGup23aAMzMbH3tdAVgZmZVnADMzNrUsEsAkg6WdJ+k+yWd0stySbowXX6XpL1bGNs2km6StFDSAkmf6mWdAyUtkTQvnWa1Kr60/Ick3Z2WPaeX5UUevx2rjss8SUslnVizTkuPn6RLJD0taX7VvM0l/VrSX9PXzeps2+d3Ncf4zpV0b/rvd5Wk8XW27fO7kGN8Z0p6rOrf8O11ti3q+P2kKraHJM2rs23ux69hETFsJmAk8DdgO2AD4E5gl5p13g78ChDwD8CfWhjfVsDe6fuxwF96ie9A4BcFHsOHgM4+lhd2/Hr5t36SpINLYccPeD2wNzC/at5XgVPS96cA59SJv8/vao7xvQUYlb4/p7f4snwXcozvTODkDP/+hRy/muX/Acwq6vg1Og23K4B9gfsj4oGIWAn8GDi0Zp1DgR9E4jZgvKStWhFcRDwREXek718AFgKvbkXZTVTY8avxJuBvETHYnuFNERG3AM/WzD4U+H76/vvAYb1smuW7mkt8EXFjRKxOP94GTGx2uVnVOX5ZFHb8KiQJeA/wo2aX2yrDLQG8Gni06nMPrzzBZlknd5ImA3sBf+pl8UxJd0r6laRprY2MAG6U1C3p+F6Wl+L4AUdR/3+8Io8fwJYR8QQkSR/Yopd1ynIcP0hyRdeb/r4LeTohraK6pE4VWhmO3+uApyLir3WWF3n8MhluCUC9zKu9zzXLOrmStAnwM+DEiFhas/gOkmqNPYCLgKtbGRuwf0TsDbwN+Lik19csL8Px2wA4BOjtSdlFH7+synAcTwNWA5fXWaW/70JevglsD+wJPEFSzVKr8OMHvJe+f/0XdfwyG24JoAfYpurzRODxQayTG0mjSU7+l0fEz2uXR8TSiHgxfX89MFpSZ6vii4jH09engatILrWrFXr8Um8D7oiIp2oXFH38Uk9VqsXS16d7Wafo7+GxwD8Bx0RaYV0rw3chFxHxVESsiYi1wLfrlFv08RsFvBv4Sb11ijp+AzHcEsDtwA6SpqS/Eo8Crq1Z51rg/endLP8ALKlcructrTP8LrAwIs6rs86r0vWQtC/Jv9HiFsW3saSxlfckjYXza1Yr7PhVqfvLq8jjV+Va4Nj0/bHANb2sk+W7mgtJBwOfAw6JiJfrrJPlu5BXfNVtSu+qU25hxy91EHBvRPT0trDI4zcgRbdCN3siuUvlLyR3CJyWzvso8NH0vYBvpMvvBma0MLYDSC5T7wLmpdPba+I7AVhAclfDbcB+LYxvu7TcO9MYSnX80vI3Ijmhb1o1r7DjR5KIngBWkfwq/RAwAfgt8Nf0dfN03a2B6/v6rrYovvtJ6s8r38Fv1cZX77vQovguS79bd5Gc1Lcq0/FL519a+c5Vrdvy49fo5KEgzMza1HCrAjIzs4ycAMzM2pQTgJlZm3ICMDNrU04AZmZtygnA2oKk/5WU+wO7JX1SyWiv9XrX5lXumZJOzrBe9QiV8yRd2Ir4rJxGFR2AWdlJGhXrBk/rz8eAt0XEg3nG1KA3RMSiooOw4vkKwEpD0uT01/O3lTwv4UZJY9Jlf/8FL6lT0kPp++MkXS3pOkkPSjpB0kmS5kq6TdLmVUW8T9KtkuanvYQrPTYvkXR7us2hVfu9UtJ1wI29xHpSup/5Sp9JIOlbJB2ArpX06Zr1RyoZh//2dJCzj6TzD5R0i5Jx+e+R9C1JI9Jl701/rc+XdE7Vvg6WdIeSAe9+W1XMLulxekDSJxv6x7D2UHRPNE+eKhMwmWRwsj3Tzz8F3pe+/1/SXsdAJ/BQ+v44kp6tY4EuYAnregWfTzLgXmX7b6fvX086vjtwdlUZ40l6lm6c7reHtBdvTZzTSXqqbgxsQtLTc6902UP0MgY8cDxwevp+Q2AOMIXk+QXLSRLHSODXwBEkvUofSf+mUcDvSIaV7iLpxTsl3Vell/GZwK3pvjtJekuP7iWOh9LY56XTp4v+d/dU3OQqICubByNiXvq+myQp9OemSJ6v8IKkJcB16fy7gd2r1vsRJGO8Sxqn5ElYbwEOqao/7wAmpe9/HRG9jQV/AHBVRLwEIOnnJEMDz+0jxrcAu0s6Iv28KbADsBL4c0Q8kO7rR+n+VwH/GxHPpPMvJ0lca4BbIq1iqonvlxGxAlgh6WlgS5IkVstVQAa4DcDKZ0XV+zXAmPT9atZVWXb0sc3aqs9rWf87XjvuSZCMbXR4RNxXvUDSa4GX6sTY21DE/RHwiYi4oaacA/uIq95+6o3fUnvs/P+39cltADZUPERS9QJJFclg/D8ASQeQjGK6BLgB+ETVCKJ7ZdjPLcBhkjZKR3p8F/D7fra5AfhXJcOBI2lqui3AvumoliPSGP9A8qCgf0zbO0aSjIB6M/DHdP6UdD+b1xZklpV/IdhQ8TXgp5L+maQ+fDCek3QrMI7kSVgAZwEXAHelSeAhknHy64qIOyRdCvw5nfWdiOir+gfgOyTVWXek5TzDukdF/hH4CrAbSXK5KiLWSjoVuInkV//1EXENgJKnS/08TRhPA2/u7w+vcZOkNen7uyLi/QPc3oYJjwZqVqC0CujkiOgz6ZjlwVVAZmZtylcAZmZtylcAZmZtygnAzKxNOQGYmbUpJwAzszblBGBm1qb+D4NDuyL8wv4yAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "a,b,a_r,b_r = gradient_descent(x,y,C=0.05,E=20)\n",
    "plt.plot(a_r,color='r',label=r'$\\alpha $')\n",
    "plt.plot(b_r,label=r'$\\beta$')\n",
    "plt.xlabel(\"number of epoch E\")\n",
    "plt.ylabel(\"etimated parameters\")\n",
    "plt.title('C = 0.05 , E = 50')\n",
    "#set_spines()\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61c27fa6",
   "metadata": {},
   "source": [
    "When learning rate is 0.05 and Iter.=20 we observe the learning jump."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "bf3e8eb6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0xd7c5c978b0>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6VklEQVR4nO3dd3hU1dbA4d+iht57C70XBUTEBnIVCyBgVxRFwd6liYKgInbFCoqiYqWDgoAiqB+K1NBBeqihh5K+vj/2cI25SZiUqVnv88yTycwpK4dhZWefvdcWVcUYY0z4yRfoAIwxxviGJXhjjAlTluCNMSZMWYI3xpgwZQneGGPClCV4Y4wJU5bgjTEmTFmCN34jIreIyFIROSEie0VktohcmMvnKCwi40XkuIjsE5HHM9m2iojMEJE9IqIiEpmD814qIimeny31o312j5nBebaLyOlUx5+b5v1bRGSHiJwUkWkiUjY3z29CiyV44xeeRPsm8CJQCagJvAd0z+VTDQfqA7WAjsAAEemSwbYpwBygVy6de4+qFk/zWJxLx06ta6rjX37mRRFpCnwI9MZd41O4a2zyKLGZrMbXRKQUsBu4U1W/8/G5zpxnruf7kUB9Vb0pk30KAIlAbVXdns3zXgp8oarVs7N/Fs6zHbhbVeen896LQKSq3uL5vi6wHiinqrG+jMsEJ2vBG39oD0QAU73dQUQGicjRjB4Z7FMGqAqsSvXyKqBpDmL3CRGZlcnPN+ssu08UkRgRmSsiLVO93pRUP7uqbgESgAY++BFMCCgQ6ABMnlAOOKiqSd7uoKovAS9l8TzFPV+PpXrtGFAii8fJrqrp/PKppqon026oqtdk8xy3AssBAR4BfhSRRqp6FPfzH0uzvT9/fhNkrAVv/OEQUN7TFeJLJzxfS6Z6rSTgr+6JPapaOs3jf5J7Tqjq76p6WlVPqeoo4ChwkeftE/z7Zwf//vwmyFiCN/6wGIgDrvV2BxEZks6IlP8+0ttHVY8Ae4HU3RYtgbU5iN0nPCOIMvr5ZmfhUIprzYP7Of/7s4tIHaAwsCn3IjehxLpojM+p6jEReRZ4V0SSgLm4m5qdgY6qOiCdfV7EjbjJqs+AoSKyFDeS5B7gzow2FpEIIL/n28IiEqGqcRls+6kntj7ZiOtfVPXKrO4jIjWBGsBfuMbZQ0B54HfPJhOBxSJyEa4bZwQwxW6w5l3Wgjd+oaqvA48DQ4EYYBfwIDAtl081DNgC7AAWAq+o6pwzb3payBel2v40/3TtbPB8n5Ea/JNM01M1ndZ4bg3BBNeX/j5wBDcqqQtwpaoeAlDVtcC9uER/wLP9/bl4fhNibJikMV4QkUK4ESotVDUx0PEY4w1L8MYYE6asi8YYY8KUJXhjjAlTluCNMSZMBdUwyfLly2tkZGSgwzDGmJCxbNmyg6paIb33girBR0ZGsnTp0kCHYYwxIUNEdmT0nnXRGGNMmLIEb4wxYcoSvDHGhKmg6oNPT2JiItHR0cTFpVseJKhERERQvXp1ChYsGOhQjDEm+BN8dHQ0JUqUIDIyEhE5+w4BoqocOnSI6OhoateuHehwjDEm+Lto4uLiKFeuXFAndwARoVy5ciHxl4YxJm8I+gQPBH1yPyNU4jTG5A0hkeCNMSZcLdl2mA8WbvHJsYO+D94YY8LR8ZNxvPTZr3y5I4GaSSe4vX0tihbK3ZRsCd4YY/wlNhbmzmXO3OU8W6gxB4uU5O7ls3i8QDRF8+Xm2jCOJXgvrV27lkceeYSdO3fSu3dvDhw4wO23307btm0DHZoxJpht3w4zZ8KsWez/axXPXtKXHxteQJOEI3zUOJ4WQ16G0qV9curQSvCPPgorV+buMVu1gjffzHSTuLg4rr/+er777jvq1KlDo0aNaN26tSV3Y8z/Sk6GJUtcUp85E9asIQXhyyvuYPQ995OQvwADO9fn7kvqUTC/b2+DhlaCD5D58+dzzjnn0LRpUwASEhJ44oknAhyVMSZonDgBc+e6hP799xATA/nzw8UX8/foMQwu2IS/9p/mgrrleLFHcyLLF/NLWKGV4M/S0vaVFStWcO655wKwZ88eihcvTocOHQISizEmSMTEwIwZMHky/PQTJCRAmTJw5ZXQtSsJ/7mc91cc4t0Ff1OkUBIvX9eC61tX9+tw6tBK8AFSuHBhoqOjARg8eDAJCQkBjsgYExC7d8O0aS6pL1wIKSlQuzY8+CB06wYdOkCBAizbcYTBn0Wxaf8JurasyrPXNKFCicJ+D9cSvBduueUWunfvTsOGDenfvz/x8fE8+uijvBmgvyiMMX60bRtMmeKS+uLF7rXGjWHIEOjVC1q2BE+rPDYukVe+X8Pnf+ygSskIxvdpQ6dGlQIWuiV4L1SvXp1ly5YFOgxjjL+sX/9PUl+xwr12zjnw/PPQs6dL8GnMW7efZ6atYX9sHHe0j+TJKxpSvHBgU6wleGOMUXUj9CZPdol9/Xr3evv28OqrLqlnUETwQGwcz81Yx/er99KwUgneu+1czq1Zxn+xZ8ISvDEmb1KFVavgq6/gu+9cV0y+fHDJJfDAA9CjB1StmsnuyrdLd/HC9+uJS0rhycsb0O/iuhQqEDwVYCzBG2Pylk2bXFL/+mvYsAEKFIDOneHpp6F7dyhf/qyH2HbwJEOmrGbx1kOcV7sso3o2p26F4n4IPmsswRtjwt+uXfDNNy6xL1/ubopecombPNmrl1dJHSAxOYVxv27lrfmbKVQgH6N6NufGNjXIly84K8lagjfGhKeYGNf18vXX8Ouv7rW2beH11+GGG6BatSwdbtWuowyaspr1e4/TpWllnuvelEolI3wQeO6xBG+MCR/Hj8PUqa6lPn++KxvQpAmMHAk33QT16mX5kKcSknh97ibG/76N8sUL88FtrenSrLIPgs99luCNMaHt9GlXHuCrr9zX+HiIjIQBA+Dmm6FZs/+OU8+qhZtieHrqaqKPnOaWdjUZ2KURpYqEzprLluCNMaFH1U06+vRT17d+/DhUrgz9+7uk3q5dtpM6wOGTCTw/ax1TVuymToVifNu/PefVLpt78fuJJfgsePbZZ5k6dSrx8fE8+eST9OvXL9AhGZO37NwJn38OEybA5s1QtChcdx3cfjtceqkr8JUDqsr0lXsYMWsdx08n8lCnejzQsR4RBXN23ECxBO+lH3/8kRUrVrBy5UrWrVvHwIEDLcEb4w8nT7rJR59+CgsWuNb7pZf+UyqgRIlcOc2uw6d4etoaFm2KoVWN0rzUqzmNKpfMlWMHiiV4L82YMYM+ffqQmJjIO++8Q69eub/6ijHGIyXFjXyZMMGNhDlxAurUgeHDoXfvDGeVZkdyivLJ79t4be4mRGBY1ybc3j6S/EE69DErQirBPzdzLev2HM/VYzapWpJhXZuedbtly5bRtm1bypUrR2RkJG+88UauxmGMAbZuhc8+c4l9+3YoXtwNaezTBy68MEf96ulZv/c4gyZHsSr6GB0bVuD5Hs2pVrpIrp4jkIJnTm0QS0lJITo6mj59+nDw4EFat27N66+/HuiwjAkPsbEwfrybeFS3LowY4YYzfv457NsHH38MF12Uq8k9LjGZV37cQNcxvxF95DRv3dSK8X3ahlVyhxBrwXvT0vaFjRs3Ur9+fQCKFClChw4d2LdvX0BiMSZsLFsGH34IX37p+tnr13fVGnv3hpo1fXbaP7YeYsiU1Ww9eJJe51Zn6NWNKVOskM/OF0ghleADZcWKFcTHx5OcnExSUhJffvklb7/9dqDDMib0xMa68eoffuhKBhQpAjfeCHffDRdckOtdMKkdO53IS7M38NWSndQoW4TP+57HRfUr+Ox8wcCnCV5EtgOxQDKQpKptfHk+X1m5ciWnT5+mbt26lC9fnvvvv5+WLVsGOixjQsfy5f+01k+ccJOPxoyB226D0qV9fvo5a/bx7PQ1HDwRzz0X1eax/zSgaKHwb9/64yfsqKoH/XAen1mxYgWff/45zZo1C3QoxoSOEydca33sWFi6FCIiXGu9f384/3yfttbP2H88jmHT1zJn7T4aVynJR3e0oUX10j4/b7AI/19huWDDhg00atQo0GEYExpWrHCt9YkTXZJv2hTeftu11sv4ZyGMlBTl6792MWr2ehKSUhjYpRF3X1Sbgvnz1rgSXyd4BeaKiAIfqurYtBuISD+gH0BNH95YyYldu3YFOgRjgtuJE65q44cf/ru13q+fWxXJD631M7bGnGDwlNX8ue0w59cpy6ieLahdvpjfzh9MfJ3gO6jqHhGpCMwTkQ2quij1Bp6kPxagTZs26uN4jDG5acMGeOcdN3Y9NjYgrfUzEpNTGLtoK2/9tJnCBfIxuldzbmhTA/HjL5dg49MEr6p7PF8PiMhU4DxgUeZ7GWOCWkoKzJnjEvmPP0KhQq61fu+9fm+tn7Fq11EGTo5iw75YrmpemeFdm1IxyGu1+4PPEryIFAPyqWqs5/nlwIjsHEtVQ+K3sKr9AWLC2PHjbobpmDGu0FeVKq7Oer9+ULFiQEI6lZDEa3M38cnv26hQojAf9m7NFU1Do1a7P/iyBV8JmOpJzAWAL1V1TlYPEhERwaFDhyhXrlxQJ3lV5dChQ0REWKvBhJnNm103zCefuG6Y88+H555zhb4KBW6C0KJNMQxJVat90JWNKBkROrXa/cFnCV5VtwI5HixevXp1oqOjiYmJyYWofCsiIoLq1asHOgxjck4V5s1z3TA//OAWpr7xRnjoITjvvICGduRkAiO/X8eU5bupUz50a7X7Q9APkyxYsCC1c7FynDEmEydOuBumY8a4G6iVKsGzz7qx61WqBDQ0VWXGqj2MmLmOY6cTebBjPR7sFLq12v0h6BO8McYPtm513TDjx8OxY9CmjSv2df31ULhwoKNj99HTPDNtDT9vOEDL6qX44u52NK4S2rXa/cESvDF52eLF8PLLMH26Ww3puuvg4Yf9NtP0bFJSlM//2MHLczaQojD06sbc2aF2WNRq9wdL8MbkNSkprl/95ZfdohplysDgwXD//VCtWqCj+6/N+2MZNGU1y3Yc4aL65XmxR3NqlC0a6LBCiiV4Y/KKhARXG+aVV2DtWqhRA958E/r2dQtrBImEpBTe/2UL7y74m6KF8/P6DS3pcU61oB5FF6wswRsT7mJjYdw4eOMNiI6G5s1d//qNN0LB4BpWuGzHEQZPiWLT/hN0a1mVZ7s2oXzxwN8DCFWW4I0JV/v3u2GO770HR4+6harHjoUuXYKifz21E/FJvPrjRiYs3k6VkhGM79OGTo0qBTqskGcJ3phw8/ff8Oqr8OmnrlumZ0946ilo1y7QkaVrwcYDDJ26hj3HTnP7+bV4qksjihe21JQb7CoaEy6WLoXRo2HyZNf10qcPPPEENGgQ6MjSdehEPCNmrWP6yj3Uq1icSfe2p3Utm7CUmyzBGxPKVGH+fBg1ChYsgFKlYNAgN9SxcnDWZFFVpq3czYiZ6zgRn8Qjl9Xn/o51KVzAJizlNkvwxoQiVZg7F4YPhz/+cMMbX30V7rkHSgbvBKDoI6cYOm0Nv2yM4ZyapRndqwUNKpUIdFhhyxK8MaFE1ZXqfe45+PNPqFkTPvjAdccEwYzTjCSnKJ8t3s4rP24EYFjXJtzePtImLPmYJXhjQoEqzJ7tEvuSJVCrlls9qU+fgFZ09Mam/bEMnBzFip1HuaRBBV7o0YzqZWzCkj9YgjcmmKnC99+7xL50KURGujHtt98e9Ik9PimZ9xZs4b1f/qZ44QK8eWMrureqahOW/MgSvDHBSBVmzXKJfdkyqF0bPvrIJfYgm5yUnmU7jjBochSbD5zg2lZVeeaaJpSzCUt+d9YE71mN6bSqpohIA6ARMFtVE30enTF5jSrMmAEjRsDy5VCnjqvweNttIZHY005Y+qRPWzo2CsxqT8a7Fvwi4CIRKQP8BCwFbgRu9WVgxuQpqq6i43PPwcqVULeuW0Hp1ltDIrHDvycs3dE+kievaGgTlgLMm6svqnpKRPoCY1T1ZRFZ4evAjMkTziT24cNh1SqoV8+te3rLLW4VpRBw+GQCI2auZZpNWAo6XiV4EWmPa7H3zcJ+xpjMLFzoJiX98QfUr+9WUrr55pBJ7KrK9JV7GDFrHbFxiTZhKQh580l6BBgMTFXVtSJSB1jg27CMCWOrVrn667NnuwlK48a54Y4hktjBrbA0dOpqFmyMoVUNN2GpYWWbsBRsMv1EiUh+oKuqdjvzmmcx7Yd9HZgxYWfrVre+6ZdfQunSbsGNBx+EIkUCHZnX0q6w9Mw1TehzgU1YClaZJnhVTRaR1v4KxpiwtH8/PP+8m5hUoAAMHOgepUsHOrIs+ftALAMn2wpLocSbvwlXiMgM4Dvg5JkXVXWKz6IyJhwcP+7qw7z+OsTFwd13uxZ81aqBjixLEpJS+GDhFt752a2w9Nr1Lel5rq2wFAq8SfBlgUNAp1SvKWAJ3pj0xMfD++/DCy/AwYNwww0wcmTQlu3NzMpdRxk4KYqN+2O5pkUVhnVtSoUSNmEpVJw1wavqnf4IxJiQl5wMX3zhWuk7d0Lnzq6Mb5s2gY4sy04lJPHa3E188vs2KpaI4KPb29C5ia2wFGq8mcnaAHgfqKSqzUSkBdBNVZ/3eXTGhIIzZQWGDIE1a6B1a/j4Y5fgQ9Cvm2MYMnU1uw6f5tZ2NRl4ZSNKRoTGZCvzb/m82GYcbphkIoCqRgE3+TIoY0LG8uVwySXQrZvrmvn2W1ftMQST+9FTCTzx7Sp6f7yEgvny8U2/83mhR3NL7iHMmz74oqq6JM0NlSQfxWNMaNi3z7XYP/0Uypd3fe59+4ZMWYHUVJUfVu9j2Iw1HDmVyAMd6/JQp/pEFLQJS6HOmwR/UETq4m6sIiLXAXt9GpUxwSouDt58091AjY93a54OHeqWygtB+47F8cz0Ncxbt59m1Uoy4a7zaFo1NH8W87+8SfAPAGOBRiKyG9iGFRozeY0qTJ0KTz4J27a5LplXX3UlBkJQSory9V+7GPXDehKSUxh8ZSP6XlibAvm96bU1ocKbBK+q2tlTNjifqsaKSG1fB2ZM0Fi1Ch59FH75BZo2hXnzQrKP/YxtB08yaHIUf247TPs65RjVszmR5YsFOizjA94k+MnAuap6MtVrkwCb4WrC24ED8MwzbqGNMmXg3XehX7+QqhmTWlJyCh/9to035m2iUIF8vNSzOTe2rWETlsJYhp9UEWkENAVKiUjPVG+VBCJ8HZgxAZOQAGPGuEU3Tp2Chx92Y9vLlAl0ZNm2ds8xBk6OYs3u41zepBIjr21GpZL23zjcZdYUaQhcA5QGuqZ6PRa4x4cxGRMYZ8azP/EEbN4MV10Fr70GjRoFOrJsi0tM5q2fNjN20VbKFC3Ee7eey5XNKlurPY/IMMGr6nRguoi0V9XFfozJGP9buxYee8z1rzdq5Er5dukS6Khy5M+thxg8ZTVbD57k+tbVefrqxpQuGtwLdZvc5U1n4iER+QmbyWrC0bFjrp/9vfegRAl46y24776QHM9+RmxcIi/N3sDEP3dSo2wRvujbjgvrlw90WCYAvEnw44CngA/BzWQVkS8BrxK8p6b8UmC3ql6T3UCNyVWqbtbpo4+6cr733ef63MuVC3RkOTJ/3X6GTlvDgdg4+l5Ymycub0DRQqF5U9jknD9msj4CrMfdnDUm8P7+Gx54AObOdXVjZs4MyYJgqR08Ec9zM9cxc9UeGlYqwQe9W9OqRulAh2UCzKczWUWkOnA18ALweHaDNCZXxMe7VZReeAEKFXIjZe67D/KH7pR8VWXqit2MmLWOk/FJPNa5AfddWpdCBWzCksn+TNbbvDz+m8AAIMPFGkWkH9APoGbNml4e1pgs+vlnl8w3bYIbb3SLcITYwhtpRR85xdNT17BwUwzn1nTrotavZOuimn94Uw9+K/CvmazeHFhErgEOqOoyEbk0k+OPxf0CoU2bNurNsY3x2v79rrzAF19A3bowZw5ccUWgo8qRM+uijp6zAYDhXZvQu72ti2r+lzf14EsDtwORQIEzffGqeraFtzsA3UTkKtzEqJIi8oWqetv6Nyb7UlJg7FgYPBhOnnQjZQYPDqkFrtPz94ETDJocxVJbF9V4wZsumh+AP4DVQIq3B1bVwbg68nha8E9acjd+sXIl3Hsv/PkndOzohkCG8GQlgMTkFD5cuIW3f/qbIoVsXVTjHW8SfISq2g1SE/xiY2HYMDeWvVw5+PxzuPVWCPEkGBV9lAGTotiwL5arW1RhuK2LarzkTYL/XETuAWYB8WdeVNXD3p5EVX8BfslqcMZ45Uwp34cfht27oX9/txZqCNeOATidkMyb8zcx7tetVChRmLG9W3N508qBDsuEEG8SfALwCvA0nqGSnq91fBWUMV7bvduNjpk5E1q0gO++g/btAx1Vji3ecojBU6LYfugUN59Xg0FXNqZUkdCdXWsCw5sE/zhQT1UP+joYY7ymChMmuJmoCQlu8Y1HHgnZUr5nHI9LZNQPG/hqyU5qlSvKl/e044K6VmbAZI83/xvWAqd8HYgxXouOdnXZZ8+Giy6C8eOhXr1AR5Vj89btZ+i01cTExtP/4jo82rkBRQqF7iQsE3jeJPhkYKWILODfffBnGyZpTO5Sdcn88cchKQneftuVHMgX2rM2D56IZ/iMtcyK2kujyiUYd3sbWlQvHeiwTBjwJsFP8zyMCZydO+Gee1z9mEsugY8/dhOXQljqMgOn4pN58vIG9L+kLgVtXVSTS7yZyTrBH4EYky5VGDfOzUZNSXHL5t17b8i32ncfPc2QKatZuCmG1rXKMLpXc+pVtDIDJnd5M5O1PjAKaEKqpfpU1UbRGN/avt212ufPh06d3NqotUN7vfeUFOWLP3cwevYGFHiuW1N6n1+LfFZmwPiAN100nwDDgDeAjsCdgH0aje+kpMCHH8KAAe77Dz5wN1VDfMLSlhhXZuCv7Ue4uEEFXuzRjOplrMyA8R1vEnwRVf1JRERVdwDDReRXXNI3Jndt2wZ9+8KCBdC5s2u116oV6KhyJDE5hbGLtvLWT5spUtDKDBj/8SbBx4lIPmCziDwI7AYq+jYsk+ekpLiaMYMGuf71ceNcog/xJLhm9zEGTIpi3d7jXN28CsO7WZkB4z/eJPhHgaLAw8BIXDfNHT6MyeQ1W7bAXXfBokWulO/YsRDiawPEJSbz1k+bGbtoK2WLFeKD21rTpZmVGTD+lWmC96yneoOqPgWcwPW/G5M7VOGTT1wNmQIF3NDHO+8M+Vb7km2HGTQ5iq0HT3JjmxoMuaoxpYpamQHjf5kmeFVNFpHWnv53W4zD5J7Dh92N08mTXUnfCROgRo1AR5UjsXGJjJ6zgS/+2EmNskWYeHc7OtSzMgMmcLzpolkBTBeR74CTZ15U1Sk+i8qEtwULoHdvt9rS6NHwxBMhvS4qwIINBxgydTX7jsfR98LaPHF5A4oWCu26OCb0efMJLAscAjqlek0BS/AmaxIS3MpKr7wC9evDH39A69aBjipHDp9MYMTMtUxbuYf6FYsz+b4LOLdmaJcpNuHDm5ms1u9ucm7jRrjlFli+3HXNvP46FCsW6KiyTVWZGbWX4TPWEhuXyMOX1eeBjnUpXCC0/xIx4cWbmawRQF+gKf+eyXqXD+My4ULVjWV/9FGIiIApU6BHj0BHlSP7jsUxdNoa5q/fT8vqpRh9XTsaVS4Z6LCM+R9eregEbACuAEYAtwLrfRmUCRMHD7pSA9OmuUlLEyZA1aqBjirbVJWv/9rFi9+vJzElhaevasxdF9Ymv5UZMEHKmwRfT1WvF5HuqjpBRL4EfvR1YCbEzZsHd9zhkvyrr8Jjj4V0gbAdh04yaPJqFm89xPl1yvJSzxZElg/dLiaTN3iT4BM9X4+KSDNgHxDps4hMaIuPhyFDXB9748bwww/QqlWgo8q25BTlk9+38ercjRTMl49RPZtzU9saVmbAhARvEvxYESkDPAPMAIp7nhvzb+vWuRupq1a5dVJffRWKhm4xrY37YhkwOYpVu47SuXFFnr+2OZVLRZx9R2OChDejaD7yPF2ILbRt0qMK77/vxrMXLw4zZkDXroGOKtsSklJ475e/eXfB35SIKMjbN59D1xZVrNVuQo43o2jKAcOBDrjx778CI1X1kG9DMyHhyBHo08cl9SuugE8/hcqhW3Nl5a6jDJwUxcb9sVzbqirPdm1K2WKFAh2WMdniTRfN18AioJfn+1uBb4DOvgrKhIilS+H662H3btfn/sgjIXsj9XRCMq/N3cj437dRqWQE4/u0oVOjSoEOy5gc8Womq6qOTPX98yJyrY/iMaFA1S3I8cgjUKmSqwJ5/vmBjirb/m/LQQZNXs3Ow6e4tV1NBl3ZiBIRVhzMhD5vEvwCEbkJ+Nbz/XXA974LyQS1kyehf3+YONF1yXzxBZQPzYJax+MSGfXDer5asovIckX5ut/5nF+nXKDDMibXeJPg+wOP4yY8AeQHTorI44Cqqk3hyyvWr4frrnNfR4yAp58O2S6Z+ev28/S01cTExtP/4jo82rkBRQpZmQETXrwZRWNLvRv4+mu4+2437HHuXDczNQQdOhHPczPXMWPVHhpVLsHY3m1oWaN0oMMyxiesnqnJXHy8G/747rvQoQN88w1UqxboqLJMVZmxag/PzVxHbFwij3VuwH2X1qVQgdD8C8QYb1iCNxnbscONkvnrL5fkR42CgqF389EVB1vN/PUHaFmjNC/3akHDyvaHqQl/luBN+n74AW67DZKT3apLPXsGOqIsS1scbOjVjbmzgxUHM3lHhgleRMpmtqOqHs79cEzAJSfDsGHwwgvQsiVMmgT16gU6qiyz4mDGZN6CX4abuSpATeCI53lpYCdQ29fBGT/bv9/Vkvn5Z+jbF8aMgSJFAh1VlqQtDvZiD1ccLJ+12k0elGGCV9XaACLyATBDVX/wfH8lNos1/Pz6K9x4oys9MH483Bl6C3lt2h/LgElRrNx1lMsaVeT5Hs2oUiq0fkEZk5u86YNvq6r3nvlGVWeLyMjMdjAhRNWVGRg4EGrXhtmzXddMCElISuH9X7bwzoLNFC9cgLduakW3llWtOJjJ87xJ8AdFZCjwBa7L5jbcItwm1MXFufVRP//c3UQdPx5KlQp0VFkSFX2UAZOi2LAvlq4tqzK8axPKFS8c6LCMCQreJPibgWHAVFyCX+R5zYSyPXvc2qhLlsDIkW5Wagi1eOMSk3lj3ibG/bqVCiUKM+72NvyniRUHMyY1b2ayHgYeEZHiqnrC2wN7FuteBBT2nGeSqg7LdqQm9/z1F1x7LRw7FpKLYC/ZdpiBk6PYdvAkN7WtweCrGlOqSOiNzzfG17ypB38B8BFuJaeaItIS6K+q959l13igk6qeEJGCwG8iMltV/8hx1Cb7vvzSjZCpVAn+7/+gRYtAR+S1E/FJjJ69gc//2EGNskWYeHc7OtQLzUJnxviDN100bwBX4JbrQ1VXicjFZ9tJVRU40+Iv6HloNuM0OZWSAkOHutmoF1/sxrdXqBDoqLy2cFMMQ6asZs+x09zZIZKnrmhI0UI2T8+YzHj1P0RVd6UZkZDszX4ikh83nr4e8K6q/pnONv2AfgA1a9b05rAmq44fd7NSZ850N1XHjIFCobFK0dFTCYyctZ7Jy6OpW6EYk+5tT+tamc7BM8Z4eJPgd3m6aVRECgEPA+u9ObiqJgOtRKQ0MFVEmqnqmjTbjAXGArRp08Za+Llt61bo1g02bHCJ/YEHQuZm6pw1exk6bS1HTiXwYMd6PNipHhEFraSvMd7yJsHfC7wFVAOigbnA2frf/0VVj4rIL0AXYM1ZNje5ZcECV79dFX78ES67LNAReSUmNp5hM9bww+p9NK1akgl3taVp1dAavmlMMPAmwTdU1VtTvyAiHYDfM9tJRCoAiZ7kXgQ3+3V0tiM1WfP++/Dww1C/vuuaqVs30BGdlaoydcVuRsxax6mEZJ66oiH9Lq5DwfxW0teY7PAmwY8BzvXitbSqABM8/fD5gG9VdVbWQzRZkpjoEvsHH8DVV7tRMyWDf9GtPUdPM2Tqan7ZGEPrWmUY3asF9SoWD3RYxoS0zKpJtgcuACp4luc7oyRu2b5MqWoUcE6OIzTeO3jQdcksXOhKD7zwAuQP7j7rlBTlyyU7eWn2BlJUGd61Cb3bR1pJX2NyQWYt+EK4se8FgNSrIxzHLbxtgsmaNe5m6p49biHsW289+z4Btv3gSQZOjuLPbYe5sF55RvVsTo2yRQMdljFhI7NqkguBhSLyqaru8GNMJqtmzHAJvUQJWLQIzjsv0BFlKjlFGf/bNl6bt5GC+fMxuldzbmhTw4qDGZPLvOmDPyUirwBNgYgzL6pqJ59FZbz3zjuuz711a5g2LejXS920P5anJkWxatdROjeuxAs9mlGpZMTZdzTGZJk3CX4i8A1wDW7I5B1AjC+DMl5ISYFBg+CVV6B7d3cztWjwdm8kJruSvmN+3kyJiIK8ffM5dG1RxVrtxviQNwm+nKp+LCKPpOq2WejrwEwm4uOhTx/4+mu4/354++2gvpm6OvoYT01aZSV9jfEzbxJ8oufrXhG5GtgDVPddSCZTR4+66o+//AIvvQQDBgTtzNS4xGTe+mkzYxdtpVyxQozt3ZrLm1YOdFjG5BneJPjnRaQU8ARu/HtJ4DGfRmXSt2sXXHklbNoU9CNllm4/zIDJUWyNOckNbarz9FVNKFXUSvoa40/e1IM/MznpGNDRt+GYDEVFwVVXQWwszJkDnYLzHvfJ+CRe+XEjExZvp2qpInx213lc3CB0qlYaE068qQdfG3gIiEy9vap2811Y5l9++sktqVeiBPz2GzRvHuiI0vXb5oMMmhJF9JHT3NG+FgO6NKJYYSvpa0ygePO/bxrwMTATSPFpNOZ/TZwId94JDRu6BbGrB9/tj2OnE3nx+/V8s3QXtcsX49v+7TmvtpX0NSbQvEnwcar6ts8jMf+mCqNHw+DB0LGjW1qvdOlAR/U/5q3bz9Bpq4mJjaf/JXV4rHMDK+lrTJDwJsG/JSLDcGWC48+8qKrLfRZVXpecDA895CpC3nwzfPIJFA6uYYWHTyYwfMZaZqzaQ8NKJRjbuw0ta5QOdFjGmFS8SfDNgd5AJ/7polHP9ya3nTrlkvqMGa5g2IsvQr7gKZerqny/ei/Dpq/l2OlEHu1cn/svrUehAsETozHG8SbB9wDqqGqCr4PJ82JioGtXWLLElSB44IFAR/QvB47HMXTaGuau20/zaqX44u52NK4S/KWIjcmrvEnwq4DSwAHfhpLHbdkCXbpAdLTrb7/22kBH9F+qyqRl0YyctY64pBQGXdmIuy+sTQFbiMOYoOZNgq8EbBCRv/h3H7wNk8wtf/3lFudISYGff4b27QMd0X/tPnqawVNWs2hTDG1qlWH0dS2oW8EW4jAmFHiT4If5PIq87OefXR33ihXdBKYGDQIdEeAW4pi4ZCcv/bAeBZ7r1pTe59ciny3EYUzI8GYmqxUW85VZs9wKTPXrw7x5UDk46rSkXoijQ71yvNSzhS3EYUwIymzJvt9U9UIRicWNmvnvW4Cqqt1dy4lvvoHbboNWrVzLvVy5QEdEcoryye/beHXuRgrms4U4jAl1ma3odKHna4mMtjHZNH483H03XHiha8UHwaLYmz0LcazcdZTLGlXkhR7NqVzKFuIwJpSddRiEiHzuzWvGS2+/DX37wuWXu5Z7gJN7YnIK7/y8mavf/o0dh07y1k2t+OiONpbcjQkD3txkbZr6GxEpALT2TThh7sUX4emnXT33r74K+OzUtXuO8dR3Uazbe5yrm1fhue5NKW8LcRgTNjLrgx8MDAGKiMhxXN87QAIw1g+xhQ9VGDLELdBx222u9ECBwFVZjE9K5p2f/+b9X7ZQumghPritNV2aBccNXmNM7smsD34UMEpERqnqYD/GFF5SUtyi2O++C/fe674GsPTAip1HGDApis0HTtDr3Oo8c01jShctFLB4jDG+400z8mkRuQ2oraojRaQGUEVVl/g4ttCXlORupk6YAE8+CS+/HLDl9U4nJPPG/E189OtWKpWM4JM729KxYcWAxGKM8Q9vEvy7uCJjnYCRwAnPa219GFfoS0hwS+pNmgTPPQfPPBOw5P7n1kMMnBzF9kOnuKVdTQZf2YgSEbZ8njHhzpsE305VzxWRFQCqekRE7G/6zJw+Db16uQU6XnsNHn88IGGcjE9i9JwNfLZ4BzXKFuHLu9txQb3yAYnFGON/3iT4RBHJj2eyk4hUwFZ2ylhsrCs9sHAhfPgh9OsXkDB+23yQgZOj2HPsNHd2iOSpKxpStJAtn2dMXuLN//i3galARRF5AbgOGOrTqELV4cNw5ZWwbBl88QXccovfQ0i9fF6d8sX4rn972kTa8nnG5EXe1KKZKCLLgMtwQyWvVdX1Po8s1Ozf7yYvbdjg+t0DUO73p/X7GTLVLZ937yV1ebRzfVs+z5g8zKu/2VV1A7DBx7GErl27oHNn93XWLPjPf/x6+iMnE3hu5lqmrdxDo8olGHd7G1pUL+3XGIwxwcc6ZXNq50645BLXPTN3rqsv40c/rN7Ls9PXcPRUIo9cVp8HOtryecYYxxJ8TuzeDZ06wZEjMH8+tPXfyNGY2Hienb6G2Wv20axaST7va8vnGWP+zRJ8du3dCx07woEDrpa7n5K7qjJ95R6Gz1zLqfhkBnRpSL+L6tjyecaY/2EJPjv273ct9z174McfoV07v5x237E4hk5bzfz1BzinZmleua4F9SpaNWdjTPoswWdVTAxcdpnre589Gzp08PkpVZXvlkYz8vt1JCan8Mw1TehzQST5bfk8Y0wmfJbgPTVrPgMq4yZGjVXVt3x1Pr84dMiNltmyBb7/Hi6+2OenjD5yisFTVvPr5oO0q12W0b1aEFm+mM/Pa4wJfb5swScBT6jqchEpASwTkXmqus6H5/SdI0fc8MeNG2HmTNdF40OpF70GGHltM249r6Ytem2M8ZrPEryq7gX2ep7Hish6oBoQegn+2DG44gpYuxamTfP5OPfUi15fVL88o3o2p3oZW/TaGJM1fumDF5FI4Bzgz3Te6wf0A6hZs6Y/wsma2Fjo0gVWroTJk10pAh/516LX+fPxcq8WXN+mui16bYzJFp8neBEpDkwGHlXV42nfV9WxeFaIatOmjfo6niw5cQKuugr++gu++w66dvXZqf4+cIIBk1axfKctem2MyR0+TfAiUhCX3Ceq6hRfnivXnTrlEvrixW791B49fHKapOQUxv26jTfmb6Joofy8eWMrureqaq12Y0yO+XIUjQAfA+tV9XVfnccnTp92JX8XLXJVIa+/3ien2bDvOE99F8Xq3ce4slllRnRvRoUStui1MSZ3+LIF3wHoDawWkZWe14ao6g8+PGfOxcW51vrPP7ul9m6+OddPkZicwnsLtvDOgs2UjCjIe7eey1XNq+T6eYwxeZsvR9H8hisvHDri4+G669zs1I8/ht69c/0Ua3Yf46lJUazfe5xuLasyvFtTyhazBbKMMbnPZrKekZgIN97oJjB9+CHcdVeuHj4+KZkxP/3N+wu3UK5YIcb2bs3lTSvn6jmMMSY1S/AASUmuK2b6dHjnnVxfZm/FziMMmBTF5gMnuK51dZ65ugmlitqi18YY37IEn5wMt9/uxri/+SY88ECuHTouMZnX523io1+3UqlkBJ/c2ZaODSvm2vGNMSYzeTvBq8LDD7thkKNHwyOP5Nqh/9p+mAGToth28CQ3n1eTwVc1omSEtdqNMf6TtxP8yJHw3nswYIB75IJTCUm8PGcjExZvp1rpIky8ux0d6pXPlWMbY0xW5N0E/8EHMGwY9OkDL72UK4f8vy0HGTg5il2HT3NH+1oM6NKIYoXz7iU2xgRW3sw+kybB/ffDNdfAuHGQw1mjsXGJvDR7AxP/3ElkuaJ80+982tUpl0vBGmNM9uS9BP/zz3DrrXDBBfDNN1AgZ5dg4aYYhkxZzZ5jp7n7wto8cXlDihTKn0vBGmNM9uWtBL9sGXTvDg0auJruRbNfgvfY6URe+H4d3y6Npm6FYky69wJa1yqTi8EaY0zO5J0Ev3mzK/VbrhzMmQNlsp+Mf1q/nyFTVxMTG8+9l9Tl0c71iShorXZjTHDJGwl+7164/HI3LHLuXKhWLVuHOXoqgREz1zFlxW4aVCrO2N5taFmjdO7GaowxuST8E/zRo27BjpgYWLDAdc9kw49r9zF02hoOn0zgoU71eLBTPQoXsFa7MSZ4hXeCP1P2d/16V2OmbdssH+LwyQSGzVjLzFV7aFylJJ/0aUuzaqV8EKwxxuSu8E3wZ+rL/Pabm6majXVUv4/ay7PT13A8LpHH/9OA+y6tS8H8+XwQrDHG5L7wTPCq0L+/Kx42ZoyrEpkFMbHxPDt9DbPX7KN5tVJMvL4djSqX9FGwxhjjG+GZ4J9+GsaPh2eegQcf9Ho3VWXGqj0Mm7GWU/HJDOjSkH4X1aGAtdqNMSEo/BL8m2/CqFGu5O9zz3m924HjcQyZuob56/dzTs3SvHJdC+pVLOG7OI0xxsfCK8FPnAiPPQY9e7oiYl6UIFBVJi/fzYiZa4lPSuHpqxpz14W1yZ8vtBajMsaYtMInwc+Z4wqHXXqpS/T5zz6Ece+x0wyesppfNsbQNrIMo3u1oE6F4j4P1Rhj/CE8Evyff0KvXtCsmbuxGhGR6eaqyrdLd/H8rPUkpSjDujbhjvaR5LNWuzEmjIR+gj90CK6+GqpUca34kpmPdok+corBU1bz6+aDnF+nLKN7taBWuWJ+CtYYY/wn9BN8uXLwyitwySVQqVKGm6WkKF8u2cmoH9ajwMjuTbm1XS1rtRtjwlboJ3iAO+/M9O1dh08xYFIUi7ce4sJ65RnVszk1yma/kqQxxoSC8EjwGUhJUT7/Ywej52wgnwijejbnprY1kBwu8GGMMaEgbBP89oMnGTA5iiXbDnNxgwqM6tmcaqWLBDosY4zxm7BL8Mkpyie/b+PVuRspmD8fL1/XgutbV7dWuzEmzwmrBL8l5gQDJkWxbMcROjWqyIs9mlO5VOZDJo0xJlyFRYJPTlE++nUrr8/bRETB/Lx+Q0t6nFPNWu3GmDwt5BP8sVOJ3PHJElbuOsp/mlTihWubUbGktdqNMSbkE3zJIgWoVa4od3aIpFvLqtZqN8YYj5BP8CLCWzedE+gwjDEm6Fihc2OMCVOW4I0xJkxZgjfGmDBlCd4YY8KUJXhjjAlTluCNMSZMWYI3xpgwZQneGGPClKhqoGP4LxGJAXZkc/fywMFcDCe3WXw5Y/HljMWXM8EcXy1VrZDeG0GV4HNCRJaqaptAx5ERiy9nLL6csfhyJtjjy4h10RhjTJiyBG+MMWEqnBL82EAHcBYWX85YfDlj8eVMsMeXrrDpgzfGGPNv4dSCN8YYk4oleGOMCVMhleBFpIuIbBSRv0VkUDrvi4i87Xk/SkTO9XN8NURkgYisF5G1IvJIOttcKiLHRGSl5/Gsn2PcLiKrPedems77AbuGItIw1XVZKSLHReTRNNv49fqJyHgROSAia1K9VlZE5onIZs/XMhnsm+nn1YfxvSIiGzz/flNFpHQG+2b6WfBhfMNFZHeqf8OrMtg3UNfvm1SxbReRlRns6/Prl2OqGhIPID+wBagDFAJWAU3SbHMVMBsQ4HzgTz/HWAU41/O8BLApnRgvBWYF8DpuB8pn8n5Ar2Gaf+99uEkcAbt+wMXAucCaVK+9DAzyPB8EjM4g/kw/rz6M73KggOf56PTi8+az4MP4hgNPevHvH5Drl+b914BnA3X9cvoIpRb8ecDfqrpVVROAr4HuabbpDnymzh9AaRGp4q8AVXWvqi73PI8F1gPV/HX+XBLQa5jKZcAWVc3uzOZcoaqLgMNpXu4OTPA8nwBcm86u3nxefRKfqs5V1STPt38A1XP7vN7K4Pp5I2DX7wxxCzzfAHyV2+f1l1BK8NWAXam+j+Z/k6c32/iFiEQC5wB/pvN2exFZJSKzRaSpfyNDgbkiskxE+qXzfrBcw5vI+D9WIK8fQCVV3QvulzpQMZ1tguU63oX7iyw9Z/ss+NKDni6k8Rl0cQXD9bsI2K+qmzN4P5DXzyuhlOAlndfSjvH0ZhufE5HiwGTgUVU9nubt5bhuh5bAGGCan8ProKrnAlcCD4jIxWneD/g1FJFCQDfgu3TeDvT181YwXMengSRgYgabnO2z4CvvA3WBVsBeXDdIWgG/fsDNZN56D9T181ooJfhooEaq76sDe7KxjU+JSEFccp+oqlPSvq+qx1X1hOf5D0BBESnvr/hUdY/n6wFgKu5P4dQCfg1x/2GWq+r+tG8E+vp57D/TbeX5eiCdbQJ6HUXkDuAa4Fb1dBin5cVnwSdUdb+qJqtqCjAug/MG+voVAHoC32S0TaCuX1aEUoL/C6gvIrU9LbybgBlptpkB3O4ZCXI+cOzMn9L+4Omz+xhYr6qvZ7BNZc92iMh5uH+DQ36Kr5iIlDjzHHczbk2azQJ6DT0ybDkF8vqlMgO4w/P8DmB6Ott483n1CRHpAgwEuqnqqQy28eaz4Kv4Ut/T6ZHBeQN2/Tw6AxtUNTq9NwN5/bIk0Hd5s/LAjfDYhLu7/rTntXuBez3PBXjX8/5qoI2f47sQ92dkFLDS87gqTYwPAmtxowL+AC7wY3x1POdd5YkhGK9hUVzCLpXqtYBdP9wvmr1AIq5V2RcoB/wEbPZ8LevZtirwQ2afVz/F9zeu//rMZ/CDtPFl9FnwU3yfez5bUbikXSWYrp/n9U/PfOZSbev365fTh5UqMMaYMBVKXTTGGGOywBK8McaEKUvwxhgTpizBG2NMmLIEb4wxYcoSvAkLIvKLiPh8UWQReVhctdCMZof66rzDReRJL7ZLXeFwpYi87Y/4THAqEOgAjAk0ESmg/xTnOpv7gStVdZsvY8qhjqp6MNBBmMCzFrzxGxGJ9LR+x4mrlz9XRIp43vtvC1xEyovIds/zPiIyTURmisg2EXlQRB4XkRUi8oeIlE11ittE5P9EZI1nluuZGYfjReQvzz7dUx33OxGZCcxNJ9bHPcdZI56a9CLyAW6CywwReSzN9vnF1WH/y1NEq7/n9UtFZJG4uuzrROQDEcnnee9mT2t7jYiMTnWsLiKyXFxBtZ9SnaaJ5zptFZGHc/SPYfKGQM+0skfeeQCRuOJXrTzffwvc5nn+C55Zs0B5YLvneR/czMwSQAXgGP/Man0DV9DtzP7jPM8vxlPfG3gx1TlK42ZGFvMcNxrPLNQ0cbbGzbQsBhTHzVQ8x/PedtKpAQ70A4Z6nhcGlgK1cfXr43C/GPID84DrcLMid3p+pgLAz7iywxVws1Bre451ZpbscOD/PMcuj5vtWzCdOLZ7Yl/peTwW6H93ewTuYV00xt+2qepKz/NluKR/NgvU1dePFZFjwEzP66uBFqm2+wpcjW8RKSluJaPLgW6p+q8jgJqe5/NUNb1a4BcCU1X1JICITMGVjl2RSYyXAy1E5DrP96WA+kACsERVt3qO9ZXn+InAL6oa43l9Iu4XUzKwSD1dQGni+15V44F4ETkAVML9kkrLumgMYH3wxv/iUz1PBop4nifxT5dhRCb7pKT6PoV/f4bT1t1QXG2dXqq6MfUbItIOOJlBjOmVqj0bAR5S1R/TnOfSTOLK6DgZ1Q9Je+3s/6/JlPXBm2CxHdc1Aq4LIztuBBCRC3FVMI8BPwIPpapAeY4Xx1kEXCsiRT2VAnsAv55lnx+B+8SVi0ZEGnj2BTjPUxUxnyfG33ALwVziud+QH1dBcyGw2PN6bc9xyqY9kTHeshaACRavAt+KSG9cf3R2HBGR/wNK4lYyAhgJvAlEeZL8dlyd9Ayp6nIR+RRY4nnpI1XNrHsG4CNcd9Nyz3li+Gcpv8XAS0Bz3C+PqaqaIiKDgQW4VvsPqjodQNzqQFM8vxAOAP852w+exgIRSfY8j1LV27O4vwkTVk3SGB/ydNE8qaqZ/lIxxhesi8YYY8KUteCNMSZMWQveGGPClCV4Y4wJU5bgjTEmTFmCN8aYMGUJ3hhjwtT/A8boUptN3+KBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "a,b,a_r,b_r = gradient_descent(x,y,C=0.001,E=20)\n",
    "plt.plot(a_r,color='r',label=r'$\\alpha $')\n",
    "plt.plot(b_r,label=r'$\\beta$')\n",
    "plt.xlabel(\"number of epoch E\")\n",
    "plt.ylabel(\"etimated parameters\")\n",
    "plt.title('C = 0.1 , E = 50')\n",
    "#set_spines()\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16ff0fe3",
   "metadata": {},
   "source": [
    "When the learning rate is 0.03 and Iter.=20 a is 5.53 but is still increased, which means the number of iterations are small, and $\\alpha$ is achieved its minimum, by increasing the iteration $\\alpha$ will achieve its limit , as shown below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "448c38ac",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6.994149589579071, 115.4291768709957)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAxkElEQVR4nO3dd3gc5bn+8e9jSZZsy1WWezcu2MaALQymd5JQTAKEEooTEpMfkECABHMggZzkAAk5hBAIJ4TmEDqBYBIIxfSOe2+4yk223CTb6s/vjx3DWpHkVdmdlfb+XNdeOzs7O+/t0VqPpr2vuTsiIiL10SrsACIi0vyoeIiISL2peIiISL2peIiISL2peIiISL2peIiISL2peIiISL2peEjCmdlFZjbdzIrNbIOZvWpmRzdxG5lm9oiZ7TSzjWZ2XQyZVpvZLjP7h5l1iXVdZvagmS0xsyozm9jI3B5kKI56/Kwx66yhjW+b2UdmttvM3qn23lAze8nMNpvZVjN7zcyGVVvmJ8F22BFsl8ymzCfNg4qHJFTwi/ce4HagO9AP+BMwoYmbug0YAvQHTgB+ZmZfqyXTSODPwCVBpt1BpljXNQe4EpjZRNkPdvfsqMdvm2i9e20l8jO4s4b3OgFTgWFEtsVnwEt73zSz04DJwEnAAGAQ8MsmzifNgOkOc0kUM+sIrAO+6+7Pxbmtve28Hrz+FTDE3S+oYdnbgQHuflHwejCwCMhx96JY12VmHwAPuftjjcjtwbqXN3Qd9Wjr+8DF7n58Hct0AQqBru5eaGZPAqvc/b+C908CnnD3HvHOK8lFex6SSOOBLODFWD9gZpPNbHttj1o+0xnoRWSPYK85wMhamhkZvay7fwGUAUMbsK6Eaci2aYBjgY3uXhi83mdbBdPdzSynidqTZkLFQxIpB9ji7hWxfsDd73T3TrU9avlYdvC8I2reDqB9HcvvqDZv7/L1XVdTmFmtEJxW00IN3DYxM7M+wP1A9Dme6ttq73Q8t4ckofSwA0hKKQS6mll6fQpIAxQHzx2AkqjpojqW71Bt3t7l67uupjAmEYet6mJmucDrwJ/c/amot6pvq73T8dwekoS05yGJ9DGRX8Bnx/oBM/uvalce7fOo6TPuvg3YABwcNftgYEEtzSyIXtbMBgGZwNIGrCthGrJtYlxvZyKFY6q7/0+1t/fZVsH0pqjDWpIiVDwkYdx9B/AL4H4zO9vM2ppZhpl93cxqvKLI3W+vduXRPo86mvsrcIuZdTaz4cAPgMdqWfYJ4EwzO8bM2gH/Dbzg7nv/mq5zXWbW2syyAAMyzCzLzGr8v2VmE81sVR25Y9bQbWNmaUHedKBVkDcjeK8D8BrwobtPruHjfwUuN7MRQZG5hdq3q7Rk7q6HHgl9AN8BpgO7gI3Av4Ajm7iNTOARYCewCbiu2vvFwDFRry8C1gSZXgK61GNd7wBe7XF8Lbl+TuTqpNpye5ChOOpxTxNvm4k15H0seO+yWjL0i/r8dcF22Ak8CmSG/Z3SI/EPXaorkkBm9jpwjbsvCjuLSGOoeIiISL3pnIeIiNSbioeIiNRb3IpH0GFagZnNj5p3l5ktNrO5ZvaimXWKeu8mM1sedDBX401RIiKSHOJ2zsPMjiVylcZf3X1UMO9U4C13rzCz3wC4+41mNgJ4ChhHpCuIN4Gh7l5ZVxtdu3b1AQMGxCW/iEhLNWPGjC3untuYdcTtDnN3f8/MBlSb93rUy0+Ac4PpCcDT7l4KrDSz5UQKycd1tTFgwACmT5/edKFFRFKAma1u7DrCPOfxPeDVYLo3sDbqvfxg3n8ws0kWGQti+ubNm+McUUREahJK8TCzm4EKInf2QuTO3OpqPJ7m7g+6e5675+XmNmqvS0REGijhHSOa2WXAGcBJ/tUJl3ygb9RifYD1ic4mIiKxSWjxCEZfuxE4zt13R701FXjSzO4mcsJ8CJERzOqtvLyc/Px8SkpK9r9wEsjKyqJPnz5kZGSEHUVEJGZxKx5m9hRwPJEuuPOBW4GbiPQT9IaZAXzi7j909wVm9iywkMjhrKv2d6VVbfLz82nfvj0DBgwgaCNpuTuFhYXk5+czcODAsOOIiMQsnldbXVjD7IfrWP5/gOrdP9dbSUlJsygcAGZGTk4OOvEvIs1Ni7zDvDkUjr2aU1YRkb00kqCISJIpr6yiqKSCopJyikoq2Bk8R88blNuOM0b3Ci2jioeISBxUVjlFJeVs313O9j3lbN9dxo49wevd5WzfU8aOau/tLRB7yvd/yvfMg3upeIiIJLvdZRUUFpdRuKuMwuLSfad3RaZ37C4LikE5O0vKqav3p+zMdDq2yaBT28ijR8cs2mdm0D4rnfZZe5/T6dAmMt0h66v3sjPTaZ0e7lkHFY84WbBgAddccw1r1qzhkksuoaCggEsvvZTDDjss7GgiEthTVklBUQmbdpayaWcJBUWlFBSVRApDcSlbd5WxpbiMwl2llJRX1biOrIxWdM3OJKddazq1bc2Aru3o1CaDjm1b0ymqOHRsk0HHNq2/nM5Ia96nnFt28bj2Wpg9u2nXecghcM89dS5SUlLCeeedx3PPPcegQYMYPnw4Y8eOVeEQSZCKyio2FZWyYfseNu0s/bJAFOwsYVPU9M6Siv/4bOu0VuRkt6ZLu9bkZGcyODf7y+mc7Nbk7J1u15qc7Na0bd2yf43WJjX/1XH25ptvcuihhzJy5EgAysrKuP7660NOJdJy7CwpZ/32Pazfvod120tYt23Pl6/Xb9/Dxp0lVFU7ZJSRZnRrn0X3DpkckJvNUYNz6NYhi+4dsujWPpPuHSLvdWyToasgY9Cyi8d+9hDiZdasWYwZMwaA9evXk52dzVFHHRVKFpHmqKyiivxtu1m9dTdrCnezunA3a7buIn/bHtZt20NR6b57DBlpRs+ObejVKYvxg7vSu1MWvTq1oUfHLHp0zKJ7+yw6tVVRaEotu3iEJDMzk/z8fABuuukmysrKQk4kknz2lFWyYksxq7bsZvXWXVFFYjcbduzZZ8+hTUYa/XPa0qdzW44YlEOvoDj0Dh5dszNp1UqFIZFUPOLgoosuYsKECQwbNowrrriC0tJSrr32Wu4JaU9IJCzuzpbiMr7YXMwXm4tZXlDMF5t38UVBMeu279ln2Zx2remX05bDBnSmX04f+ndpS/+ctvTLaUtudqb2GpKMikcc9OnThxkzZoQdQyRh3J3NxaUs3lDEko1FLNlUFCkYBcX7nJRuk5HG4G7tyBvQmfNz+zIotx0Du7ajX5e2tM9S56DNiYqHiNRLSXklSzcVsXhjEYs3FLF4406WbCyicNdXh2dz20dOSp91SC8G52ZHHt2y6dkhS4eXWggVDxGpVVFJOfPX7WT+uh3MXbeDBet3sGrLri/PR2RltGJY9/acfGB3hvVoz/Ce7RneowNd2rUON7jEnYqHiABQXFrBgnU7mLf3kb+DFVt2ffl+r45ZjOzdkTNG9+LAHu0Z1qM9/XPakaY9iZSk4iGSgtydFVt2MWP1Nmau3saM1dtYvrn4y+40enTI4qA+HTn70N4c1KcjB/XuSNfszHBDS1JR8RBJAXvKKpmTv/3LYjFzzTa27S4HoENWOmP6d+b00T0Z3acjo3p3pFv7rJATS7JT8RBpgXaVVvD5qq18smIrH68oZMG6HVQEJyoG57bjlBHdGdu/M2P7d2ZQ12ydxJZ6U/EQaQF2l1UwfdU2PllRyMcrCpmbv4PKKicjzTikbyeuOG4QY/t35tC+nemsk9nSBFQ8RJqhqipn3rodvLt0M+8t3cyc/O2UVzrprYzRfTryw+MGMX5QV8b075SyHfdJfOlbFUe/+MUvePHFFyktLeWGG25g0qRJYUeSZqygqIT3l27h3aWbeX/ZZrbtLscMDurdkcuPHsT4wTnk9e9Mu0z9t5b407csTl577TVmzZrF7NmzWbhwITfeeKOKh9RLVZUza+12pi3axLtLN7Ng/U4AumZncsLwbhw3NJejD+hKjq6CkhCoeMTJ1KlTmThxIuXl5dx3332cc845YUeSZqCkvJIPl2/hjYWbeHNRAVuKS0lvZYzt35mfnjaM44bmMqJnB53gltC16OLxy5cXsDD4a62pjOjVgVvPHLnf5WbMmMFhhx1GTk4OAwYM4Pe//32T5pCWY/vuMqYtKuCNhZE9jD3llWRnpnP8sFxOGdGd44d1o2Mb9fskyaVFF4+wVFVVkZ+fz8SJEzn//PO54ooruPvuu7nlllvCjiZJYseecl5fsJF/zt3Ah8u3UFHl9OiQxTlje3PKiB4cMagLmelpYccUqVWLLh6x7CHEw5IlSxgyZAgAbdq04aijjmLjxo2hZJHkUVxawZsLN/HPuet5b+kWyiqr6NO5Dd8/ZhDfOKgHB/XuqG7Hpdlo0cUjLLNmzaK0tJTKykoqKip48sknuffee8OOJSEor6zi7cUFvDhrHW8tLqC0ooqeHbO4dHx/zji4Fwf3UcGQ5iluxcPMHgHOAArcfVQwrwvwDDAAWAV82923Be/dBFwOVAI/dvfX4pUt3mbPns2ePXsYPHgwXbt25corr+Tggw8OO5YkiLuzYP1O/j4zn5dmr2frrjK6ZrfmgsP6cubBvRjTr7NOeEuzF889j8eA+4C/Rs2bDExz9zvNbHLw+kYzGwFcAIwEegFvmtlQd6+MY764mTVrFo8//jijRo0KO4okUEFRCS/NWs/fZ+azeGMRrdNacfKIbpw7tg/HDsklPa1V2BFFmkzcioe7v2dmA6rNngAcH0xPAd4BbgzmP+3upcBKM1sOjAM+jle+eFq8eDHDhw8PO4YkQFWV88HyLTz56RreWLSJyirnkL6d+NXZozhzdE86tVVXINIyJfqcR3d33wDg7hvMrFswvzfwSdRy+cG8/2Bmk4BJAP369Ytj1IZbu3Zt2BEkzgqLS3luRj5PfrqGNVt306Vda75/9EDOy+vLAd2yw44nEnfJcsK8pgPAXtOC7v4g8CBAXl5ejcuIxIO78/mqbTzx6WpenbeRssoqxg3swvWnDuVro3ro0lpJKYkuHpvMrGew19ETKAjm5wN9o5brA6xPcDaRGpVVVPHPuet5+IOVLFi/k/ZZ6Vx0eD++c3g/hnRvH3Y8kVAkunhMBS4D7gyeX4qa/6SZ3U3khPkQ4LOGNuLuzebyR3ftPCWrbbvKePKzNUz5aBUFRaUc0C2bO751EGcf0ps2rbWXIaktnpfqPkXk5HhXM8sHbiVSNJ41s8uBNcB5AO6+wMyeBRYCFcBVDb3SKisri8LCQnJycpK+gLg7hYWFZGVp1LZksmJzMQ99sJIXZuZTUl7FMUO68ttzR3Pc0Nyk/06JJIo157988/LyfPr06fvMKy8vJz8/n5KSkpBS1U9WVhZ9+vQhI0N9F4Vt4fqd3P/Ocl6Zt4GMtFZ885DefO/ogQzroUNT0rKY2Qx3z2vMOpLlhHmTycjIYODAgWHHkGZkxupt3P/2ct5aXEB2Zjo/PG4w3ztqILnt1dW5SG1aXPEQiYW78/EXhfzxreV8vKKQzm0zuP6UoVx65AD1YCsSAxUPSTkzVm/lrteW8MmKrXTvkMktpx/IRYf303CtIvWg/y2SMuav28HvXl/CO0s20zU7k9vOHMGFh/fT/RkiDaDiIS3esk1F3P3GUl6dv5FObTOY/PXhXDZ+gC63FWkEFQ9psQqKSvj9G0t55vO1tG2dzjUnDeHyYwbSIUvnNEQaS8VDWpw9ZZU8/MEKHnjnC0orqrjsyAH86MQhdGmnTgpFmoqKh7QYVVXOP2av467XlrBhRwmnjezO5K8fyMCu7cKOJtLiqHhIizBrzTZunbqAufk7OKh3R+45/xAOH5QTdiyRFkvFQ5q1rbvK+O2/F/P052vp3iGTu799MGcf0lsj9YnEmYqHNEtVVc7Tn6/lt68tpqikgh8cM5BrTh5Kdqa+0iKJoP9p0uzMy9/BLS/NZ87a7Ywb2IVfTRil/qdEEkzFQ5qNPWWV3P3GEh7+YCVd2mXy+/Mjh6jU061I4ql4SLPw8ReFTH5hLqsLd3PR4f248WvD1QeVSIhUPCSp7Swp545XFvPUZ2von9OWp35wBOMH6yoqkbDtt3iYWTtgj7tXmdlQYDjwqruXxz2dpLS3Fxdw0wvzKCgqYdKxg/jJyUPVpYhIkohlz+M94Bgz6wxMA6YD5wPfiWcwSV27yyr49b8W8eSnaxjWvT1/vmQsB/ftFHYsEYkSS/Ewd98dDB37R3f/rZnNincwSU2z1mzjJ8/MZvXW3Vxx3CCuO2Woer0VSUIxFQ8zG09kT+PyenxOJGbllVXc99Zy7nt7OT06ZPHUD47gCN0hLpK0YikC1wA3AS+6+wIzGwS8Hd9YkkpWbdnFNc/MZs7a7XxrTG9uO2uker4VSXJ1Fg8zSwPOdPez9s5z9xXAj+MdTFLDP+euZ/Lf55HWyvjTd8bwjYN6hh1JRGJQZ/Fw90ozG5uoMJI6Ssor+fW/FvK3T9ZwaL9O/PHCQ+nTuW3YsUQkRrEctpplZlOB54Bde2e6+wtxSyUt2sotu7jqiZks3LCTK44dxA2nDSMjrVXYsUSkHmIpHl2AQuDEqHkOqHhIvf1z7npufH4uGemteGRiHicO7x52JBFpgP0WD3f/biKCSMtWUVnFXa8t4c/vrWBMv07cd9EYenVqE3YsEWmg/R4rMLOhZjbNzOYHr0eb2S2NadTMfmJmC8xsvpk9ZWZZZtbFzN4ws2XBc+fGtCHJY9uuMiY++jl/fm8FFx/Rj6cnjVfhEGnmYjnQ/Bcil+qWA7j7XOCChjZoZr2JXK2V5+6jgLRgfZOBae4+hMid7JMb2oYkj4Xrd3LW/R/w2cqt/Oacg/j12QfROl3nN0Sau1j+F7d198+qzatoZLvpQBszSwfaAuuBCcCU4P0pwNmNbENC9vKc9XzrgQ8pq6jimSuO4PzD+oUdSUSaSCwnzLeY2WAiJ8kxs3OBDQ1t0N3XmdnvgDXAHuB1d3/dzLq7+4ZgmQ1m1q2hbUi43J173lzGH6YtI69/Z/508Ri6tc8KO5aINKFYisdVwIPAcDNbB6ykEZ0iBucyJgADge3Ac2Z2cT0+PwmYBNCvn/6STTalFZXc+Pxc/jF7PeeM6cMd39JhKpGWKJbi4e5+ctA1eyt3LzKzgY1o82RgpbtvBjCzF4AjgU1m1jPY6+gJFNQS5kEixYy8vDxvRA5pYlt3lXHF49P5fNU2fnraMK48frBG+RNpoWL5k/DvAO6+y92LgnnPN6LNNcARZtbWIr9ZTgIWAVOBy4JlLgNeakQbkmArNhfzzT99yJz8HfzxwkO56oQDVDhEWrBa9zzMbDgwEuhoZt+KeqsD0OAD2O7+qZk9D8wkcuJ9FpE9iWzg2aDr9zXAeQ1tQxJr+qqtXD5lOumtjKd+cARj++sqa5GWrq7DVsOAM4BOwJlR84uAHzSmUXe/Fbi12uxSInsh0oy8uXATVz05k96d2vDYd8fRL0f9U4mkglqLh7u/BLxkZuPd/eMEZpJm4rnpa5n8wjxG9erAIxMPIyc7M+xIIpIgsZzzKGzqO8yl+fvzu1/w0+fncuTgHJ78wREqHCIpJuF3mEvz5u7c/soi7nh1MWeM7snDlx1Gu0wNLCmSamL5X9/W3T+rduVMY+8wl2aoqsq5+R/zeOqztVw2vj+3njmSVq10RZVIKkr4HebSPFVWOT97fi5/n5nP1SccwPWnDtWluCIprKF3mMd8R7g0fxWVVVz37BymzlnPdacM5ccnDQk7koiELJbxPFYA+9xhHv9YkizKKqq45ulZvDp/I5O/PpwfHjc47EgikgT2WzzMrBNwKTAASN97qMLdfxzPYBK+0opKrnpiJm8uKuDnZ4zg8qMb0yuNiLQksRy2egX4BJgHVMU3jiSLsoqqLwvHryaM5JLxA8KOJCJJJJbikeXu18U9iSSNisoqrn1mlgqHiNQqlvs8HjezH5hZz2Co2C5m1iXuySQUlVXOT5+fyyvzNnLL6QeqcIhIjWLZ8ygD7gJuJrhcN3geFK9QEo6qKufmF+fx4qx13HDqUL5/jH7EIlKzWIrHdcAB7r4l3mEkPO7OL19ewNOfr+XqEw7g6hN1Oa6I1C6Ww1YLgN3xDiLhuvuNpUz5eDXfP3og1586NOw4IpLkYtnzqARmm9nbRLpNB3Spbksy5aNV/PGt5VxwWF9uPv1A3TkuIvsVS/H4R/CQFuifc9dz28sLOGVEd3599igVDhGJSSx3mE9JRBBJvA+Xb+Enz8wmr39n/njhoaSnxXIUU0QktjvMhwB3ACOIGn7W3XUpTjM2f90Ornh8BoO6ZvPQpYeRlZEWdiQRaUZi+VPzUeABIt2wnwD8FXg8nqEkvtYU7mbio5/RsU0GU743jo5tM8KOJCLNTCzFo427TwPM3Ve7+23AifGNJfGyY085333sMyqqnCnfG0ePjln7/5CISDWxnDAvMbNWwDIzuxpYB3SLbyyJh/LKKq58YgZrtu7m8csP54Bu2WFHEpFmKpY9j2uBtsCPgbFExvK4LI6ZJA7cnVtenM+Hywu541ujOWJQTtiRRKQZq3PPw8zSgG+7+0+BYuC7CUklTe7B91bwzPTI3ePnju0TdhwRaebq3PNw90pgrOni/2bt3/M3cOe/F3PG6J5cd4ruHheRxovlnMcs4CUzew7YtXemu78Qt1TSZBas38G1z8zmkL6d+N15B9Oqlf4OEJHGi6V4dAEK2fcKKwdUPJLc1l1lTPrrDDq1ac2Dl+TpXg4RaTKx3GHe5Oc5gqFtHwJGESlE3wOWAM8QGe52FZFzLduauu1UUVEZGQlwc3Epz10xntz2mWFHEpEWJJY7zLOAy4GR7HuH+fca0e4fgH+7+7lm1prI1Vz/BUxz9zvNbDIwGbixEW2ktNtfWczHKwr53/MO5uC+ncKOIyItTEwjCQI9gNOAd4E+QFFDGzSzDsCxwMMA7l7m7tuBCcDefrSmAGc3tI1U98LMfB75cCUTjxzAObqySkTiIJbicYC7/xzYFXSSeDpwUCPaHARsBh41s1lm9pCZtQO6u/sGgOC5xhsRzWySmU03s+mbN29uRIyWaW7+dia/MI8jBnXh5tMPDDuOiLRQsRSP8uB5u5mNAjoSOS/RUOnAGOABdz+UyBVck2P9sLs/6O557p6Xm5vbiBgtz7ZdZfy/v80kNzuT+y8aQ4Z6yRWROInlt8uDZtYZ+DkwFVgI/KYRbeYD+e7+afD6eSLFZJOZ9QQIngsa0UbKqapyrn9uDpuLSnng4jHkZOsEuYjETyxXWz0UTL5L5JBTo7j7RjNba2bD3H0JcBKRgrSQSLcndwbPLzW2rVTy4PsreGtxAb88aySj+3QKO46ItHCxXG2VA9wGHEXkstr3gV+5e2Ej2v0R8ERwpdUKIt2etAKeNbPLgTXAeY1Yf0r5fNVW7nptCacf1JNLx/cPO46IpIBYbhJ8GngPOCd4/R0i92Oc3NBG3X02kFfDWyc1dJ2pqrC4lKufnEnfzm2445yDNIysiCRELOc8urj7r9x9ZfD4NdApzrkkBlVVzrXPzGbb7nLu/84YOmRpUCcRSYxYisfbZnaBmbUKHt8G/hXvYLJ/D7z7Be8v28JtZ45kZK+OYccRkRQSS/G4AngSKA0eTwPXmVmRme2MZzip3ey127n7jaWcMbonF47rG3YcEUkxsVxt1T4RQSR2xaUVXPP0LHp0yOJ/vqnzHCKSeLGcMJck88upC1i7dTdPTxpPxzY6zyEiiadbkJuZf83dwHMz8rny+AMYN7BL2HFEJEWpeDQj67bv4aYX5nJI305cc/KQsOOISAqr9bCVmdX5Z627b236OFKbyirnumdmU1nl/OGCQ9RvlYiEqq5zHjOI3FFuQD9gWzDdicgd4APjHU6+8uiHK/l05VZ+e+5o+ue0CzuOiKS4Wv98dfeB7j4IeA040927unsOcAYagjahlhcUc9drSzj5wO6cp/E5RCQJxHLs4zB3f2XvC3d/FTgufpEkWmWVc8Nzc2jTOo3bvzVKl+WKSFKI5VLdLWZ2C/A3IoexLgYa0ymi1MNf3l/B7LXb+cMFh9Ctfdb+PyAikgCx7HlcCOQCLwaP3GCexNnSTUXc/fpSvjayB2cd3CvsOCIiX4rlDvOtwDVmlu3uxQnIJEB5ZRXXPzuH7Kx0fv1NHa4SkeSy3z0PMzvSzPYO1oSZHWxmf4p7shT353e/YN66Hfxqwii6alRAEUkysRy2+j1wGsF5DnefAxwbz1CpbnlBMfdOW87pB/Xk9NE9w44jIvIfYrrTzN3XVptVGYcsQmSMjv96YR5ZGa247ayRYccREalRLFdbrTWzIwEPho39MbAovrFS1zPT1/LZqq385pyDyG2vw1Uikpxi2fP4IXAV0BvIBw4BroxjppRVsLOE219ZxBGDuvDtPI3RISLJK5Y9j2Hu/p3oGWZ2FPBhfCKlrl++vJDSiipu1xgdIpLkYtnz+GOM86QR3ly4iX/N28CPTzyAQbnZYccREalTXb3qjgeOBHLN7LqotzoAafEOlkqKSyv4+UvzGda9PZOOHRx2HBGR/arrsFVrIDtYJnoo2p3AufEMlWrueWMpG3eWcN9FY2idrq7WRST51Vo83P1d4F0ze8zdVycwU0pZuqmIRz9axQWH9WVs/85hxxERiUksJ8x3m9ldwEjgy5753P3EuKVKEe7OL16aT/usdH562vCw44iIxCyWYyRPAIuJDP70S2AV8HkcM6WMqXPW88mKrdxw6jC6tGsddhwRkZjFUjxy3P1hoNzd33X37wFHNLZhM0szs1lm9s/gdRcze8PMlgXPLfoYTnFpBbe/sohRvTtw4bh+YccREamXWIpHefC8wcxON7NDgaYYzu4a9r1TfTIwzd2HANOC1y3WvdOWsWlnKb+aMIq0VrqnQ0Sal1iKx6/NrCNwPXAD8BDwk8Y0amZ9gNODde01AZgSTE8Bzm5MG8ls2aYiHvlgJefn9eXQfi16B0tEWqhYxvP4ZzC5Azihidq9B/gZ+14C3N3dNwRtbjCzbjV90MwmAZMA+vVrfod73J1bpy6gbes0fva1YWHHERFpkP0WDzMbCPwIGBC9vLuf1ZAGzewMoMDdZ5jZ8fX9vLs/CDwIkJeX5w3JEKbXF27ioy8K+e8JI8nROB0i0kzFcqnuP4CHgZeBqiZo8yjgLDP7BpFLfzuY2d+ATWbWM9jr6AkUNEFbSaWsooo7XlnEkG7ZXKST5CLSjMVSPErc/d6matDdbwJuAgj2PG5w94uDe0kuA+4Mnl9qqjaTxV8/XsWqwt089t3DSE/TneQi0nzFUjz+YGa3Aq8DpXtnuvvMJs5yJ/CsmV0OrAHOa+L1h2rbrjLunbaMY4fmcvywGk/niIg0G7EUj4OAS4AT+eqwlQevG8Xd3wHeCaYLgZMau85kdc+bSykureCW0w8MO4qISKPFUjy+CQxy97J4h2mplhcU87dP13DhuH4M7d5+/x8QEUlysRx4nwN0inOOFu32VxbRNiON604ZGnYUEZEmEcueR3dgsZl9zr7nPBp0qW6qeX/ZZt5aXMBNXx+uS3NFpMWIpXjcGvcULVRVlXPHK4vp07kNE48aEHYcEZEmE8sd5u8mIkhL9PLc9SzcsJN7zj+EzHQNvigiLUddw9B+4O5Hm1kRkaurvnwLcHfvEPd0zVhZRRX/+/pShvdoz1kH9wo7johIk6prJMGjg2ddHtQAz3y+hjVbd/PoxMNopV5zRaSF2e/VVmb2eCzz5Cu7Siv4w7TljBvYheOH5YYdR0SkycVyqe7I6Bdmlg6MjU+cluGRD1aypbiUG782HDPtdYhIy1Nr8TCzm4LzHaPNbKeZFQWvN9EC+51qKlt3lfHgeys4ZUR3xvbXWB0i0jLVWjzc/Y7gfMdd7t7B3dsHj5ygc0OpwZ/eXs6usgp+eprG6hCRliuWw1Y3m9nFZvZzADPra2bj4pyrWdq4o4S/frKab43po25IRKRFi6V43A+MBy4KXhcH86SaP72znKoq55qThoQdRUQkrmK5w/xwdx9jZrMA3H2bmbWOc65mZ/32PTz92VrOy+tL3y5tw44jIhJXsex5lJtZGsGNgmaWS9OMKNii3P/2chzn6hMPCDuKiEjcxVI87gVeBLqZ2f8AHwC3xzVVM5O/bTfPTl/Lt/P60rtTm7DjiIjEXSx9Wz1hZjOIDNRkwNnuvijuyZqR+9/+AsO46gTtdYhIaojlnAfuvhhYHOcszdLarbt5bvpaLhzXj17a6xCRFBHLYSupw31vLadVK+PKEwaHHUVEJGFUPBphTeFunp+Zz0Xj+tGzo/Y6RCR1qHg0wgPvLietlfH/jtdeh4ikFhWPBtq4o4TnZ+Rzfl5funfICjuOiEhCqXg00F/eX0GVw6RjB4UdRUQk4VQ8GmDrrjKe/HQNEw7ppbvJRSQlqXg0wKMfrqSkopIrda5DRFJUwotH0Cvv22a2yMwWmNk1wfwuZvaGmS0LnpNyMIyiknIe+2gVp43owQHd1HOuiKSmMPY8KoDr3f1A4AjgKjMbAUwGprn7EGBa8Drp/O2TNRSVVOi+DhFJaQkvHu6+wd1nBtNFwCKgNzABmBIsNgU4O9HZ9qekvJKHP1jBMUO6MrpPp7DjiIiEJtRzHmY2ADgU+BTo7u4bIFJggG61fGaSmU03s+mbN29OWFaAZ6evZUtxGVerDysRSXGhFQ8zywb+Dlzr7jtj/Zy7P+juee6el5ubG7+A1VRUVvHnd1cwtn9nxg3skrB2RUSSUSjFw8wyiBSOJ9z9hWD2JjPrGbzfEygII1ttXpm/kXXb9/DD4wZjZmHHEREJVRhXWxnwMLDI3e+OemsqcFkwfRnwUqKz1cbd+ct7KxjUtR0nDa/xaJqISEoJY8/jKOAS4EQzmx08vgHcCZxiZsuAU4LXSeHTlVuZt24Hlx8zkFattNchIhLTeB5Nyd0/IDKoVE1OSmSWWD30/gq6tGvNOWP6hB1FRCQp6A7z/VheUMybiwq45Ij+ZGWkhR1HRCQpqHjsx8MfrKR1eisuGd8/7CgiIklDxaMOW4pL+fvMfM4Z04eu2ZlhxxERSRoqHnV4/OPVlFVUcfnRA8OOIiKSVFQ8alFSXsnjn6zm5AO7cUC37LDjiIgkFRWPWrw4ax1bd5Xx/WM02JOISHUqHjVwdx77cBUH9uzA4eqKRETkP6h41OCTFVtZsqmI7x45QF2RiIjUQMWjBlM+WkXnthmcdUivsKOIiCQlFY9q1m3fw+sLN3L+Yf10U6CISC1UPKr52yerAbj4iH4hJxERSV4qHlFKyit56rM1nDqiB306tw07johI0lLxiDJ19nq27y7nsiMHhB1FRCSpqXgE3J3HPlrFsO7tOWKQLs8VEamLikdg+uptLNywk8t0ea6IyH6peASmfLSKjm0yOPtQXZ4rIrI/CR8MKhltLirl3/M3MvHIAbRdtgQ+/RQWL4bNm2HrVtizB8rLoaLiq4d73Svd3/uxLiMiUpNTT4U7wxtwVcUDeG7GWiqqnAv/+0r4/N3IzMxMyM2FLl2gXTtIT4eMDGjbFtLSoFUMO22xHP7SITIRaYicnFCbT/niUVVZxVOvzmL8mmUM3lMIDzwAJ50EgwZFioSIiPyHlC8e7//mz6y1fvysJ/C32SoYIiIxSO0T5qtW8eSM9eRU7OG039yowiEiEqOULh6bbv8dbw4+jHPH9aO1+rESEYlZ6haPnTt5ZulOKlulcdFJI8NOIyLSrKRs8ah87nmeHnECx3TLoH9Ou7DjiIg0KylbPN59bx7rO3TjolNGhR1FRKTZSc3iUVnJE2U55Fbu4eQRPcJOIyLS7CRd8TCzr5nZEjNbbmaT49HG+vc/4+2+ozm/Zysy0pJuE4iIJL2k+s1pZmnA/cDXgRHAhWY2oqnb2d27H8d1qOT8sw9v6lWLiKSEZLtJcByw3N1XAJjZ08AEYGFTNnLAkN48ess3m3KVIiIpJan2PIDewNqo1/nBvC+Z2SQzm25m0zdv3pzQcCIiEpFsxaOmXgL36XrW3R909zx3z8vNzU1QLBERiZZsxSMf6Bv1ug+wPqQsIiJSi2QrHp8DQ8xsoJm1Bi4ApoacSUREqkmqE+buXmFmVwOvAWnAI+6+IORYIiJSTVIVDwB3fwV4JewcIiJSu2Q7bCUiIs2AioeIiNSbufv+l0pSZrYZWN2IVXQFtjRRnKaWzNkgufMlczZQvsZI5mzQfPL1d/dG3evQrItHY5nZdHfPCztHTZI5GyR3vmTOBsrXGMmcDVIrnw5biYhIval4iIhIvaV68Xgw7AB1SOZskNz5kjkbKF9jJHM2SKF8KX3OQ0REGibV9zxERKQBVDxERKTeUrJ4JGKo2/2039fM3jazRWa2wMyuCebfZmbrzGx28PhG1GduCvIuMbPTEpBxlZnNC3JMD+Z1MbM3zGxZ8Nw5jHxmNixqG802s51mdm2Y28/MHjGzAjObHzWv3tvLzMYG2325md1rZjUNU9AU2e4ys8VmNtfMXjSzTsH8AWa2J2ob/l88s9WRr94/ywRuu2eicq0ys9nB/DC2XW2/S+L/3XP3lHoQ6XDxC2AQ0BqYA4xIcIaewJhguj2wlMiwu7cBN9Sw/IggZyYwMMifFueMq4Cu1eb9FpgcTE8GfhNWvmo/z41A/zC3H3AsMAaY35jtBXwGjCcyts2rwNfjlO1UID2Y/k1UtgHRy1VbT5NnqyNfvX+Widp21d7/X+AXIW672n6XxP27l4p7Hl8OdevuZcDeoW4Txt03uPvMYLoIWES1EROrmQA87e6l7r4SWE7k35FoE4ApwfQU4Oyo+WHlOwn4wt3r6mkg7vnc/T1gaw3txry9zKwn0MHdP/bI/+a/Rn2mSbO5++vuXhG8/ITI2Dm1ile22vLVIfRtt1fwl/m3gafqWkect11tv0vi/t1LxeKx36FuE8nMBgCHAp8Gs64ODiU8ErWrGUZmB143sxlmNimY193dN0DkSwt0CzHfXhew73/eZNl+UP/t1TuYTnTO7xH5S3OvgWY2y8zeNbNjgnlhZKvPzzKMfMcAm9x9WdS80LZdtd8lcf/upWLx2O9Qt4liZtnA34Fr3X0n8AAwGDgE2EBklxjCyXyUu48Bvg5cZWbH1rFsKNvUIgOGnQU8F8xKpu1Xl9ryJDynmd0MVABPBLM2AP3c/VDgOuBJM+sQQrb6/izD+BlfyL5/uIS27Wr4XVLrorVkqXfGVCweSTHUrZllEPlhP+HuLwC4+yZ3r3T3KuAvfHVoJeGZ3X198FwAvBhk2RTs3u7dFS8IK1/g68BMd98UZE2a7Reo7/bKZ9/DR3HNaWaXAWcA3wkOVRAczigMpmcQOSY+NNHZGvCzTPS2Swe+BTwTlTmUbVfT7xIS8N1LxeIR+lC3wbHSh4FF7n531PyeUYt9E9h7hcdU4AIzyzSzgcAQIie34pWvnZm13ztN5OTq/CDHZcFilwEvhZEvyj5/+SXL9otSr+0VHF4oMrMjgu/IpVGfaVJm9jXgRuAsd98dNT/XzNKC6UFBthWJzBa0Xa+fZaLzAScDi939y0M9YWy72n6XkIjvXlOc8W9uD+AbRK5K+AK4OYT2jyaySzgXmB08vgE8DswL5k8FekZ95uYg7xKa6EqNOvINInJFxhxgwd5tBOQA04BlwXOXMPIF7bUFCoGOUfNC235EitgGoJzIX3GXN2R7AXlEflF+AdxH0AtEHLItJ3Lse+/37/+CZc8JfuZzgJnAmfHMVke+ev8sE7XtgvmPAT+stmwY26623yVx/+6pexIREam3VDxsJSIijaTiISIi9abiISIi9abiISIi9abiISIi9abiISnLzN4xs7wEtPPjoNfTJ/a/dJO2e5uZ3RDDctE9KM82s3sTkU+at/SwA4g0R2aW7l91LLg/VxK5nn5lPDM10gnuviXsENJ8aM9DkppFxkhYZGZ/CcYreN3M2gTvfbnnYGZdzWxVMD3RzP5hZi+b2Uozu9rMrgs6rPvEzLpENXGxmX1kZvPNbFzw+XZBZ3yfB5+ZELXe58zsZeD1GrJeF6xnvpldG8z7PyI3XU41s59UWz7NIuNqfB50AHhFMP94M3vPIuNsLDSz/zOzVsF7FwZ7CfPN7DdR6/qamc00szlmNi2qmRHBdlphZj9u1A9DJFpT32mrhx5N+SAyRkIFcEjw+lng4mD6HSAvmO4KrAqmJxK5g7o9kAvsILgbGPg9kc7j9n7+L8H0sQRjMQC3R7XRiUhvBO2C9eYTdbduVM6xRO6IbgdkE7nT+NDgvVVUGxslmD8JuCWYzgSmExlj4XighEjRSQPeAM4FegFrgn9TOvAWkW6zc4ncLT4wWFeX4Pk24KNg3V2J3JGfUUOOVUH22cHjJ2H/3PVI/ocOW0lzsNLdZwfTM4gUlP152yPjGxSZ2Q7g5WD+PGB01HJPQWTcBjPrYJER9U4Fzoo6X5AF9Aum33D3msZ3OBp40d13AZjZC0S67J5VR8ZTgdFmdm7wuiORvobKiPQ3tCJY11PB+suBd9x9czD/CSJFrxJ4z4PDYtXy/cvdS4FSMysAurNv19t76bCV1IuKhzQHpVHTlUCbYLqCrw69ZtXxmaqo11Xs+72v3j/P3u6pz3H3JdFvmNnhwK5aMjZkWFEDfuTur1Vr5/g6ctW2ntr6Gaq+7fR/XpqEznlIc7aKyOEiiBzWaYjzAczsaGCHu+8AXgN+FPQuipkdGsN63gPONrO2QU/E3wTe389nXgP+n0W61MbMhgafhcjobgODcx3nAx8QGeTnuOD8ThqRXoXfBT4O5g8M1tOlekMiTU1/hUhz9jvgWTO7hMjx/4bYZmYfAR2IjKgH8CvgHmBuUEBWERn3olbuPtPMHuOrrt4fcve6DlkBPETkENzMoJ3NfDX058fAncBBRArTi+5eZWY3AW8T2dt4xd1fArDIaI8vBMWmADhlf//wat42s8pgeq67X1rPz0uKUa+6IkkmOGx1g7vXWbBEwqTDViIiUm/a8xARkXrTnoeIiNSbioeIiNSbioeIiNSbioeIiNSbioeIiNTb/wfdmBCYe7U+XwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "a,b,a_r,b_r = gradient_descent(x,y,C=0.001,E=1920)\n",
    "plt.plot(a_r,color='r',label=r'$\\alpha $')\n",
    "plt.plot(b_r,label=r'$\\beta$')\n",
    "plt.xlabel(\"number of epoch E\")\n",
    "plt.ylabel(\"etimated parameters\")\n",
    "plt.title('C = 0.001 , E = 120')\n",
    "#set_spines()\n",
    "plt.legend()\n",
    "a,b"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0f27a12",
   "metadata": {},
   "source": [
    "<h2 id='rates-iterations'>1.3Introduction in Ridge Regression in context of simple Linear Regression </h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10419f32",
   "metadata": {},
   "source": [
    "When the model fit training data too much well,that is the very bad sign.When the model fit training data too much well,that is the very bad sign.In general this mean the biases is fitted by model function.In Machine Learning it called overfit.Let's get below graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "941873cd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-58.55494602525627, -515.389216344578)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADnCAYAAAC9roUQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAvSUlEQVR4nO3deVyVZf7/8ddhVRQ1UxQ8WuJBZNXkuLWZCwYtWllGY6N9yyj99rWyNGfMX42V2KpOGTO0jTYz0bSMVBqkOTaNZghpJVaQSwNHVFyQVGS9fn9cShrnAAfPOTfL5/l4+JjxPtvnvom317nuazEppRBCCOEZXkYXIIQQ7YmErhBCeJCErhBCeJCErhBCeJCErhBCeJBPI4/L0AbRYiUkJJCZmWl0GULYY3L0gLR0Rat16NAho0sQwmkSukII4UESukII4UESukII4UESukII4UESukII4UESusLlCgsLGTNmDBEREURFRbF8+XIAjhw5Qnx8PGFhYcTHx3P06NG616SkpGCxWAgPDycrK8uo0kUbtHqbjcuWbKD//DVctmQDq7fZDK1HQle4nI+PD88//zzfffcdW7ZsYcWKFezcuZMlS5Ywbtw4CgoKGDduHEuWLAFg586dpKenk5eXR2ZmJrNmzaKmpsbgsxBtweptNn73/rfYSstRgK20nN+9/62hwdvY5AghnBYcHExwcDAAgYGBREREYLPZyMjIYOPGjQBMnz6dq666iqeffpqMjAySkpLw9/enf//+WCwWsrOzGTVqlIFnIVqi1dtsPJv1A/tKywnp1pG5V4dzwyV9HD7/2awfKK869x/w8qoans36oe7xpr6Xq0joCrfau3cv27ZtY8SIERw4cKAujIODgzl48CAANpuNkSNH1r3GbDZjs9lviaSlpZGWlgZASUmJm6sXLcmZVuuZED3TagUchuW+0nK7x8+89tfvpZSiqkbxwrp89pedoo8bwlhCV7jN8ePHmTx5MsuWLaNLly4On2dvIX2Tyf4syuTkZJKTkwGwWq2uKbQdaKiF6Gzr0dnPaM7723uNM63WhycMZFxkL4IC/Tnwc0W99zedfu2v3+vBf3x9zrGmBLuzJHSFW1RVVTF58mSmTp3KTTfdBECvXr0oLi4mODiY4uJigoKCAN2yLSwsrHttUVERISEhhtTdFjXUQgScbj068xk5Px3hvVybw/e3F672anrkvW+oqK61+9m20nLm/GM7teqXv/86PH/NmUVlzgS7q0LX1Mh2PbLgjXCaUorp06fTvXt3li1bVnd87ty5XHjhhcyfP58lS5Zw5MgRnnnmGfLy8vjNb35DdnY2+/btq7vZ5u3t3eDnWK1WcnJy3Hw2rd9lSzZgs/M1u0+3jgAOH9s0f+x5f4a3yUSNnYzp3smPqyN78U5uEdW1vzzuZdKt0BoXJE+XDj4svC6S7/f/TMZ2G4eOV9K7SwcemjCQZesL7NbriAnYs+RaZz7e4YI30tIVLrdp0ybefPNNYmJiGDJkCACLFy9m/vz5TJkyhddee41+/frxzjvvABAVFcWUKVOIjIzEx8eHFStWNBq4bY2rvuLb46hf09FxR48ppThRWUPpyUpKT1ZxrLyK0pNVlJZXOgwwe4ELcOREJW9tLax3vLaRsPX39qKi5pcWb0df73rdBGf8fKqaW6x9AVh4XeQ5j/l6e53Tmj7zXh18vTh6sqree4Wc/gfKFaSlK1qt1tbSbajP014ApNwU43TwOuoLtReKvQL9qVVQcrx+n2dHX29GDbhQB2x5FWWnA7a6sVT8FRP2Q6RnZ3+7n9uQPmedT1POr7HWelO6NqDZPwuHLV0JXdFqNSd0XXVTpzlh6OiXubmhUVldS2l5JcdOVlFaXkXWjv2s/GIvVWd9N/c2mQjq4s/+slM4s/F3905+BPh5U/JzBRXVtQT4eXPpgAsZ3r873Tr60TXAl64dfbkgwI9uAb589kMJj32QV+/8Jsf1OadPtynn3a2jLxXVtU0OPlf+o3Xm/VzwrUNCV7Q9zoZuc345XfUL7ajPM6RrB/YdO+XwdWMHBbFl92FOVtbg5+NFj85+oODQiUoqHdxY+jVfbxOWoM7sPXSS8qoaAv19GB/ZiyvCetAtwJevC4/xVvZ/OfhzBSFdOzAvYRDguMUH9se3Ojt6oaFr6+gzHHFn90wzSeiK1s3eL9WTd09yKnQbuqFk72vrDZf0afA1Z7dCa2sVP5+q5lh5Fau321i5eS+HT1TStaMvlw24kLU79jfvxH/FywQX9+jEfw+fPOervp+3F5U19kO4GTeBHJ63s63QxrTAsHQVCV1x/oz6BXHUIqp67xF+3Nnw0KCz9Z+/xuF/0L++IePv48W9owew/NMCh+83uG83jp3V5+lkdycAnf19GBPek8y8/ed0C/j7eNHR14vS8up6r3E0IsDRcWdHIkDD18qe5nxGGyejF8T5ac5sIFdxNCj+UJnjr+X/2FpYN6uoR2c/Jg4OoWtHX0rL69+ZPvN+Z6uorm0wcH29TXTt6Eu/7gFcEOBLt46+dA3w48UNBZTaufvtqIX45A3RDr+CP/j2druf7WhEQI1S9f7x6OjrXXeDyBkh3To6NaSqoZEQ4lwSuqJJGpoN5Io77PaC5+EJAxkf2cvhL39VTS1z3/ma0vKq0zeT9FCmoycrz2k1Hjpeyeub9tp9Dy9Tw8OUHhwfxssbd50zML+hfsdjdgIX4Fh5FUtvHeLwm8INl/Spdx0d3WhqqEXrqJvEWXOvDjdsSFVbJ90LbZyzXQKOnu/o62ZD/YVn3stWWk7vLh248/KLOVlRw8sbd53T/+jtZaJf9478dPikU1/Ri1c+wKB7V3CioprqWkUHXy+GmLuxY18Zxyvqfy3v1cWfOeMHsvzTAoqPnWrykCNnhhY5CqXmfP121K3S0IgAV37rcPOQqrZO+nTbI2fvvDdnWFO3jr6MDu/Jhu8O8nNFNR19venXPYCTldUUHS0/7/+AunTw4dqYYN77ynZOUHfw8eJY+sN0u+35evU6Gizv6B+I5oxQMPpGk5E3oNrwzS9XktBtK5z5JXQUlMFdO3D3FaGkbtxFyfEKugf4MT4yiMwd+yk7Zf/GjbeXyeHd8V8zmfRrnB1Ib/e90EFp7/xuv34sPW5/wW69zt5QcjZIGmr5N9SNINoNCV1XMbqF8esWmb+PF2MG9WTDdyX1vrLXuCD0zrj7iv7sKz3Ff348xLHyKi7s5Mcdl17MX7f8ZHcVJ2c1Jyj9g8MInr7M7mP2bii58itwU4eSiXZLRi+4gqvv4K/eZuOZzO/Zd+wUQYH+JA3rS6y5G6XlVZSerORY+dnz26v4Ytehc24Qgb7LnrnjQL33bihwHU3N9ALstWX7dOvIgmsj7TwCL6zLd3yCdjj6+u2on7KhO+++3vY3PnHlDSVHHN1oas5IAdG+uCV0tz71En2fe4Kg0hIOdutJ4cMLGbbgPnd8VD3u7ANzdAf/mczvuTqqd93dc70YSGVdWG7de4QvdulZRf4+XvTo7M/Jyupzbrgc/LmCP274sd5nepn0lMw+FwTUC9ymsNfic9TnWevg+Q0FiaOhRY7C9fGJUYD92UbWi7o79TPq3aUDvg7qtTcawJXOvLd0Iwhnubx7YetTLxH9h4fpWPXLV85yX392PPac08HbnDvvzt7thV9+cYK7duC+sRYut/T8JUDLqzh2utX53CfOteoc8TKBj5eJSjsh2tnfm6oaZXeIkquGEDV0t97ZFqIrp3I6y2q18ugrGRJ8oiXyXJ/u/gt60bv0YP3j3YLofbT+12BwfmgK2P9ldtTP5uhrs4+XDqumLgTi6Gt5Z38frgrvSZYTs4qcdSYQXTGEqIUuEOK01rbKmGhXPBe6tSYvvOy8rBbwsvNZjgLA0XjHjr7eVNXU1lv4OKRrR4pcNCsmwNeb6lpVb4jSzVaz0ysmuUpDd/Gb033SFob9SOiKFsy4lq46/ell/gFsmjGX6o0bMVXXUNg9mMLx1/OBKcjuQHZndfDxwmQy2e2vdNTSdVZDX7+dnavuqM/TlYPr2zoJXdGCeW70QuHDCyl5cRn3Xz+XYx0687N/ALd+8wkPff5X4l9+gjeHXsvSy6dS1qEzVIJ3bQV4nf8uARXVtSy9dYhTX78dBZwj+0rLHd6gcdUNJbDfrSJ3xYVoG1weusMW3EdmrTf9v9lHr7JD+Kpa1KhRfHi4kKt+3ModuR/y26/WcNKvA37VVRzu1I3LZr5R732cbQmGdOvY4B1le3fGwbmpnA3NL3fU39rQ3XpwPNSstX/1F0LY57HJEQ319UY9+nGTb5iBa+d+u3J+eVvoJ21NpHtBtGDGz0hzNKoBoDDxRu4bfCvfqM4t5iaQBGjLJ6ErWjDjQ9fu+F0fP45eNoaQLRvBxwcWLIA5c8Df31UfK9owCV3RgjkMXfvzKN1g2IL72PHYc+zvFkQtJvZ3C2LH488TsjETdu6E+Hj4/e8hKgo+/BCndtETbUJmZibh4eFYLBaWLFlidDlCuEXLWvBm3Tq4/3747jtISIBlyyBc7tq3BzU1NQwcOJB169ZhNpsZNmwYb731FpGR9td8AGnpihbN+JZuk8THw9dfw9KlsHkzREfD3LlQVmZ0ZcLNsrOzsVgshIaG4ufnR1JSEhkZGUaXJYTLtazQBfD1hQcegIICmD4dnn8eBg6Ev/wFal0xxUG0RDabjb59+9b93Ww2Y7PZDKxICPdosHshISFBHTp0qNlvXlJSQs+ePZv9egBOnIDCQv2/AQHQrx906nReb+mSutygPdd19OhRysrKuOiiiwA4fPgwJ06coF+/fvVqOfPfZEVFBUOGDHFrXc3Rnn+OzdEW68rNzc1SSiXYfVAp1dCf8xIXF3e+b6HV1Ci1cqVSvXsrBUrdcYdSxcXG1+Vi7bmuzZs3qwkTJtT9ffHixWrx4sUNviYgIMDdZTVLe/45Nkcbrcthrra87gV7vLxg2jTIz4d58+Bvf9NdDs89B5WVRlcnXGDYsGEUFBSwZ88eKisrSU9PZ+LEiUaXJYTLtY7QPSMwEJ5+GnbsgCuu0DfZYmMhM9PoysR58vHx4aWXXuLqq68mIiKCKVOmEBUVZXRZQricW0M3OTnZPW88cCCsWQMffaRvriUmwqRJsGuXsXWdp/Ze1zXXXEN+fj67du1iwYIFjT6/R48eHqjKee395+is9lZXyxqn2xwVFbB8OTzxhO5qeOghPcmic2ejKxNuJuN0RQvWSsbpNoe/v+7nzc+HpCRISdETKv7+d5nVJoRocc4rdN955x2ioqLw8vKq1+JISUnBYrEQHh5OVlaW3dcfOXKE+Ph4wsLCiI+P5+jRo80vJjgYVq7UkyqCg2HqVN3vu20bt956K0OGDGHIkCFcfPHFDocZXXzxxcTExDBkyBCsVmvza2mixx9/nD59+tTVtnbtWrvP8/T02Llz5zJo0CBiY2O58cYbKS0ttfs8T1yvxs5dKcXs2bOxWCzExsby1VdfuaWOXyssLGTMmDFEREQQFRXF8uXL6z1n48aNdO3ate7nu2jRIo/U1tjPxYhr9sMPP9RdhyFDhtClSxeWLVt2znM8db3uvPNOgoKCiI6OrjvW1Cxyye9iQ0MbGhsTsXPnTvX999+r0aNHq61bt9Ydz8vLU7GxserUqVNq9+7dKjQ0VFVXV9d7/dy5c1VKSopSSqmUlBQ1b9688xihcZaaGqVefVWpnj2VMpmUSk5W6uBBpZRSc+bMUX/4wx/svuyiiy5SJSUlrqmhCR577DH17LPPNvic6upqFRoaqnbt2qUqKipUbGysysvLc2tdWVlZqqqqSiml1Lx58xz+XNx9vRo797i4OLVmzRqVkJCgamtr1RdffKGGDx/utnrOtm/fPpWbm6uUUqqsrEyFhYXV+7n861//Utdee61H6jlbYz8Xo67ZGdXV1apXr15q79695xz31PX67LPPVG5uroqKiqo71pQscvJ30T1DxiIiIgi3szZCRkYGSUlJ+Pv7079/fywWC9nZ2XafN336dACmT5/O6tWrz6ecX3h5wV136S6HBx6A116DgQNRy5fz/ttvc9ttt7nmczzAiOmxEyZMwMdHr28/cuRIioqK3Pp5jjTl3DMyMpg2bRomk4mRI0dSWlpKcXGx22sLDg5m6NChAAQGBhIREdFqZtAZdc3O+PTTTxkwYEDdRBhPu/LKK+nevfs5x5qSRa76XXRLn25Tp3QeOHCA4OBgQP9HfPCg/fV2m61bN3jhBfjmG7BaMT3wAOsPHybsv/+1+3STycSECROIi4sjLS3NtbU48NJLLxEbG8udd95p9yuN0dNjX3/9dRITE+0+5u7r1ZRzN/r6AOzdu5dt27YxYsSIeo998cUXDB48mMTERPLy8jxST2M/F6OvWXp6usOGjxHXC5qWRa66bo1u1zN+/Hj2799f7/hTTz3FpEmT7L5G2bmBZTI5vJnnEo3W+ckn/Ckxkduys2H8eJg8WU+uuPjiuudu2rSJkJAQDh48SHx8PIMGDeLKK690W10zZ85k4cKFmEwmFi5cyEMPPcTrr79+zvPcdS2b8nN96qmn8PHxYerUqXbfwx3X62xNOXcj/ls72/Hjx5k8eTLLli2jS5cu5zw2dOhQfvrpJzp37szatWu54YYbKCgocHtNjf1cjLxmlZWVfPDBB6SkpNR7zKjr1VSuum6Nhu769eudflOz2UxhYWHd34uKiggJCan3vF69elFcXExwcDDFxcUEBQU5/VlNrbO6pobHtm3jui+/pOvbb+tRDmvW6JEPjzwCAQF1NQYFBXHjjTeSnZ193iHS1Ot39913c91119U73tRr6eq6Vq5cyUcffcSnn37q8D8sd1yvszXl3N11fZqiqqqKyZMnM3XqVG666aZ6j58dwtdccw2zZs3i0KFDbh9f3NjPxchr9vHHHzN06FB69epV7zGjrhc0LYtcdd3c0r0wceJE0tPTqaioYM+ePRQUFDB8+HC7z1u5ciWgf8kdtZxdYf369QwaNAhzWBg8+ih8/72eULFoEQwaxKk33+Tn00tInjhxgk8++eScu5vucHY/2j//+U+7n2fE9NjMzEyefvppPvjgAwICAuw+58SJE/z88891/98d16sp5z5x4kRWrVqFUootW7bQtWvXuq+J7qSU4q677iIiIoI5c+bYfc7+/fvrWkfZ2dnU1tZy4YUXurWupvxcjLpmAG+99ZbDrgUjrtcZTckil/0uNnSXrbG7gO+//77q06eP8vPzU0FBQecsWPLkk0+q0NBQNXDgQLV27dq643fddVfdSIdDhw6psWPHKovFosaOHasOHz7c2Ec22/Tp01Vqauo5x2w2m5o7YoRSgwcrBSq7Uyd1U1iYioyMVE8++aTbajnj9ttvV9HR0SomJkZdf/31at++fXV1JSYm1j1vzZo1KiwsTIWGhnqkrgEDBiiz2awGDx6sBg8erO655556de3atUvFxsaq2NhYt14ve+eempqqUlNTVVxcnKqtrVWzZs1SoaGhKjo6+pxRNO70+eefK0DFxMTUXac1a9bU1aaUUi+++KKKjIxUsbGxasSIEWrTpk1ur8vRz+Xsuoy6ZidOnFDdu3dXpaWldceMuF5JSUmqd+/eysfHR/Xp00e9+uqrDrPoPH4XHeZq65+R5go1NfDKK3qPttJSmDlTt4B/dYdTtCwyI020YG14RporeHvDvffqhdNnzoTUVL2+w5/+pANZCCFcREL3bN27w0svwbZtEBOjAzguDj7/3OjKhBBthISuPbGxsGED/OMfcOQIXHkl3Hab3sFCCCHOg4SuIyYT3HKLHuXw2GOwejUMGgRPPQWnThldnRCilZLQbUxAADz++C/bwj/6KERG6hCWVcyEEE6S0G2qiy+G996D9et1EN94I0yYADt3Gl2ZEKIVkdB11rhx+kbb8uWQk6P7fx98UA81E0KIRkjoNoevL8yerVcxu+suHcADB+rVzGprja5OCNGCSeiej5494c9/1i3egQNhxgwYMQK++MLoyoQQLZSErisMHarH8v7tb7BvH1x6qd4y3oNrlAohWgcJXVcxmeA3v4EffoDf/Q7eflu3fp95Rm+eKYQQSOi6XufOsHgx5OXB2LF62ciYGHCw/1lb09D+ao72zcvNzSUmJgaLxcLs2bPtrlsqRFshoesuFgtkZMDHH+vtg669Fq67Tq/v0IbFx8ezY8cOvvnmGwYOHFi3WPXOnTtJT08nLy+PzMxMZs2aRc3pdS1mzpxJWloaBQUFFBQUkJmZaeQpCOFWErrulpCgtwt67jn4978hKkq3fk+vedrWONpfzdG+ecXFxZSVlTFq1ChMJhPTpk1z3V55QrRAErqe4OcHDz2kh5jdfrvu5x04EFatatNDzM7eX83R/lI2mw2z2VzvuCNpaWlYrVasVislJSXuK14IN5HQ9aTeveH11+HLL6FfP5g+HS67DLZuNboyp4wfP57o6Oh6f87eGfXX+6vZ66c1mUxO7zuVnJxMTk4OOTk59OzZ0wVnI4RnNbpHmnCD4cP1WN4339RdDSNGwP/8j74BZ2fvqJamOfurOdpfymw2n7PFuyf36xLCCNLSNYqXl27p5ufrroc339RdDkuXQlWV0dU1m6P91RztmxccHExgYCBbtmxBKcWqVavculeeEEaT0DValy7w7LPw7bd6UsWcOTB4MKxbZ3RlzXLffffx888/Ex8fz5AhQ7j33nsBiIqKYsqUKURGRpKQkMCKFSvw9vYGIDU1lRkzZmCxWBgwYEBdP7AQbZHskdaSKAUffaQX0Nm1C264AZ5/HkJDja6sRZI90kQLJnuktQomE1x/vZ5YkZKiW7uRkXoN3xMnjK5OCOECErotkb8/zJ+vpxTffLPerWLQIEhPl4XThWjlJHRbsj594K9/1Yvp9Oyp92kbPRq2bze6MiFEM0notgaXX67H8v75z3rboLg4mDULDh82ujIhhJMkdFsLb29ITtZDzO67D9LSICwMVqyA6mqjqxNCNJGEbmtzwQV6p4rt2+GSS3QADx0KGzcaXZkQogkkdFur6Gi9SeZ770FZGYwZA7feCv/9r9GVCSEaIKHbmplMcNNNup930SL48EM9ymHRIigvN7o6IYQdErptQceOsHAhfP+9Huf72GMQEaFbwTLETIgWRUK3LenXT28T9K9/6enFN98M48fryRZCiBZBQrctuuoq+OorePFF2LZNr+Vw//1w9KjRlQnR7knotlU+PnpkQ36+Hmr20kt6FbO0NDi9TY4QwvMkdNu6Hj3g5ZchN1f3895zDwwbBps2GV2ZEO2ShG57MWQIfPYZvPUWlJToWW5Tp0IDW+MIIVxPQrc9MZkgKUmPcnj0UT26ITxcr2h26pTR1QnRLkjotkedOsETT8DOnRAfD7//vZ5s8eGHMsRMCDeT0G3PQkPhn/+ETz7ROxZPnAjXXKOXlBRCuIWErtCt3a+/1vuzbd6sW70PP6ynFwshXEpCV2i+vvDAA1BQAHfcAS+8oIeY/eUvUFtrcHFCtB0SuuJcQUHwyiuQna27H/7nf2DUKP13IcR5k9AV9lmt8J//wKpVUFgII0boAN6/3+jKhGjVJHSFY15e8Nvf6htrjzwCf/ub7nJ4/nmorGzwpc899xwmk4lDhw7VHUtJScFisRAeHk5WVlbd8dzcXGJiYrBYLMyePZtGdqgWolWT0BWNCwyEJUv0wjlXXqlvssXGQmam3acXFhaybt06+vXrV3ds586dpKenk5eXR2ZmJrNmzaLm9HTkmTNnkpaWRkFBAQUFBWQ6eF8h2gIJXdF0YWHw0UewZo2+uZaYqIeZ/fjjOU978MEHeeaZZzCZTHXHMjIySEpKwt/fn/79+2OxWMjOzqa4uJiysjJGjRqFyWRi2rRprF692sMnJoTnSOgK511zDezYAc88o5eRjIqC3/0Ojh/ngw8+oE+fPgwePPicl9hsNvr27Vv3d7PZjM1mw2azYTab6x13JC0tDavVitVqpaSkxPXnJoSb+RhdgGh9xo8fz/7TN9R6hITw4IEDTFqyhPK0NHK6dWNRbm6919jrpzWZTA6PO5KcnExycjIAVqu1uacghGEkdIXT1q9fX//gF1+gZsxg0c6dbO3dm8cvuICikhKGDh1KdnY2ZrOZwsLCuqcXFRUREhKC2WymqKio3nEh2irpXhCuMWoUAd9+C6+9xrAuXVhz4ACrOnZk2yef0Lt3byZOnEh6ejoVFRXs2bOHgoIChg8fTnBwMIGBgWzZsgWlFKtWrWLSpElGn40QbiOhK1zHywvuvFMvnP7AA9xy/DgXjBwJf/wjUeHhTJkyhcjISBISElixYgXe3t4ApKamMmPGDCwWCwMGDCAxMdHgExHCfUyNjImUAZOi+b77Tm8TtG6dvtm2fDmMG+eyt7dareTk5Ljs/YRwIYc3JqSlK9wnIgKysmD1ajh5Um+SOXky7N1rdGVCGEZCV7iXyQSTJum1e598Uk+oiIjQ28SfPGl0dUJ4nISu8IwOHWDBAj2l+MYbYdEiGDQI3nlHFk4X7YqErvAssxn+/nf497/hwgthyhQYMwa++cboyoTwCAldYYwrroCcHPjTn/Tstksu0VvGHzlidGVCuJWErjCOt7feEj4/H2bNgtRUvb5DaiqcXgxHiLZGQlcYr3t3ePFF2L5dr142axbExekuCCHaGAld0XLExMCGDfCPf8DRozB6NNx2m15EXYg2QkJXtCwmE9xyi55Y8dhjeozvoEF6uNmpU0ZXJ8R5k9AVLVNAADz+uA7fxERYuBAiI3UIyxAz0YpJ6IqW7eKL4d134dNPdRDfeCNMmKAnWwjRCknoitZh7Fh9o+3FF/VQs9hY3ddbWmp0ZUI4RUJXtB4+Pnosb34+zJgBBw/qjTJfe01vHyREKyCrjIlWyxoZSU737rBpkx5i9sc/wqWXGl2WECCrjIk2KSAAPv8c/vpXKC6Gyy6DadNg3z6jKxPCIQld0bqZTDB1ql5I53e/g7ffhvBwePppqKgwujoh6pHQFW1D586weDHk5embbvPnQ3S03i5eiBZEQle0LRYLZGTAxx/rtR2uu05vGZ+fb3RlQgASuqKtSkjQy0U+/zz85z+61TtvHpSVGV2ZaOckdEXb5ecHc+boVu7tt8Ozz+r+3lWrZIiZMIyErmj7eveG11+HL7+Eiy6C6dP1SIetW42uTLRDErqi/Rg+HDZvhjfegD17YMSIXyZZCOEhErrC5V588UXCw8OJiopi3rx5dcdTUlKwWCyEh4eTlZVVdzw3N5eYmBgsFguzZ8+mkQk758fLC+64Q3c5PPQQrFypF05fuhSqqtz3uUKcoZRq6I8QTtmwYYMaN26cOnXqlFJKqQMHDiillMrLy1OxsbHq1KlTavfu3So0NFRVV1crpZQaNmyY2rx5s6qtrVUJCQlq7dq1TfqsuLi48y/4u++USkhQCpSKiFAqK+v831OIBnJVWrrCpVJTU5k/fz7+/v4ABAUFAZCRkUFSUhL+/v70798fi8VCdnY2xcXFlJWVMWrUKEwmE9OmTWP16tWeK3jQIFi7Fj78ECor4eqr4YYbYPduz9Ug2hUJXeFS+fn5fP7554wYMYLRo0ez9fTNKpvNRt++feueZzabsdls2Gw2zGZzveOOpKWlYbVasVqtlJSUuKZok0mP583L0xMs1q/Xa/c++iicOOGazxDiNAld4bTx48cTHR1d709GRgbV1dUcPXqULVu28OyzzzJlypS6r1W/ZjKZHB53JDk5mZycHHJycujZs6dLzwt/fz2V+Icf4Oab4amndEs4PV0WThcu42N0AaL1Wb9+vcPHUlNTuemmmzCZTAwfPhwvLy8OHTqE2Wym8Ky9zoqKiggJCcFsNlNUVFTvuKH69NGL6MycCbNn633aXn5Zr2I2ZIixtYlWT1q6wqVuuOEGNmzYAOiuhsrKSnr06MHEiRNJT0+noqKCPXv2UFBQwPDhwwkODiYwMJAtW7aglGLVqlVMmjTJ4LM47bLLIDsb0tL0tkFxcTqIDx0yujLRiknoCpe688472b17N9HR0SQlJbFy5UpMJhNRUVFMmTKFyMhIEhISWLFiBd7e3oBuHc+YMQOLxcKAAQNITEw0+CzO4u0Nd9+th5j93//BK6/ohdNfegmqq42uTrRCsoi5aLWsVis5OTme/dC8PLj/fr1nW3S07nIYM8azNYjWQBYxF8IloqJg3Tp47z04flwvI3nLLfDTT0ZXJloJCV0hnGUywU036R2JFy3Sa/YOGgR/+AOUlxtdnWjhJHSFaK6OHWHhQvj+e5g0CR5/HCIi9JbxMsRMOCChK8T56tdPj+XduBG6dNHdDePGwY4dRlcmWiAJXSFcZfRo+OorWLECtm/XY3pnz4ajR42uTLQgErpCuJKPD8yaBQUFcM89OoDDwvRY35oao6sTLYCErhDucOGFOnC/+kqPeLjnHhg2DDZtMroyYTAJXSHcafBg3debng4lJXD55XrL+AYW9RFtm4SuEO5mMsGtt+pRDo8+qsf4hodDSgqcOmV0dcLDJHSF8JROneCJJ/Q6DhMmwO9/r2e1ffCBDDFrRyR0hfC0/v3h/ff1zDZ/fz3GNzFRt4RFmyehK4RRxo/XQ8uWLYMtWyAmRu/bduyY0ZUJN5LQFcJIvr56AZ38fL1h5tKlehWzN96A2lqjqxNuIKErREsQFKSXjdy6FQYMgDvvhJEj4csvja5MuJiErhAtSVycHsv75ptQVKSD9447YP9+oysTLiKhK0RLYzLB7bfrvdoeeQT+/nfd5fDcc3rHYtGqSegK0VIFBsKSJXrh9NGjYe5cfbPt44+NrkycBwldIVq6sDD48EO9bq9ScM01cP318OOPRlcmmkFCV4jW4ppr9HKRzzyjpxZHRekt448fN7oy4QQJXSFaEz8/3c2Qnw9JSbr7ITwc/vY3mdXWSkjoCtEaBQfDypXwxRcQEqJvvF1+OeTmGl2ZaISErnCp7du3M3LkSIYMGYLVaiU7O7vusZSUFCwWC+Hh4WRlZdUdz83NJSYmBovFwuzZs2lkh2pxtjNjeV97Ta/hO2wYJCfrFc1Ey6SUauiPEE6Jj49Xa9euVUoptWbNGjV69GillFJ5eXkqNjZWnTp1Su3evVuFhoaq6upqpZRSw4YNU5s3b1a1tbUqISGh7vWNiYuLc8s5tFpHjyr14INK+fgo1bWrUsuWKVVZaXRV7ZXDXJWWrnApk8lEWVkZAMeOHSMkJASAjIwMkpKS8Pf3p3///lgsFrKzsykuLqasrIxRo0ZhMpmYNm0aq1evNvAMWrFu3eCFF+Cbb2D4cHjgAb1l0Pr1BhcmziahK1xq2bJlzJ07l759+/Lwww+TkpICgM1mo2/fvnXPM5vN2Gw2bDYbZrO53nFH0tLSsFqtWK1WSuQrtH0REZCVBatX6y3h4+P1lvF79hhdmUBCVzTD+PHjiY6OrvcnIyOD1NRUli5dSmFhIUuXLuWuu+4CsNtPazKZHB53JDk5mZycHHJycujZs6frTqqtMZn0kpE7d8JTT+kQjojQW8afOGF0de2aj9EFiNZnfQNfV6dNm8by5csBuOWWW5gxYwagW7CFhYV1zysqKiIkJASz2UxRUVG948JFOnTQi6VPmwbz5sGTT8Jf/qKnFE+ZosNZeJS0dIVLhYSE8NlnnwGwYcMGwsLCAJg4cSLp6elUVFSwZ88eCgoKGD58OMHBwQQGBrJlyxaUUqxatYpJkyYZeQptk9ms13D497+hRw89xveqq+Drr42urN2Rlq5wqVdeeYX777+f6upqOnToQFpaGgBRUVFMmTKFyMhIfHx8WLFiBd7e3gCkpqZyxx13UF5eTmJiIomJiUaeQtt2xRWQkwOvvgoLFsDQoXDvvbBokd7BWLidyV6f2llkwKRosaxWKzk5OUaX0XodOQKPPw4vvwxdu+quh+RkOP2PoTgvDvttpHtBiPaqe3f44x/1lkGDB8OsWbrle7p7SLiHhK4Q7V10NHz6Kbz7LpSW6r7epCQ468ancB0JXSGEHsUwebLeHv7xxyEjQy+k88QTeqyvcBkJXSHELwIC4LHH9Hbw114L/+//QWQk/POfsoqZi0joCiHqu+gieOcd3e3QqZOe0TZhgp5sIc6LhK4QwrGxY/WNthdf1EPNYmPhwQd1369oFgldIUTDfHzgvvv0wukzZsDy5XqjzFdfhZoao6trdSR0hRBN07Mn/OlPeqH08HC4+24YMQI2bza6slZFQlcI4ZxLLtHTif/+d9i/Hy67TK/tsG+f0ZW1ChK6QgjnmUxw2216lMPvfw9vv61bv08/DRUVRlfXoknoCiGar3NnvXTkzp0wbhzMn68nW6xZY3RlLZaErhDi/A0YoBdNz8zUN96uu05vGZ+fb3RlLY6ErhDCda6+Wi8X+fzz8J//6FbvvHlwegsnIaErhHA1Pz+YM0fvTvzb38Kzz+r+3pUrobbW6OoMJ6ErhHCPXr301vBffqlnuN1xB1x6KWzdanRlhpLQFUK41/DheizvG2/A3r3673fdBQcOGF2ZISR0hRDu5+WlW7r5+fDww/Dmm3pW29KlUFVldHUeJaErhPCcLl10H++33+quhjlz9ALqn3xidGUeI6ErhPC88HBYuxY+/FC3dK++Gm64AXbvNroyt5PQFUIYw2TS43l37IAlS2D9er1274IFcPy40dW5jYSuEMJY/v7wyCO6v3fKFFi8GAYN0ms7tMGF0yV0RbO88847REVF4eXlVW9H3pSUFCwWC+Hh4WRlZdUdz83NJSYmBovFwuzZszmzE3VFRQW33norFouFESNGsHfvXk+eimgpQkJg1SrYtAl694apU+HKK2HbNqMrcykJXdEs0dHRvP/++1x55ZXnHN+5cyfp6enk5eWRmZnJrFmzqDm95urMmTNJS0ujoKCAgoICMjMzAXjttde44IIL+PHHH3nwwQd55JFHPH4+ogW59FI9tveVV+CHHyAuDu69Fw4dMroyl5DQFc0SERFBeHh4veMZGRkkJSXh7+9P//79sVgsZGdnU1xcTFlZGaNGjcJkMjFt2jRWr15d95rp06cDcPPNN/Ppp5/WtYJFO+XtrRdMz8+H++/XC6aHhekdLKqrja7uvEjoCpey2Wz07du37u9msxmbzYbNZsNsNtc7/uvX+Pj40LVrVw4fPuzZwkXL1K2bHsv7zTe6xTt7tl7Pd8MGoytrNgld4dD48eOJjo6u9ycjI8Pha+y1UE0mk8PjDb3GnrS0NKxWK1arlZKSkqaeimjtIiNh3Tp4/309smHcOLjlFvjpJ6Mrc5qP0QWIlmv9+vVOv8ZsNlNYWFj396KiIkJCQjCbzRQVFdU7fvZrzGYz1dXVHDt2jO7du9t9/+TkZJKTkwGwWq1O1ydaMZMJbrwREhL0KmaLF8NHH+mRD/Pm6e3jWwFp6QqXmjhxIunp6VRUVLBnzx4KCgoYPnw4wcHBBAYGsmXLFpRSrFq1ikmTJtW9ZuXKlQC8++67jB071mFLVwg6doRHH9W7VkyaBH/4A0REwLvvto4hZkqphv4IYdf777+v+vTpo/z8/FRQUJCaMGFC3WNPPvmkCg0NVQMHDlRr166tO75161YVFRWlQkND1f/+7/+q2tpapZRS5eXl6uabb1YDBgxQw4YNU7t27WpSDXFxca49KdE6bdyoVGysUqDUmDFKffut0RUp1UCumlTD/zK0gn82RHtltVrrjREW7VR1NaSlwcKFcOwYzJqlW8AXXGBURQ6/qkn3ghCi9fPx0UGbnw/33AMrVughZn/+M5weJ95SSOgKIdqOCy/UgfvVVxAVpSdVWK1666AWQkJXCNH2DB4MGzdCerqeyXbFFfCb38BZI2iMIqErhGibTCa49VY9yuHRR/UY3/BwPdTs1CnDypLQFUK0bZ06wRNPwHff6XV7FyzQXQ8ffGDIEDMJXSFE+9C/v27trlsHHTroMb6Jibol7EESukKI9mX8eNi+HZYtgy1bICYGHnpIDzXzAAldIUT74+urVy/Lz9cbZi5dqjfKfOMNqK1160dL6Aoh2q+gIL1u79atMGAA3HknjByp1/N1EwldIYSIi9M7Vvz1r3pY2ciRcN99bvkoCV0hhAA9xGzqVL1bxfz5EBrqlo+RpR2FEOJsgYGQkuK2t5eWrhBCeJCErhBCeJCErhBCeJCErhBCeJCErhBCeJCErhBCeJCErhBCeJCErhBCeFBjG1MK0WKZTKZMpVSC0XUI4QwJXSGE8CDpXhBCCA+S0BVCCA+S0BVCCA+S0BVCCA+S0BVCCA/6/9h9CI+cw9CVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, 7*x +  120)  \n",
    "plt.scatter(x, y)\n",
    "a,b,a_r,b_r = gradient_descent(x[0:2],y[0:2],C=0.005,E=505620)\n",
    "plt.scatter(x[0:2],y[0:2],color='r')\n",
    "plt.plot(x, a*x +  b,color='r')\n",
    "set_spines()\n",
    "a,b"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5646b0d",
   "metadata": {},
   "source": [
    "The red line is a regression line coming from the fitting of only the first two points. <br>\n",
    "We can see that when we train the model over the first two points, that points have been feet too well,but at all the model is awful because the red line has a  high bias by the rest points.This simple example shows how we can achieved overfit model.The reasons for overfitting could have many different origins and they could be different by itself.In the above example because we train the model over too few points, the model trains the bias of data, which leads to very bad prediction for the rest points.To resolve this problem we have to involve an artificially Bias in our Loss function LSR. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f376cc80",
   "metadata": {},
   "source": [
    " $$L = \\sum_i(y_i - w_0- w_1 x_i)^2 + \\lambda(w_0^2 + w_1^2) $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ce54076",
   "metadata": {},
   "source": [
    "Therefore, ridge regression puts further constraints on the parameters, $w_j$'s, in the linear model. In this case, what we are doing is that instead of just minimizing the residual sum of squares we also have a penalty term on the $w$'s. This penalty term is $λ$ (a pre-chosen constant) times the squared norm of the $w$ vector. This means that if the $w_j$'s take on large values, the optimization function is penalized. We would prefer to take smaller $w_j$'s, or $w_j$'s that are close to zero to drive the penalty term small.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32e20a2f",
   "metadata": {},
   "source": [
    "Appling Gradient Descent over penalty Loss function we obtain  :"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12ea289e",
   "metadata": {},
   "source": [
    "\n",
    "$\\nabla_{w_1}(J(w_1,w_0)=\\frac{\\partial{L}}{\\partial{{\\alpha}}}=\\frac{\\partial{\\sum_i(y_i - w_0 - w_1 x_i)^2}}{\\partial{{w_1}}} + \\frac{\\partial{\\lambda(w_0^2 + w_1^2)  }}{w_1}  =-\\frac{2}{n}\\sum_{n=1}^{n}{(y_i-w_1*x_i - w_0)*x_i} + 2\\lambda w_1 $ <br>\n",
    "$\\nabla_{w_0}(J(w_1,w_0)=\\frac{\\partial{L}}{\\partial{{\\beta }}}=\\frac{\\partial{\\sum_i(y_i - w_0 - w_1 x_i)^2}}{\\partial{{w_0}}} + \\frac{\\partial{ \\lambda(w_0^2 + w_1^2) }}{w_1}  =- \\frac{2}{n}\\sum_{n=1}^{n}{(y_i-a*x_i - b)} + 2\\lambda w_0 $ <br> <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b376b10",
   "metadata": {},
   "source": [
    "the new gradient descent implementation :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "097863e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradient_descent(X,Y,L = 0 , C=0.1,E=100):\n",
    "    '''\n",
    "    \n",
    "    Mimumization of cost (error) function of simple linear regression\n",
    "   \n",
    "    Parrameters : \n",
    "    L penalty coeficient (L1 regularization)\n",
    "    X(array or number) : X arguments, independent variable\n",
    "    Y (array or number) : Y arguments, actual dependent variable \n",
    "    C (number) :  learning rate\n",
    "    I(int) : number of iteration(Epoch)\n",
    "    Returns :\n",
    "      number of cost function\n",
    "   '''\n",
    "   \n",
    "    a =1 \n",
    "    b = 1 \n",
    "    a_args = []\n",
    "    b_args =   []\n",
    "    n = X.size\n",
    "    \n",
    "    for i in range(E):\n",
    "        a = a + (2/n)*(np.sum((Y - a*X - b)*X))*C + L*a\n",
    "        b =  b + (2/n)*(np.sum((Y - a*X -b)))*C + L*b\n",
    "        #print(a,b)\n",
    "        a_args.append(a)\n",
    "        b_args.append(b)\n",
    "    return a,b,a_args,b_args "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "75ad9246",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADnCAYAAAC9roUQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAzOElEQVR4nO3deVzVVf748ddlEVGRTUHgqly8hsgi6hW0xcpcy1HTMtNGS5Mym76TZTnTOFmTQtuMNZVKZT+dGp2cVJoWNSvLcVTUtFLSKEHZVBAVN5Dl8/vjyFXkXta7sLyfjweP8nKXcz/g23PPeb/fR6dpGkIIIRzDxdkDEEKI1kSCrhBCOJAEXSGEcCAJukII4UASdIUQwoHcavm+pDaIJmvkyJFs2LDB2cMQwhKdtW/ITFc0WwUFBc4eghD1JkFXCCEcSIKuEEI4kARdIYRwIAm6QgjhQBJ0hRDCgSToCpvLysri1ltvJSIigsjISF577TUACgsLGTZsGD179mTYsGGcOnXK/JjExESMRiPh4eFs3LjRWUMXLdD6vTnckPQVhnmfckPSV6zfm+PU8UjQFTbn5ubGq6++yk8//cSOHTt48803SUtLIykpidtuu4309HRuu+02kpKSAEhLS2P16tUcOHCADRs28Mgjj1BeXu7kdyFagvV7c/jD2h/JOX0RDcg5fZE/rP3RqYFXgq6wuaCgIPr16weAl5cXERER5OTkkJKSwrRp0wCYNm0a69evByAlJYVJkybh4eGBwWDAaDSSmprqrOGLFuTljYe4WFr1H/CLpeW8vPGQk0YkQVfYWWZmJnv37iU+Pp7jx48TFBQEqMB84sQJAHJycujatav5MXq9npwcyzOR5ORkTCYTJpOJ/Px8+78B0azlnr5Yr9sdQYKusJtz584xYcIEFi9eTMeOHa3ez1IjfZ3OchVlQkICu3fvZvfu3XTu3NlmYxUtU7CPZ71udwQJusIuSktLmTBhAlOmTGH8+PEABAYGkpeXB0BeXh4BAQGAmtlmZWWZH5udnU1wcLDjBy1anLkjwvF0d61ym6e7K3NHhDttg02CrrA5TdOYMWMGERERzJkzx3z7mDFjWLFiBQArVqxg7Nix5ttXr15NSUkJGRkZpKenExcX55Sxi5ZlXN8QEsdHE+LjiQ4I8fEkcXw0gNM22HS1nJEmXcZEvf33v//lpptuIjo6GhcX9e/6okWLiI+PZ+LEiRw9epRu3bqxZs0a/Pz8AFi4cCHLly/Hzc2NxYsXM2rUqFpfx2QysXv3bru+F9Ey3ZD0FTkW1nUDvDz4v6E9Sc0oJNS/PY8Pu66hL2G1y5gEXdFsSdBtfdbvzeHljYfIPX2RYB9P5o4IZ1zfkHo/j2HepzUGt85eHkzop2feqF4NHarVoFtbP10hhGgSKnNuK1PAKpcEgDoF3tLyCg7kFrHz8Ek83FwoLquodh8fT3fWz76B7v7trG7mNpYEXSFEs1BTzq2loFtSVs73WWdIzTjJzoxC9hw5xYVL6vGdvTwoPXeJ8qs+6Xu6u7JgTCT7sk4z5Z2djZ5NWyNBVwjRLNSWc3vhUhl7j55m52EVZPdmnebS5dlseKAXd/XXE2fwI87gR4BXW4tLFUCjZtN1IWu6otmSNd3WxdrmVwcPN64L7MAP2Wcoq9Bw0UFksDfxlwPsgFA/fNu3adRrhPh4sm3ekPoMV9Z0hWjNbLUB5SiWxjvrlh48/580LpVXXYu9WFqOTqdj5uAw4gx+mLr74tXWvUGv64gKNgm6QrRwtW1A2SogW3ue+j6/pfE+/q991T52e3m48dtB3fndkJ5sPHCMlzceYumWXxv1HoJ9PC3OdG1ZwSZBV4gWrramL7ZYw7QW2HcfKeSjPTl1ev6swgukZhQyP2V/tfFqQMe2biy/fwDRem883K5UmTU2q+Fqc0eEV3kuuFLBZisSdIVo4Wr6yFzfjABrrD3Pqp1ZVTIEKm9/acNBYvTe7MwoJPXyl6UZ5tXOFpdhCvWr82vX9z3AlSBtz6UYCbpCtHA1fWS21RqmtftfG3DN9z9TzJBXvwHARQcVGnh7uvPQ4DDe33GE3DPFFsdbn9du6DrsuL4hdl3vlt4LQrRwNTV9sVUXLmv3t7aF7+nuykSTHg83Fyoux+UzF0v5+1e/MCQiwOp46/PazuwkVhMJukK0cNaavozrG1JjQK6Px4f2pI1r9XCiUT3wtnVzIXF8NNt+OUlJWfVMhK8P5lsdryW2eg+OInm6otmSPF3baEj2QnFpOd8dPWVej/3u6CmKS68E0PZtXJnQX88jtxjZcfikxee31v9AB2Qk3WH392Bn0vBGtDwSdB3nXEkZe46cYufhk6RmFPJ99mlKyzV0Ougd1JE4gx/xBn8GhPri38GjTs9pw0KEpkiKI4QQdXfmQim7MgvZmaGC7P7cIsorNFxddESHeDP9RgPxBj/6d/fD27NhhQiOSM9qiiToCiHIP1uiguzlvgWHjp9F06CNqwuxXX145JYexBn86NfNl/YetgkbjkjPaopkeUE0W7K80HC5py+SmlHIzgw1mz2cfx5QM83+3X3NfQv6dPWh7TWbVKJOZHlBiKbMnhtBmqZxtPACOw+rIJuaeZKsQrWW6tXWjQGhfkw0dSXe4EdUiDfuFrIQhO1I0BV2MX36dD755BMCAgLYv38/AAsWLODtt982n+K7aNEibr/9dgASExN59913cXV15fXXX2fEiBFOG7uj2bKMFVSQ/eXEucuz2EJSM05yvKgEAL/2bYgL9eOB6w3EGfyICOqIq4t9mnULyyToCru4//77efTRR5k6dWqV2x9//HGefPLJKrelpaWxevVqDhw4QG5uLkOHDuXnn3/G1bVlfay1NpttbBlreYXGT3lFpGYUsn5vDvtzz5gLDrw93bn5us6Xswv8MAZ0sNuJCKJuJOgKuxg8eDCZmZl1um9KSgqTJk3Cw8MDg8GA0WgkNTWVQYMG2XeQDlTTbLa+Zayl5RXszzljXpPdlVnI2eIyQC0kXr0RU1JazpBeAS1+c6o5kaArHOqNN95g5cqVmEwmXn31VXx9fcnJyWHgwIHm++j1enJyLB+FnZycTHJyMgD5+fkOGbMt1DSbra2dYHFpOT9kn1E5splVj50J69Se0TFBxBv8Sfz8J/MyQqXisooGNX4R9iNBVzjMrFmzmD9/Pjqdjvnz5/PEE0+wfPlyLGXQWPsInJCQQEJCAqCyF5qLmmazf7sntlq+ahtXFyKDOzJx2Xb2XXXsTK8uXtzdX0+cwZ8BBl8CvNqaH/P4v/bV67WFc0jQFQ4TGBho/v+ZM2cyevRoQM1ss7KyzN/Lzs4mODjYLmNwVrloTbPZW3sFcN/AbqxKzeJciVomuFReweafjhMV4s3Ugd2JD1PVXj7trB87Y+sG3E2wtLZFkKArHCYvL4+goCAA1q1bR1RUFABjxoxh8uTJzJkzh9zcXNLT04mLi7P56zc0S8AWwcdS9VVl1kDs85vQNHB31WHq7kt8mB9xBn/6d/elQz0KEWxZ4WXrjApxhQRdYRf33nsvW7ZsoaCgAL1ez3PPPceWLVvYt28fOp2O0NBQli1bBkBkZCQTJ06kd+/euLm58eabb9olc6EhWQK2CD7Hi4pxcdHRt5sPqRmFlF1OLXDRQXf/dkw0dSXO4Effbo0rRLBlhZctG4OLqqQiTTRb9a1Ia0hXq/o2ZdE0jexTldVeqm9B5skLgDq11hTqy4BQPwaG+REd4kMbt5oLEWx17lh92bIDWCslFWlCNGTNs7Z0Lk3TOFxwXgXZyx24Kk898PZ0J87gx30DuxNv8CciyAu3elR72eLcsYaq6VrJWm/jSNAVzYKlv+j11ZA1T2vBp6OnO7M/+I6dGYUUnFNpWp06eBBv8GOQuwtb0ws4cbaEtNwi7ogOIlrvXe/x1vfcMVt+9Ld2rW7t1VnWehtJgq5o8qzN+EovlNbreWpa87Q2e5sz7Dr+uPZHSsqrnnBw5mIpe4+e4qaenczVXoZO7UnZl2uzoFTvc8dsmBpm7VrJWm/jyZquaPKsrasWvD+Hc9mHGv381wZ1ADcXHcaADmQVXuD8pSu3t2vjyrjYEB65tQd633Z1HmtDGnNbey5Xnc5i4HVE829Z660zWdMVzVdN5bCNdfFSOS98klZt9lZWoZrG3BvXjbjLbQ4DO7a18iy1j7Uhs1BrH/En9A+psqZbebsjmn/bOhe4NZKgK5o8a3/RG9KC8FxJGbszC819C364fOyMJeUVGn8ZF2WTsTYkKNW0HGLq7ueUzazWetqDLcnygqgzZ+1aW/r47+nuSulHT/NL2vc1Pvb0hUvmwxNTMwvZn6M6cLm56IjWexNv8GfN7ixOnr9U7bEN+bhubaw1nWbb3Ej2Qp3I8oJoHGdWKFmb8b3wefWzuU6cLWZXxilzjuzBY2cBaOPmQt+uPjx6q5E4gz/9uvvQro369e/Vxctms7fWcATNuL4hLer9OJrMdEWdNMWTW00mEymbt5J6OcDuPFzI4QJ17Ey7NurYmbhQP+LD/OnT1RsPN+vVXjJ7EzYmM13ROLbcIGooTdPIPHmB1Ax1eOLBY2e5IekrQB07Exfqx6S4rsQZ/IkM7livNV+ZvQlHkaAr6sQZu9YVFRq/5J8zn1CbmlHIibOqEMG/fRvatXFlwW96E2fwJ7yLV5VjZ2TmKpoqCbqiTmzdwcpSQKw8dqbyXK/UjEJOXS6A6NKxLQPD/IkP8yPe4E+Pzu0ZkNKO+28wWHx+R6w/S2AXDSFruqLObBFkrBUiXBfoxeGCcxSXqtxbVxcd/bv5cpdJz0CDP139PKs1NrfW8MYR68+tIUtBNIqs6YrGa+y6Z3FpOS98arkQ4eCxoipBtbxC48ecM0yO70Y3/+qVXzVxxPqzlMOKhpKgK+zmfEkZ3x09xc7Daj12X9ZpLlmpIqvQABs1cXHE+nNT2FgUzZMEXVFFY5YQzlwoZfcRVem1M6OQH7NPm48Cd3fVcaOxE99nn6HQQiGCNbYsn7Vl1ZSUw4qGkqDbwtUniNZ3A+rkuRJzOa1K4SpC09ShinpfT1x0Oiouz15LyzV2HC602jegrbuLedPsarYun7UVKYcVDSUbaS1YfTd7atuAOnam2FzptTOjkF9OnAOgrbsL/br5Em9Q2QWxXX247dVvrD5XZYvAa3vj1ndjymQy8ae3U5yWQSDZC6IGVjfSJOi2YPXdxbfWtg/UWV5Hrjl2Jt7gT5zBj+gQ72rHznjFDOPCr7twbedN8Iy3ACi/eJaClBfRu58jNDSUDz/8EF9fXwASExN5/a1lnC4ux/vWmfTod2OtQczYuw/uE16UDALRFEn2QmtUn80eTdPo7OVhLj64mk4H4YFe/PbysTO9gzvWWojQfdAdnOw3mpOf/tV8v6IdawjoZSJ9w9skJSWRlJTEiy++SFpaGqtXrybzl0Pk5uYydOhQvl31TK2HUx4rKqaTZBCIZkaCbgtW02ZPRYXGwWNnzSW1qRmFFjttebi5kDg+mvH99BZfw9o68IRRt7Hqy+84edV9i3/ZybOrPgZg2rRp3HLLLbz44oukpKQwadIkPDw8MBgMGI1GUlNTGTRoUI3vz1o/XckgEE2ZBN0WzNJmj5uLDp927sQ+v4mi4jJALTfcHN6ZeIMfRRfLeG9bBnlniuu0TmktX/Xrg/k8PTKc369xQYcK9PmXzjJ9eD8AgoKCOHHiBAA5OTkMHDjQ/Hi9Xk9OTo7F10tOTiY5OVn9objI4n0kg0A0ZRJ0W6iSsnJCfD25tVdnvvzpBCVlalZYVqFx8VI5t0cHmU9EuPbYmZmDw+r8OjUtYQyPDCOsU3v2Xz7GxSfJ8jKXpX2Fa6vPKiUkJJCQkABcXtN1d5UMAtGsSNBtZqztmH+4K4uXNhyk4Pwl2ri5oGma+USE8EAvBhguZxcY/Ajo2NZmO+/1yVcNDAwkLy+PoKAg8vLyCAgIANTMNisry3y/7OxsgoODa31tn3bu/Gl8tGQQiGZFsheaEWt9C3zauVNwrup6rKuLjmmDuvO7IT3xbd+m1udp6K5/Tc8V61vK6NGj2b9/PwBz587F39+fefPmkZSURGFhIS+99BIHDhxg8uTJpKamkpuby2233UZ6enqtG2nWei8I0QRYzV6o/yFTwilOnb/E8/+x3Lfg2oALqnfBxgPHqwVcqLlvQH2N6xtC4vhoQnw80aHWhxPHR/Ovl55k0KBBHDp0CL1ez7vvvsu8efP44osv6NmzJ1988QXz5s0DIDIykokTJ9K7d29GjhzJm2++WWvAFaK5kpluE3WiqNicVZCaUcih42fr/RzWjsVuKcdoy0xXNGGSp2sr9qpCyj51wXzkTGpmIRmXj51p38aV/qF+jIkN5r1tGRZnta46HeUW/vG0tosvfQOEcB4JuvVgq+bYmqaRUXDePIvdmVFoDoId27oRZ/Bjclw34gx+RAZ3xO3ysTMhPp4W10+t9TOwtosvfQOEcB4JuvXQ0B6qFRUa6SfOkZpxkh2XA23+VcfOxBn8mHmTgTiDP726eOHiYvmTSU2NXEzd/eo8A28NJ9YK0VTJmm491HUttLxCIy23yNwcZldm1WNnKo+ciTP40aNze6s5qaJmsqYrmjBZ07UFa2uhQd5t2XPklDnI7sk8xdkSVe3V3b8dQyMCiTOoQHv1sTPr9+YwbXmqzDaFaEVkplsPlnJSXXQqJ7ayEMEY0IH4y5Ve8QZ/uni3rfNzSYes+pGZrmjCHDvT3bXwDbq+8hcCTudzwqczWU/OZ8Azj9rjpaqxll3QmKyDcyVl7DlyivQTZ+ni3dacWQAQ5O3J8MhA4g1+DAj1w7+DR53GJGdsCdE62Xymu2vhG0Q99ySepVdaBF5092D/s6/YPfBamz1a2923Nqs8c6GUXZmF5uWC/blFlFdouLroiA7xNs9kTaF+eHu6N2hM1wbcSjrgb/fEyiZXHchMVzRhjmtifsw3kC6nT1S/3SeALqeO1/fp6sVa025reayVzbwLLh87k5pRyI7DJzl0/Kz52Jk+Xb3NJyL06+ZLe4/6fTio75h8PN0pKauQZYc6kKArmjDHLS8EnM6v1+22ZK3jlaXgBirPNm7hZnPjbh3QM7ADjw+9jjiDOnamrXvjylFrGtO1M15Pd1d0OmTZQYgWzOa9F074dK7X7bZkraKqpjdZcO7KMogGZBVepJtfOwaG+Tc64NY0psoeBdf2LDht4XBGkMbcQrQUNg+6WU/O56J71c2ki+4eZD0539YvVc3cEeG0dav+liydL+Dh5oKPp5v5iPBKDW38UtOYPK8J3pXVX+P6hrBt3hAyku5g27whjOsbUmPprhCi+bP58sKAZx5lF9gke6EuGQflFRoHjxWpngUZhbi66kClyOKigz5dfRjfT8/54jJWbs+sciLC4//aZ/F1GzqrrGm8dd0YkxJdIVo2++TplpfDK6/AlCmgt3y2Vm2s7fq/MC6KsM7tzT0LdmUWcvbysTN6X8/LlV6qYXd3/3Y1VnvVdFqupWPCa1pTtXWPWsleqJ1spIkmzMFHsG/fDjfcAC4u8JvfwKxZMHSo+rMF1vJYLQVE3VWDCuvUnvgwv8vHzvgTUs+P4LZKMYP6H3cuGk+CrmjCHNzEfNAg+PVXePJJ2LYNRoyA666Dl1+GgoIqd60MfDmnL6KhMgqe+vcPFgMYqIB7//WhdOmoihS+/bkAHbp6B1yw3oD764P59W7yXZ/jzoUQrZf9y4BLSmDtWliyBLZuhTZt4O671ez3+usZlPQVeWeK6/x0jshjbUiTb5npOp7MdEUT5sTjejw84N574dtv4ccfKb5/OmXrU+DGG8kIMXLblrV0KLlg8aGWdv1rymO1lYZkENSUpSCqCg0NJTo6mtjYWEwmEwCFhYUMGzaMnj17MmzYME6dOuXkUQphH3YPuseLivn4+1z+tP5Hhn9xkl6+o4lJeI/5o36Hzs2VFza9xY63prFw4xtEnDhsfpwz81gbEkCtLVXIBphlX3/9Nfv27TPPVJOSkswHUt52220kJSU5eYRC2IddlheyCi/w96/SSc0oJPOkmsW2b+OKKbSy+5Yf0XpvPFxd+OYfn1D46uuMSvuWtmWX2BPciw9No7nhDw8zZmCPas/tqI/xkkFgP6GhoezevZtOnTqZbwsPD2fLli3m49lvueUWDh2q+dOLLC+IJsyx2Qu5py8y6rWtDAj1Y+Dl7ILeQR3Zm/SWxfzd9XtzWLpuFzds+5RpP2ygW0E2+PvDAw/AQw+B0Wh+bmmJ2PwZDAZ8fX3R6XQ89NBDJCQk4OPjw+nTp8338fX1tbjEkJycTHJyMgD5+fkcOXLEUcMWoj6s56pqmlbTV4OVl1dU+XPqC3/XLrh7aBqYvy64e2ipL/y96gMrKjRt82ZNu+suTXNzU/cdNkzT1q7VtNJSTdM0bd132dr1iV9qoU9/ol2f+KW27rvsxgxVOFhOTo6maZp2/PhxLSYmRvvmm280b2/vKvfx8fGp9Xn69+9vj+EJYQtW46rDmpg3qPtYXh688w68/TZkZUFwMMycqb5CZFbbEixYsIAOHTrw9ttvy/KCaEmcmL1wWYO6jwUFwfz5cPgwpKRATAw8/zx07w7jx8OmTVBhqbOCaKrOnz/P2bNnzf+/adMmoqKiGDNmDCtWrABgxYoVjB071pnDFMJumvZM15LDh2HZMli+XBVaGI1q3feBB9Q6sGjSDh8+zJ133glAWVkZkydP5plnnuHkyZNMnDiRo0eP0q1bN9asWYOfn1+NzyUzXdGEObgM2AKbnyhRUgIffaSKLv77X5UPPHGiKroYOBDkhN0WT4KuaMKcv7ww4JlH2f/sKxzzCaACHcd8Ahp3hI+HB0yerKrcfvgBZsyA9evh+uuhb19YuhQuf4wVQoimomWdBnzuHPzzn2r2u28feHnBffep2W90tLNHJ2xMZrqiCXP+TNchOnSAhAT47jvV6ezOO9Xab0wM3HgjvP8+FNe9z4MQQthaywq6lXQ6ta67YgXk5sKrr8KJE/Db36r+vk89pbqgCSGEg7XMoHs1Pz+YMwcOHoQvvoCbb4a//lVlPYwYodaBy8qcPUohRCvR8oNuJRcX1Uj9o4/gyBF47jk4cEAtQYSGqj/n5Dh7lEKIFq71BN2rhYTAn/8MmZmwbh1ERsKCBaroYsIENSOWogshhB20zqBbyc0Nxo2DjRshPV0tQ3zzDQwfDr16qbXgkyedPUohRAvSuoPu1YxGeOklyM6Gf/wDOndWxw2FhMC0abBjh2rTI4QQjSBB91pt26rc3m3b4PvvYfp0ddzQoEGq6GLZMpUPLIQQDSBBtyYxMfDWWyrtbMkSNdN9+GHV7Wz2bNi/39kjFEI0MxJ068LLSwXbffvgf/9T68Dvvquq3G66CT74QPWCEEKIWkjQrQ+dTi0zrFyp1n5ffln1/L3vPlV08fTTqguaEEJYIUG3oTp1UhttP/+s+vredJPKdjAaYeRI1f9Xii6EENeQoNtYLi4wbJjabDtyBJ59Vq31jhsHBgP85S9qNiyEEEjQta2QEBV0MzNVEO7dWxVhdOsGd90FX34paWdCtHISdO3BzU2VF1cWXfz+97BliypD7tUL/vY3KCx09iiFEE4gQdfejEa14VZZdNGpk6p8CwmB+++HnTtl9nvZhg0bCA8Px2g0kpSU5OzhCGEXEnQd5eqii337VMD96CPVgrJ/f3Xi8fnzzh6l05SXlzN79mw+//xz0tLSWLVqFWlpac4elhA2J0HXGfr0UcUWubmq+KKsTDVfDw6GRx9V3c9amdTUVIxGI2FhYbRp04ZJkyaRkpLi7GEJYXMSdJ3Jy0sdJfT992oGPGaMmvFGRcHgwbBqVaspusjJyaFr167mP+v1enKk1aZogWo8I23kyJFaQUFBg588Pz+fzp07N/jx9tKkx+Xrq46WLyhQAdfNTa0Dd+qkDuN01rjsfL1OnTpFUVER3bt3B+DkyZOcP3+ebt26VRtL5e9kSUkJsbGxdh1XQzTp3y8ZV501Zlx79uzZqGnaSIvf1DStpq9G6d+/f2Ofwi6axbjKyzVt40ZNGzdO01xcNE2n07RRozQtJUXTysqcNy47+d///qcNHz7c/OdFixZpixYtqvEx7dq1s/ewGqRZ/H41IS10XFbjqiwvNFUuLqqv77p1quhi/ny1ATd2bIssuhgwYADp6elkZGRw6dIlVq9ezZgxY5w9LCFsToJuc6DXq+OEjhxRGQ/h4VeKLu6+G776qtmnnbm5ufHGG28wYsQIIiIimDhxIpGRkc4elhA252bPJ09ISLDn0zdYsx2XuzuMH6++fv5Z9fZ97z34979VIH74YdVw3dfXseOykdtvv53bb7+9zvfv1KmTHUfTcM3298tJWtu4atxIA5r39Kk1uHgR1qxRKWg7dqh84EmTVFbEgAGqM1oLZTKZ2L17t7OHIYQlVv/iyfJCc+fpCVOnwvbtsHevmumuWQPx8WAywTvvtOqiCyGamkYF3TVr1hAZGYmLi0u1GUdiYiJGo5Hw8HA2btxo8fGFhYUMGzaMnj17MmzYME6dOtWY4Vh1zz33EBsbS2xsLKGhoVbTjEJDQ4mOjiY2NhaTyWSXsVxtwYIFhISEmMf22WefWbxfnctjY2Nh6VJVdPHmm3DpEsycqUqOH3sM6ljhNXfuXHr16kVMTAx33nknp0+ftng/R1yv2t67pmk89thjGI1GYmJi+O677+wyjmtlZWVx6623EhERQWRkJK+99lq1+2zZsgVvb2/zz/f55593yNhq+7k445odOnTIfB1iY2Pp2LEjixcvrnIfR12v6dOnExAQQFRUlPm2usYim5Sq15TaUFtORFpamnbw4EHt5ptv1nbt2mW+/cCBA1pMTIxWXFysHT58WAsLC9PKLKQ5zZ07V0tMTNQ0TdMSExO1p556qhEZGnUzZ84c7bnnnrP4ve7du2v5+fl2H0OlZ599Vnv55ZdrvE9ZWZkWFham/frrr1pJSYkWExOjHThwoG4vUFGhaVu3atqUKZrWpo2mgaYNHqxpq1ZpWkmJ1Ydt3LhRKy0t1TRN05566imrPxd7X6/a3nv//v21Tz/9VBs5cqRWUVGhbd++XYuLi7PbeK6Wm5ur7dmzR9M0TSsqKtJ69uxZ7efy9ddfa3fccYdDxnO12n4uzrpmlcrKyrTAwEAtMzOzyu2Oul7ffPONtmfPHi0yMtJ8W11iUT3/LtonZSwiIoLw8PBqt6ekpDBp0iQ8PDwwGAwYjUZSU1Mt3m/atGkATJs2jfXr1zdmOLXSNI0PP/yQe++9166vY0uNKo/V6eDGG+H991XDnRdfVP+9917o2hX++EfVhvIaw4cPx81N7bEOHDiQ7OxsG76juqvLe09JSWHq1KnodDoGDhzI6dOnyXNAKl1QUBD9+vUDwMvLi4iIiGZTQeesa1bpyy+/pEePHuZCGEcbPHgwfn5+VW6rSyyyVam6XdZ061rSefz4cYKCggD1S3zixAl7DMds69atBAYG0rNnT4vf1+l0DB8+nP79+5OcnGzXsVR64403iImJYfr06RY/0tisPLZzZ3jqKdVqcsMGdezQiy9CWBjccQd88gmUl1d72PLlyxk1apTFp7T39arLe28K5cOZmZns3buX+Pj4at/bvn07ffr0YdSoURxwUE+N2n4uzr5mq1evtjrxccb1grrFIltdt1pTxoYOHcqxY8eq3b5w4ULGjh1r8TGahYwInZ130esyzlWrVtU4y922bRvBwcGcOHGCYcOG0atXLwYPHmy3cc2aNYv58+ej0+mYP38+TzzxBMuXL69yP5tfSxcXGDGCoS+/DEYjE06dYsKmTQR89hm57u6cmTiRiFdegS5dWLhwIW5ubkyZMsXiU9njel2tLu/dGb9rVzt37hwTJkxg8eLFdOzYscr3+vXrx5EjR+jQoQOfffYZ48aNIz093e5jqu3n4sxrdunSJT7++GMSExOrfc9Z16uubHXdag26mzdvrveT6vV6srKyzH/Ozs4mODi42v0CAwPJy8sjKCiIvLw8AgIC6v1adR1nWVkZa9euZc+ePVbvUznGgIAA7rzzTlJTUxsdROp6/WbOnMno0aOr3V7Xa9mocZWWwscfE7xkCcEffAD/+hcZ/fqRc/YsH+zaZfUXyx7X62p1ee/2uj51UVpayoQJE5gyZQrjx4+v9v2rg/Dtt9/OI488QkFBgd3zi2v7uTjzmn3++ef069ePwMDAat9z1vWCusUiW103uywvjBkzhtWrV1NSUkJGRgbp6enExcVZvN+KFSsAWLFihdWZsy1s3ryZXr16odfrLX7//PnznD171vz/mzZtqrK7aQ9Xr6OtW7fO4us5pDzW3R0mTIDNm+HQITLGjMFv927e+ukn2plM8NprcE0GgyOuV13e+5gxY1i5ciWaprFjxw68vb3NHxPtSdM0ZsyYQUREBHPmzLF4n2PHjplnR6mpqVRUVODv72/XcdXl5+KsawY1f9p0xvWqVJdYZLO/izXtstW2C7h27VotJCREa9OmjRYQEFClYckLL7yghYWFadddd5322WefmW+fMWOGOdOhoKBAGzJkiGY0GrUhQ4ZoJ0+erO0lG2zatGnakiVLqtyWk5OjjRo1StM0Tfv111+1mJgYLSYmRuvdu7f2wgsv2G0sle677z4tKipKi46O1n7zm99oubm51calaWq3uWfPnlpYWJhDxtWjRw/NGBKi/alrV+37du1U1oOnp3Zu0iTtseuv1zTNcdfL0ntfsmSJtmTJEq1///5aRUWF9sgjj2hhYWFaVFRUlSwae9q6dasGaNHR0VqfPn20Pn36aJ9++ql5bJqmaX//+9+13r17azExMVp8fLy2bds2u4/L2s/l6nE565qdP39e8/Pz006fPm2+zRnXa9KkSVqXLl00Nzc3LSQkRHvnnXesxqJG/F20GlelIk3Ubu9eVfH2wQdw4YI66WLWLFX51r6904YlFWmiCZOKNNEIfftCcrIqunjjDSguhgcfvFJ08dNPzh6hEM2GBF1Rd97eMHs2/PgjfPutSjVbtkwdNX/rrfCvf6kqOCGEVRJ0Rf3pdHDTTWq5ISsLkpJU28lJk2osuhBCSNAVjRUQAE8/Db/8Ap99pk43riy6GD0aPv3UYtGFEK2VBF1hGy4uMGoUpKRARgY88wzs2aMCb48esGgRHD/u7FEK4XQSdIXtdeumjhM6ehQ+/FAF3WeeUUsPkybBli3N/qQLIRpKgq6wH3d3dZzQl1/CwYNqE27jRrXpFhkJr79erehCiJZOgq5wjPBwFnh708PTkz937cqPR47A//2fSjt78EHemz271v7LQrQEEnSFQ82aM4fnjx4l+vx5teY7ZQoV//wnD7z1Fj/7+PDf6dOZ8/DDlMvmm2ihJOgK5+nXD5KTWTx3Lpt+8xtcLl6k87x57MjO5sTkyWpJQogWRoKucChL/YMPnzzJiYkTYf9++PZbDnTrRuBHH0FEhFr//fBDc9FFcnIyJpMJk8lEfn6+M9+KEA0ivReETdXUP3jgwIF06tTJ3D84Ly+P5cuXM3v2bAYNGsR9990HwIwZM7jzhhsYfeKEqnjLzITAQFV6PHMmXD5xQHoviCbMau+FWvvpClEfDekfbKlPqX9EBEyfDnPnwqZNquFOYqL6uv121XBHiGZIlheEw1jrH1xj/2VXV1V08fHHqujiD3+AXbtU34cff1RB2M7HPAlhS7K8IBzmt7/9Lfv27UOn0xEaGsqyZcvMzbMXLlzI8uXLcXNzY/HixVbPZQPU+u769ZgefJDdZ89eacL+8MMweLDqDSGEc1n9JZSgK5otk8nE7vffh6VLYcUKVWjRu7cKvlOnqq5oQjiH9NMVLVSvXrB4MeTkwPLlqqn6Y49BcLDadKvhTDwhnEGCrmgZ2rWDBx6A1FTYvRvuvVe1njSZIC4O3ntPnXohhJNJ0BUtT//+8M476qSL11+Hc+dUJkRICPz+91J0IZxKgq5ouXx84He/gwMHVGezESPgrbdU0cWQIbBmjTp+XggHkqArWj6dDm6+GVavViddLFoEhw/DxImqDeWf/qTaUArhABJ0ResSGKhyfX/9FT75RC1FLFoEBgOMGQOffy4nXQi7kqArWidXV1Vg8cknquhi3jzYuVNVuxmN6tw3KboQdiBBV4ju3WHhQrX0sHq1+vMf/gB6PUyeDFu3ykkXwmYk6ApRqU0buOcetemWlqb6O3z2mapyi46GN96AM2ecPUrRzEnQFcKSiAh47TWVdvbuu+DpqTIhQkIgIQH27nX2CEUzJUFXiJq0a6dyfHftUl+TJsH776sG7AMHwv/7f3DxorNHKZoRCbpC1JXJdKXo4rXX1FLDAw+o2e+cOfDzz84eoWgGJOgKUV8+Pqq/Q1oafP01DB+u1nvDw2HoUPjoIym6EFZJ0BWioXQ6uOWWK0UXCxdCejrcdZfKgPjzn9XtQlxFgq4QthAYCH/8o6p0+89/oG9feOEFCA2FceNgwwaoqHD2KEUTIEFXCFtydYXRo+HTT1UAfvpp2L5dnX7Rsye89BLIgZqtmgRdIewlNFSVGGdlwapVqtji6afVf6dMgf/+V4ouWiEJusKm1qxZQ2RkJC4uLtVO6k1MTMRoNBIeHs7GjRvNt+/Zs4fo6GiMRiOPPfYYtZxm0vy0aaNSzb75Rh0z/9BDqvz4ppsgJkZ1PisqcvYohYNI0BU2FRUVxdq1axk8eHCV29PS0li9ejUHDhxgw4YNPPLII5Rfbiwza9YskpOTSU9PJz09nQ0bNjhj6I4RGal6/Obmwttvq4A8e7Y66eKhh2DfPmePUNiZBF1hUxEREYSHh1e7PSUlhUmTJuHh4YHBYMBoNJKamkpeXh5FRUUMGjQInU7H1KlTWb9+veMH7mjt28ODD6pTLlJTVZvJlSvVBtygQer/peiiRZKgKxwiJyeHrl27mv+s1+vJyckhJycHvV5f7fZWQ6eDAQPU+W65ufC3v8GpUzBtmlr7feIJlYYmWgwJuqLehg4dSlRUVLWvlJQUq4+xtE6r0+ms3m5NcnIyJpMJk8lEfkvLAvD1VccJ/fQTfPkl3HabWoq47jopumhB3Jw9ANH8bN68ud6P0ev1ZF1VKJCdnU1wcDB6vZ7s7Oxqt1uTkJBAQkICoI5gb5F0OnWc0JAhkJenGu4kJ6uii6AgdcrxzJlqJiyaHZnpCocYM2YMq1evpqSkhIyMDNLT04mLiyMoKAgvLy927NiBpmmsXLmSsWPHOnu4TUdQkDpO6PBhSEmBPn3gL39R6Wh33gkbN0rRRTMjQVfY1Lp169Dr9Wzfvp077riDESNGABAZGcnEiRPp3bs3I0eO5M0338TV1RWAJUuW8OCDD2I0GunRowejRo1y5ltomtzcrhwn9Msv8OSTKs935Ei1/PDyy1BQ4OxRijrQ1ZIT2cISJkVLYjKZquUCtyolJbB2LSxZok63aNMG7r5bNV+//nq1TCGcxerFl5muEM2Vhwfcey98+y38+KNa5/34Y7jxRrUMsWQJnD3r7FGKa0jQFaIliIpS7SVzc9Wmm5sbPPKIKrqYNQu+/97ZIxSXSdAVoiXp0EHNePfsUacb33WXOt0iNlYtOfzjH1Bc7OxRtmoSdIVoiXQ6iIuD996DnBz461/h5EmYOlWlms2dqzbkhMNJ0BWipfPzg8cfh4MHYfNm1Xj9b39TrSaHD4d166CszNmjbDUk6ArRWuh0qsrt3/+Go0fh+edV9dv48eqkiwUL1KxY2JUEXSFao+BgmD8fMjJU0UVMjArC3burIPzFF1J0YScSdIVoza4tunjiCZXzO3y4OmjzlVfUWrCwGQm6QgglLAxefBGys+GDD6BLF7XhFhKiNuD+9z856cIGJOgKIary8IDJk9WM94cfYMYMWL8ebrhBpZ4tXSpFF40gQVcIYV10NLz5ptpgW7YMXFxUsUVl0cUPPzh7hM2OBF0hRO28vCAhAb77Tp1uPH68ygHu00eKLupJgq4Qou50Ohg4EFasULPfV19V3c2k6KLOJOgKIRrG3x/mzFFFF198UbXoYsQItQ4sRRfVSNAVQjSOi4s6Tujqoou0NNVkPTQUnntONeIRgARdIYQtXV10sX696n62YAF06wYTJqgy5FZedCFBVwhhe25uMHYsbNhwpeji229h2DDo1UutBbfSogsJukII++rR40rRxfvvQ0CAOm4oJEQdNb99e6squpCgK2xqzZo1REZG4uLiUuUonczMTDw9PYmNjSU2NpaHH37Y/L09e/YQHR2N0Wjkscces3gsu2gBPDxgyhR1ttv338P06eq4oeuvh759VR7wuXPOHqXdSdAVNhUVFcXatWsZPHhwte/16NGDffv2sW/fPpYuXWq+fdasWSQnJ5Oenk56ejobNmxw5JCFM8TEwFtvqQ22pUvVTPfhh9Wa8OzZ6vihFkqCrrCpiIgIwsPD63z/vLw8ioqKGDRoEDqdjqlTp7J+/Xr7DVA0LV5e8NBDsG+f6u0wbhy8+64KyjfeqHpAlJQ4e5Q2JUFXOExGRgZ9+/bl5ptvZuvWrQDk5OSg1+vN99Hr9eTU0NM1OTkZk8mEyWQiPz/f7mMWDqLTwaBBsHKlKrp45RU4fhzuu08VXTz9NBw+7OxR2oQEXVFvQ4cOJSoqqtpXSkqK1ccEBQVx9OhR9u7dy1//+lcmT55MUVGRxfVbXQ1HhyckJLB79252795N586dbfJ+RBPj76+yHQ4dgk2b4KabVLZDjx4wcqTq/9uMiy7cnD0A0fxs3ry53o/x8PDAw8MDgP79+9OjRw9+/vln9Ho92dnZ5vtlZ2cTHBxss7GKZszFRaWYDRumZr/vvKNOOh43Drp2VQdwPvggBAU5e6T1IjNd4RD5+fmUl5cDcPjwYdLT0wkLCyMoKAgvLy927NiBpmmsXLmSsWPHOnm0oskJCYFnn4UjR1TGQ0QE/PnPquji7rvhyy+bTdqZBF1hU+vWrUOv17N9+3buuOMORowYAcC3335LTEwMffr04a677mLp0qX4+fkBsGTJEh588EGMRiM9evRg1KhRznwLoilzc1PlxRs3Qno6/P738PXXqgy5Vy/V+6Gw0NmjrJGulpzI5vFPh2iVTCZTlVxg0UoVF6u+D0uWqAyItm3hnntUv9+4OLVJ53hWX1RmukKI5q1tW5XlsG2bKrq4/3746CPVgrJ/f7UO3ISKLiToCiFajpgYNePNzVXFF2VlKg+4suhi/35nj1CCrhCiBfLyUssL33+vZsBjx6rsh+holYL2z386rehCgq4QouXS6a4cJ5STAy+9BHl5qgeEk4ouJOgKIVqHTp3UcUI//6yyHyqLLoxGGDUKPv4YLqc12pMEXSFE6+LiAsOHq3zfzEyV7/vDD2oJwmCAF16AY8fs9/J2e2YhhGjq9Hp1skVmpsp4CA9XJ1907QqPPmqXl5QyYCGEcHdXx8qPH6+KLpYtgy5d7PJSEnSFEOJqPXuqLmd2IssLQgjhQBJ0hRDCgSToCiGEA0nQFUIIB5KgK4QQDiRBVwghHEiCrhBCOJAEXSGEcKDaTo4QosnS6XQbNE0b6exxCFEfEnSFEMKBZHlBCCEcSIKuEEI4kARdIYRwIAm6QgjhQBJ0hRDCgf4/zSbcdilnzKAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, 7*x +  120)  #L = 0.0028\n",
    "plt.scatter(x, y)\n",
    "a,b,a_r,b_r = gradient_descent(x[0:2],y[0:2],L=0.00258 ,C=0.005,E=1800)\n",
    "plt.scatter(x[0:2],y[0:2],color='r')\n",
    "plt.plot(x, a*x +  b,color='r')\n",
    "set_spines()\n",
    "#a,b\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a92164a3",
   "metadata": {},
   "source": [
    "<h1 id='m-linear-r'> Multiple Linear Regression from scratch</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2a3dbf3",
   "metadata": {},
   "source": [
    "Multiple Linear Regression also known as multiple regression, is a statistical technique that uses several explanatory variable to predict the outcome of response variable.In essence, MLR is an extension of ordinary least-squared regression.The equation that we are searching for is defined by : <br> <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84a5089b",
   "metadata": {},
   "source": [
    "$$\\hat{ y}^i= h(x{^i}) =  \\omega_0 + \\omega_1 x^i_{1} + \\omega_2 x_{2}^i+ ...\\omega_p x_{p}^i +ε^i  $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ef4220b",
   "metadata": {},
   "source": [
    "where , $i$ is number of obeservation,\n",
    "$y_i$ dependet(target) value,\n",
    "$x_p $ feature values,\n",
    "$\\omega_0$ intercept,\n",
    "$\\omega_{p}$ slope coeff.(weight) for each explanority variable $\\epsilon_i$  is referred to as the error.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e0d2634",
   "metadata": {},
   "source": [
    "The above hypotesis can also be represented by  \n",
    " $$\\hat{Y} = X\\Omega + \\epsilon$$  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9b6177a",
   "metadata": {},
   "source": [
    "Consider an experiment in which p characteristics of n samples are measured. The data from this experiment are\n",
    "denoted $X$.\n",
    " where  $$X = \\begin{bmatrix} \n",
    "1 &x_1^1 & x_1^2 & x_1^p \\\\\n",
    "\\vdots & \\ddots & \\vdots & \\vdots \\\\\n",
    "1& x_n^1 & \\dots & x_n^p \\\\\n",
    "\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae729f8b",
   "metadata": {},
   "source": [
    "$\\Omega = \\begin{bmatrix} \n",
    "\\omega_1 \\\\\n",
    "\\vdots & \\\\\n",
    "\\omega_p \\\\\n",
    "\\end{bmatrix} $ \n",
    "is the regression parameter or weights,or predictors. The parameter $\\omega_j$ , $j = 1, . . . , p,$ represents\n",
    "the effect size of covariate j on the response. That is, for each unit change in covariate j (while keeping the\n",
    "other covariates fixed) the observed change in the response is equal to $\\omega_j$ ."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e472d6c",
   "metadata": {},
   "source": [
    "$\\epsilon = \\begin{bmatrix} \n",
    "\\epsilon_1 \\\\\n",
    "\\vdots & \\\\\n",
    "\\epsilon_p \\\\\n",
    "\\end{bmatrix} $ are the bias normaly distributed as $\\epsilon_ ∼ N (\\omega_p, σ2)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53b22d44",
   "metadata": {},
   "source": [
    "To define and measure the error of our model we define the Lost function as the sum of the squares of the residuals as in simple linear regression. The Lost is the same :\n",
    "    $$ \\hspace{1cm} J(\\Omega) = \\frac{1}{2m}\\sum_{i=0}^m(h(x^i) - y^i)^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8664334e",
   "metadata": {},
   "source": [
    "We have to initialize the model parameter with some random values(random initialization).To use Gradient Descent we need to measure how the lost function changes with change in it's parametes.Therefore we compute the partial derivatives of lost finction $\\hspace{1cm}  J(\\omega_0,\\omega_1,...,\\omega_n)$ <br> <br>\n",
    "$$ \\frac{\\partial{J(\\Omega)}}{\\partial{\\omega_j}} = \\frac{1}{m}\\sum_{i=0}^m(h(x^i) - y^i)x_j^i$$\n",
    "In more compatable form using matrix in order to be implemnted using <mark>NUMPY</mark>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "612909f0",
   "metadata": {},
   "source": [
    "Using the <a href='https://en.wikipedia.org/wiki/Einstein_notation' > Eistein notaion </a> we can rewrite eq. 4) <br> <br> <br>\n",
    "$$ \\frac{\\partial{J(\\omega_k)}}{\\partial{\\omega_j}} = \\frac{1}{m}(\\omega_p x_p^i - y^i)x_j^i $$ in matrix form <br> <br> <br>\n",
    " $$5) \\hspace{1cm}  \\vec{\\nabla} J(\\Omega)=((X\\Omega^T- Y)^T.X)^T $$ <br> <br>\n",
    "  applying the rule  $(AB)^T$ the eq can be reformed\n",
    "   $$6) \\hspace{1cm}  \\vec{\\nabla} J(\\Omega)= X^T(X\\Omega^T- Y) $$  <br> <br>\n",
    " $$ where  \\hspace{1cm} \\vec{\\nabla} = \\frac{\\partial}{\\partial{\\omega_j}\\vec{e_j}}$$ <br> <br>\n",
    "\n",
    " eq. 6 is extremely comfortable because it can be implemented very simple in <mark>NUMPY</mark> which is many times faster than common Python\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07b58705",
   "metadata": {},
   "source": [
    "The Gradient Descent alg finaly will looks like that : <br>\n",
    " $$\\theta = \\theta - h\\vec{\\nabla} J(\\Omega) $$ h is learnig rate\n",
    "$h$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a543208",
   "metadata": {},
   "source": [
    "Calcaulation of accuracy of Multilinear regression by $R^2$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4107fe6b",
   "metadata": {},
   "source": [
    "We will be using Root mean squared error(RMSE) of Determination($R^2$ score)  to evaluate our model.<br>\n",
    "RMSE is a square root of average of sum of suares of residualas.\n",
    "RMSE is difined by : <br>\n",
    "$$ RMSE =\\sqrt{\\frac{1}{2}\\sum_{i=1}^m(h(x^i)- y ^i)} $$\n",
    "R² score or the coefficient of determination explains how much the total variance of the dependent variable can be reduced by using the least square regression.<br>\n",
    "$R^2$ is determed by \n",
    "    $$R^2 = 1 - \\frac{SS_r}{SS_t}$$\n",
    "\n",
    "$SS_t$  is the total sum of errors if we take the mean of the observed values as the predicted value.\n",
    "$$SS_t =\\sum_{i=1}^m( y ^i -\\bar{y})^2 $$ <Br>\n",
    "    $$ SS_r =\\sum_{i=1}^m(h(x^i)- y ^i)^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6fdabe9",
   "metadata": {},
   "source": [
    "<h2>Implementation of Multiple Linear regression in NUMPY</h2>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "7997930a",
   "metadata": {},
   "outputs": [],
   "source": [
    "class MultipleLinearRegression:\n",
    "    \"\"\"\n",
    "    Multiple Linear regression\n",
    "\n",
    "    Parameters\n",
    "    ..........\n",
    "\n",
    "    C : float\n",
    "        learning rate\n",
    "    I : int\n",
    "        number of iterations (Epochs)\n",
    "    Attributes\n",
    "    ..........\n",
    "     w_ : weights (Predictors)\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, C=0.001, I=50):\n",
    "        self.C = C\n",
    "        self.I = I\n",
    "        self.w_ = None\n",
    "\n",
    "    def fit(self, X, Y , L2=0):\n",
    "        \"\"\"Fit tthe training data\n",
    "\n",
    "        Parameters\n",
    "        ..........\n",
    "\n",
    "        X : array, shape = [N_samples,n_features]\n",
    "            Training samples\n",
    "        Y  : array, shape = [n_samples, n_target_values]\n",
    "\n",
    "        Returns\n",
    "        .......\n",
    "        self : object\n",
    "\n",
    "        \"\"\"\n",
    "\n",
    "        X, Y = self._validate(X, Y)\n",
    "        self.w_ = np.ones((X.shape[1], 1))  # inicialized predictors\n",
    "        M = X.shape[0]  # numbers of samples\n",
    "        for _ in range(self.I):\n",
    "            gradient = self.C * (1 / M) *( X.T.dot((X.dot(self.w_) - Y)) +  np.abs(L2*self.w_))\n",
    "            #print( L*self.w_)\n",
    "            if np.isnan(np.sum(gradient)) or np.isinf(np.sum(gradient)):\n",
    "                return self\n",
    "            self.w_ = self.w_ - gradient\n",
    "\n",
    "        return self\n",
    "\n",
    "    def predict(self, x):\n",
    "        \"\"\" Predicts the value after the model has been trained.\n",
    "        Parameters\n",
    "        ----------\n",
    "        x : array-like, shape = [n_samples, n_features]\n",
    "            Test samples\n",
    "        Returns\n",
    "        -------\n",
    "        Predicted value\n",
    "        \"\"\"\n",
    "        x = np.append(np.ones((x.shape[0], 1)), x, axis=1)\n",
    "        return np.dot(x, self.w_)\n",
    "\n",
    "    def score(self, X, Y):\n",
    "        \"\"\"Calculation of accuracy using (R^2 score)\n",
    "             x : array-like, shape = [n_samples, n_features]\n",
    "             y : array-like, shape = [n_samples, n_features]\n",
    "            Test samples\n",
    "        Returns\n",
    "        float : score\n",
    "        -------\n",
    "        Predicted value\n",
    "\n",
    "        \"\"\"\n",
    "        # X = np.append(np.ones((X.shape[0], 1)), X, axis=1)\n",
    "        # Y = np.array(Y).reshape(X.shape[0], 1)\n",
    "        X, Y = self._validate(X, Y)\n",
    "        y_prime = np.dot(X, self.w_)\n",
    "        ssr = np.sum((y_prime - Y) ** 2)\n",
    "        sst = np.sum((Y - np.mean(y_prime)) ** 2)\n",
    "        r2_score = 1 - (ssr / sst)\n",
    "        return r2_score\n",
    "\n",
    "    def _validate(self, X, y):\n",
    "        '''Added tow with one number to X data\n",
    "        reshape Y data\n",
    "\n",
    "\n",
    "        :param X: array-like, shape = [n_samples, n_features]\n",
    "        :param y: array-like, shape = [n_samples, n_target_values]\n",
    "        :return: validated X,Y\n",
    "        '''\n",
    "        return np.append(np.ones((X.shape[0], 1)), X, axis=1), \\\n",
    "               np.array(y).reshape(y.shape[0], 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36a5a3a1",
   "metadata": {},
   "source": [
    "<h2>Perform out implementation in insurence.csv data set</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce3a99ae",
   "metadata": {},
   "source": [
    "We will test our implementation over insurence.csv data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "3f85c821",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>bmi</th>\n",
       "      <th>children</th>\n",
       "      <th>smoker</th>\n",
       "      <th>region</th>\n",
       "      <th>charges</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19</td>\n",
       "      <td>female</td>\n",
       "      <td>27.900</td>\n",
       "      <td>0</td>\n",
       "      <td>yes</td>\n",
       "      <td>southwest</td>\n",
       "      <td>16884.92400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>18</td>\n",
       "      <td>male</td>\n",
       "      <td>33.770</td>\n",
       "      <td>1</td>\n",
       "      <td>no</td>\n",
       "      <td>southeast</td>\n",
       "      <td>1725.55230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>male</td>\n",
       "      <td>33.000</td>\n",
       "      <td>3</td>\n",
       "      <td>no</td>\n",
       "      <td>southeast</td>\n",
       "      <td>4449.46200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>33</td>\n",
       "      <td>male</td>\n",
       "      <td>22.705</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>northwest</td>\n",
       "      <td>21984.47061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>32</td>\n",
       "      <td>male</td>\n",
       "      <td>28.880</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>northwest</td>\n",
       "      <td>3866.85520</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age     sex     bmi  children smoker     region      charges\n",
       "0   19  female  27.900         0    yes  southwest  16884.92400\n",
       "1   18    male  33.770         1     no  southeast   1725.55230\n",
       "2   28    male  33.000         3     no  southeast   4449.46200\n",
       "3   33    male  22.705         0     no  northwest  21984.47061\n",
       "4   32    male  28.880         0     no  northwest   3866.85520"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"../../../resources/data/insurance.csv\")  \n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c156c7b",
   "metadata": {},
   "source": [
    "Converting Categories to Numbers. The linear regression can be performed only on numbers, so we should convert these categorical features into numbers. To do that, we can make use of a function called get_dummies. So let’s convert the “sex,” “smoker,” and “region” columns into numerically represented features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "1eb5d87b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>bmi</th>\n",
       "      <th>children</th>\n",
       "      <th>charges</th>\n",
       "      <th>sex_male</th>\n",
       "      <th>smoker_yes</th>\n",
       "      <th>region_northwest</th>\n",
       "      <th>region_southeast</th>\n",
       "      <th>region_southwest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19</td>\n",
       "      <td>27.900</td>\n",
       "      <td>0</td>\n",
       "      <td>16884.92400</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>18</td>\n",
       "      <td>33.770</td>\n",
       "      <td>1</td>\n",
       "      <td>1725.55230</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>33.000</td>\n",
       "      <td>3</td>\n",
       "      <td>4449.46200</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>33</td>\n",
       "      <td>22.705</td>\n",
       "      <td>0</td>\n",
       "      <td>21984.47061</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>32</td>\n",
       "      <td>28.880</td>\n",
       "      <td>0</td>\n",
       "      <td>3866.85520</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age     bmi  children      charges  sex_male  smoker_yes  region_northwest  \\\n",
       "0   19  27.900         0  16884.92400         0           1                 0   \n",
       "1   18  33.770         1   1725.55230         1           0                 0   \n",
       "2   28  33.000         3   4449.46200         1           0                 0   \n",
       "3   33  22.705         0  21984.47061         1           0                 1   \n",
       "4   32  28.880         0   3866.85520         1           0                 1   \n",
       "\n",
       "   region_southeast  region_southwest  \n",
       "0                 0                 1  \n",
       "1                 1                 0  \n",
       "2                 1                 0  \n",
       "3                 0                 0  \n",
       "4                 0                 0  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols = ['sex', 'smoker', 'region']\n",
    "new_df = pd.get_dummies(df, cols, drop_first= True)\n",
    "new_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc8cfea1",
   "metadata": {},
   "source": [
    "Now, let’s only select the features that are the most relevant. Feature selection is one of the important tasks in any machine learning project. You must know which features are most correlated with the targets (the “charges” column in our case) and must only use those features that have a high correlation with your target. This can be done through experimentation. For example, in this problem, I tried using the “sex” and “region” features to predict “charges” but didn’t find much of an improvement in the prediction performance of the model. So I decided to omit these features from the model. Through small experimentation like that, I found the “age,” “bmi,” and “smoker” columns to be most relevant when predicting insurance costs (the “charges” column in our data frame)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "50b4ad2c",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = new_df[['age', 'bmi', 'smoker_yes']]\n",
    "y = new_df['charges']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da6623e4",
   "metadata": {},
   "source": [
    "We will perfomr standarlization over X "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "64c40e3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "X = scaler.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03b47621",
   "metadata": {},
   "source": [
    "Splliting data to Train and Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "d419446c",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "632e42f1",
   "metadata": {},
   "source": [
    "End let to train the data using our implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "aee9824c",
   "metadata": {},
   "outputs": [],
   "source": [
    "model =  MultipleLinearRegression(C=0.01,I=1000)\n",
    "z = model.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "59b44ccc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7265465111570096"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.score(X_test,y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "789d3e52",
   "metadata": {},
   "source": [
    "We've achived according to 𝑅2  score 71% accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14682d44",
   "metadata": {},
   "source": [
    "Let see how will be in standart LinearRegresion in sklearn in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "fa3a2269",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7265172376603916"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr = LinearRegression(fit_intercept=True)\n",
    "lr.fit(X_train,y_train)\n",
    "lr.score(X_test,y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f836072",
   "metadata": {},
   "source": [
    "The score using imprlemntation in sklearn in Python is the same as our implementation , Let to compare  the  predicted valuea"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "3b958e6c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6063.029119</th>\n",
       "      <td>6062.677849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14917.758785</th>\n",
       "      <td>14916.855252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9770.373461</th>\n",
       "      <td>9768.977799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17206.575031</th>\n",
       "      <td>17205.641626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10098.229799</th>\n",
       "      <td>10097.502775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5170.289266</th>\n",
       "      <td>5170.648315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10820.196701</th>\n",
       "      <td>10818.809988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16295.712563</th>\n",
       "      <td>16295.255050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6072.281549</th>\n",
       "      <td>6072.317732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12284.604700</th>\n",
       "      <td>12283.600256</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>268 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                         0\n",
       "6063.029119    6062.677849\n",
       "14917.758785  14916.855252\n",
       "9770.373461    9768.977799\n",
       "17206.575031  17205.641626\n",
       "10098.229799  10097.502775\n",
       "...                    ...\n",
       "5170.289266    5170.648315\n",
       "10820.196701  10818.809988\n",
       "16295.712563  16295.255050\n",
       "6072.281549    6072.317732\n",
       "12284.604700  12283.600256\n",
       "\n",
       "[268 rows x 1 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "our_predicted_data = model.predict(X_test)[:,0]\n",
    "python_predicted_data = lr.predict(X_test)\n",
    "pd.DataFrame(our_predicted_data ,python_predicted_data  )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "480c850d",
   "metadata": {},
   "source": [
    "The diferences is too small,but we can perform $R^2 score$ over the both Python and Our predicted  data  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "ddd906e4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999999900767778"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ssr = np.sum((our_predicted_data - python_predicted_data) ** 2)\n",
    "sst = np.sum((our_predicted_data - np.mean(python_predicted_data)) ** 2)\n",
    "r2_score = 1 - (ssr / sst)\n",
    "r2_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1106c09f",
   "metadata": {},
   "source": [
    "0.9999999976969305 shows that the difference is negligible"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df31ba43",
   "metadata": {},
   "source": [
    "<h2>Polynomial Regression </h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "959db131",
   "metadata": {},
   "source": [
    "In statistics , polynomial regression is a form of regression analysis in which the relationship between the independent variable $x$ and the dependent variable $y$ is modelled as an $n^{th}$ degree polynomial in $x$.In Machine learning the polynomial regression is just the extended Linear Regression.This extension we achieve by the tranform the dataset as shown below. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f9ce1a3",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "ecc1ed5a",
   "metadata": {},
   "source": [
    "First Let's generate some random dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "f32d8749",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVQAAAEvCAYAAAAadzm3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAV/ElEQVR4nO3df4xddXrf8ffTsZdaSSD8mJ24Nl47YVJ1WLUQrhyi1hEVa+ysqhgikIyiYKlIThArt/8VRAXprpBCqhTVUWHlFARGiYGQbO12l+zOQlb4DwcYagJmWHdmlyVMsIZJTAlVXWLPPv3jfgeuh+sfzHzv3Dsz75d0dM99zvmeeQbbH77nnDtzIjORJM3fP+h2A5K0VBioklSJgSpJlRioklSJgSpJlRioklTJim430CmXXXZZrl+/vtttSFpiXnnllb/JzP5225ZsoK5fv56RkZFutyFpiYmIt8+0zVN+SarEQJWkSgxUSarEQJWkSgxUSarEQJWkSqoEakQ8GhHvRcSRltpvR8RfR8SrZflyy7a7I2I8Io5GxJaW+jUR8XrZtjsiotQviIinSv3FiFhfo29JqqnWDPUxYGub+oOZeVVZvgUQEUPAduDKMuahiOgr+z8M7AQGyzJzzNuB9zPzCuBB4IFKfUtSNVUCNTNfAI6f5+7bgCcz86PMfAsYBzZGxGrgwsw8lM3fer0XuLFlzONl/Rng+pnZqyTN1fDoJPfuP8Lw6GSV43X6GupXIuK1ckng4lJbA7zTss9Eqa0p67Prp43JzFPAB8Cls79YROyMiJGIGJmamqr7nUhaUoZHJ9m17zB7D73Nrn2Hq4RqJwP1YeDngKuAY8DvlXq7mWWepX62MacXMvdkZiMzG/39bX/UVpIAODg2xYmT0wCcODnNwbH5T8I6FqiZOZmZ05n5Y+APgI1l0wRwecuua4F3S31tm/ppYyJiBXAR53+JQZI+ZdNgP6tWNm/frFrZx6bB+U/COhao5ZrojJuAmU8AHAC2lzv3G2jefHopM48BH0bEteX66G3A/pYxO8r6zcDz6dMFJc3D5qEBdt96Nbf90hfYfevVbB4amPcxq/y2qYjYB1wHXBYRE8B9wHURcRXNU/MfAb8JkJlvRMTTwChwCrgzM6fLoe6g+YmBVcCzZQF4BHgiIsZpzky31+hb0vK2eWigSpDOiKU60Ws0Gumv75NUW0S8kpmNdtv8SSlJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKqgRqRDwaEe9FxJGW2iURMRwRY+X14pZtd0fEeEQcjYgtLfVrIuL1sm13RESpXxART5X6ixGxvkbfklRTrRnqY8DWWbW7gOcycxB4rrwnIoaA7cCVZcxDEdFXxjwM7AQGyzJzzNuB9zPzCuBB4IFKfUtSNVUCNTNfAI7PKm8DHi/rjwM3ttSfzMyPMvMtYBzYGBGrgQsz81BmJrB31piZYz0DXD8ze5WkXtHJa6gDmXkMoLx+vtTXAO+07DdRamvK+uz6aWMy8xTwAXBpxzqXpDnoxk2pdjPLPEv9bGNOP3DEzogYiYiRqampebQoSZ9dJwN1spzGU17fK/UJ4PKW/dYC75b62jb108ZExArgIj59iYHM3JOZjcxs9Pf3V/xWJOncOhmoB4AdZX0HsL+lvr3cud9A8+bTS+WywIcRcW25PnrbrDEzx7oZeL5cZ5WknrGixkEiYh9wHXBZREwA9wG/AzwdEbcDfwXcApCZb0TE08AocAq4MzOny6HuoPmJgVXAs2UBeAR4IiLGac5Mt9foW5JqiqU60Ws0GjkyMtLtNiQtMRHxSmY22m3zJ6UkqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqZKOB2pE/CgiXo+IVyNipNQuiYjhiBgrrxe37H93RIxHxNGI2NJSv6YcZzwidkdEdLp3SfosFmqG+i8z86rMbJT3dwHPZeYg8Fx5T0QMAduBK4GtwEMR0VfGPAzsBAbLsnWBepek89KtU/5twONl/XHgxpb6k5n5UWa+BYwDGyNiNXBhZh7KzAT2toyRpJ6wEIGawHci4pWI2FlqA5l5DKC8fr7U1wDvtIydKLU1ZX12XZJ6xooF+Br/PDPfjYjPA8MR8f2z7NvuumiepX764GZg7wRYt27dXHqVpDnr+Aw1M98tr+8B3wA2ApPlNJ7y+l7ZfQK4vGX4WuDdUl/bpj77a+3JzEZmNvr7+2t/K5I6bHh0knv3H2F4dLLbrcxJRwM1In4iIn5qZh24ATgCHAB2lN12APvL+gFge0RcEBEbaN58eqlcFvgwIq4td/dvaxkjaQkYHp1k177D7D30Nrv2HV6UodrpU/4B4BvlE04rgD/KzD+LiJeBpyPiduCvgFsAMvONiHgaGAVOAXdm5nQ51h3AY8Aq4NmySFoiDo5NceJk85/7iZPTHBybYvPQQJe7+mw6GqiZ+UPgn7Wp/y1w/RnG3A/c36Y+Anyxdo+SesOmwX7+eGSCEyenWbWyj02Di++y3ULclJKkc9o8NMDuW6/m4NgUmwb7F93sFAxUST1k89DAogzSGf4svyRVYqBKUiUGqiRVYqBKUiUGqiRVYqBKUiUGqiRVYqBKUiUGqiRVYqBKUiUGqiRVYqBKUiUGqiRVYqBKUiUGqiRV4u9DlRaJ4dHJRf3Ll5cDZ6jSIrAUHmC3HBio0iLQ7gF26j0GqrQIbBrsZ9XKPoBF+wC75cBrqNIisBQeYLccGKjSIrHQD7DzJthn5ym/pE/xJtjcGKiSPsWbYHNjoEr6FG+CzY3XUCV9ijfB5sZAldTWQt8EWwoMVGmOvAuu2byGKs2Bd8HVjoEqzYF3wdWOgSrNgXfB1Y7XUKU58C642llUgRoRW4H/DPQB/zUzf6fLLWkZ8y64Zls0p/wR0Qf8F+BXgCHg1ogY6m5XkvSJRROowEZgPDN/mJl/DzwJbOtyT5L0scUUqGuAd1reT5SaJPWExRSo0aaWp+0QsTMiRiJiZGrKj7FIWliLKVAngMtb3q8F3m3dITP3ZGYjMxv9/X6MRdLCWkyB+jIwGBEbIuJzwHbgQJd7kqSPLZqPTWXmqYj4CvBtmh+bejQz3+hyW5L0sUUTqACZ+S3gW93uQ5LaWUyn/JLU0wxUSarEQJWkSgxUSarEQJWkShbVXX4tLj4iRMuNM1R1hI8I0XJkoKojfESIliMDVR3hI0K0HHkNVR3hI0K0HBmo6hgfEaLlxlN+SarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSarEQJWkSgxUSaqkY4EaEb8dEX8dEa+W5cst2+6OiPGIOBoRW1rq10TE62Xb7oiIUr8gIp4q9RcjYn2n+pakuer0DPXBzLyqLN8CiIghYDtwJbAVeCgi+sr+DwM7gcGybC3124H3M/MK4EHggQ73LUmfWTdO+bcBT2bmR5n5FjAObIyI1cCFmXkoMxPYC9zYMubxsv4McP3M7FWaMTw6yb37jzA8OtntVrRMdTpQvxIRr0XEoxFxcamtAd5p2Wei1NaU9dn108Zk5ingA+DSTjauxWV4dJJd+w6z99Db7Np32FBVV8wrUCPiuxFxpM2yjebp+88BVwHHgN+bGdbmUHmW+tnGzO5nZ0SMRMTI1NTUZ/12tIgdHJvixMlpAE6cnObgmH/+Wngr5jM4M790PvtFxB8A/6O8nQAub9m8Fni31Ne2qbeOmYiIFcBFwPE2/ewB9gA0Go1PBe5yNjw6ycGxKTYN9rN5aKDb7VS3abCfPx6Z4MTJaVat7GPTYH+3W9Iy1Mm7/Ktb3t4EHCnrB4Dt5c79Bpo3n17KzGPAhxFxbbk+ehuwv2XMjrJ+M/B8uc6q87AcToc3Dw2w+9arue2XvsDuW69ekv/TUO+b1wz1HH43Iq6ieWr+I+A3ATLzjYh4GhgFTgF3ZuZ0GXMH8BiwCni2LACPAE9ExDjNmen2Dva95LQ7HV6KgbN5aGBJfl9aPDoWqJn5G2fZdj9wf5v6CPDFNvX/B9xStcFlxNNhaWF0coaqHjFzOryUr6FKvcBAXSY8HZY6z5/ll6RKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqsRAlaRKDFRJqmRFtxtYroZHJzk4NsWmwX42Dw10ux1JFThD7YLh0Ul27TvM3kNvs2vfYYZHJ7vdkqQKDNQuODg2xYmT0wCcODnNwbGpLnckqQYDtQs2DfazamUfAKtW9rFpsL/LHUmqwWuoXbB5aIDdt17tNVRpiTFQu2Tz0IBBKi0xnvJLUiXzCtSIuCUi3oiIH0dEY9a2uyNiPCKORsSWlvo1EfF62bY7IqLUL4iIp0r9xYhY3zJmR0SMlWXHfHqWpE6Z7wz1CPBrwAutxYgYArYDVwJbgYcioq9sfhjYCQyWZWup3w68n5lXAA8CD5RjXQLcB/wisBG4LyIunmffklTdvAI1M9/MzKNtNm0DnszMjzLzLWAc2BgRq4ELM/NQZiawF7ixZczjZf0Z4Poye90CDGfm8cx8HxjmkxCWpJ7RqWuoa4B3Wt5PlNqasj67ftqYzDwFfABcepZjSVJPOedd/oj4LvAzbTbdk5n7zzSsTS3PUp/rmNO/aMROmpcTWLdu3Rlak6TOOGegZuaX5nDcCeDylvdrgXdLfW2beuuYiYhYAVwEHC/162aN+d4Zet0D7AFoNBptQ1eSOqVTp/wHgO3lzv0GmjefXsrMY8CHEXFtuT56G7C/ZczMHfybgefLddZvAzdExMXlZtQNpSZJPWVeH+yPiJuA3wf6gW9GxKuZuSUz34iIp4FR4BRwZ2ZOl2F3AI8Bq4BnywLwCPBERIzTnJluB8jM4xHxNeDlst9XM/P4fPqWpE6I5iRw6Wk0GjkyMtLtNiQtMRHxSmY22m3zJ6UkqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqMVAlqRIDVZIqWdHtBnrF8OgkB8em2DTYz+ahgW63I2kRmtcMNSJuiYg3IuLHEdFoqa+PiBMR8WpZvt6y7ZqIeD0ixiNid0REqV8QEU+V+osRsb5lzI6IGCvLjvn03M7w6CS79h1m76G32bXvMMOjk7W/hKRlYL6n/EeAXwNeaLPtB5l5VVl+q6X+MLATGCzL1lK/HXg/M68AHgQeAIiIS4D7gF8ENgL3RcTF8+z7NAfHpjhxchqAEyenOTg2VfPwkpaJeQVqZr6ZmUfPd/+IWA1cmJmHMjOBvcCNZfM24PGy/gxwfZm9bgGGM/N4Zr4PDPNJCFexabCfVSv7AFi1so9Ng/01Dy9pmejkNdQNEXEY+Dvg32fmQWANMNGyz0SpUV7fAcjMUxHxAXBpa73NmCo2Dw2w+9arvYYqaV7OGagR8V3gZ9psuicz959h2DFgXWb+bURcA/y3iLgSiDb75syXOsO2s42Z3etOmpcTWLdu3Rlaa2/z0IBBKmlezhmomfmlz3rQzPwI+KisvxIRPwB+nubscm3LrmuBd8v6BHA5MBERK4CLgOOlft2sMd87w9fdA+wBaDQabUNXkjqlI59DjYj+iOgr6z9L8+bTDzPzGPBhRFxbro/eBszMcg8AM3fwbwaeL9dZvw3cEBEXl5tRN5SaJPWUeV1DjYibgN8H+oFvRsSrmbkF+GXgqxFxCpgGfiszj5dhdwCPAauAZ8sC8AjwRESM05yZbgfIzOMR8TXg5bLfV1uOJUk9I5qTwKWn0WjkyMhIt9uQtMRExCuZ2Wi3zR89laRKDFRJqsRAlaRKDFRJqsRAlaRKluxd/oiYAt6eVb4M+JsutPNZ2Wddi6VPWDy9Luc+v5CZbX/hx5IN1HYiYuRMH3foJfZZ12LpExZPr/bZnqf8klSJgSpJlSy3QN3T7QbOk33WtVj6hMXTq322sayuoUpSJy23GaokdcySD9SI+FpEvFYeFvidiPhHLdvuLg8FPBoRW7rc53+MiO+XXr8RET/dsq1n+iz9tH04Y9nWa71uLb2MR8Rd3e6nVUQ8GhHvRcSRltolETFcHkg5XPv5aXPo8fKI+POIeLP8mf+bXuyz9PQPI+KliPjL0ut/WPBeM3NJLzSfYTWzvgv4elkfAv4SuADYAPwA6OtinzcAK8r6A8ADvdhn6emfAP+Y5i/6brTUe6pXoK/08LPA50pvQ938bzerv18GfgE40lL7XeCusn7XzN+DLva4GviFsv5TwP8qf8491WfpI4CfLOsrgReBaxey1yU/Q83Mv2t5+xN88viUbcCTmflRZr4FjNN8qmpXZOZ3MvNUefsXfPJkg57qE876cMZe63UjMJ6ZP8zMvweeLD32hMx8gebv/m3V+rDKx/nkIZZdkZnHMvN/lvUPgTdpPtOtp/oEyKb/U96uLEuygL0u+UAFiIj7I+Id4NeBe0u54w//m4d/zSe/eLuX+5yt13rttX7Ox0A2n2xBef18l/v5WESsB66mOfPryT4joi8iXgXeo/m05AXtdUkEakR8NyKOtFm2AWTmPZl5OfCHwFdmhrU5VEc/8nCuPss+9wCnSq9d6fN8e203rE2tmx8j6bV+Fq2I+EngT4B/O+usr6dk5nRmXkXzDG9jRHxxIb9+Jx8jvWDy/B8k+EfAN4H7+OShgDNaHxjYEefqMyJ2AP8KuD7LBR+60CfM7eGMdKnXs+i1fs7HZESszsxjEbGa5kyrqyJiJc0w/cPM/NNS7rk+W2Xm/46I7wFbWcBel8QM9WwiYrDl7a8C3y/rB4DtEXFBRGyg+SDBlxa6vxkRsRX4d8CvZub/bdnUU32eQ6/1+jIwGBEbIuJzNJ9TdqCL/ZyP1odV7uCTh1h2RXmY5iPAm5n5n1o29VSf8PHDQX+6rK8CvkTz3/vC9drtO3MLcOfvT4AjwGvAfwfWtGy7h+Zd4KPAr3S5z3Ga1/teLcvXe7HP0s9NNGd/HwGTwLd7uNcv07wz/QPgnm73M6u3fcAx4GT573k7cCnwHDBWXi/pco//guZlktda/m5+udf6LL3+U+Bw6fUIcG+pL1iv/qSUJFWy5E/5JWmhGKiSVImBKkmVGKiSVImBKkmVGKiSVImBKkmVGKiSVMn/B94KevwJuhI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.rcParams[\"figure.figsize\"] = (5,5)\n",
    "np.random.seed(0)\n",
    "x = 2 - 3 * np.linspace(-10,10,num=10)  \n",
    "y = x - 2 * (x ** 2) + 0.5 * (x ** 3)  +100*np.random.normal(-30, 20, size=x.size)  \n",
    "#y = x - 2 * (x ** 2) + 0.5 * (x ** 3) +200*np.random.normal(-10, 30,  size=x.size)\n",
    "x = x[:, np.newaxis]\n",
    "y = y[:, np.newaxis]\n",
    "plt.scatter(x,y, s=10)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83396056",
   "metadata": {},
   "source": [
    "Let's apply our Linear regression model to the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "abe027c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "model =  MultipleLinearRegression()\n",
    "z = model.fit(x,y)\n",
    "y_pred = model.predict(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "77bda238",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0xd7c6b7e340>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVQAAAEvCAYAAAAadzm3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAnc0lEQVR4nO3deXxU9b3G8c8Xwhb2fV9lkzVIBNTq1aKAWoNa1ND2yq22WKuC+1Ktu71qa1Faq9K6Vk1AkJIq2IBLbRVFMAHCHvaw70RCIMvv/jGH2xHDlpzJmZk879drXnPmd+ZMHiE8njln5nfMOYeIiFRctaADiIjECxWqiIhPVKgiIj5RoYqI+ESFKiLiExWqiIhPEoIOECnNmjVznTp1CjqGiMSZBQsW7HTONS9rXdwWaqdOnZg/f37QMUQkzpjZ+mOt01t+ERGfqFBFRHyiQhUR8YkKVUTEJypUERGfqFBFRHziS6Ga2Stmtt3McsLGHjazTWaW7d0uCVt3n5nlmtkKMxseNj7QzBZ76yaamXnjtcxssjf+pZl18iO3iIif/NpDfQ0YUcb4BOdcknebCWBmvYBUoLe3zZ/MrLr3/BeAsUA373bkNa8H9jjnugITgKd8yi0i4htfCtU59ymw+ySfPhJId84dcs6tBXKBQWbWGmjgnJvrQrNevwFcHrbN697yVGDokb1XEZFyy8iAm28O3fsg0sdQbzazRd4hgcbeWFtgY9hz8ryxtt7y0ePf2sY5VwzsA5oe/cPMbKyZzTez+Tt27PD3v0RE4ktGBoweDc8/H7r3oVQjWagvAKcBScAW4BlvvKw9S3ec8eNt8+0B5yY555Kdc8nNm5f5VVsRkZDMTCgoCC0XFIQeV1DECtU5t805V+KcKwX+DAzyVuUB7cOe2g7Y7I23K2P8W9uYWQLQkJM/xCAi8l3DhkFiYmg5MTH0uIIiVqjeMdEjrgCOfAIgA0j1ztx3JnTyaZ5zbguQb2ZDvOOj1wIzwrYZ4y2PAj5yurqgiFRESgqkpcFNN4XuU1Iq/JK+zDZlZmnA+UAzM8sDHgLON7MkQm/N1wE3ADjnlpjZFGApUAzc5Jwr8V7qRkKfGKgDzPJuAC8DfzWzXEJ7pql+5BaRKi4lxZciPcLidUcvOTnZafo+EfGbmS1wziWXtU7flBIR8YkKVUTEJypUERGfqFBFRHyiQhUR8YkKVUTEJypUERGfqFBFRHyiQhUR8YkKVUTEJypUERGfqFBFRHyiQhUR8YkKVUTEJypUERGfqFBFRHyiQhUR8YkKVUTEJypUERGfqFBFRHyiQhUR8YkKVUSqrG37CzlcXOrb66lQRaRKylyyleHPfsqEOSt9e80E315JRCQGFBaV8Pj7S3nziw30aduAqwa28+21VagiUmUs37qfW97OYtX2bxh7XhfuHNaDmgn+vVFXoYpI3HPO8cbc9TwxcxkNatfgjesGcV735r7/HBWqiMS1Xd8c4u6pi/hw+Xa+37MFT4/qR7N6tSLys1SoIhK3/r1qJ7dPyWbvwSIevqwXY87uhJlF7OepUEUk7hwuLuWZzBW89Okauraox+vXDeL01g0i/nNVqCISV9bs+Ibx6dks3rSPHw/uwAOX9qJOzeqV8rNVqCISF5xzvLMgj4czllAzoRov/mQgI/q0qtQMKlQRiXn7DhZx//TFvLdoC0O6NGHCNUm0blin0nP48gEsM3vFzLabWU7YWBMzm21mq7z7xmHr7jOzXDNbYWbDw8YHmtlib91E844em1ktM5vsjX9pZp38yC0isW/+ut1c8ty/mJWzlbuG9+Ctnw0JpEzBv6+evgaMOGrsXuBD51w34EPvMWbWC0gFenvb/MnMjhzgeAEYC3Tzbkde83pgj3OuKzABeMqn3CISo4pLSnluziqufmku1arB1F+cxU0XdKV6tcidxT8RXwrVOfcpsPuo4ZHA697y68DlYePpzrlDzrm1QC4wyMxaAw2cc3Odcw5446htjrzWVGCoRfKzDyIS1TbtPcjoP3/BhDkrSenfhpnjzmVAh8Yn3jDCInkMtaVzbguAc26LmbXwxtsCX4Q9L88bK/KWjx4/ss1G77WKzWwf0BTYGbn4IhKNZi7ewr3TFlFS6phwTX+uGODfd/ErKoiTUmXtWbrjjB9vm2+/sNlYQocM6NChQ3nziUgUKjhczCMZS5k8fyP92zdiYmoSHZvWDTrWt0SyULeZWWtv77Q1sN0bzwPahz2vHbDZG29Xxnj4NnlmlgA05LuHGHDOTQImASQnJ3+ncEUkNuVs2se4tCzW7jrAL88/jdsu6k6N6tE3+2gkE2UAY7zlMcCMsPFU78x9Z0Inn+Z5hwfyzWyId3z02qO2OfJao4CPvOOsIhLHSksdf/nXGq7402ccOFzMW9cP5u4RPaOyTMGnPVQzSwPOB5qZWR7wEPAkMMXMrgc2AFcBOOeWmNkUYClQDNzknCvxXupGQp8YqAPM8m4ALwN/NbNcQnumqX7kFpHotT2/kDvfWcSnK3dwUa+WPP3DfjSuWzPoWMdl8bqjl5yc7ObPnx90DBEph49XbOeudxaSX1jMr3/Qix8P7hDRSU1OhZktcM4ll7VO35QSkahxqLiEJ2ct59XP1tGzVX3e/vkQuresH3Ssk6ZCFZGokLs9n1vSslm2ZT//c3Yn7r24J7VrVM6kJn5RoYpIoJxzpM3byKPvLSGxZgIvj0lm6Oktg45VLipUEQnM3oLD3DttMR8s2cr3ujbj91f3p0WD2kHHKjcVqogE4os1u7htcjY78g9x38U9+fm5XagW4Pfw/aBCFZFKVVRSysQPV/HHj3Pp1LQu7/7ybPq1axR0LF+oUEWk0mzcXcC49CyyNuzlqoHteDilN3VrxU8Nxc9/iYhEtRnZm3hgeg4Y/GH0AC7r3yboSL5ToYpIRH1zqJgHZ+Tw7tebGNixMc9ek0T7JolBx4oIFaqIRMzCjXsZl57Fxt0FjB/ajVu+35WEKP0evh9UqCLiu9JSx0ufruGZzBW0qF+L9LFnMahzk6BjRZwKVUR8tW1/IbdNzubz1bu4tG9rfnNFXxom1gg6VqVQoYqIb2Yv3cbdUxdSWFTKUz/sy9XJ7aNmUpPKoEIVkQorLCrhifeX8dcv1tO7TQMmjh7Aac3rBR2r0qlQRaRCVmzNZ1xaFiu25fPzcztz5/Ae1EqIrUlN/KJCFZFycc7x1y/W8/j7y2hQuwavXzeI/+rePOhYgVKhisgp233gMHdPXcicZds5v0dzfndVf5rVqxV0rMCpUEXklHyWu5PbJmezt6CIB3/Qi5+e06lKnXg6HhWqiJyUw8Wl/H72Sl76dDVdmtXltZ8OolebBkHHiioqVBE5oXU7DzAuPYtFefsYPagDD/6gF3VqVs0TT8ejQhWRY3LOMe3rTTw0I4eE6tV48SdnMKJP66BjRS0VqoiUaX9hEQ9MzyFj4WYGd27ChGuSaNOoTtCxopoKVUS+Y8H6PYxPz2LLvkLuuKg7v7ygK9VjfDb9yqBCFZH/V1Lq+NPHuTz74SpaN6zNlBvOYmDHxkHHihkqVBEBYPPeg9w6OZt5a3czMqkNj13ehwa1q8akJn5RoYoIH+Rs4Z5piykuKeWZq/pz5Rlt9dnSclChilRhBYeLeey9paTN20j/dg15LnUAnZrVDTpWzFKhilRRSzbvY1xaFmt2HuDG80/jtgu7UzMhfmfTrwwqVJEqxjnHK5+t46lZy2mUWIM3rx/MOV2bBR0rLqhQRaqQHfmHuGvqQj5ZsYMLT2/J06P60aRuzaBjxQ0VqkgV8c+VO7hjykLyC4t4bGRvfjKko048+UyFKhLnDhWX8NsPVvCXf6+lR8v6vPWzwfRoVT/oWHFJhSoSx1bv+IZxaVks2byfa8/qyK8uOZ3aNTSpSaRE/JSema0zs8Vmlm1m872xJmY228xWefeNw55/n5nlmtkKMxseNj7Qe51cM5toeq8ickzOOdLnbeAHE//N5r0H+cu1yTw6so/KNMIq6zMSFzjnkpxzyd7je4EPnXPdgA+9x5hZLyAV6A2MAP5kZkd+A14AxgLdvNuISsouElP2FRRx09tfc++7izmjYyM+uPU8LuzVMuhYVUJQb/lHAud7y68DnwD3eOPpzrlDwFozywUGmdk6oIFzbi6Amb0BXA7MqtTUIlFu3trd3Jqexfb8Q9x7cU/GntuFaprUpNJURqE6INPMHPCSc24S0NI5twXAObfFzFp4z20LfBG2bZ43VuQtHz0uIkBxSSkTP8rljx+tokOTRKbdeDb92zcKOlaVUxmFeo5zbrNXmrPNbPlxnlvW/0rdcca/vbHZWEKHBejQoUN5sorEnI27C7h1cjYL1u/hh2e045GRvalXS+ebgxDxP3Xn3GbvfruZTQcGAdvMrLW3d9oa2O49PQ9oH7Z5O2CzN96ujPGjf9YkYBJAcnLydwpXJN78feFmfjV9MTh4LjWJkUkx/sYtIwMyM2HYMEhJCTrNKYvoSSkzq2tm9Y8sA8OAHCADGOM9bQwww1vOAFLNrJaZdSZ08mmed3gg38yGeGf3rw3bRqTKOXComDvfWcgtaVl0a1GPmePPjY8yHT0ann8+dJ+REXSiUxbpPdSWwHTvE04JwNvOuQ/M7CtgipldD2wArgJwzi0xsynAUqAYuMk5V+K91o3Aa0AdQiejdEJKqqRFeXsZn57N+l0HGPf9rowb2o2E6nEwqUlmJhQUhJYLCkKPY2wvNaKF6pxbA/QvY3wXMPQY2zwBPFHG+Hygj98ZRWJFaanjz/9aw+8yV9CsXi3Sfj6EwV2aBh3LP8OGwauvhso0MTH0OMboyLVIDNi+v5Dbpyzk37k7GdG7FU/+sC+NEuNsUpOUFEhLi+ljqCpUkSj34bJt3DV1EQWHi/nNFX0ZPah9/E5qkpISk0V6hApVJEoVFpXw5KzlvPb5Onq1bsDE0QPo2qJe0LHkOFSoIlFo5bZ8xqVlsXxrPted05l7Lu5BrQR9Dz/aqVBFoohzjje/3MDj7y2lfu0EXv3pmVzQo8WJN5SooEIViRJ7DhzmnmmLyFy6jfO6N+eZq/rTvH6toGPJKVChikSBz1fv5LbJ2ew+cJgHLj2d687prElNYpAKVSRARSWlTJi9khf+uZrOzery8pgz6dO2YdCxpJxUqCIBWb/rAOPSs1m4cS+pZ7bnwct6kVhT/yRjmf72RAIwPSuPX/9tCdUM/vTjM7ikb+ugI4kPVKgilSi/sIhf/y2Hv2VvZlCnJkxITaJtozpBxxKfqFBFKsnXG/YwPj2LzXsLuf2i7tx0QVeq68RTXFGhikRYSanjxX+u5vezV9KqQW2m3DCEgR2bBB1LIkCFKhJBW/Yd5LbJ2XyxZjeX9W/D45f3oWGdGkHHkghRoYpEyD+WbOWeaYs4XFzKb0f1Y9TAdvE7qYkAKlQR3x08XMJj7y/l7S830LdtQyaOHkDnZnWDjiWVQIUq4qNlW/YzLi2LVdu/4YbzunDHsB7UTIiD2fTlpKhQRXzgnOO1z9fxv7OW07BODd68fjDf69bM3x8S4xewqwpUqCIVtOubQ9w1dREfLd/O0J4teHpUP5rW83lSkyMXsCsoCF0mJC1NpRqFVKgiFfDpyh3c8c5C9h0s4pGU3lx7VsfInHiKgwvYVQU6uCNSDoeLS/nNzGVc+8o8GtWpwYybzmHM2Z0idxZ/2LDQhesgZi9gVxVoD1XkFK3Z8Q3j0rPI2bSfnwzpwAOX9qJ2jQjPph8HF7CrClSoIifJOcc7C/J4OGMJNROqMem/BzKsd6vKC1DZF7DTSbBTpkIVOQn7Dhbxq+mLeX/RFs7q0pQJ1yTRqmHtoGNFjk6ClYsKVeQE5q/bzfj0bLbtL+TuET244bzT4n9SE50EKxedlBI5huKSUp6ds5KrX5pL9WrG1BvP5pfnV5EZonQSrFy0hypShrw9Bdw2OZuv1u3hygFteWRkb+rXrkKTmugkWLmoUEWO8v6iLdz77iKcg2evSeLyAW2DjhSMyj4JFgdUqCKegsPFPJyxhCnz80hq34iJqQPo0DTx2BvoLLgcRYUqAuRs2se4tCzW7jrAzRd0ZfyF3ahR/TinGHQWXMqgk1JSpZWWOv786Rqu+NNnFBwu4e2fDeHO4T2OX6ZQ9llwqfJUqFJlbc8vZMyr83hi5jK+37MFH9x6Lmed1vTkNtZZcCmD3vJLlfTx8u3c+c5CDhwu5jdX9GX0oPan9j18nQWXMsRUoZrZCOA5oDrwF+fckwFHkhhTWFTCUx8s59XP1tGzVX3SRw+hW8v65XsxnQWXo8RMoZpZdeB54CIgD/jKzDKcc0uDTSaxInd7PrekZbNsy35+ek4n7hnRM/KTmkiVEjOFCgwCcp1zawDMLB0YCahQ5bicc6TN28ij7y2hbs0EXv2fM7mgZ4ugY0kciqVCbQtsDHucBwwOKIvEiL0Fh7l32mI+WLKVc7s145mr+9OifhxPaiKBiqVCLeuMgfvWE8zGAmMBOnToUBmZJIrNXb2L2yZns+vAIR649HSuO6cz1arC9/AlMLFUqHlA+7DH7YDN4U9wzk0CJgEkJyd/q2yl6igqKeW5Oat4/pNcOjety1/GnEOftg2DjiVVQCwV6ldANzPrDGwCUoEfBRtJos2GXQWMn5xF1oa9XJPcngcv60XdWrH0ay6xLGZ+05xzxWZ2M/APQh+besU5tyTgWBJFZmRv4v7pOZjBH380gB/0axN0JKliYqZQAZxzM4GZQeeQ6JJfWMRDM5bwbtYmkjs25tnUJNo1Ps6kJiIRElOFKnK07I17GZeWRd6eAm69sBs3X9CVhBN9D18kQlSoEpNKSh0v/nM1E2avpGWD2ky54SySOzUJOpZUcSpUiTlb9xVy2+Rs5q7ZxaX9WvObK/rSsE4Vmk1fopYKVWJK5pKt3D1tEYeLS3l6VD+uGtju1CY1EYkgFarEhMKiEh5/fylvfrGBPm0bMDF1AF2a1ws6lsi3qFAlcny6RMjyrfu55e0sVm3/hrHndeHOYT2omaATTxJ9VKgSGT5cIsQ5xxtz1/PEzGU0qF2DN64bxHndm0cosEjFqVAlMsq6RMgpFOqubw5x99RFfLh8O9/v2YKnR/WjWb1aEQor4g+9b5LIqMAlQv69aicXP/cv/pW7k4cv68XLY5JVphITtIcqkVGOS4QcLi7lmcwVvPTpGrq2qMfr1w3i9NYNKiGsiD9UqBI5p3CJkLU7DzAuLYvFm/bx48EdeODSXtSpqdn0JbaoUCVQzjmmLsjjoYwl1Eyoxos/GciIPq2CjiVSLipUCcy+g0U88Lcc/r5wM0O6NGHCNUm0blgn6Fgi5aZClUAsWL+bcWnZbN1fyF3De/CL/zqN6ppNX2KcClUqVXFJKc9/vJqJH62iTaPavPOLszijQ+OgY4n4QoUqlWbT3oPcmp7FV+v2MDKpDY9d3ocGtTWpicQPFapUipmLt3DvtEWUlDp+f3V/rjyjXdCRRHynQpWIKjhczCMZS5k8fyP92zdiYmoSHZvWDTqWSESoUCVicjbtY1xaFmt3HeDG80/j9ou6U0Oz6UscU6GK70pLHa98tpanPlhOk7o1eev6wZzdtVnQsUQiToUqvtqeX8id7yzi05U7uPD0ljw9qh9N6tYMOpZIpVChim8+XrGdu95ZSH5hMY9d3oefDO6g2fSlSlGhSoUdKi7hyVnLefWzdfRoWZ+3fjaEHq3qBx1LpNKpUKVCcrfnc0taNsu27GfMWR2575LTqV1Dk5pI1aRClXJxzpH+1UYe+fsS6tSozl+uTebCXi2DjiUSKBWqnLK9BYe5793FzMrZyve6NuOZq/vTskHtoGOJBE6FKqfkyzW7uHVyNjvyD3HfxT35+bldqKZJTUQAFaqcpOKSUp77cBXPf5xLx6Z1efeXZ9OvXaOgY4lEFRWqnNDG3QWMT8/i6w17uWpgOx5O6U3dWvrVETma/lXIcc3I3sQD03MAmDh6ACn92wScSCR6qVClTN8cKuahGUuY9nUeAzs25tlrkmjfJDHoWCJRTYUq37Fw417Gp2exYXcB44Z2Y9z3u5KgSU1ETihi/0rM7GEz22Rm2d7tkrB195lZrpmtMLPhYeMDzWyxt26ied9bNLNaZjbZG//SzDpFKndVVlrqePGfq/nhC59zuLiU9LFncftF3VWmIicp0nuoE5xzvwsfMLNeQCrQG2gDzDGz7s65EuAFYCzwBTATGAHMAq4H9jjnuppZKvAUcE2Es1cp2/YXcvuUbD7L3cUlfVvxv1f0o2GiZtMXORVB7HqMBNKdc4ecc2uBXGCQmbUGGjjn5jrnHPAGcHnYNq97y1OBoaZZN3wzZ+k2Rjz7KV+v38tTP+zL8z86IzbLNCMDbr45dC8SgEjvod5sZtcC84E7nHN7gLaE9kCPyPPGirzlo8fx7jcCOOeKzWwf0BTYGdn48a2wqITfzFzGG3PX06t1AyaOHkDXFvWCjlU+GRkwejQUFMCrr0JaGqSkBJ1KqpgK7aGa2RwzyynjNpLQ2/fTgCRgC/DMkc3KeCl3nPHjbXN0nrFmNt/M5u/YseNU/3OqlBVb8xn5x894Y+56fva9zky/6ezYLVOAzMxQmULoPjMz2DxSJVVoD9U5d+HJPM/M/gy85z3MA9qHrW4HbPbG25UxHr5NnpklAA2B3WXkmQRMAkhOTv5O4VZpGRmQmYm7aBhvtujH4+8vo37tBF776Zmc36NF0Okqbtiw0J5pQQEkJoYei1SyiL3lN7PWzrkt3sMrgBxvOQN428x+T+ikVDdgnnOuxMzyzWwI8CVwLfCHsG3GAHOBUcBH3nFWORne2+HdLoG7tzdjTpfqnN+jOb8d1Z/m9WsFnc4fKSmht/mZmaEy1dt9CUAkj6E+bWZJhN6arwNuAHDOLTGzKcBSoBi4yTvDD3Aj8BpQh9DZ/Vne+MvAX80sl9CeaWoEc8efzEw+b9aV235wB3vqNODXh5bx0zGXxN+kJikpKlIJVMQK1Tn338dZ9wTwRBnj84E+ZYwXAlf5GrCKKCop5Zkew3mp7nC67N7EK+89Se8/PAnxVqYiUUDflIpj63YeYHx6Fgs3VWN0q1J+fWAZiX94UntxIhGiQo1Dzjne/XoTD87IIaF6NV748Rlc3Lc1cFnQ0UTimgo1zuwvLOLXf8thRvZmBnVuwrPXJNGmUZ2gY4lUCSrUOPL1hj2MT89i895C7rioO7+8oCvVdaxUpNKoUONASanjhU9ymTBnFa0b1mbKDWcxsGPjoGOJVDkq1Bi3ee9BbpuczZdrd5PSvw2PX9GHBrVj8Hv4InFAhRrDPsjZwj3TFlNcUsozV/XnyjPaojljRIKjQo1BBw+X8Oh7S0mbt4F+7RoyMXUAnZrVDTqWSJWnQo0xSzfv55a0r1mz8wC/+K/TuP2i7tRM0ATQItFAhRojnHO8+tk6npy1nEaJNXjz+sGc07VZ0LFEJIwKNQbs/OYQd76zkE9W7ODC01vw9Kj+NKlbM+hYInIUFWqU++fKHdwxZSH7C4t4dGRv/ntIR514EolSKtQodai4hN9+sIK//Hst3VvW462fDaZHq/pBxxKR41ChRqHVO75hXFoWSzbv59qzOvKrS06ndo3qQccSkRNQoUYR5xyTv9rII39fSu0a1fjztclc1Ktl0LFE5CSpUKPEvoIi7pu+iJmLt3L2aU35/dVJtGpYO+hYInIKVKhRYN7a3dyansX2/EPcM6InN5zXJf5m0xepAlSoASouKWXiR7n88aNVtG+SyNQbzyapfaOgY4lIOalQA7JxdwG3Ts5mwfo9XHlGWx4d2Yd6tfTXIRLL9C84AH9fuJlfTV+Mc/BcahIjk9oGHUlEfKBCrUQHDhXzUMYSpi7II6l9IyamDqBD08SgY4mIT1SolWRR3l7Gp2ezbtcBbvl+V8YN7UaN6prURCSeqFAjrLTU8ed/reF3mStoVq8WaT8fwpAuTYOOJSIRoEKNoO37C7l9ykL+nbuTEb1b8eQP+9IoUZOaiMQrFWqEfLhsG3dNXUTB4WL+98q+pJ7ZXpOaiMQ5FarPCotKeHLWcl77fB2nt27AH0Yn0bWFJjURqQpUqD5auS2fcWlZLN+az3XndObuET2OPalJRgZkZsKwYZCSUrlBRSQiVKg+cM7x5pcbePy9pdSrlcCr/3MmF/RscewNMjJg9GgoKIBXX4W0NJWqSBxQoVbQngOHuWfaIjKXbuPcbs145ur+tKh/gklNMjNDZQqh+8xMFapIHNAHISvg89U7GfHcp3y8YjsPXHo6r/900InLFEJv8xO9D/QnJoYei0jM0x5qORSVlDJh9kpe+OdqOjery8tjzqRP24Yn/wIpKaG3+TqGKhJXVKinaP2uA4xLz2bhxr1ck9yeh1J6kVizHH+MKSkqUpE4o0I9BdOz8vj135ZQzeD5H53Bpf1aBx1JRKJIhY6hmtlVZrbEzErNLPmodfeZWa6ZrTCz4WHjA81ssbduonmfdjezWmY22Rv/0sw6hW0zxsxWebcxFclcHvmFRdyansVtkxfSq3UDZt16nspURL6jonuoOcCVwEvhg2bWC0gFegNtgDlm1t05VwK8AIwFvgBmAiOAWcD1wB7nXFczSwWeAq4xsybAQ0Ay4IAFZpbhnNtTwewn5esNexifnsXmvYXcflF3brqgK9U1m76IlKFCe6jOuWXOuRVlrBoJpDvnDjnn1gK5wCAzaw00cM7Ndc454A3g8rBtXveWpwJDvb3X4cBs59xur0RnEyrhiCopdTz/cS5XvTiX0lKYcsMQxg3tpjIVkWOK1DHUtoT2QI/I88aKvOWjx49ssxHAOVdsZvuApuHjZWwTEVv2HeS2ydl8sWY3l/Vvw+OX96FhnRqR/JEiEgdOWKhmNgdoVcaq+51zM461WRlj7jjj5d3m2z/UbCyhwwl06NDhGNGO7x9LtnLPtEUcLi7lt6P6MWpgO01qIiIn5YSF6py7sByvmwe0D3vcDtjsjbcrYzx8mzwzSwAaAru98fOP2uaTY2SdBEwCSE5OLrN0j+Xg4RIee38pb3+5gb5tGzJx9AA6N6t7Ki8hIlVcpL4plQGkemfuOwPdgHnOuS1AvpkN8Y6PXgvMCNvmyBn8UcBH3nHWfwDDzKyxmTUGhnljvrp9SjZvf7mBG87rwrQbz1aZisgpq9AxVDO7AvgD0Bx438yynXPDnXNLzGwKsBQoBm7yzvAD3Ai8BtQhdHZ/ljf+MvBXM8sltGeaCuCc221mjwFfec971Dm3uyK5yzJuaDd+PLgj3+vWzO+XFpEqwkI7gfEnOTnZzZ8/P+gYIhJnzGyBcy65rHWaHEVExCcqVBERn6hQRUR8okIVEfGJClVExCcqVBERn6hQRUR8okIVEfGJClVExCcqVBERn6hQRUR8okIVEfGJClVExCcqVBERn6hQRUR8okIVEfGJClVExCcqVBERn6hQRUR8okIVEfGJClVExCcqVBERn6hQRUR8okIVEfGJClVExCcqVBERn6hQRUR8okIVEfGJClVExCcqVBERn6hQRUR8okIVEfGJCvWIjAy4+ebQvYhIOVSoUM3sKjNbYmalZpYcNt7JzA6aWbZ3ezFs3UAzW2xmuWY20czMG69lZpO98S/NrFPYNmPMbJV3G1ORzGXKyIDRo+H550P3KlURKYeK7qHmAFcCn5axbrVzLsm7/SJs/AVgLNDNu43wxq8H9jjnugITgKcAzKwJ8BAwGBgEPGRmjSuY+9syM6GgILRcUBB6LCJyiipUqM65Zc65FSf7fDNrDTRwzs11zjngDeByb/VI4HVveSow1Nt7HQ7Mds7tds7tAWbznxL2x7BhkJgYWk5MDD0WETlFCRF87c5mlgXsBx5wzv0LaAvkhT0nzxvDu98I4JwrNrN9QNPw8TK28UdKCqSlhfZMhw0LPRYROUUnLFQzmwO0KmPV/c65GcfYbAvQwTm3y8wGAn8zs96AlfFcd+RHHWPd8bY5OutYQocT6NChwzGiHUNKiopURCrkhIXqnLvwVF/UOXcIOOQtLzCz1UB3QnuX7cKe2g7Y7C3nAe2BPDNLABoCu73x84/a5pNj/NxJwCSA5OTkMktXRCRSIvKxKTNrbmbVveUuhE4+rXHObQHyzWyId3z0WuDIXm4GcOQM/ijgI+846z+AYWbW2DsZNcwbExGJKhU6hmpmVwB/AJoD75tZtnNuOHAe8KiZFQMlwC+cc7u9zW4EXgPqALO8G8DLwF/NLJfQnmkqgHNut5k9BnzlPe/RsNcSEYkaFtoJjD/Jyclu/vz5QccQkThjZgucc8llrdM3pUREfKJCFRHxiQpVRMQnKlQREZ+oUEVEfBK3Z/nNbAew/qjhZsDOAOKcKuX0V6zkhNjJWpVzdnTONS9rRdwWalnMbP6xPu4QTZTTX7GSE2Inq3KWTW/5RUR8okIVEfFJVSvUSUEHOEnK6a9YyQmxk1U5y1CljqGKiERSVdtDFRGJmLgvVDN7zMwWeRcLzDSzNmHr7vMuCrjCzIYHnPO3ZrbcyzrdzBqFrYuanF6eMi/O6K2LtqwjvCy5ZnZv0HnCmdkrZrbdzHLCxpqY2WzvgpSzfb9+2qlnbG9mH5vZMu/vfHw05vQy1TazeWa20Mv6SKVndc7F9Y3QNayOLI8DXvSWewELgVpAZ2A1UD3AnMOABG/5KeCpaMzpZTod6EFoou/ksPGoygpU9zJ0AWp62XoF+Wd3VL7zgDOAnLCxp4F7veV7j/weBJixNXCGt1wfWOn9PUdVTi+HAfW85RrAl8CQyswa93uozrn9YQ/r8p/Lp4wE0p1zh5xza4FcQldVDYRzLtM5V+w9/IL/XNkgqnLCcS/OGG1ZBwG5zrk1zrnDQLqXMSo45z4lNPdvuPCLVb7Ofy5iGQjn3Bbn3Nfecj6wjNA13aIqJ4AL+cZ7WMO7OSoxa9wXKoCZPWFmG4EfAw96w5G/+F/5Xcd/Jt6O5pxHi7as0ZbnZLR0oStb4N23CDjP/zOzTsAAQnt+UZnTzKqbWTawndDVkis1a1wUqpnNMbOcMm4jAZxz9zvn2gNvATcf2ayMl4roRx5OlNN7zv1AsZc1kJwnm7WszcoYC/JjJNGWJ2aZWT1gGnDrUe/6oopzrsQ5l0ToHd4gM+tTmT8/kpeRrjTu5C8k+DbwPvAQ/7ko4BHhFwyMiBPlNLMxwA+Aoc474EMAOaF8F2ckoKzHEW15TsY2M2vtnNtiZq0J7WkFysxqECrTt5xz73rDUZcznHNur5l9AoygErPGxR7q8ZhZt7CHKcBybzkDSDWzWmbWmdCFBOdVdr4jzGwEcA+Q4pwrCFsVVTlPINqyfgV0M7POZlaT0HXKMgLMczLCL1Y5hv9cxDIQ3sU0XwaWOed+H7YqqnLC/18ctJG3XAe4kNC/98rLGvSZuUo48zcNyAEWAX8H2oatu5/QWeAVwMUB58wldLwv27u9GI05vTxXENr7OwRsA/4RxVkvIXRmejVwf9B5jsqWBmwBirw/z+uBpsCHwCrvvknAGb9H6DDJorDfzUuiLaeXtR+Q5WXNAR70xistq74pJSLik7h/yy8iUllUqCIiPlGhioj4RIUqIuITFaqIiE9UqCIiPlGhioj4RIUqIuKT/wPg2X3kAo8S0wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y_pred)  #L = 0.0028\n",
    "plt.scatter(x,y, s=10,color='r')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "55f5eddf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8289508861040764"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.score(x,y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9917e95",
   "metadata": {},
   "source": [
    "We can see that the straight line is unable to capture the patterns in the data. This is an example of <mark>under-fitting</mark>. Computing the RMSE and R²-score of the linear line gives:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "b0b3e4ea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7076986792979332"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "0.7076986792979332"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6225302d",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2de86810",
   "metadata": {},
   "source": [
    "We can add new features to the dataset, which are just the  $𝑛_{𝑡ℎ}$  degree polynomial in existing ones.\n",
    "$$X = \\begin{bmatrix} \n",
    " x_1 \\\\\n",
    "\\vdots   \\\\\n",
    " x_n \\\\\n",
    "\\end{bmatrix} -- >  \n",
    "X_{polyData} = T(X) = \\begin{bmatrix} \n",
    "1 &x_1 & x_1^2 & \\dots & x_1^p \\\\\n",
    "\\vdots & \\ddots &  \\ddots & \\vdots & \\vdots \\\\\n",
    "1& x_n & x_n^2  & \\dots & x_n^p \\\\\n",
    "\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3df1e4c9",
   "metadata": {},
   "source": [
    "The new Dataset 𝑇(𝑋) can be trained in Multiple Linear regression and the weights we achieve will correspond to in $w_p $ in $Y = \\omega_0  + \\omega_1 x^1 + ... + \\omega_p x^p$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35bbabfe",
   "metadata": {},
   "source": [
    "To convert the original features into their higher order terms we will use the PolynomialFeatures class provided by <mark>scikit-learn</mark>.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "58c023cf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1.],\n",
       "       [1., 2., 4.],\n",
       "       [1., 3., 9.]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "polynomial_features= PolynomialFeatures(degree=2)\n",
    "polynomial_features.fit_transform(np.array([[1],[2],[3]]))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae78c24a",
   "metadata": {},
   "source": [
    " $$X = \\begin{bmatrix} \n",
    " 1 \\\\\n",
    " 2   \\\\\n",
    " 3 \\\\\n",
    "\\end{bmatrix} -- >  \n",
    "X_{polyData} = T(X) = \\begin{bmatrix} \n",
    "1 & 1 & 1   \\\\\n",
    "2 & 2 &  4 \\\\\n",
    "1& 3 & 9 \\\\\n",
    "\\end{bmatrix} $$ T(X) 2 degree"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be4b47d6",
   "metadata": {},
   "source": [
    "Let's returned back to out dataset "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cacd9531",
   "metadata": {},
   "source": [
    "We see that if we using linear eqaution the result is underfitting,Let's compare the 1,2,3 degree polynomial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "cf64d93a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def poly_regression(degree,L2_reguralization=0):\n",
    "    polynomial_features= PolynomialFeatures(degree=degree)\n",
    "    x_poly = polynomial_features.fit_transform(x)\n",
    "    x_poly = scaler.fit_transform(x_poly)  # standart scaller \n",
    "    model =  MultipleLinearRegression(C=0.001,I=2000)\n",
    "    model.fit(x_poly,y,L2=L2_reguralization ) \n",
    "    return model.predict(x_poly),model.score(x_poly,y),model.w_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "c83191d9",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "too many values to unpack (expected 2)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-36-8d8cd7238063>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[1;32min\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m2\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m3\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m     \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfigsize\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m50\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m50\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 5\u001b[1;33m     \u001b[0my_pred\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mscor\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpoly_regression\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      6\u001b[0m     \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my_pred\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mlabel\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m\"$n^{}$,score={:.2f}\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mscor\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m  \u001b[1;31m#L = 0.0028\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m     \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mscatter\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0ms\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m10\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mcolor\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'r'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mlabel\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m\"data set\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: too many values to unpack (expected 2)"
     ]
    }
   ],
   "source": [
    "#fig, axs = plt.subplots(size, size,figsize=(25,25)) \n",
    "plt.rcParams[\"figure.figsize\"] = (10,10)\n",
    "for i in 1,2,3:\n",
    "    plt.figsize=(50,50)\n",
    "    y_pred,scor = poly_regression(i)\n",
    "    plt.plot(x, y_pred,label=\"$n^{}$,score={:.2f}\".format(i,scor))  #L = 0.0028\n",
    "    plt.scatter(x,y, s=10,color='r',label=\"data set\")\n",
    "    plt.legend()\n",
    "plt.show()    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3aea38f",
   "metadata": {},
   "source": [
    "From the above graphic we see that the Polynomial $n^3$ is the best with an accuracy 0.82 and the green line fit data very well and red and blue we can conclude they underfit the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "401d5326",
   "metadata": {},
   "source": [
    "Let see what's happen with n^5,n^10,n^20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2df667c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams[\"figure.figsize\"] = (10,10)\n",
    "for i in 1,3,25:\n",
    "    plt.figsize=(50,50)\n",
    "    y_pred,scor = poly_regression(i)\n",
    "    plt.plot(x, y_pred,label=\"$degree={}$,score={:.2f}\".format(i,scor))  #L = 0.0028\n",
    "    plt.scatter(x,y, s=10,color='r',label=\"data set\")\n",
    "    plt.legend()\n",
    "plt.show()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34411872",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "c3c17bc0",
   "metadata": {},
   "source": [
    "For degree=60 the model fits training data too much well,that is the very bad sign.In general this means the biases or  noise is captured too, that is called <mark>over-fitting</mark>.This is also mean that the model will fail for new unseen data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bcf7850",
   "metadata": {},
   "source": [
    "<h2>Bias and Variance</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a42f930",
   "metadata": {},
   "source": [
    "Bias refers to the error due to the model’s simplistic assumptions in fitting the data. A high bias means that the model is unable to capture the patterns in the data and this results in under-fitting.<br> <br>\n",
    "Variance refers to the error due to the complex model trying to fit the data. High variance means the model passes through most of the data points and it results in over-fitting the data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20bf7d08",
   "metadata": {},
   "outputs": [],
   "source": [
    "High Bias : low Variance "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ac34fb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams[\"figure.figsize\"] = (3,3)\n",
    "y_pred,scor = poly_regression(1)\n",
    "plt.plot(x, y_pred,label=\"$degree={}$,score={:.2f}\".format(i,scor))\n",
    "plt.scatter(x,y, s=10,color='r',label=\"data set\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c7e2084",
   "metadata": {},
   "source": [
    "In case when degree=1 it called has high Bias and low variance or Underfit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04bd5fb7",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams[\"figure.figsize\"] = (3,3)\n",
    "y_pred,scor = poly_regression(3)\n",
    "plt.plot(x, y_pred,label=\"$degree={}$,score={:.2f}\".format(i,scor))\n",
    "plt.scatter(x,y, s=10,color='r',label=\"data set\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67fb7cfd",
   "metadata": {},
   "source": [
    "In case when degree=3 that is correct fit,that has low bais and low variance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8306778",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams[\"figure.figsize\"] = (3,3)\n",
    "y_pred,scor = poly_regression(25)\n",
    "plt.plot(x, y_pred,label=\"$degree={}$,score={:.2f}\".format(i,scor))\n",
    "plt.scatter(x,y, s=10,color='r',label=\"data set\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ef1b3f1",
   "metadata": {},
   "source": [
    "In case when degree=25 that is overfit,that has low bais and high variance."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df4e8aa8",
   "metadata": {},
   "source": [
    "Ideally, a machine learning model should have low variance and low bias.Therefore to achieve a good model that performs well both on the train and unseen data, a trade-off is made."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c31177e",
   "metadata": {},
   "source": [
    "<img src='https://randlow.github.io/images/ml/tradeoff_cost_bias.png' alt=\"Italian Trulli\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02164a2e",
   "metadata": {},
   "source": [
    "<h2>Reguralization </h2> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d5312dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred,scor,weight = poly_regression(3,L2_reguralization=0)\n",
    "plt.plot(x, y_pred)  #L = 0.0028\n",
    "plt.scatter(x,y, s=10,color='r',label=\"data set\")\n",
    "weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d07e1a4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred,scor,weight = poly_regression(5,L2_reguralization=100)\n",
    "plt.plot(x, y_pred)  #L = 0.0028\n",
    "plt.scatter(x,y, s=10,color='r',label=\"data set\")\n",
    "weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "584ab59d",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "616e6a4e",
   "metadata": {},
   "source": [
    "<h2> References </h2>\n",
    "<ul>\n",
    "    <li> </li>\n",
    "<li><a href='https://www.analyticsvidhya.com/blog/2017/06/a-comprehensive-guide-for-linear-ridge-and-lasso-regression/'>A comprehensive beginners guide for Linear, Ridge and Lasso Regression in Python and R </a></li>\n",
    "<li> <a href='https://www.youtube.com/watch?v=Q81RR3yKn30&ab_channel=StatQuestwithJoshStarmer'> Regularization Part 1: Ridge (L2) Regression </a> </li>\n",
    "    <li><a href='https://arxiv.org/pdf/1509.09169;Lecture'>Lecture notes on ridge regression</a> </li>\n",
    "    <li> <a href='https://medium.com/@minions.k/ridge-regression-l1-regularization-method-31b6bc03cbf'>Ridge</a> </li>\n",
    "   <li> <a href= 'https://www.southampton.ac.uk/~mb1a10/stats/FEEG6017_lecture-Interaction_terms_etc_brendan.pdf'>Transformations,polynomial fitting,and interaction terms </a>  </li>\n",
    "    <li> <a href='https://towardsdatascience.com/polynomial-regression-bbe8b9d97491'>Polynomial Regression </a>\n",
    "     <li> <a href='    http://scott.fortmann-roe.com/docs/BiasVariance.html '> Understanding the Bias-Variance Tradeoff </a> </li>\n",
    "    </ul>\n",
    "    https://www.ajol.info/index.php/gjmas/article/view/50810\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eafc7c47",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.array([[ 1],\n",
    "       [2],\n",
    "       [4]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "600cdb99",
   "metadata": {},
   "outputs": [],
   "source": [
    "poly = PolynomialFeatures(2)\n",
    "poly.fit_transform(X)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}