AFML 5.3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fractional differentiate\n",
    "\n",
    "This notebook will cover exercise answer.\n",
    "\n",
    "* Exercise 5.6\n",
    "\n",
    "As we go along, there will be some explanations.\n",
    "\n",
    "Stationarity is a key concept in time-series, by now the idea itself has been demostrated in previous notebooks (Feat Importance).\n",
    "\n",
    "Most of the functions below can be found under research/Features and /Ensemble\n",
    "\n",
    "Contact: boyboi86@gmail.com"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num of CPU core:  4\n",
      "Machine info:  Windows-10-10.0.18362-SP0\n",
      "Python 3.7.4 (default, Aug  9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)]\n",
      "Numpy 1.17.3\n",
      "Pandas 1.0.3\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Wei_X\\Anaconda3\\lib\\site-packages\\statsmodels\\tools\\_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
      "  import pandas.util.testing as tm\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 1500x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import research as rs\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "from statsmodels.tsa.stattools import adfuller\n",
    "from statsmodels.tsa.stattools import coint\n",
    "\n",
    "p = print\n",
    "\n",
    "#pls take note of version\n",
    "#numba 0.49.1 #https://github.com/numba/numba/issues/4255\n",
    "#numpy 1.17.3\n",
    "#pandas 1.0.3\n",
    "#sklearn 0.21.3\n",
    "\n",
    "dollar = pd.read_csv('./research/Sample_data/dollar_bars.txt', \n",
    "                 sep=',', \n",
    "                 header=0, \n",
    "                 parse_dates = True, \n",
    "                 index_col=['date_time'])\n",
    "\n",
    "# For most part of the func we only use 'close'\n",
    "\n",
    "close = dollar['close'].to_frame()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "ADF pVal: 0.04165 with critical value: 5%\n"
     ]
    }
   ],
   "source": [
    "ffd_series = close.apply(np.log).cumsum()\n",
    "ffd_series = rs.fracDiff_FFD(ffd_series, \n",
    "                    d = 1.99999889, \n",
    "                    thres=1e-5\n",
    "                   ).dropna()\n",
    "\n",
    "adf_pval = adfuller(ffd_series.squeeze(), \n",
    "                    maxlag = 1,\n",
    "                    regression='c', \n",
    "                    autolag=None)[1]\n",
    "\n",
    "p(\"\\nADF pVal: {0:.5f} with critical value: {1}%\".format(adf_pval, 5)) #confirm stationarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>close</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-02 09:35:57.204</th>\n",
       "      <td>-0.001693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-02 12:59:42.176</th>\n",
       "      <td>0.000243</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-02 14:19:33.847</th>\n",
       "      <td>0.001329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-02 14:33:39.311</th>\n",
       "      <td>0.000355</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-02 14:42:28.315</th>\n",
       "      <td>0.000714</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            close\n",
       "date_time                        \n",
       "2015-01-02 09:35:57.204 -0.001693\n",
       "2015-01-02 12:59:42.176  0.000243\n",
       "2015-01-02 14:19:33.847  0.001329\n",
       "2015-01-02 14:33:39.311  0.000355\n",
       "2015-01-02 14:42:28.315  0.000714"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ffd_series.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>close</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2016-12-30 20:59:58.848</th>\n",
       "      <td>-0.101709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-30 21:00:00.352</th>\n",
       "      <td>-0.102272</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-30 21:00:24.294</th>\n",
       "      <td>-0.101718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-30 21:03:03.027</th>\n",
       "      <td>-0.102393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-30 21:13:31.990</th>\n",
       "      <td>-0.102062</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            close\n",
       "date_time                        \n",
       "2016-12-30 20:59:58.848 -0.101709\n",
       "2016-12-30 21:00:00.352 -0.102272\n",
       "2016-12-30 21:00:24.294 -0.101718\n",
       "2016-12-30 21:03:03.027 -0.102393\n",
       "2016-12-30 21:13:31.990 -0.102062"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ffd_series.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "839\n"
     ]
    }
   ],
   "source": [
    "cs_event = rs.cs_filter(data = ffd_series, limit=(ffd_series.std() * 0.2))\n",
    "\n",
    "p(len(cs_event))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**\n",
    "\n",
    "This is the real reason why cs_filter was created, it was meant to be applied to stationary series.\n",
    "\n",
    "Our stationary series can act as our anchor for our mean-reversion strategy, that is why we are using cs_filter on this series to sample feature matrix.\n",
    "\n",
    "However, due to the lack of data sample/ resource, I cannot use 2 x std. (In fact, we would have been able to increase our F1 score in previous chapters, if we had enough data samples.)\n",
    "\n",
    "Based on my previous trials, when 1 x std is applied, the series itself will reduce heteroscedasticity even if it was not stationary.\n",
    "\n",
    "If you have the means to sample 5 - 10 years data, feel free to change to 2 x std."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>close</th>\n",
       "      <th>ffd_series</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-04 23:20:12.567</th>\n",
       "      <td>2040.75</td>\n",
       "      <td>-0.003825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-07 10:55:23.194</th>\n",
       "      <td>2008.00</td>\n",
       "      <td>0.004166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-07 15:10:50.900</th>\n",
       "      <td>2014.50</td>\n",
       "      <td>-0.002360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08 01:48:57.964</th>\n",
       "      <td>2037.25</td>\n",
       "      <td>0.006857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08 05:47:32.006</th>\n",
       "      <td>2032.75</td>\n",
       "      <td>-0.003119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-11-28 01:31:48.252</th>\n",
       "      <td>2205.75</td>\n",
       "      <td>-0.099969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-04 23:32:49.403</th>\n",
       "      <td>2183.50</td>\n",
       "      <td>-0.102717</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-05 02:06:52.025</th>\n",
       "      <td>2189.00</td>\n",
       "      <td>-0.096321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-08 12:46:05.346</th>\n",
       "      <td>2233.00</td>\n",
       "      <td>-0.101778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-14 20:09:50.768</th>\n",
       "      <td>2248.25</td>\n",
       "      <td>-0.103682</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>839 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                           close  ffd_series\n",
       "2015-01-04 23:20:12.567  2040.75   -0.003825\n",
       "2015-01-07 10:55:23.194  2008.00    0.004166\n",
       "2015-01-07 15:10:50.900  2014.50   -0.002360\n",
       "2015-01-08 01:48:57.964  2037.25    0.006857\n",
       "2015-01-08 05:47:32.006  2032.75   -0.003119\n",
       "...                          ...         ...\n",
       "2016-11-28 01:31:48.252  2205.75   -0.099969\n",
       "2016-12-04 23:32:49.403  2183.50   -0.102717\n",
       "2016-12-05 02:06:52.025  2189.00   -0.096321\n",
       "2016-12-08 12:46:05.346  2233.00   -0.101778\n",
       "2016-12-14 20:09:50.768  2248.25   -0.103682\n",
       "\n",
       "[839 rows x 2 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This portion requires combining events based on ffd_series with original dollar close using filtered index\n",
    "# since filter is based on ffd, therefore index matrix must also be based on filtered ffd\n",
    "df_mtx = pd.DataFrame(index = cs_event).assign(close = close,\n",
    "                                                ffd_series = ffd_series).drop_duplicates().dropna()\n",
    "df_mtx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_mtx['volatility'] = rs.vol(df_mtx.close, span0 = 50) #one of our features, since we do not have a side\n",
    "\n",
    "df_mtx.dropna(inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "vb = rs.vert_barrier(data = df_mtx.close, events = cs_event, period = 'days', freq = 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Wei_X\\Desktop\\Python\\research\\Labels\\triple_barrier_method.py:75: UserWarning: Data and events index shape must be same, reindex data to fit events\n",
      "  warnings.warn('Data and events index shape must be same, reindex data to fit events')\n",
      "C:\\Users\\Wei_X\\Desktop\\Python\\research\\Labels\\triple_barrier_method.py:112: UserWarning: Not Recommended: No side prediction provided\n",
      "  warnings.warn('Not Recommended: No side prediction provided')\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[                                             t1                      sl  \\\n",
      "2015-01-07 15:10:50.900 2015-01-12 16:02:08.112                     NaT   \n",
      "2015-01-08 01:48:57.964 2015-01-13 09:38:58.103                     NaT   \n",
      "2015-01-08 05:47:32.006 2015-01-13 09:38:58.103                     NaT   \n",
      "2015-01-09 14:48:46.704 2015-01-14 19:14:20.771 2015-01-14 04:31:40.468   \n",
      "2015-01-12 14:36:34.243 2015-01-19 09:36:49.301 2015-01-14 19:14:20.771   \n",
      "...                                         ...                     ...   \n",
      "2015-09-01 19:46:17.742 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-01 22:05:09.069 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-02 00:20:04.277 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-02 02:09:17.333 2015-09-07 04:40:25.376                     NaT   \n",
      "2015-09-02 08:08:50.931 2015-09-08 01:27:51.915                     NaT   \n",
      "\n",
      "                                             pt  \n",
      "2015-01-07 15:10:50.900 2015-01-08 01:48:57.964  \n",
      "2015-01-08 01:48:57.964                     NaT  \n",
      "2015-01-08 05:47:32.006                     NaT  \n",
      "2015-01-09 14:48:46.704                     NaT  \n",
      "2015-01-12 14:36:34.243                     NaT  \n",
      "...                                         ...  \n",
      "2015-09-01 19:46:17.742                     NaT  \n",
      "2015-09-01 22:05:09.069                     NaT  \n",
      "2015-09-02 00:20:04.277                     NaT  \n",
      "2015-09-02 02:09:17.333                     NaT  \n",
      "2015-09-02 08:08:50.931                     NaT  \n",
      "\n",
      "[279 rows x 3 columns]] this out\n",
      "[                                             t1                      sl  \\\n",
      "2015-01-07 15:10:50.900 2015-01-12 16:02:08.112                     NaT   \n",
      "2015-01-08 01:48:57.964 2015-01-13 09:38:58.103                     NaT   \n",
      "2015-01-08 05:47:32.006 2015-01-13 09:38:58.103                     NaT   \n",
      "2015-01-09 14:48:46.704 2015-01-14 19:14:20.771 2015-01-14 04:31:40.468   \n",
      "2015-01-12 14:36:34.243 2015-01-19 09:36:49.301 2015-01-14 19:14:20.771   \n",
      "...                                         ...                     ...   \n",
      "2015-09-01 19:46:17.742 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-01 22:05:09.069 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-02 00:20:04.277 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-02 02:09:17.333 2015-09-07 04:40:25.376                     NaT   \n",
      "2015-09-02 08:08:50.931 2015-09-08 01:27:51.915                     NaT   \n",
      "\n",
      "                                             pt  \n",
      "2015-01-07 15:10:50.900 2015-01-08 01:48:57.964  \n",
      "2015-01-08 01:48:57.964                     NaT  \n",
      "2015-01-08 05:47:32.006                     NaT  \n",
      "2015-01-09 14:48:46.704                     NaT  \n",
      "2015-01-12 14:36:34.243                     NaT  \n",
      "...                                         ...  \n",
      "2015-09-01 19:46:17.742                     NaT  \n",
      "2015-09-01 22:05:09.069                     NaT  \n",
      "2015-09-02 00:20:04.277                     NaT  \n",
      "2015-09-02 02:09:17.333                     NaT  \n",
      "2015-09-02 08:08:50.931                     NaT  \n",
      "\n",
      "[279 rows x 3 columns],                                              t1  sl  pt\n",
      "2016-01-21 20:15:15.382 2016-01-27 00:04:03.812 NaT NaT\n",
      "2016-01-22 01:19:34.937 2016-01-27 13:41:54.306 NaT NaT\n",
      "2016-01-22 04:21:15.807 2016-01-27 13:41:54.306 NaT NaT\n",
      "2016-01-22 06:19:21.200 2016-01-27 13:41:54.306 NaT NaT\n",
      "2016-01-22 08:48:09.889 2016-01-27 13:41:54.306 NaT NaT\n",
      "...                                         ...  ..  ..\n",
      "2016-11-28 01:31:48.252 2016-12-04 23:32:49.403 NaT NaT\n",
      "2016-12-04 23:32:49.403 2016-12-14 20:09:50.768 NaT NaT\n",
      "2016-12-05 02:06:52.025 2016-12-14 20:09:50.768 NaT NaT\n",
      "2016-12-08 12:46:05.346 2016-12-14 20:09:50.768 NaT NaT\n",
      "2016-12-14 20:09:50.768                     NaT NaT NaT\n",
      "\n",
      "[279 rows x 3 columns]] this out\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2020-05-29 12:52:42.887067 33.33% _pt_sl_t1 done after 0.27 minutes. Remaining 0.53 minutes.\r",
      "2020-05-29 12:52:42.989733 66.67% _pt_sl_t1 done after 0.27 minutes. Remaining 0.13 minutes.\r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[                                             t1                      sl  \\\n",
      "2015-01-07 15:10:50.900 2015-01-12 16:02:08.112                     NaT   \n",
      "2015-01-08 01:48:57.964 2015-01-13 09:38:58.103                     NaT   \n",
      "2015-01-08 05:47:32.006 2015-01-13 09:38:58.103                     NaT   \n",
      "2015-01-09 14:48:46.704 2015-01-14 19:14:20.771 2015-01-14 04:31:40.468   \n",
      "2015-01-12 14:36:34.243 2015-01-19 09:36:49.301 2015-01-14 19:14:20.771   \n",
      "...                                         ...                     ...   \n",
      "2015-09-01 19:46:17.742 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-01 22:05:09.069 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-02 00:20:04.277 2015-09-07 01:34:00.944                     NaT   \n",
      "2015-09-02 02:09:17.333 2015-09-07 04:40:25.376                     NaT   \n",
      "2015-09-02 08:08:50.931 2015-09-08 01:27:51.915                     NaT   \n",
      "\n",
      "                                             pt  \n",
      "2015-01-07 15:10:50.900 2015-01-08 01:48:57.964  \n",
      "2015-01-08 01:48:57.964                     NaT  \n",
      "2015-01-08 05:47:32.006                     NaT  \n",
      "2015-01-09 14:48:46.704                     NaT  \n",
      "2015-01-12 14:36:34.243                     NaT  \n",
      "...                                         ...  \n",
      "2015-09-01 19:46:17.742                     NaT  \n",
      "2015-09-01 22:05:09.069                     NaT  \n",
      "2015-09-02 00:20:04.277                     NaT  \n",
      "2015-09-02 02:09:17.333                     NaT  \n",
      "2015-09-02 08:08:50.931                     NaT  \n",
      "\n",
      "[279 rows x 3 columns],                                              t1  sl  pt\n",
      "2016-01-21 20:15:15.382 2016-01-27 00:04:03.812 NaT NaT\n",
      "2016-01-22 01:19:34.937 2016-01-27 13:41:54.306 NaT NaT\n",
      "2016-01-22 04:21:15.807 2016-01-27 13:41:54.306 NaT NaT\n",
      "2016-01-22 06:19:21.200 2016-01-27 13:41:54.306 NaT NaT\n",
      "2016-01-22 08:48:09.889 2016-01-27 13:41:54.306 NaT NaT\n",
      "...                                         ...  ..  ..\n",
      "2016-11-28 01:31:48.252 2016-12-04 23:32:49.403 NaT NaT\n",
      "2016-12-04 23:32:49.403 2016-12-14 20:09:50.768 NaT NaT\n",
      "2016-12-05 02:06:52.025 2016-12-14 20:09:50.768 NaT NaT\n",
      "2016-12-08 12:46:05.346 2016-12-14 20:09:50.768 NaT NaT\n",
      "2016-12-14 20:09:50.768                     NaT NaT NaT\n",
      "\n",
      "[279 rows x 3 columns],                                              t1  sl  pt\n",
      "2015-09-03 00:05:02.346 2015-09-08 01:27:51.915 NaT NaT\n",
      "2015-09-03 06:00:48.819 2015-09-08 12:08:06.840 NaT NaT\n",
      "2015-09-03 07:52:14.021 2015-09-08 12:08:06.840 NaT NaT\n",
      "2015-09-03 09:57:43.822 2015-09-08 12:08:06.840 NaT NaT\n",
      "2015-09-03 12:33:59.011 2015-09-09 03:06:59.837 NaT NaT\n",
      "...                                         ...  ..  ..\n",
      "2016-01-21 16:00:11.522 2016-01-27 00:04:03.812 NaT NaT\n",
      "2016-01-21 18:20:55.061 2016-01-27 00:04:03.812 NaT NaT\n",
      "2016-01-21 19:14:12.535 2016-01-27 00:04:03.812 NaT NaT\n",
      "2016-01-21 19:36:37.429 2016-01-27 00:04:03.812 NaT NaT\n",
      "2016-01-21 20:07:23.306 2016-01-27 00:04:03.812 NaT NaT\n",
      "\n",
      "[279 rows x 3 columns]] this out\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2020-05-29 12:52:43.368618 100.0% _pt_sl_t1 done after 0.28 minutes. Remaining 0.0 minutes.\n"
     ]
    }
   ],
   "source": [
    "# triple barrier events based on filter while data is also based on filtered index\n",
    "tb = rs.tri_barrier(data = df_mtx.close, \n",
    "                    events = cs_event, \n",
    "                    trgt = df_mtx['volatility'], \n",
    "                    min_req = 0.0002, \n",
    "                    num_threads = 3, \n",
    "                    ptSl= [2,2], #2x barriers\n",
    "                    t1 = vb, \n",
    "                    side = None)\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>t1</th>\n",
       "      <th>trgt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-07 15:10:50.900</th>\n",
       "      <td>2015-01-08 01:48:57.964</td>\n",
       "      <td>0.002252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08 01:48:57.964</th>\n",
       "      <td>2015-01-13 09:38:58.103</td>\n",
       "      <td>0.007576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08 05:47:32.006</th>\n",
       "      <td>2015-01-13 09:38:58.103</td>\n",
       "      <td>0.006867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-09 14:48:46.704</th>\n",
       "      <td>2015-01-14 04:31:40.468</td>\n",
       "      <td>0.009708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-12 14:36:34.243</th>\n",
       "      <td>2015-01-14 19:14:20.771</td>\n",
       "      <td>0.008815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-11-14 02:12:45.857</th>\n",
       "      <td>2016-11-28 01:31:48.252</td>\n",
       "      <td>0.022494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-11-28 01:31:48.252</th>\n",
       "      <td>2016-12-04 23:32:49.403</td>\n",
       "      <td>0.022526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-04 23:32:49.403</th>\n",
       "      <td>2016-12-14 20:09:50.768</td>\n",
       "      <td>0.022102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-05 02:06:52.025</th>\n",
       "      <td>2016-12-14 20:09:50.768</td>\n",
       "      <td>0.021669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-08 12:46:05.346</th>\n",
       "      <td>2016-12-14 20:09:50.768</td>\n",
       "      <td>0.021819</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>836 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             t1      trgt\n",
       "2015-01-07 15:10:50.900 2015-01-08 01:48:57.964  0.002252\n",
       "2015-01-08 01:48:57.964 2015-01-13 09:38:58.103  0.007576\n",
       "2015-01-08 05:47:32.006 2015-01-13 09:38:58.103  0.006867\n",
       "2015-01-09 14:48:46.704 2015-01-14 04:31:40.468  0.009708\n",
       "2015-01-12 14:36:34.243 2015-01-14 19:14:20.771  0.008815\n",
       "...                                         ...       ...\n",
       "2016-11-14 02:12:45.857 2016-11-28 01:31:48.252  0.022494\n",
       "2016-11-28 01:31:48.252 2016-12-04 23:32:49.403  0.022526\n",
       "2016-12-04 23:32:49.403 2016-12-14 20:09:50.768  0.022102\n",
       "2016-12-05 02:06:52.025 2016-12-14 20:09:50.768  0.021669\n",
       "2016-12-08 12:46:05.346 2016-12-14 20:09:50.768  0.021819\n",
       "\n",
       "[836 rows x 2 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlabel = rs.meta_label(data = df_mtx.close, \n",
    "                       events = tb, \n",
    "                       drop = 0.05) # because we do not have side, we need to drop rare labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       " 1.0    489\n",
       "-1.0    345\n",
       "Name: bin, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlabel['bin'].value_counts() #834"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>826</th>\n",
       "      <th>827</th>\n",
       "      <th>828</th>\n",
       "      <th>829</th>\n",
       "      <th>830</th>\n",
       "      <th>831</th>\n",
       "      <th>832</th>\n",
       "      <th>833</th>\n",
       "      <th>834</th>\n",
       "      <th>835</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-01-07 15:10:50.900</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08 01:48:57.964</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-08 05:47:32.006</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-09 14:48:46.704</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-12 14:36:34.243</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-11-28 01:31:48.252</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-04 23:32:49.403</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-05 02:06:52.025</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-08 12:46:05.346</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-12-14 20:09:50.768</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>837 rows × 836 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                         0    1    2    3    4    5    6    7    8    9    \\\n",
       "2015-01-07 15:10:50.900  1.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2015-01-08 01:48:57.964  1.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2015-01-08 05:47:32.006  0.0  1.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2015-01-09 14:48:46.704  0.0  1.0  1.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2015-01-12 14:36:34.243  0.0  1.0  1.0  1.0  1.0  0.0  0.0  0.0  0.0  0.0   \n",
       "...                      ...  ...  ...  ...  ...  ...  ...  ...  ...  ...   \n",
       "2016-11-28 01:31:48.252  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2016-12-04 23:32:49.403  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2016-12-05 02:06:52.025  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2016-12-08 12:46:05.346  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "2016-12-14 20:09:50.768  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0   \n",
       "\n",
       "                         ...  826  827  828  829  830  831  832  833  834  835  \n",
       "2015-01-07 15:10:50.900  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "2015-01-08 01:48:57.964  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "2015-01-08 05:47:32.006  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "2015-01-09 14:48:46.704  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "2015-01-12 14:36:34.243  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "...                      ...  ...  ...  ...  ...  ...  ...  ...  ...  ...  ...  \n",
       "2016-11-28 01:31:48.252  ...  1.0  1.0  1.0  1.0  1.0  1.0  1.0  0.0  0.0  0.0  \n",
       "2016-12-04 23:32:49.403  ...  0.0  0.0  0.0  0.0  0.0  0.0  1.0  1.0  0.0  0.0  \n",
       "2016-12-05 02:06:52.025  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0  1.0  0.0  \n",
       "2016-12-08 12:46:05.346  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0  1.0  1.0  \n",
       "2016-12-14 20:09:50.768  ...  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0  1.0  1.0  \n",
       "\n",
       "[837 rows x 836 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "troo = rs.mp_idx_matrix(data = df_mtx.close, events = tb)\n",
    "#np.arange(troo.shape[1])\n",
    "troo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quick Reference Guide/ Summary to func usage**\n",
    "\n",
    "1. Apply cs_filter using stationary series (FFD)\n",
    "\n",
    "2. Apply bband_as_side using close price series\n",
    "\n",
    "3. Create a new dataframe, using both outputs with index = cs_filter output.\n",
    "\n",
    "4. Apply func tri_barrier, Vert_barrier using new dataframe, before applying meta_label func.\n",
    "\n",
    "5. Feed these meta lables and features to random forest.\n",
    "\n",
    "Always include indicators, stationary series, volatility as part of your feature matrix if possible, since these are the features on how you derive meta-labels. \n",
    "\n",
    "As seen in previous exercises, ML models are smart enough to know what was the key parameters which meta-labels was labelled."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Wei_X\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:143: FutureWarning: The sklearn.ensemble.bagging module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.\n",
      "  warnings.warn(message, FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.ensemble.bagging import BaggingClassifier\n",
    "\n",
    "base_estimate = DecisionTreeClassifier()\n",
    "\n",
    "# common parameters to use, this ensemble method mimics monte-carlos as it randomly selects samples to form \"forest\"\n",
    "n_estimate = 10\n",
    "random_state = 42\n",
    "max_samples = 35"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>close</th>\n",
       "      <th>ffd_series</th>\n",
       "      <th>volatility</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2015-08-26 10:34:40.144</th>\n",
       "      <td>1899.75</td>\n",
       "      <td>-0.027225</td>\n",
       "      <td>0.017240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-08-25 11:37:16.214</th>\n",
       "      <td>1943.25</td>\n",
       "      <td>-0.024960</td>\n",
       "      <td>0.019059</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-01-08 10:53:07.196</th>\n",
       "      <td>1949.00</td>\n",
       "      <td>-0.054259</td>\n",
       "      <td>0.011619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-09-23 13:11:44.559</th>\n",
       "      <td>1928.75</td>\n",
       "      <td>-0.039258</td>\n",
       "      <td>0.012964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-07-01 10:25:32.469</th>\n",
       "      <td>2070.75</td>\n",
       "      <td>-0.023448</td>\n",
       "      <td>0.010666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-03-23 08:14:01.389</th>\n",
       "      <td>2096.00</td>\n",
       "      <td>-0.014996</td>\n",
       "      <td>0.008650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-05-29 10:32:41.319</th>\n",
       "      <td>2120.75</td>\n",
       "      <td>-0.014521</td>\n",
       "      <td>0.009525</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-09-01 13:48:58.967</th>\n",
       "      <td>1934.75</td>\n",
       "      <td>-0.033187</td>\n",
       "      <td>0.021461</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-12-22 09:01:07.048</th>\n",
       "      <td>2015.50</td>\n",
       "      <td>-0.050555</td>\n",
       "      <td>0.010494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-05-12 14:41:49.319</th>\n",
       "      <td>2093.50</td>\n",
       "      <td>-0.013817</td>\n",
       "      <td>0.009930</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>583 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                           close  ffd_series  volatility\n",
       "2015-08-26 10:34:40.144  1899.75   -0.027225    0.017240\n",
       "2015-08-25 11:37:16.214  1943.25   -0.024960    0.019059\n",
       "2016-01-08 10:53:07.196  1949.00   -0.054259    0.011619\n",
       "2015-09-23 13:11:44.559  1928.75   -0.039258    0.012964\n",
       "2015-07-01 10:25:32.469  2070.75   -0.023448    0.010666\n",
       "...                          ...         ...         ...\n",
       "2015-03-23 08:14:01.389  2096.00   -0.014996    0.008650\n",
       "2015-05-29 10:32:41.319  2120.75   -0.014521    0.009525\n",
       "2015-09-01 13:48:58.967  1934.75   -0.033187    0.021461\n",
       "2015-12-22 09:01:07.048  2015.50   -0.050555    0.010494\n",
       "2015-05-12 14:41:49.319  2093.50   -0.013817    0.009930\n",
       "\n",
       "[583 rows x 3 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = df_mtx.reindex(mlabel.index)\n",
    "y = mlabel['bin']\n",
    "\n",
    "# Split data into train and test sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, shuffle=True, stratify=None)\n",
    "\n",
    "X_train #quick check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ave uniqueness for grp sample 1: 0.647674\n",
      "Ave uniqueness for grp sample 2: 0.854561\n",
      "Ave uniqueness for grp sample 3: 0.849644\n",
      "Ave uniqueness for grp sample 4: 0.856805\n",
      "Ave uniqueness for grp sample 5: 0.867614\n",
      "Ave uniqueness for grp sample 6: 0.857027\n",
      "Ave uniqueness for grp sample 7: 0.804856\n",
      "Ave uniqueness for grp sample 8: 0.812000\n",
      "Ave uniqueness for grp sample 9: 0.904695\n",
      "Ave uniqueness for grp sample 10: 0.800973\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "BaggingClassifier(base_estimator=DecisionTreeClassifier(),\n",
       "                  data=2015-01-07 15:10:50.900    2014.50\n",
       "2015-01-08 01:48:57.964    2037.25\n",
       "2015-01-08 05:47:32.006    2032.75\n",
       "2015-01-09 14:48:46.704    2049.75\n",
       "2015-01-12 14:36:34.243    2030.50\n",
       "                            ...   \n",
       "2016-11-28 01:31:48.252    2205.75\n",
       "2016-12-04 23:32:49.403    2183.50\n",
       "2016-12-05 02:06:52.025    2189.00\n",
       "2016-12-08 12:46:05.346    2233.00\n",
       "2016-12-14 20:09:50.768    2...\n",
       "2015-01-12 14:36:34.243 2015-01-14 19:14:20.771  0.008815\n",
       "...                                         ...       ...\n",
       "2016-11-14 02:12:45.857 2016-11-28 01:31:48.252  0.022494\n",
       "2016-11-28 01:31:48.252 2016-12-04 23:32:49.403  0.022526\n",
       "2016-12-04 23:32:49.403 2016-12-14 20:09:50.768  0.022102\n",
       "2016-12-05 02:06:52.025 2016-12-14 20:09:50.768  0.021669\n",
       "2016-12-08 12:46:05.346 2016-12-14 20:09:50.768  0.021819\n",
       "\n",
       "[836 rows x 2 columns],\n",
       "                  max_samples=35, n_jobs=1, oob_score=True, random_state=42)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#warning, this is a heavy algo, will take up to 15 mins to complete with current parameter\n",
    "\n",
    "rs_bag_cls=rs.BaggingClassifier(data = df_mtx.close,\n",
    "                                events = tb,\n",
    "                                base_estimator = base_estimate,\n",
    "                                n_estimators = n_estimate, # the number of trees as a estimate\n",
    "                                max_samples = max_samples, #if you put max_sample = 1.0, pls mentally get ready to wait\n",
    "                                max_features = 1.0,\n",
    "                                bootstrap = True, #must always be True\n",
    "                                bootstrap_features=False,\n",
    "                                oob_score = True, # You will need to have bootstrap = True\n",
    "                                warm_start = False,\n",
    "                                n_jobs = 1, # depending your machine. might become slower if you use all processors\n",
    "                                random_state = random_state, #seed figure\n",
    "                                verbose = 0)\n",
    "\n",
    "\n",
    "\n",
    "# Fit and score\n",
    "rs_bag_cls.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Out-of-bag Score: 0.655232\n"
     ]
    }
   ],
   "source": [
    "p(\"Out-of-bag Score: {0:.6f}\".format(rs_bag_cls.oob_score_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Classification Report\n",
      "=======================================================\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "        -1.0       0.55      0.63      0.59       102\n",
      "         1.0       0.72      0.65      0.68       149\n",
      "\n",
      "    accuracy                           0.64       251\n",
      "   macro avg       0.64      0.64      0.64       251\n",
      "weighted avg       0.65      0.64      0.64       251\n",
      "\n",
      "\n",
      " Confusion Matrix\n",
      "=======================================================\n",
      "[[TN, FP]\n",
      " [FN, TP]]\n",
      "\n",
      "[[64 38]\n",
      " [52 97]]\n",
      "\n",
      " Accuracy Score\n",
      "=======================================================\n",
      "0.6414342629482072\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxM5/fA8c+RhFgidmpNYovYSe2119JStKVaXbRBUbpqy6+bqvaL2kpjbVWrWm2V0larC62iVOyqliCILRGEIJHl+f0xQ1MShmRyMzPn/XrNy70zd+aem8Q98zzPvecRYwxKKaU8Vx6rA1BKKWUtTQRKKeXhNBEopZSH00SglFIeThOBUkp5OE0ESinl4TQRKKWUh9NEoNyOiESJyEURSRCR4yIyV0QKXbVNMxFZISLnRCReRL4VkZCrtiksIpNF5JD9syLt6yVy9oiUci5NBMpddTXGFALqAfWBEZdfEJGmwE/AEqAsEAhsBdaISJB9m7zAr0BNoBNQGGgGxAGNnBW0iHg767OVyowmAuXWjDHHgeXYEsJl44BPjDHvGWPOGWNOGWNeBdYBI+3bPApUBHoYY3YaY9KMMTHGmLeMMcsy2peI1BSRn0XklIicEJH/sz8/V0RGp9uutYhEp1uPEpGXRWQbcF5EXhWRhVd99nsiMsW+7C8iH4rIMRE5IiKjRcQriz8q5cE0ESi3JiLlgc5ApH29ALZv9l9lsPmXwJ325fbAj8aYBAf34wf8AvyIrZVRBVuLwlEPAncDRYB5wF0iUtj+2V5AL+Az+7YfAyn2fdQHOgD9bmJfSv2HJgLlrr4RkXPAYSAGeMP+fDFsf/fHMnjPMeBy/3/xTLbJTBfguDFmgjEm0d7SWH8T759ijDlsjLlojDkIbAK6219rC1wwxqwTkdLYEtuzxpjzxpgYYBLQ+yb2pdR/aCJQ7qq7McYPaA0E8+8J/jSQBtyWwXtuA07al+My2SYzFYB9txSpzeGr1j/D1koAeIh/WwOVAB/gmIicEZEzwEygVBb2rTycJgLl1owxvwNzgfH29fPAn0DPDDbvxb/dOb8AHUWkoIO7OgxUzuS180CBdOtlMgr1qvWvgNb2rq0e/JsIDgNJQAljTBH7o7AxpqaDcSp1DU0EyhNMBu4UkcsDxsOBx0TkaRHxE5Gi9sHcpsCb9m3mYTvpfi0iwSKSR0SKi8j/ichdGezjO6CMiDwrIvnsn9vY/toWbH3+xUSkDPDsjQI2xsQCvwEfAQeMMf/Ynz+G7YqnCfbLW/OISGURaXULPxelAE0EygPYT6qfAK/Z11cDHYF7sY0DHMQ26NrCGLPXvk0StgHjXcDPwFngL2xdTNf0/RtjzmEbaO4KHAf2Am3sL8/DdnlqFLaT+BcOhv6ZPYbPrnr+USAvsBNbV9dCbq4bS6n/EJ2YRimlPJu2CJRSysNpIlBKKQ+niUAppTycJgKllPJwLlfgqkSJEiYgIMDqMJRSyqVs3LjxpDGmZEavuVwiCAgIICIiwuowlFLKpYjIwcxe064hpZTycJoIlFLKw2kiUEopD+dyYwQZSU5OJjo6msTERKtDUTfJ19eX8uXL4+PjY3UoSnkst0gE0dHR+Pn5ERAQgIhYHY5ykDGGuLg4oqOjCQwMtDocpTyW07qGRGSOiMSIyI5MXhcRmWKfEHybiDS41X0lJiZSvHhxTQIuRkQoXry4tuSUspgzxwjmYpv0OzOdgar2xwBgelZ2pknANenvTSnrOa1ryBizSkQCrrNJN2wTiBtgnYgUEZHb7PXWlVLKpaSlGc5fSiEhKYWExBTOJaVwLtG2nJCUbFtOSiEt7eYrPl9KTubChQvc16QadSsUyfbYrRwjKMd/p+eLtj93TSIQkQHYWg1UrFgxR4K7WV5eXtSuXZuUlBQCAwOZN28eRYpk/RcWFRVFly5d2LEjwx62WzZy5Ehmz55NyZK2Gw07derEmDFjsnUfl23ZsoWjR49y110ZzeeilLXS0gwXklM5l5h85QSekGg/iac7gf97Ur98kk++sp6QmELCpRQcqep/041gA8akAVC1fCm3SwQZ/Tgy/DEaY2YBswBCQ0Nz5QQK+fPnZ8uWLQA89thjhIeH88orr1gc1fU999xzDBs27Kbfl5qaipeXl8Pbb9myhYiICE0EKltdPoHbTtrJV07gCfaT9Ll0J+lz6U7gCYnJ/z2xO3gCL5jXi0K+3hTK500hXx8K+3pTprCvfd0bP18f/OzLhfJ54+drexTK5/Pv+/J545XHsUxw5swZXnzxRT744AOqVKnCBx98QKumAVn7oWXCykQQjW3C78vKA0ctiiVbNW3alG3btgGQkJBAt27dOH36NMnJyYwePZpu3boRFRVF586dadGiBWvXrqVcuXIsWbKE/Pnzs3HjRp544gkKFChAixYtrnxuYmIigwYNIiIiAm9vbyZOnEibNm2YO3cu33zzDampqezYsYMXXniBS5cuMW/ePPLly8eyZcsoVqyYQ7H/+uuvDBs2jJSUFG6//XamT59Ovnz5CAgI4IknnuCnn35iyJAh3H777Tz11FPExsZSoEABZs+eTXBwMF999RVvvvkmXl5e+Pv788svv/D6669z8eJFVq9ezYgRI3jggQec8nNX7iUlNY2Dpy4QGZPAvtgE+7/niT2baDuJO3gCL5DXy35Ctp3A/fJ5Uzr9CTyf7ST+70k+g+du4gSeHVJTU2nWrBm7d+/mpZdeYuTIkeTPn99p+7MyESwFhojIAqAxEJ8d4wNvfvs3O4+ezXJw6YWULcwbXR2bGzw1NZVff/2VsLAwwHad/OLFiylcuDAnT56kSZMm3HPPPQDs3buXzz//nNmzZ9OrVy++/vprHn74YR5//HGmTp1Kq1atePHFF698dnh4OADbt29n165ddOjQgT179gCwY8cONm/eTGJiIlWqVGHs2LFs3ryZ5557jk8++YRnn712mtxJkybx6aefAjB27FhatWpF3759+fXXX6lWrRqPPvoo06dPv/JeX19fVq9eDUC7du2YMWMGVatWZf369QwePJgVK1YwatQoli9fTrly5Thz5gx58+Zl1KhRRERE8P7779/Kj1+5ufNJKeyPPU9k7Dn2xZwnMiaByNgEDsadJzn13zN96cL5qFKqEFWrlLB9277y7dvHdqJP95yfr48lJ/CsiouLo1ixYnh5efH2229ToUIFQkNDnb5fpyUCEfkcaA2UEJFo4A3AB8AYMwNYBtwFRAIXgMedFUtOuHjxIvXq1SMqKoqGDRty5513ArZr5f/v//6PVatWkSdPHo4cOcKJEycACAwMpF4923zqDRs2JCoqivj4eM6cOUOrVra5yB955BF++OEHAFavXs3QoUMBCA4OplKlSlcSQZs2bfDz88PPzw9/f3+6du0KQO3ata+0Tq52ddfQ1q1bCQwMpFq1asC/XVyXE8Hlb/IJCQmsXbuWnj17XnlvUlISAM2bN6dv37706tWLe++9N0s/U+U+jDGcTLh05Zv95W/5+2ISOBr/7+XDXnmESsULULlkIe4MKU3lkoWoUqoQlUsWxM/XfW86NMYwf/58nnnmGcaMGUP//v3p0aNHju3fmVcNPXiD1w3wVHbv19Fv7tnt8hhBfHw8Xbp0ITw8nKeffpr58+cTGxvLxo0b8fHxISAg4Mp18/ny5bvyfi8vLy5evIgxJtNLKq83v3T6z8qTJ8+V9Tx58pCSkuLQMdxo/uqCBQsCkJaWRpEiRa6MiaQ3Y8YM1q9fz/fff0+9evUy3Ea5r9Q0Q/TpC1ed8G3f8uMvJl/ZrkBeLyqXLESjwGJUKXX5ZF+ISsULktfbsyrfHD58mIEDB7Js2TKaNGlC8+bNczwGt7izODfx9/dnypQpdOvWjUGDBhEfH0+pUqXw8fFh5cqVHDyYaSVYAIoUKYK/vz+rV6+mRYsWzJ8//8prLVu2ZP78+bRt25Y9e/Zw6NAhqlevzqZNm7Il9uDgYKKiooiMjKRKlSrMmzfvSsskvcKFCxMYGMhXX31Fz549Mcawbds26taty759+2jcuDGNGzfm22+/5fDhw/j5+XHu3LlsiVHlDonJqfbuHNu3+sv/7j95nkspaVe2K1EoL5VLFqJLnduufLuvUqoQZQr7kseFumyc5fPPP+fJJ58kNTWVyZMnM2TIkJu6ECO7aCJwgvr161O3bl0WLFhAnz596Nq1K6GhodSrV4/g4OAbvv+jjz66MljcsWPHK88PHjyYgQMHUrt2bby9vZk7d+5/WgJZ5evry0cffUTPnj2vDBYPHDgww23nz5/PoEGDGD16NMnJyfTu3Zu6devy4osvsnfvXowxtGvXjrp161KxYkXGjBlDvXr1dLDYxZy5cOk/XTmX+++jT1+8MlArAhWKFqBKqUK0rFaSyiULXvmGX6RAXmsPIJcrWrQojRs3ZtasWZaWWZEbdQfkNqGhoebqiWn++ecfatSoYVFEKqv092e9i5dS+Svq1H+v0IlJIO78pSvb5PPOQ1DJQldO9JdP9oElCuLrk/PfYl1RSkoKkyZN4tKlS1cuL79ed3B2EpGNxpgMR561RaCUB7uUksYXGw4xZUUksedsA/5FCvhQpWQh2tco/Z8Tfrmi+V3qCpzcZuvWrYSFhbFx40Z69ep1JQHkhjIrmgiU8kCpaYYlW44w6Zc9HD51kUaBxXj3/jrULudPsYJ5c8XJyV0kJSUxevRoxowZQ7Fixfjqq6+47777ctXP2G0SQU41r1T2crWuSVdnjGH53yeY8NNu9sYkUKtcYUY/UZuWVUvo/x8n2bt3L2PHjuWhhx5i4sSJFC9e3OqQruEWicDX15e4uDgtRe1iLs9H4Ovra3UoHmH13pO8u3wXW6PjCSpZkGl9GtC5Vhn9P+MECQkJLFmyhD59+lCrVi127dpFUFCQ1WFlyi0SQfny5YmOjiY2NtbqUNRNujxDmXKezYdO8+7y3azdF0e5IvkZd38d7q1fDm8vz7peP6f8/PPPDBgwgIMHD9KgQQNq1KiRq5MAuEki8PHx0RmulLrK7uPnGP/Tbn7eeYLiBfPyRtcQHmpckXzeeoWPM5w+fZphw4YxZ84cqlWrxu+//+4yV8O5RSJQSv3rUNwFJv2yh2+2HKFQXm+GdajG480DKZhP/7s7S2pqKs2bN2fPnj2MGDGC119/3aW6PPUvQyk3ceJsIlNX7GXBX4fx9hKebFmZga2C9KYuJzp58uSVInHvvPMOFStWpEGDW5511zKaCJRycWcuXGL67/v4eG0UKamGBxtVZGjbKpQq7DrfSF2NMYZ58+bx7LPPMmbMGAYMGED37t2tDuuWaSJQykWdT0phzuoDzFq1n4RLKfSoV45n21ejYvECVofm1g4ePMiTTz7J8uXLadasGS1btrQ6pCzTRKCUi0lMTuWz9YcIXxlJ3PlLdAgpzQsdqlO9jJ/Vobm9Tz/9lEGDBmGMYerUqQwePJg8eVz/6itNBEq5iJTUNBZtOsLkX/ZwND6RZpWL82LH6tSvWNTq0DxGyZIlad68OTNnzqRSpUpWh5Nt3KLonFLuLC3N8MOO40z4eTf7Y89Tt0IRXupYneZVSlgdmttLTk5mwoQJJCcn89prrwGuW8VAi84p5YKMMfy+J5bxP+1mx5GzVC1ViJmPNKRDSGmXPBG5ms2bNxMWFsbmzZvp3bt3rioSl900ESiVC0VEnWLcj7v5K+oU5YvmZ2KvunSrV06rf+aAxMRERo0axbhx4yhRogRff/2120+7qolAqVzk76PxjF++m5W7Yynpl4+3utXkgdsretz0jVaKjIxk/PjxPProo0yYMIGiRd1/DEYTgVK5wIGT55n48x6+3XoU//w+vNwpmMeaVaJAXv0vmhMSEhJYvHgxjzzyCLVq1WL37t0eVbZG/8qUslBammH67/uY+PMe8nrlYUibKvRvGYR/fh+rQ/MYy5cvZ8CAARw+fJjQ0FBq1KjhUUkANBEoZZlzicm88OVWftp5gi51buONrjUp6Zd9c1Cr64uLi+P555/nk08+ITg4mD/++MNlisRlN00ESlkgMuYcA+Zt5GDcBV7rEsITzQPc8mqU3OpykbjIyEheeeUVXn31VZcqEpfdNBEolcN+3HGcF77cgq+PF5+GNaZp5dw3Y5W7io2NpXjx4nh5eTF27FgqVapEvXr1rA7LcnopglI5JDXNMO7HXQz8dCNVSvvx3dMtNAnkEGMMH330EdWqVWP27NkAdOvWTZOAnbYIlMoBp89f4ukFm/lj70kebFSBkffU1AlickhUVBQDBgzg559/5o477qBNmzZWh5TraCJQysl2HIln4KcbiTmbxP/urc2DjSpaHZLHmDdvHoMGDUJEmDZtGk8++aRbFInLbpoIlHKiRZuiGbFoO8UK5uXLgU2pV6GI1SF5lNKlS9OyZUtmzJhBxYqagDOjiUApJ7iUksbb3+/k4z8P0jiwGOF9GlCikF4a6mzJycmMGzeO1NRUXn/9dTp06ECHDh2sDivX00SgVDaLOZfIU/M3sSHqNP1aBDK8czDeXtod4WybNm3iiSeeYOvWrTz00EMuWyXUCvrXqVQ22njwNF2mrGbHkbNMebA+r3YJ0STgZBcvXmT48OE0atSIEydOsHjxYubPn69J4CY49S9URDqJyG4RiRSR4Rm8XlFEVorIZhHZJiJ3OTMepZzFGMO8dQfpPetPfH28WDS4GffULWt1WB5h//79TJw4kb59+7Jz506XnjvYKk7rGhIRLyAcuBOIBjaIyFJjzM50m70KfGmMmS4iIcAyIMBZMSnlDInJqbz6zQ4WboymdfWSvPdAffwLaK0gZzp79iyLFi2ib9++1KxZk71797rVjGE5zZktgkZApDFmvzHmErAA6HbVNgYobF/2B446MR6lsl306Qv0nPEnCzdG83S7qsx57HZNAk62bNkyatWqRVhYGP/88w+AJoEscmYiKAccTrcebX8uvZHAwyISja01MDSjDxKRASISISIRsbGxzohVqZu2eu9Juk5dTdTJ83zwaCjP31mNPDpxjNOcPHmSRx55hLvvvhs/Pz/WrFnjsUXispszE0FG/yOuniD5QWCuMaY8cBcwT0SuickYM8sYE2qMCS1ZsqQTQlXKccYYZvy+j0fnrKdEoXwsGdKc9iGlrQ7LrV0uErdgwQJef/11Nm3aRJMmTawOy2048/LRaKBCuvXyXNv1EwZ0AjDG/CkivkAJIMaJcSl1yxKSUnhp4VaWbT/O3bVvY9z9dSiYT6/CdpYTJ05QsmRJvLy8GD9+PJUqVaJOnTpWh+V2nNki2ABUFZFAEckL9AaWXrXNIaAdgIjUAHwB7ftRudL+2AR6hK/hxx3HGdE5mPcfqq9JwEmMMXz44YdUr16dWbNmAdC1a1dNAk7itL9iY0yKiAwBlgNewBxjzN8iMgqIMMYsBV4AZovIc9i6jfoaY67uPlLKcj/vPMHzX2zB20uYF9aY5lVKWB2S29q/fz/9+/dnxYoVtGrVivbt21sdkttz6tcZY8wybIPA6Z97Pd3yTqC5M2NQKivS0gyTf93LlF/3UrucP9MfbkD5ogWsDsttffzxxwwePBgvLy9mzJhB//79tUhcDtB2rVKZiL+QzLNfbGbl7ljub1ie0d1r4eujpaOdqWzZsrRt25bp06dTvnx5q8PxGJoIlMrAP8fO8uS8jRyLv8hb3WvxcOOKWrLACS5dusSYMWNIS0tj5MiR3Hnnndx5551Wh+VxtM2l1FWWbDnCvdPWkpicyoIBTXikSSVNAk6wYcMGGjZsyBtvvMH+/fvR4UHraCJQyi45NY23vtvJMwu2UKtcYb57ugUNKxWzOiy3c+HCBYYNG0aTJk04ffo0S5cu5ZNPPtFkayHtGlIKiD2XxJDPNrH+wCn6Ngvg/+6qQV5v/Z7kDAcOHGDq1Kn079+fsWPH4u/vb3VIHk8TgfJ4Ww6fYeC8jZy+cImJvepybwMdpMxu8fHxLFq0iMcff5yaNWsSGRlJhQoVbvxGlSP0K4/yaAv+OkSvGX/i7SV8PaiZJgEn+P7776lZsyb9+vVj165dAJoEchlNBMojJaWkMmLRNoYv2k7joGJ8O6QFtcppF0V2io2NpU+fPnTp0oWiRYvy559/EhwcbHVYKgPaNaQ8zrH4iwz8dBNbD59hcOvKvNChOl5aNTRbpaam0qJFCw4cOMCbb77J8OHDyZs3r9VhqUxoIlAe5c99cQz5bBOJyanMeLgBnWrdZnVIbuX48eOUKlUKLy8vJkyYQEBAALVq1bI6LHUD2jWkPIIxhg/+2M/DH67Hv4APS4Y01ySQjdLS0pg5cybVqlVj5syZAHTp0kWTgIu4YYtARPIDzwKVjDEDRaQKUNUY84PTo1MqG1y4lMLwr7ezdOtROoSUZkKvuvj56ixi2SUyMpL+/fvz22+/0bZtWzp27Gh1SOomOdI1NAfYDrSwrx8FvgI0Eahc72DceZ6ct5HdJ87xYsfqDGpVWWcRy0YfffQRgwcPJm/evMyePZuwsDC9McwFOZIIqhpjHhSRngDGmAuiv2nlAlbujuGZzzcjIsx9vBGtqunsdtmtYsWKdOzYkfDwcMqVu3omWuUqHEkEl+wzhxkAEQkELjk1KqWyIC3N8P7KSCb9socaZQoz85GGVCimpaOzQ1JSEv/73/9IS0tj1KhRtGvXjnbt2lkdlsoiRxLBW8CPQHkR+RhoBfRzalRK3aKzick8/8VWfvnnBD3ql+OdHrXJn1dLR2eH9evXExYWxt9//81jjz2GMUa7gdzEDROBMeYHEYkAmmGbkP5FY4zOKaxynT0nzvHkvI0cPnWBkV1DeKxZgJ6ossH58+d57bXXmDx5MuXKleO7777j7rvvtjoslY0cuWroJ2NMB2BJBs8plSt8v+0YLy7cSoG83nzWvwmNArVqaHY5ePAg06ZNY+DAgYwZM4bChQtbHZLKZpkmAvuE875AaRHxw9YaACgMVMyB2JS6oZTUNN79aTczf99P/YpFmN6nIWX8fa0Oy+WdOXOGhQsX0q9fP0JCQoiMjNQZw9zY9VoETwHPA6WAv/k3EZwFZjg5LqVu6NT5Swz9fBNrIuPo07gir3cNIZ+3jgdk1ZIlSxg0aBAxMTG0aNGC4OBgTQJuLtM7i40xk4wxFYCXjTEVjTEV7I+axpjJORijUtfYHh1P16mr2RB1mnH31eHtHrU1CWRRTEwMvXv3pnv37pQsWZJ169ZpkTgP4chg8WQRCQZCsHUVXX7+M2cGplRmFm6M5v8Wb6dEwbwsHNiUOuWLWB2Sy0tNTaV58+YcOnSI0aNH89JLL+Hjo3dfewpHBotfBToAwcByoCOwGtBEoHLUxUupvL1sJ5+uO0SzysWZ+mB9ihfKZ3VYLu3o0aOUKVMGLy8v3nvvPQICAggJCbE6LJXDHCk69wDQBjhmjHkEqItWLVU57M99cXR6bxWfrjvEgJZBfPJEI00CWZCWlsb06dMJDg5mxgzbkN9dd92lScBDOXJCv2iMSRWRFPvVQ8eBICfHpRQA5xKT+d8Pu/hs/SEqFS/A5/2b0LRycavDcml79uyhf//+rFq1ivbt29O5c2erQ1IWcyQRbBaRItiKz0Vgu2pok1OjUgpYuSuG/1u8nRNnE+nXIpAXOlTXu4Sz6MMPP2TIkCH4+voyZ84c+vbtqzfdqesnAntxuZHGmDNAuIgsBwobYzQRKKc5ff4So77byeLNR6haqhDTBjWjfsWiVoflFgICAujcuTPh4eHcdpvOx6BsxBhz/Q1ENhpjGuZQPDcUGhpqIiIirA5DOYExhmXbj/PG0h2cuZDM4NaVeaptFb0sNAuSkpJ46623ABg9erTF0Sgr2c/loRm95kjX0F8i0kBbAcqZYs4m8tqSHSz/+wS1y/kzL6wxNW7TUgZZsXbtWsLCwti1axdPPPGEFolTmXIkEbQA+ovIPuA8tjuMjTGmgVMjUx7BGMPCjdG89d1OElPSGN45mH4tAvH20llUb1VCQgKvvPIKU6dOpUKFCvz44486a5i6LkcSQfdb/XAR6QS8B3gBHxhjxmSwTS9gJLb5DrYaYx661f0p1xJ9+gL/t3gHq/bEcntAUcbeV4egkoWsDsvlHTp0iJkzZ/LUU0/xzjvv4OfnZ3VIKpdz5M7ifbfywSLiBYQDdwLRwAYRWWqM2Zlum6rACKC5Mea0iJS6lX0p15KWZpi37iBjf9wFwKhuNXm4cSWdQjILTp8+zVdffcWAAQMICQlh//79lC1b1uqwlItw5o1hjYBIY8x+ABFZAHQDdqbbpj8Qbow5DaDzHLi//bEJvPz1NjZEneaOqiX43721KV9UZw/LisWLFzN48GBiY2Np1aoV1atX1ySgboozO2LLAYfTrUfbn0uvGlBNRNaIyDp7V9I1RGSAiESISERsbKyTwlXOlJKaxvTf9tHpvT/Yffwc795fh0+eaKRJIAuOHz9Oz549uffeeylTpgx//fUX1atXtzos5YIcahGISHlsk9ivFJF8gLcx5vyN3pbBc1dfq+oNVAVaA+WBP0Sklv2+hX/fZMwsYBbYLh91JGaVe/xz7CwvLdzG9iPxdKxZmre61aJUYZ0zICtSU1O54447OHz4MO+88w7Dhg3TInHqljlSdO4JYAjgD1QGKgHTgPY3eGs0UCHdenngaAbbrDPGJAMHRGQ3tsSwwaHoVa6WlJJK+IpIpv22jyIFfAh/qAF31S6jlzBmQXR0NGXLlsXLy4spU6YQGBiopaJVljnSNfQ00ARbaQmMMXuwTVZzIxuAqiISaJ/trDew9KptvsFW0A4RKYGtq2i/Y6Gr3GzzodN0mbKaKSsi6Vq3LD8/14q769ymSeAWpaWlMXXqVIKDg5k+fToAnTt31iSgsoUjXUOJxphLl/8D268GuuH/ZmNMiogMwVa62guYY4z5W0RGARHGmKX21zqIyE4gFXjRGBN3i8eicoGLl1KZ8NNu5qw5QOnCvnzU93baBOvFYFmxa9cu+vXrx5o1a+jYsSNdunSxOiTlZhxJBGtE5CXAV0TaYJvC8jtHPtwYswxYdtVzr6dbNtimw3ze4YhVrvXnvjiGL9rGwbgL9FzqwaQAACAASURBVGlckeGdg/Hz1X7rrPjggw8YMmQIBQoU4OOPP+aRRx7RVpXKdo4kgpeAAcAu4Bls3+JnOjMo5Vq0VLTzVK5cma5du/L+++9TunRpq8NRbsqRonNdgR/tA7qW06JzuUv6UtFhLQJ5/k4tFZ0ViYmJjBo1CoB33nnH4miUO7le0TlHBot7AZEi8pGIdLSPESgPd/r8JZ5dsJnH527Az9ebRYOb88rdIZoEsmDNmjXUq1eP//3vf8TGxnKjL2lKZZcbJgL79JTVgG+BJ4D9IjLD2YGp3MkYw3fbjtJ+4u98t+0YT7eryrdDW1Cvgk4gf6vOnTvH0KFDueOOO0hKSmL58uXMnj1bxwJUjnHohjJjTJKILAEuYrsCqBcw0JmBqdwn5mwir36zg5922kpFf9pPS0Vnh+joaD744AOGDh3K22+/TaFCWnhP5SxHbihrj+0egPbAGuATQCuEehBjDF9tjGb0dztJSkljROdgwrRUdJbExcXx5ZdfMmjQIGrUqMH+/ft1xjBlGUdaBAOBBcBQY8xFJ8ejcpnDpy7wf4u388fekzQKKMaY+2prqegsMMbw9ddf89RTT3Hq1Cnatm1L9erVNQkoSzlShvr+nAhE5S7pS0UL8Fa3mvTRUtFZcuzYMZ566ikWL15Mw4YN+emnn7RInMoVMk0EIvK7MaaViJzmv8XiLs9QVszp0SlL7ItNYLi9VHTLaiV5p0ctrRKaRZeLxB05coRx48bx3HPP4e3tzCrwSjnuen+Jbez/lsiJQJT1UlLTmP3HASb9sof8Pl6M71mX+xqU06tXsuDw4cOUK1cOLy8vwsPDCQwMpFq1alaHpdR/ZDraZ4xJsy9+aIxJTf8APsyZ8FRO2Xn0LN2nrWHsj7toW70UPz/fkvsbltckcItSU1OZMmXKf4rEdezYUZOAypUcaZvWSb9iv6HsdueEo3JaUkoq76+IZLq9VPS0Pg24q7YOXGbFP//8Q1hYGH/++SedO3ema9euVoek1HVdb4zgZWA44Ccipy4/jW28QFsEbmBb9Ble+HIre2MSuLd+OV7rEkLRgnmtDsulzZo1i6FDh+Ln58e8efPo06ePtqpUrne9FsE4YALwP2wJAQB715BycfEXknlszl/4+nhpqehsVLVqVXr06MGUKVMoVUp/pso1XC8RVDHG7BWReUDNy09e/nZjjNnm5NiUE72/ci9nLibzXb/G1Czrb3U4LuvixYuMHDkSEWHMmDG0adOGNm3a3PiNSuUi10sEw4EwIDyD1wzQ0ikRKac7GHeeuWuj6NmwvCaBLFi1ahX9+vVj7969DBw4EGOMdgMpl5RpIjDGhNn/vSPnwlE5YcwPu/DxysMLHfRmpltx9uxZhg8fzvTp0wkKCuLXX3+lbdu2Voel1C27YbEYEblXRPzsy8NF5EsRqev80JQz/HXgFD/sOM7AVpUpXdjX6nBc0tGjR5k7dy7PP/8827Zt0ySgXJ4jVcNGGmPOiUgzoCvwBTpDmUtKSzOM/n4nZQr70v+OIKvDcSknT55k2rRpAAQHB3PgwAEmTJhAwYIFLY5MqaxzJBFcvkqoCzDNGPM1kM95ISlnWbL1CNui43mpk84i5ihjDF988QUhISE8++yz7NmzB0CnjVRuxZFEcExEwrGVol4mInkdfJ/KRS5eSmXcj7upXc6f7vXKWR2OSzh69Cjdu3end+/eVKpUiY0bN+qdwcotOXJncS/gLmCqMea0iJQl3X0FyjV88Md+jsUnMvmBelpB1AGpqam0bNmSI0eOMH78eJ555hktEqfcliNlqBNEZCfQWkRaA38YY35wemQq28ScTWT67/voVLMMjYOKWx1Ornbw4EHKly+Pl5cX06ZNIygoiCpVqlgdllJO5chVQ0OAL4GK9seXIjLY2YGp7DPhpz0kp6YxvHOw1aHkWqmpqUycOJEaNWpcKRLXoUMHTQLKIzjS1h0ANDLGJACIyDvAWmCaMwNT2WPn0bN8ufEwYc0DCSihV7hkZMeOHYSFhfHXX3/RpUsXunfvbnVISuUoRwZ9BUhOt55sf07lcsbYLhf1z+/D0LZVrQ4nV5oxYwYNGjRg//79fPbZZyxdupTy5ctbHZZSOcqRFsE8YJ2IfI0tAXQHPnZqVCpbrNgVw9p9cYzsGoJ/AR+rw8lVLpeDqFGjBj179mTy5MmULFnS6rCUsoQYY268kcjtwOVSE38YYzY4NarrCA0NNREREVbt3mUkp6bRafIqjIHlz7XEx0uv+AW4cOECr7/+Ol5eXowdO9bqcJTKMSKy0RgTmtFrjp4dkuyPi/Z/VS73+V+H2Bd7nhF31dAkYPfbb79Rp04dJkyYQEJCAo58CVLKEzhy1dArwOfAbUB54DMRGeHswNSti7+YzKSf99A0qDjta2hN/Pj4eJ588skr5aFXrFhBeHi4VgpVys6RMYKHgYbGmAsAIvI2sBHbhDUqFwpfGcmZi8m8cncNPdkBx44d49NPP2XYsGG8+eabFChQwOqQlMpVHOkzOMh/E4Y3sN+RDxeRTiKyW0QiRSTTu5FF5H4RMSKSYf+VctzBuPPMXRPF/Q3KU6uc5841EBsby9SpUwFbkbioqCjeffddTQJKZcCRRHAB+FtEPhCR2cB24IyITBSRiZm9yT7JfTjQGQgBHhSRkAy28wOeBtbfygGo/xr74y688gjDOnrmXAPGGD777DNq1KjBCy+8cKVInF4RpFTmHEkE3wMjgT+BdcAoYAXwt/2RmUZApDFmvzHmErAA6JbBdm9hmx850fGwVUY2RJ1i2XbPnWvg8OHDdO3alT59+lClShU2b96sReKUcoAjtYY+vMXPLgccTrceDTROv4GI1AcqGGO+E5FhmX2QiAzAdoczFStWvMVw3FtammH0d/a5BloGWh1OjktJSaF169YcP36cSZMmMXToULy8tNS2Uo5wZjnFjEYpr1yvJyJ5gElA3xt9kDFmFjALbPcRZFN8bmXp1qNsjY5nQs+6FMjrOVUyo6KiqFChAt7e3sycOZOgoCCCgnTSHaVuhjMvMI8GKqRbLw8cTbfuB9QCfhORKKAJsFQHjG/exUupjP1xF7XKFaZHfc+YayAlJYXx48dTo0aNKzOHtW/fXpOAUrfA4a+OIpLPGHMzN5NtAKqKSCBwBNvENg9dftEYEw+USPf5vwHDjDF62/BN+nC1ba6BSR4y18C2bdsICwsjIiKCbt26cd9991kdklIuzZEbyhqJyHZgr329rohMvdH7jDEpwBBgOfAP8KUx5m8RGSUi92QxbmUXcy6Rab/to2PN0jTxgLkGpk2bRsOGDTl48CBffPEFixcvpmzZslaHpZRLc6RFMAXbfMXfABhjtopIG0c+3BizDFh21XOvZ7Jta0c+U/3XxCtzDdSwOhSnulwkrlatWvTu3ZtJkyZRokSJG79RKXVDjiSCPMaYg1fdoZqa2cYq5+w8epYvIg7zRPNAAt10roHz58/z6quv4u3tzbvvvkvLli1p2bKl1WEp5VYcGSw+LCKNACMiXiLyLLDHyXGpGzDG8PYy21wDT7vpXAO//vortWvXZvLkySQlJWmROKWcxJFEMAh4Hts0lSewXd0zyJlBqRtbuTuGNZFxPNOuqtvNNXDmzBn69etH+/bt8fb2ZtWqVUyZMkXrJinlJI7cUBaD7YoflUskp6bx9vf/EFSiIA83qWR1ONnuxIkTLFiwgJdffpk33niD/PnzWx2SUm7thonAXl/omja5MWaAUyJSN3R5roHZj4a6zVwDl0/+zzzzDNWrVycqKkoHg5XKIY6cRX4BfrU/1gCl0MlpLBN/MZnJv+ylSVAxt5hrwBjDp59+SkhICC+99BJ79+4F0CSgVA5ypGvoi/TrIjIP+NlpEanrmrYyktMXLvHq3SEu32d+6NAhBg4cyA8//EDTpk358MMPqVrVPQe+lcrNbqUoTSDgfh3TLuBQ3AU+WhPFfW4w18DlInExMTFMmTKFwYMHa5E4pSziyBjBaf4dI8gDnAIynWRGOc+VuQY6uO5cA/v376dSpUp4e3sze/ZsKleuTEBAgNVhKeXRrjtGILa+h7pASfujqDEmyBjzZU4Ep/4VEXWK77cf48lWQZTxd725BlJSUhg7diwhISGEh4cD0K5dO00CSuUC120RGGOMiCw2xjTMqYDUtdLSDG99/w+lC+djQEvXq665ZcsWwsLC2LRpEz169KBnz55Wh6SUSseRq4b+EpEGTo9EZerbbUfZevgML3YMdrm5Bt5//31uv/12jhw5wsKFC1m0aBG33Xab1WEppdLJ9KwiIt72CqItgP4isg84j23CGWOM0eSQAxKTUxn7g22ugXtdaK6By0Xi6tSpQ58+fZg4cSLFihWzOiylVAau9/XyL6AB0D2HYlEZ+HD1AY7GJzKhl2vMNZCQkMArr7yCj48P48eP1yJxSrmA63UNCYAxZl9GjxyKz6PFnEtk2spIOoSUpmnl3D/XwE8//UStWrWYOnUqycnJWiROKRdxvRZBSRF5PrMXjTETnRCPSuf9FZEkpaQx4q7cPdfA6dOnef7555k7dy7Vq1dn1apVtGjRwuqwlFIOul6LwAsohG1u4YweyonS0gzLth+jU60yuX6ugZiYGBYuXMiIESPYsmWLJgGlXMz1WgTHjDGjciwS9R/bjsRzMuES7XJpPaHjx4/z+eef89xzz10pEle8eO7vvlJKXeuGYwTKGit2xZBHoFW13JUIjDF8/PHHhISEMGLEiCtF4jQJKOW6rpcI2uVYFOoaK3fFUL9iUYoVzGt1KFdERUXRqVMn+vbtS0hICFu2bNEicUq5gUy7howxp3IyEPWvmLOJbD8Sz4sdc09NoZSUFNq0acPJkycJDw9n4MCB5MnjHnMhKOXpXOs2VQ+xcncMAG2qW98tFBkZSWBgIN7e3syZM4egoCAqVdLis0q5E/1Klwut2BXDbf6+1LjNuouzkpOTeeedd6hZs+aVInFt2rTRJKCUG9IWQS6TlJLK6r0nuadeOcsmntm0aRNhYWFs2bKFnj178sADD1gSh1IqZ2iLIJfZcOA05y+l0jbYmm6hKVOm0KhRI44fP86iRYv48ssvKV26tCWxKKVyhiaCXGbFrhjyeueheZWcvRzzcjmI+vXr8+ijj7Jz50569OiRozEopayhXUO5zMrdMTQNKp5j5abPnTvHiBEjyJcvHxMmTOCOO+7gjjvuyJF9K6VyB20R5CIHTp7nwMnzOdYt9OOPP1KrVi2mTZuGMUaLxCnloTQR5CIrdtkuG3V2IoiLi+Oxxx6jc+fOFCxYkDVr1jBx4kTLBqeVUtbSRJCLrNh1giqlClGhWAGn7icuLo7Fixfz2muvsXnzZpo2berU/SmlcjenJgIR6SQiu0UkUkSGZ/D68yKyU0S2icivIuKxF6knJKXw14FTTmsNHDt2jPHjx2OMoVq1ahw8eJBRo0aRL18+p+xPKeU6nJYIRMQLCAc6AyHAgyISctVmm4FQY0wdYCEwzlnx5Har98aSnGqy/W5iYwxz5syhRo0avPbaa0RGRgJQtGjRbN2PUsp1ObNF0AiINMbsN8ZcAhYA3dJvYIxZaYy5YF9dB5R3Yjy52opdMfj5ehMakH0n6AMHDtChQwfCwsKoW7cuW7du1SJxSqlrOPMaxXLA4XTr0UDj62wfBvyQ0QsiMgAYAFCxYsXsii/XSEszrNwdS8uqJfHxyp7cnJKSQtu2bYmLi2P69OkMGDBAi8QppTLkzESQ0SUoGV6fKCIPA6FAq4xeN8bMAmYBhIaGut01jn8fPUvsuaRsGR/Yu3cvQUFBeHt789FHH1G5cmUqVKiQDVEqpdyVM78iRgPpz0DlgaNXbyQi7YFXgHuMMUlOjCfXWrErBhFoXb3kLX9GcnIyo0ePplatWrz//vsAtG7dWpOAUuqGnNki2ABUFZFA4AjQG3go/QYiUh+YCXQyxsQ4MZZcbcXuGOqWL0LxQrd2BU9ERARhYWFs27aN3r178+CDD2ZzhEopd+a0FoExJgUYAiwH/gG+NMb8LSKjROQe+2bvAoWAr0Rki4gsdVY8uVXsuSS2Hj5zy91C7733Ho0bN+bkyZMsWbKEzz//nFKlrJ/HQCnlOpxa0MYYswxYdtVzr6dbbu/M/buC33bf2t3ExhhEhNDQUMLCwhg3bhxFihRxRohKKTenRecstnJ3DKX88lGzbGGHtj979iwvv/wyvr6+TJo0iebNm9O8eXMnR6mUcmd6PaGFklPT+GPPSdpUL+VQnZ9ly5ZRs2ZNZs2ahbe3txaJU0plC00EFtoQdYpzSSm0uUG30MmTJ3n44Ye5++678ff3Z+3atbz77rtaJE4plS00EVho5a4Y8nrloUXVEtfd7vTp03z77be88cYbbNq0icaNr3dfnlJK3RwdI7DQil0xNA4qRqF81/4ajhw5wvz583nxxRepWrUqBw8e1MFgpZRTaIvAIofiLrAv9vw1ReaMMcyePZuQkBBGjhzJvn37ADQJKKWcRhOBRVbsOgH897LRffv20a5dOwYMGECDBg3Ytm0bVapUsSpEpZSH0K4hC/yy8wTjf9pDcBk/AkoUBGxF4tq1a8epU6eYOXMm/fr10yJxSqkcoYkgB6WlGcJXRjLxlz3UKuvPzEcasnv3bipXroy3tzcff/wxlStXpnx5j63GrZSygH7lzCHnk1IYPH8TE37eQ/d65Zj/RENmvzeO2rVrEx4eDkCrVq00CSilcpy2CHLAobgL9P8kgr0x53j17hrUzhtL8yaN2LFjBw899BB9+vSxOkSllAfTFoGTrd57kq7vr+b42UQ+fqIRCRuX0qxZsyv3BsyfP58SJa5/H4FSSjmTtgicxBjDh6sP8M6yf6hayo+ZjzQgoEQhvBo1on///owdOxZ/f3+rw1RKKcTV6tWEhoaaiIgIq8O4rsTkVEYs2s7izUdoV704rPsEv/x5mTx5stWhKaU8lIhsNMaEZvSadg1ls6NnLtJr5p8s3nyEuyum8eOr9zP3g5nky5dPi8QppXIl7RrKRhuiTjHo041cvJRCcOxvTBs7ntq1a/PNN4u5/fbbrQ5PKaUypC2CbDJ//UEemr0OP18fpnStyLqvZ/Pmm28SERGhSUAplatpiyCLLqWkMfLbv/ls/SEqeJ/lm8Ht8S+Ql4MHD+pgsFLKJWiLIAtizyXx0Ox1fLb+EBc3LWHT5P7EHj0EoElAKeUytEVwi7ZFnyHso/WcPHuR2O8n0bSsD7O2bSMoKMjq0JRS6qZoIrgFizdHM/zr7STFn+TcsvFMfeUZHn/8cZ0xTCnlkjQR3ISU1DRemr+WRTvjaRJUjEeCClB/xA+ULVvW6tCUUuqWaSJw0InTCXQft5Rjxp96Bc4wL6wzPl46xKKUcn2aCBzw5fLVvPzdftLyFab6ub/48MWnNQkopdyGJoIbGPTOTL4/WRwRL166PS9PPfCm1SEppVS20kSQiZSUVKasiOSHs+Upluc0C1/uSJVyJa0OSymlsp0mgqucOXOGZ4a9zK4iTTjhXYqeDcvzVvdO+Pp4WR2aUko5hXZ0p/PNN98Q0rgNv0h9YrxLMrJrCOPur6NJQCnl1rRFAMTExDBkyBC+3bifMt1foVDBAsx8rBHNKuuEMUop96eJAIiPj+f3mHyU7jmS4DKFmfVoKBWKFbA6LKWUyhEe2zV06NAh3n77bS4kpTAlIoH8TXrTpU5ZFg1urklAKeVRnNoiEJFOwHuAF/CBMWbMVa/nAz4BGgJxwAPGmChnxpSWlsaMGTN4+eWXoWBxVuRtzP5TSbzcKZiBrYK0TIRSyuM4LRGIiBcQDtwJRAMbRGSpMWZnus3CgNPGmCoi0hsYCzzgrJh2795N//79+eOPP2jeoy/n6jxAzPlU5vS9nTbVSzlrt0oplas5s0XQCIg0xuwHEJEFQDcgfSLoBoy0Ly8E3hcRMU6Y0/Hz9VGMmPsHqcF9CG39AsdSfAjw82X2o6EElSyU3btTSimX4cxEUA44nG49Gmic2TbGmBQRiQeKAyfTbyQiA4ABABUrVrylYIoX8qVRcAUKFixE/vy+lC7sy/N3VsPP1+eWPk8ppdyFMxNBRp3tV3/Td2QbjDGzgFkAoaGht9Ra6FCzDB1qlrmVtyqllFtz5lVD0UCFdOvlgaOZbSMi3oA/cMqJMSmllLqKMxPBBqCqiASKSF6gN7D0qm2WAo/Zl+8HVjhjfEAppVTmnNY1ZO/zHwIsx3b56BxjzN8iMgqIMMYsBT4E5olIJLaWQG9nxaOUUipjTr2PwBizDFh21XOvp1tOBHo6MwallFLX57F3FiullLLRRKCUUh5OE4FSSnk4TQRKKeXhxNWu1hSRWODgLb69BFfdtewB9Jg9gx6zZ8jKMVcyxmQ4367LJYKsEJEIY0yo1XHkJD1mz6DH7BmcdczaNaSUUh5OE4FSSnk4T0sEs6wOwAJ6zJ5Bj9kzOOWYPWqMQCml1LU8rUWglFLqKpoIlFLKw7llIhCRTiKyW0QiRWR4Bq/nE5Ev7K+vF5GAnI8yezlwzM+LyE4R2SYiv4pIJSvizE43OuZ0290vIkZEXP5SQ0eOWUR62X/Xf4vIZzkdY3Zz4G+7ooisFJHN9r/vu6yIM7uIyBwRiRGRHZm8LiIyxf7z2CYiDbK8U2OMWz2wlbzeBwQBeYGtQMhV2wwGZtiXewNfWB13DhxzG6CAfXmQJxyzfTs/YBWwDgi1Ou4c+D1XBTYDRe3rpayOOweOeRYwyL4cAkRZHXcWj7kl0ADYkcnrdwE/YJvhsQmwPqv7dMcWQSMg0hiz3xhzCVgAdLtqm27Ax/blhUA7Eclo2kxXccNjNsasNMZcsK+uwzZjnCtz5PcM8BYwDkjMyeCcxJFj7g+EG2NOAxhjYnI4xuzmyDEboLB92Z9rZ0J0KcaYVVx/psZuwCfGZh1QRERuy8o+3TERlAMOp1uPtj+X4TbGmBQgHiieI9E5hyPHnF4Ytm8UruyGxywi9YEKxpjvcjIwJ3Lk91wNqCYia0RknYh0yrHonMORYx4JPCwi0djmPxmaM6FZ5mb/v9+QUyemsUhG3+yvvkbWkW1cicPHIyIPA6FAK6dG5HzXPWYRyQNMAvrmVEA5wJHfsze27qHW2Fp9f4hILWPMGSfH5iyOHPODwFxjzAQRaYpt1sNaxpg054dniWw/f7ljiyAaqJBuvTzXNhWvbCMi3tiak9driuV2jhwzItIeeAW4xxiTlEOxOcuNjtkPqAX8JiJR2PpSl7r4gLGjf9tLjDHJxpgDwG5sicFVOXLMYcCXAMaYPwFfbMXZ3JVD/99vhjsmgg1AVREJFJG82AaDl161zVLgMfvy/cAKYx+FcVE3PGZ7N8lMbEnA1fuN4QbHbIyJN8aUMMYEGGMCsI2L3GOMibAm3GzhyN/2N9guDEBESmDrKtqfo1FmL0eO+RDQDkBEamBLBLE5GmXOWgo8ar96qAkQb4w5lpUPdLuuIWNMiogMAZZju+JgjjHmbxEZBUQYY5YCH2JrPkZiawn0ti7irHPwmN8FCgFf2cfFDxlj7rEs6Cxy8JjdioPHvBzoICI7gVTgRWNMnHVRZ42Dx/wCMFtEnsPWRdLXlb/Yicjn2Lr2StjHPd4AfACMMTOwjYPcBUQCF4DHs7xPF/55KaWUygbu2DWklFLqJmgiUEopD6eJQCmlPJwmAqWU8nCaCJRSysNpIlC5loikisiWdI+A62wbkFm1xpwmIqEiMsW+3FpEmqV7baCIPJqDsdRz9Wqcyvnc7j4C5VYuGmPqWR3EzbLftHb5xrXWQAKw1v7ajOzen4h422tmZaQetpIiy7J7v8p9aItAuRT7N/8/RGST/dEsg21qishf9lbENhGpan/+4XTPzxQRrwzeGyUiY+3b/SUiVezPVxLbPA6X53OoaH++p4jsEJGtIrLK/lxrEfnO3oIZCDxn3+cdIjJSRIaJSA0R+euq49pmX24oIr+LyEYRWZ5RZUkRmSsiE0VkJTBWRBqJyFqx1eRfKyLV7XfijgIesO//AREpKLZ69xvs22ZUsVV5Gqtrb+tDH5k9sN0Zu8X+WGx/rgDga1+uiu3uUoAA7PXbgalAH/tyXiA/UAP4FvCxPz8NeDSDfUYBr9iXHwW+sy9/CzxmX34C+Ma+vB0oZ18uYv+3dbr3jQSGpfv8K+v24wqyL78MvIrtDtK1QEn78w9gu5v26jjnAt8BXvb1woC3fbk98LV9uS/wfrr3vQM8fDleYA9Q0OrftT6sfWjXkMrNMuoa8gHeF5F62BJFtQze9yfwioiUBxYZY/aKSDugIbDBXmIjP5BZzaXP0/07yb7cFLjXvjwP2xwHAGuAuSLyJbDoZg4OW6G0XsAYbCf8B4Dq2Irl/WyP0wvIrI7MV8aYVPuyP/CxvfVjsJckyEAH4B4RGWZf9wUqAv/cZOzKjWgiUK7mOeAEUBdb1+Y1E84YYz4TkfXA3cByEemHrXTvx8aYEQ7sw2SyfM02xpiBItLYvq8t9gTlqC+w1X5aZPsos1dEagN/G2OaOvD+8+mW3wJWGmN62LukfsvkPQLcZ4zZfRNxKjenYwTK1fgDx4yt1vwj2L4x/4eIBAH7jTFTsFVqrAP8CtwvIqXs2xSTzOdtfiDdv3/al9fyb3HCPsBq++dUNsasN8a8Dpzkv+WBAc5hK4l9DWPMPmytmtewJQWwlY0uKba6+oiIj4jUzCTO9PyBI/blvtfZ/3JgqNibG2KrSqs8nCYC5WqmAY+JyDps3ULnM9jmAWCHiGwBgrFN67cTWx/8T/ZB2Z+BzKb3y2dvUTyDrQUC8DTwuP29j9hfA3hXRLbbL11dhW1O3fS+BXpcHizOYF9fAA/zbz39S9hKo48Vka3YxhGuGRDPwDjgfyKyhv8mx5VAyOXBYmwtBx9gmz3mtxz4bOXmtPqoUumINXJk8AAAAD9JREFUbRKbUGPMSatjUSqnaItAKaU8nLYIlFLKw2mLQCmlPJwmAqWU8nCaCJRSysNpIlBKKQ+niUAppTzc/wM4/xJtl5whEQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred = rs_bag_cls.predict(X_test)\n",
    "y_prob = rs_bag_cls.predict_proba(X_test)[:,1] #we only want true positive probability\n",
    "\n",
    "rs.report_matrix(y_test, y_pred, y_prob)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BaggingClassifier(base_estimator=DecisionTreeClassifier(), max_samples=35,\n",
       "                  n_jobs=-1, oob_score=True, random_state=42)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "skl_bag_cls = BaggingClassifier(base_estimator = base_estimate,\n",
    "                                n_estimators = n_estimate, # the number of trees as a estimate\n",
    "                                max_samples = max_samples, #if you put max_sample = 1.0, pls mentally get ready to wait\n",
    "                                max_features = 1.0,\n",
    "                                bootstrap = True, #must always be True\n",
    "                                bootstrap_features=False,\n",
    "                                oob_score = True, # You will need to have bootstrap = True\n",
    "                                warm_start = False,\n",
    "                                n_jobs = -1, # For parallel computering, -1 means use all\n",
    "                                random_state = random_state, #seed figure\n",
    "                                verbose = 0)\n",
    "\n",
    "# Fit and score\n",
    "skl_bag_cls.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Classification Report\n",
      "=======================================================\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "        -1.0       0.64      0.62      0.63       102\n",
      "         1.0       0.74      0.76      0.75       149\n",
      "\n",
      "    accuracy                           0.70       251\n",
      "   macro avg       0.69      0.69      0.69       251\n",
      "weighted avg       0.70      0.70      0.70       251\n",
      "\n",
      "\n",
      " Confusion Matrix\n",
      "=======================================================\n",
      "[[TN, FP]\n",
      " [FN, TP]]\n",
      "\n",
      "[[ 63  39]\n",
      " [ 36 113]]\n",
      "\n",
      " Accuracy Score\n",
      "=======================================================\n",
      "0.701195219123506\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hU1dbA4d8ioRtCbwkhoabQCSAdREEURFQQQQQJIGDH3hCRi6BSBOldREQQFBVFP7xeFKSEXqSEkJBQQwsECGn7+2MGjDGBATI5mcx6n2cezsycmbNOEmbNLmdtMcaglFLKfeWzOgCllFLW0kSglFJuThOBUkq5OU0ESinl5jQRKKWUm9NEoJRSbk4TgVJKuTlNBCrPEZEoEbksIgkiclxE5onIHRn2aSYiv4rIBRGJF5HvRCQ4wz7FRGSCiBy2v1eE/X7pnD0jpZxLE4HKqzobY+4A6gH1gTeuPiEiTYGfgW+BikAAsB1YKyJV7PsUAFYDIcC9QDGgGXAaaOysoEXE01nvrVRWNBGoPM0YcxxYhS0hXPUh8Jkx5hNjzAVjzBljzNvAemC4fZ8nAD+gqzFmjzEmzRhz0hjzvjFmZWbHEpEQEflFRM6IyAkRedP++DwRGZluvzYiEpvufpSIvCYiO4CLIvK2iCzN8N6fiMhE+7a3iMwWkWMickRERoqIx23+qJQb00Sg8jQR8QU6AhH2+0WwfbNfksnuXwH32LfvBn4yxiQ4eBwv4P+An7C1Mqpha1E46jHgfqA4sAC4T0SK2d/bA+gOfGHfdz6QYj9GfaA90P8mjqXUP2giUHnVNyJyAYgBTgLv2h8vie3v/lgmrzkGXO3/L5XFPlnpBBw3xow1xiTaWxobbuL1E40xMcaYy8aYaGAL8KD9ubuAS8aY9SJSDltie8EYc9EYcxIYD/S4iWMp9Q+aCFRe9aAxxgtoAwTy9wf8WSANqJDJayoAp+zbp7PYJyuVgIO3FKlNTIb7X2BrJQD05O/WQGUgP3BMRM6JyDlgOlD2No6t3JwmApWnGWP+B8wDPrbfvwj8CXTLZPfu/N2d839ABxEp6uChYoCqWTx3ESiS7n75zELNcH8J0MbetdWVvxNBDHAFKG2MKW6/FTPGhDgYp1L/oolAuYMJwD0icnXA+HWgj4g8JyJeIlLCPpjbFHjPvs8CbB+6X4tIoIjkE5FSIvKmiNyXyTG+B8qLyAsiUtD+vk3sz23D1udfUkTKAy/cKGBjTBzwGzAXOGSM+cv++DFsM57G2qe35hORqiLS+hZ+LkoBmgiUG7B/qH4GvGO//wfQAXgI2zhANLZB1xbGmAP2fa5gGzDeC/wCnAc2Yuti+lffvzHmAraB5s7AceAA0Nb+9AJs01OjsH2IL3Yw9C/sMXyR4fEngALAHmxdXUu5uW4spf5BdGEapZRyb9oiUEopN6eJQCml3JwmAqWUcnOaCJRSys25XIGr0qVLG39/f6vDUEopl7J58+ZTxpgymT3nconA39+f8PBwq8NQSimXIiLRWT2nXUNKKeXmNBEopZSb00SglFJuzuXGCDKTnJxMbGwsiYmJVoeiblKhQoXw9fUlf/78VoeilNvKE4kgNjYWLy8v/P39ERGrw1EOMsZw+vRpYmNjCQgIsDocpdyW07qGRGSOiJwUkV1ZPC8iMtG+IPgOEWlwq8dKTEykVKlSmgRcjIhQqlQpbckpZTFnjhHMw7bod1Y6AtXtt4HA1Ns5mCYB16S/N6Ws57SuIWPMGhHxv84uXbAtIG6A9SJSXEQq2OutK6VUtou/nMyeo+fZfTSe85eTrQ7HYUnJyVy6dImH76xB3UrFs/39rRwj8OGfy/PF2h/7VyIQkYHYWg34+fnlSHA3y8PDg9q1a5OSkkJAQAALFiygePHb/4VFRUXRqVMndu3KtIftlg0fPpyZM2dSpoztQsN7772X0aNHZ+sxrtq2bRtHjx7lvvsyW89FKec4lXCF3UfPs+tIPLuPxrPryHkOn7l07XmXaYwaMCYNgOq+ZfNcIsjs15Dp4gjGmBnADIDQ0NBcuYBC4cKF2bZtGwB9+vRh8uTJvPXWWxZHdX0vvvgiL7/88k2/LjU1FQ8PD4f337ZtG+Hh4ZoIlFMYYzgWn/ivD/3j5/8ee6pcqgi1fbzp0bgSIRW9CalYjNJ3FLQw6hs7d+4cr7zyCrNmzaJatWrMmjWL1k39nXIsKxNBLLYFv6/yBY5aFEu2atq0KTt27AAgISGBLl26cPbsWZKTkxk5ciRdunQhKiqKjh070qJFC9atW4ePjw/ffvsthQsXZvPmzfTr148iRYrQokWLa++bmJjI4MGDCQ8Px9PTk3HjxtG2bVvmzZvHN998Q2pqKrt27eKll14iKSmJBQsWULBgQVauXEnJkiUdin316tW8/PLLpKSk0KhRI6ZOnUrBggXx9/enX79+/PzzzzzzzDM0atSIp59+mri4OIoUKcLMmTMJDAxkyZIlvPfee3h4eODt7c3//d//MWzYMC5fvswff/zBG2+8waOPPuqUn7vK+9LSDIfPXLJ96B+Nt3/wn+fMxSQA8glULXMHTauWIqRiMUIqehNcsRjehV1renJqairNmjVj3759vPrqqwwfPpzChQs77XhWJoIVwDMi8iXQBIjPjvGB977bzZ6j5287uPSCKxbj3c6OrQ2emprK6tWrCQsLA2zz5JcvX06xYsU4deoUd955Jw888AAABw4cYNGiRcycOZPu3bvz9ddf8/jjj/Pkk08yadIkWrduzSuvvHLtvSdPngzAzp072bt3L+3bt2f//v0A7Nq1i61bt5KYmEi1atUYM2YMW7du5cUXX+Szzz7jhRf+vUzu+PHj+fzzzwEYM2YMrVu3pm/fvqxevZoaNWrwxBNPMHXq1GuvLVSoEH/88QcA7dq1Y9q0aVSvXp0NGzYwZMgQfv31V0aMGMGqVavw8fHh3LlzFChQgBEjRhAeHs6nn356Kz9+5aZSUtOIPHXx2of9riPx7Dl6ngtXUgDI7yHUKOfFPUHlqOVTjOCK3gRV8KJIAdedFX/69GlKliyJh4cH//nPf6hUqRKhoaFOP67TfmIisghoA5QWkVjgXSA/gDFmGrASuA+IAC4BTzorlpxw+fJl6tWrR1RUFA0bNuSee+4BbM3WN998kzVr1pAvXz6OHDnCiRMnAAgICKBePdt66g0bNiQqKor4+HjOnTtH69a2tch79+7Njz/+CMAff/zBs88+C0BgYCCVK1e+lgjatm2Ll5cXXl5eeHt707lzZwBq1659rXWSUcauoe3btxMQEECNGjWAv7u4riaCq9/kExISWLduHd26dbv22itXrgDQvHlz+vbtS/fu3XnooYdu62eq3MeVlFQOnEj4+0P/aDx/HTtPYrKtb7xQ/nwEVSjGg/V9CKlYjFo+3lQvdwcFPR3voszNjDEsXLiQ559/ntGjRzNgwAC6du2aY8d35qyhx27wvAGezu7jOvrNPbtdHSOIj4+nU6dOTJ48meeee46FCxcSFxfH5s2byZ8/P/7+/tfmzRcs+HcfpYeHB5cvX8YYk+WUyuutL53+vfLly3ftfr58+UhJSXHoHG60fnXRokUBSEtLo3jx4tfGRNKbNm0aGzZs4IcffqBevXqZ7qPc26WkFP46doHdR+PZfcT2ob//xAWSU21/f14FPQmuWIxeTSpf+9CvUroonh55syJOTEwMgwYNYuXKldx55500b948x2Nw3TZULuXt7c3EiRPp0qULgwcPJj4+nrJly5I/f37++9//Eh2dZSVYAIoXL463tzd//PEHLVq0YOHChdeea9WqFQsXLuSuu+5i//79HD58mJo1a7Jly5ZsiT0wMJCoqCgiIiKoVq0aCxYsuNYySa9YsWIEBASwZMkSunXrhjGGHTt2ULduXQ4ePEiTJk1o0qQJ3333HTExMXh5eXHhwoVsiVG5lvTTNa927xyMSyDN/p2jZNEChFQsRv+WVWwf+hW98StZhHz5XGVKz+1ZtGgRTz31FKmpqUyYMIFnnnnmpiZiZBdNBE5Qv3596taty5dffkmvXr3o3LkzoaGh1KtXj8DAwBu+fu7cudcGizt06HDt8SFDhjBo0CBq166Np6cn8+bN+0dL4HYVKlSIuXPn0q1bt2uDxYMGDcp034ULFzJ48GBGjhxJcnIyPXr0oG7durzyyiscOHAAYwzt2rWjbt26+Pn5MXr0aOrVq6eDxXlYxumau4+eJ/r039M1yxcrRC2fYtxXu8K1b/oVvAu59UWFJUqUoEmTJsyYMcPSMityo+6A3CY0NNRkXJjmr7/+IigoyKKI1O3S35/rOZ+YzIbIM//40D8W//d0Tb+SRajlU+zaVM2Qit6U8crd0zVzQkpKCuPHjycpKena9PLrdQdnJxHZbIzJdORZWwRKKYfFnLnE3LVRLN50mItJqYh9umaTgJLU8rFN1Qyp4I13EdearpkTtm/fTlhYGJs3b6Z79+7XEkBuaBFpIlBKXZcxhi2HzzLr90Os2n2cfCJ0qlOBHo39qOPr7dLTNXPClStXGDlyJKNHj6ZkyZIsWbKEhx9+OFckgKvyzG8wp5pXKnu5WtekO0lJTeOn3ceZ9fshtsWco1ghTwa2qkqfZpWp4O28i5vymgMHDjBmzBh69uzJuHHjKFWqlNUh/UueSASFChXi9OnTWoraxVxdj6BQoUJWh6LSOZ+YzFebYpi7Nooj5y7jX6oII7qE8HADX4oWzBMfGU6XkJDAt99+S69evahVqxZ79+6lSpUqVoeVpTzxW/X19SU2Npa4uDirQ1E36eoKZcp6V/v/vwqPIeFKCo0DSvJu52DaBZXDw02mc2aHX375hYEDBxIdHU2DBg0ICgrK1UkA8kgiyJ8/v65wpdQt2hx9ltl/RPLTrr/7/8NaVKG2r7fVobmUs2fP8vLLLzNnzhxq1KjB//73P5eZDZcnEoFS6uakpKaxavcJZv0RydbD2v9/u1JTU2nevDn79+/njTfeYNiwYS7V5amJQCk3krH/v3KpIrz3QAiPNNT+/1tx6tSpa0XiRo0ahZ+fHw0a3PKqu5bR37xSbiDmzCXmrYti8Sbt/88OxhgWLFjACy+8wOjRoxk4cCAPPvig1WHdMk0ESuVhm6PPMuePQ/y46xj5RLi/TgXCWgRQxzf7V7lyF9HR0Tz11FOsWrWKZs2a0apVK6tDum2aCJTKY7T/33k+//xzBg8ejDGGSZMmMWTIEPLlc/2qqJoIlMojLiQms1j7/52qTJkyNG/enOnTp1O5cmWrw8k2eaLonFLuLObMJeavi+LLdP3/YS0CuFv7/29bcnIyY8eOJTk5mXfeeQdw3SoGWnROqTxoy+GzzP5d+/+dZevWrYSFhbF161Z69OiRq4rEZTdNBEq5kJTUNH7ec4JZv0eyxd7/P6BVFfo09adice3/zw6JiYmMGDGCDz/8kNKlS/P111/n+WVXNREo5QLiLyWzdEssc9ceIvas9v87U0REBB9//DFPPPEEY8eOpUSJElaH5HT6F6RULpWaZvgj4hRLwmP4ec8JklLSaOxfknc6BWv/fzZLSEhg+fLl9O7dm1q1arFv3z63KlujiUCpXCYyLoGlm2NZtuUIx88nUrxIfno29uORhr7U8tH6P9lt1apVDBw4kJiYGEJDQwkKCnKrJACaCJTKFS4kJvPDjmMs3RxLePRZ8gm0qVmWYZ2DaRdUloKeOb+geV53+vRphg4dymeffUZgYCC///67yxSJy26aCJSySFqaYf2h0ywNj+XHXce5nJxK1TJFeb1jIA/V96FsMdcpWuZqrhaJi4iI4K233uLtt992qSJx2U0TgVI5LObMJZZujuXrLbHEnr2MVyFPujbwoVtDX+pVKp4npyfmFnFxcZQqVQoPDw/GjBlD5cqVqVevntVhWU4TgVI54FJSCj/uPM7SzbH8GXkaEWhRrTSvdKhJh5DyFMqvXT/OZIxh3rx5DB06lNGjR/PUU0/RpUsXq8PKNTQRKOUkxhjCo8+yNDyWH3YeI+FKCpVLFeGle2rwUENffHTef46Iiopi4MCB/PLLL7Rs2ZK2bdtaHVKuo4lAqWx2LP4yy7YcYenmWA6dukiRAh7cX7sC3UIr0ci/hHb95KAFCxYwePBgRIQpU6bw1FNP5YkicdlNE4FS2SAxOZWf95xgSXgMf0ScwhhoElCSIW2qcl/tCnrRl0XKlStHq1atmDZtGn5+flaHk2tp0TmlbpExhu2x8SzdHMOKbUc5n5iCT/HCPNzAh4cb+lK5VFGrQ3Q7ycnJfPjhh6SmpjJs2DCrw8lVtOicUtno5IVEvtl6hCXhsRw4mUBBz3x0rFWebqGVaFqlFPn0il9LbNmyhX79+rF9+3Z69uzpslVCraCJQCkHJKWk8eveEywJj+W3/XGkphka+BXng4dqc3+dChQrlN/qEN3W5cuXee+99/j4448pU6YMy5cvd+llI63g1EQgIvcCnwAewCxjzOgMz/sB84Hi9n1eN8asdGZMSt2M3UfjWRIey7fbjnD2UjJlvQoyoGUVHmnoS7Wyd1gdngIiIyMZN24cffv25aOPPnKLInHZzWmJQEQ8gMnAPUAssElEVhhj9qTb7W3gK2PMVBEJBlYC/s6KSSlHnLmYxLfbbF0/e46dp4BHPu4JLscjob60rFYaTw+ddWK18+fPs2zZMvr27UtISAgHDhzIUyuG5TRntggaAxHGmEgAEfkS6AKkTwQGKGbf9gaOOjEepbKUkprG//bHsSQ8ltV7T5Ccaqjt482ILiE8ULcixYsUsDpEZbdy5UoGDRrEkSNHaNKkCUFBQZoEbpMzE4EPEJPufizQJMM+w4GfReRZoChwd2ZvJCIDgYGATgFT2erAiQsssVf6PJVwhVJFC9CnqT+PhPoSWL7Yjd9A5ZhTp07x4osv8vnnnxMcHMzatWvdtkhcdnNmIshsuD7jXNXHgHnGmLEi0hRYICK1jDFp/3iRMTOAGWCbPuqUaJXbiL+czIrtR1m6OZbtMefwzCe0DSxLt4a+tA0sS37t+sl1rhaJi4yMZNiwYbz55psULFjQ6rDyDGcmgligUrr7vvy76ycMuBfAGPOniBQCSgMnnRiXckOpaYa1EadYsjmWVbuPk5SSRmB5L96+P4gH6/tQ+g79UMmNTpw4QZkyZfDw8ODjjz+mcuXK1KlTx+qw8hxnJoJNQHURCQCOAD2Anhn2OQy0A+aJSBBQCIhzYkzKzRw6dZGlm2NYtuUIx+IT8S6cn8caVeKRhpWo5VNM55nnUsYY5syZw0svvcTo0aMZNGgQnTt3tjqsPMtpicAYkyIizwCrsE0NnWOM2S0iI4BwY8wK4CVgpoi8iK3bqK9xtUudVa6TcCWFH3bYun42RdkWeWldowxv3x/M3cG6yEtuFxkZyYABA/j1119p3bo1d9+d6dChykZOvY7Afk3AygyPDUu3vQdo7swYlHtISzNsOHSGJZtj+HGnbZGXKmWK8tq9gTzUwIdyusiLS5g/fz5DhgzBw8ODadOmMWDAAC0SlwP0ymLl0mLOXOLrLbZFXmLOXMaroCcP1vehW6gv9XWRF5dTsWJF7rrrLqZOnYqvr6/V4bgNLTqnXNLBuASGr9jN7wdOIQLNq5bmkYa+dAgpT+EC2vXjKpKSkhg9ejRpaWkMHz7c6nDyNC06p/KMtDTD3HVRfPjTXgoX8GDoPTV4qIEPviWKWB2aukmbNm2iX79+7Nq1i969e2uROAtpIlAuI+bMJV5esp0Nh87QLrAsHzxUWxd4d0GXLl1i2LBhjB8/ngoVKrBixQqdEWQxTQQq1zPGsGhjDCN/2IOHCB89UodHGvrqt0cXdejQISZNmsSAAQMYM2YM3t7eVofk9jQRqFztWPxlXvt6J2v2x9GiWmnGPFJH1/p1QfHx8Sxbtownn3ySkJAQIiIiqFSp0o1fqHKEJgKVKxljWLblCMO/201KquH9LiH0alJZF31xQT/88ANPPfUUx44do2nTpgQGBmoSyGV0gq7KdeIuXGHggs28tGQ7geW9+PH5lvRu6q9JwMXExcXRq1cvOnXqRIkSJfjzzz8JDAy0OiyVCW0RqFxl5c5jvLV8JxeTUnnrviD6tQjAQxOAy0lNTaVFixYcOnSI9957j9dff50CBbSUd26liUDlCucuJTHs292s2H6UOr7ejOtel2plvawOS92k48ePU7ZsWTw8PBg7diz+/v7UqlXL6rDUDWjXkLLc6r9OcM/4NazceYyX7qnBssHNNAm4mLS0NKZPn06NGjWYPn06AJ06ddIk4CJu2CIQkcLAC0BlY8wgEakGVDfG/Oj06FSedj4xmZHf7+Gr8FgCy3sx78lGhFTUqYSuJiIiggEDBvDbb79x11130aFDB6tDUjfJka6hOcBOoIX9/lFgCaCJQN2ytRGneGXJdo6fT+TptlV5rl11rQrqgubOncuQIUMoUKAAM2fOJCwsTK/vcEGOJILqxpjHRKQbgDHmkuhvWt2iS0kpjP5xL5/9GU2VMkX5enAz6vuVsDosdYv8/Pzo0KEDkydPxsfHx+pw1C1yJBEk2VcOMwD2hWaSnBqVypPCo87w0pLtHD5zibAWAbzSoSaF8msrwJVcuXKFDz74gLS0NEaMGEG7du1o166d1WGp2+RIIngf+AnwFZH5QGugv1OjUnlKYnIq437Zz8zfI/EtUZhFA+7kziqlrA5L3aQNGzYQFhbG7t276dOnjxaJy0NumAiMMT+KSDjQDNuC9K8YY3RNYeWQHbHnGPrVdiJOJtCziR9v3hfEHQV11rIruXjxIu+88w4TJkzAx8eH77//nvvvv9/qsFQ2cmTW0M/GmPbAt5k8plSmklLS+PTXA0z+7SBl7ijI/H6NaV2jjNVhqVsQHR3NlClTGDRoEKNHj6ZYsWJWh6SyWZaJQEQKYFtMvpyIeGFrDQAUA/xyIDblovYeP8/QxdvZc+w8DzXw4d3OIXgXzm91WOomnDt3jqVLl9K/f3+Cg4OJiIjQFcPysOu1CJ4GhgJlgd38nQjOA9OcHJdyQSmpaUxfE8mE/9uPd+H8zOjdkPYh5a0OS92kb7/9lsGDB3Py5ElatGhBYGCgJoE8LstEYIwZD4wXkReMMRNyMCblgg7GJfDSV9vZFnOO+2tX4P0Ha1GyqNaWcSUnT57kueeeY/HixdSpU4cVK1ZokTg34chg8QQRCQSCsXUVXX38C2cGplxDWpph3rooxtiXjpz0WH06161odVjqJqWmptK8eXMOHz7MyJEjefXVV8mfX7vz3IUjg8VvA+2BQGAV0AH4A9BE4OZ06UjXd/ToUcqXL4+HhweffPIJ/v7+BAcHWx2WymGOFJ17FGgLHDPG9AbqolVL3Zoxhi82HKbDhDXsPnqeDx+pw6w+oZoEXEhaWhpTp04lMDCQadNsQ3733XefJgE35cgH+mVjTKqIpNhnDx0Hqjg5LpVL6dKRrm///v0MGDCANWvWcPfdd9OxY0erQ1IWcyQRbBWR4tiKz4VjmzW0xalRqVzHGMPyrUd4d4UuHenKZs+ezTPPPEOhQoWYM2cOffv21auD1fUTgb243HBjzDlgsoisAooZYzQRuJG4C1d4c/lOftlzgkb+Jfjokbr4ly5qdVjqFvj7+9OxY0cmT55MhQoVrA5H5RJijLn+DiKbjTENcyieGwoNDTXh4eFWh+E20i8d+Ur7mrp0pIu5cuUK77//PgAjR460OBplJftneWhmzznSNbRRRBpoK8C9ZFw6cmy3ulQvp6uGuZJ169YRFhbG3r176devnxaJU1lyJBG0AAaIyEHgIrYrjI0xpoFTI1OW+XXvCV77eidnLybx0j01GNymKp4euqqpq0hISOCtt95i0qRJVKpUiZ9++klXDVPX5UgiePBW31xE7gU+ATyAWcaY0Zns0x0Yjm29g+3GmJ63ejx1e3TpyLzh8OHDTJ8+naeffppRo0bh5aUtOXV9jlxZfPBW3lhEPIDJwD1ALLBJRFYYY/ak26c68AbQ3BhzVkTK3sqx1O3TpSNd29mzZ1myZAkDBw4kODiYyMhIKlbUK7yVY5x5YVhjIMIYEwkgIl8CXYA96fYZAEw2xpwF0HUOct4/lo4srUtHuqLly5czZMgQ4uLiaN26NTVr1tQkoG6KMzt+fYCYdPdj7Y+lVwOoISJrRWS9vSvpX0RkoIiEi0h4XFyck8J1P+FRZ+j4ye8sWB9Nv+YB/PBcS00CLuT48eN069aNhx56iPLly7Nx40Zq1qxpdVjKBTnUIhARX2yL2P9XRAoCnsaYizd6WSaPZZyr6glUB9oAvsDvIlLLft3C3y8yZgYwA2zTRx2JWWVNl450fampqbRs2ZKYmBhGjRrFyy+/rEXi1C1zpOhcP+AZwBuoClQGpgB33+ClsUCldPd9gaOZ7LPeGJMMHBKRfdgSwyaHolc3TZeOdG2xsbFUrFgRDw8PJk6cSEBAgJaKVrfNka6h54A7sZWWwBizH9tiNTeyCaguIgH21c56ACsy7PMNtoJ2iEhpbF1FkY6Frm5GUkoa437eR9cp60hITGF+v8aM6lpbk4CLSEtLY9KkSQQGBjJ16lQAOnbsqElAZQtHPgUSjTFJVy9Esc8GuuFVKcaYFBF5Blvpag9gjjFmt4iMAMKNMSvsz7UXkT1AKvCKMeb0LZ6LyoIuHena9u7dS//+/Vm7di0dOnSgU6dOVoek8hhHEsFaEXkVKCQibbEtYfm9I29ujFkJrMzw2LB02wbbcphDHY5YOUyXjnR9s2bN4plnnqFIkSLMnz+f3r1769XBKts5kgheBQYCe4HnsX2Ln+7MoNTti4xLYKh96cj7apdn5IO1delIF1S1alU6d+7Mp59+Srly5awOR+VRjhSd6wz8ZB/QtZwWnbuxnbHxPD57AwDvP1iLznUq6LdIF5GYmMiIESMAGDVqlMXRqLzkekXnHBks7g5EiMhcEelgHyNQudTWw2fpOWs9XoU8+f7ZFjxQt6ImARexdu1a6tWrxwcffEBcXBw3+pKmVHa5YSKwL09ZA/gO6AdEisg0Zwembl541Bl6z95IyaIFWPxUUyqVLGJ1SMoBF9U0a5MAAB89SURBVC5c4Nlnn6Vly5ZcuXKFVatWMXPmTE3gKsc4dGWxMeYK8C0wD9u00O5OjEndgg2Rp3lizkbKehVk8cCmunykC4mNjWXWrFk8++yz7Ny5k/bt21sdknIzN0wEInK3iMwCDgKPA58BOvUkF1kbcYo+czdSsXhhvhx4J+W9dRH53O706dPXrgcICgoiMjKSTz75hDvuuMPiyJQ7cqRFMAj4CQgyxvQyxqwwxiQ5OS7loP/tj6PfvE34lyrKlwPvpGwxTQK5mTGGpUuXEhwczHPPPce+ffsAdNlIZSlHxggeMcYsNcZczomAlON+3XuCAfPDqVrmDr4YcCel7yhodUjqOo4dO8bDDz9Mt27dqFSpEuHh4VokTuUKWV5HICL/M8a0FpGz/LNY3NUVyko6PTqVpZ93H+fpL7YQWL4YC8IaU7yIXiOQm10tEnfkyBE+/PBDXnzxRTw9tbyHyh2u95fY1v5v6ZwIRDlu5c5jPLdoK7V8vJnfr7GWi8jFYmJi8PHxwcPDg8mTJxMQEECNGjWsDkupf8iya8gYk2bfnG2MSU1/A2bnTHgqoxXbj/Lsoq3Uq1ScBWGaBHKr1NRUJk6c+I8icR06dNAkoHIlR9qmddLfsV9Q1sg54ajrWbYllpeXbKeRf0nm9G1EUa0cmiv99ddfhIWF8eeff9KxY0c6d+5sdUhKXVeWLQIRec0+PlBHRM7Yb2eBODIUklPO99WmGF5asp2mVUsx90lNArnVjBkzqFevHvv372fBggX88MMP+Pn5WR2WUtd1vVlDHwJlgPH2f8sApY0xJY0xr+REcMpm4YZoXv16By2rl2F2n0YUKaBJILeqXr06Xbt2Zc+ePTz++ON6dbByCVkWnROR6saYAyJSJ7PnjTE7nBpZFtyt6Nz8dVG8u2I3dwWWZUqvBhTKr6WecpPLly8zfPhwRITRo0dbHY5SWbpe0bnrfbV8HQgDJmfynAFaZUNs6jpm/R7JyB/+on1wOT7t2YACng5VBFE5ZM2aNfTv358DBw4waNAgjDHaAlAuKctEYIwJs//bMufCUVdN/e0gY37ay/21KzChRz3ye2gSyC3Onz/P66+/ztSpU6lSpQqrV6/mrrvusjospW6ZI7WGHhIRL/v26yLylYjUdX5o7mvi6gOM+WkvD9StyCeaBHKdo0ePMm/ePIYOHcqOHTs0CSiX58gnzHBjzAURaQZ0BhajK5Q5hTGGcT/vY9wv+3movg/jH62HpyaBXOHUqVNMmTIFgMDAQA4dOsTYsWMpWrSoxZEpdfsc+ZRJtf/bCZhijPka0KI22cwYw4er9jHx1wgeDa3ER93q4pFP+5utZoxh8eLFBAcH88ILL7B//34AXTZS5SmOJIJjIjIZ6AGsFJECDr5OOcgYw39++Iupvx2kVxM/PniotiaBXODo0aM8+OCD9OjRg8qVK7N582a9MljlSY5MSO8O3AdMMsacFZGK2GYUqWxgjGH4it3M/zOavs38ebdzsM48yQVSU1Np1aoVR44c4eOPP+b555/XInEqz7rhX7YxJkFE9gBtRKQN8Lsx5kenR+YG0tIMb3+7iy82HKZ/iwDeuj9Ik4DFoqOj8fX1xcPDgylTplClShWqVatmdVhKOZUjs4aeAb4C/Oy3r0RkiLMDy+tS0wyvL9vBFxsOM7hNVU0CFktNTWXcuHEEBQVdKxLXvn17TQLKLTjS1h0INDbGJACIyChgHTDFmYHlZalphleWbGfZ1iM81646L95dXZOAhXbt2kVYWBgbN26kU6dOPPjgg1aHpFSOcmTQV4DkdPeT7Y+pW5CSmsYLi7exbOsRht5Tg6H31NAkYKFp06bRoEEDIiMj+eKLL1ixYgW+vr5Wh6VUjnKkRbAAWC8iX2NLAA8C850aVR6VnJrGc4u28uOu47x2byCD21S1OiS3dbUcRFBQEN26dWPChAmUKVPG6rCUskSWRef+sZNII+BqqYnfjTGbnBrVdbhq0bkrKak888VWftlzgrfvD6J/yypWh+SWLl26xLBhw/Dw8GDMmDFWh6NUjrle0TlHrwe4Yr9dtv+rbkJiciqDP9/CL3tO8N4DIZoELPLbb79Rp04dxo4dS0JCAo58CVLKHTgya+gtYBFQAfAFvhCRN5wdWF6RmJzKgM/C+XXvSf7TtRZ9mvlbHZLbiY+P56mnnqJtW9sy3L/++iuTJ0/WsRml7BwZI3gcaGiMuQQgIv8BNgMfODOwvOBSUgr954fzZ+RpPny4Dt0bVbI6JLd07NgxPv/8c15++WXee+89ihQpYnVISuUqjnQNRfPPhOEJRDry5iJyr4jsE5EIEcnyamQReUREjIhk2n/lihKupNB37ibWR55mbLe6mgRyWFxcHJMmTQJsReKioqL46KOPNAkolQlHEsElYLeIzBKRmcBO4JyIjBORcVm9yL7I/WSgIxAMPCYiwZns5wU8B2y4lRPIjS4kJtNnzkY2R59l/KP1eKiBTkfMKcYYvvjiC4KCgnjppZeuFYnTGUFKZc2RrqEf7Ler1jv43o2BCGNMJICIfAl0AfZk2O99bOsjv+zg++Zq8ZeTeWLORnYfiWfSY/W5r3YFq0NyGzExMQwePJgffviBJk2aMHv2bC0Sp5QDHKk1NPsW39sHiEl3PxZokn4HEakPVDLGfC8iWSYCERmI7Qpn/Pz8bjEc5zt3KYnHZ29g3/ELTOnVgPYh5a0OyW2kpKTQpk0bjh8/zvjx43n22Wfx8ND1nZVyhDPLKWY2JePafD0RyQeMB/re6I2MMTOAGWC7jiCb4stWpxOu8PjsjRw8mcD03g25K1Dr1eeEqKgoKlWqhKenJ9OnT6dKlSpUqaLTc5W6Gc5cVyAWSD9C6gscTXffC6gF/CYiUcCdwApXHDCOu3CFnjM3EBmXwMw+oZoEckBKSgoff/wxQUFB11YOu/vuuzUJKHULHG4RiEhBY8zNXEy2CaguIgHAEWwL2/S8+qQxJh4one79fwNeNsa41GXDJ88n8tjM9Rw9l8jcvo1oVq30jV+kbsuOHTsICwsjPDycLl268PDDD1sdklIuzZELyhqLyE7ggP1+XRGZdKPXGWNSgGeAVcBfwFfGmN0iMkJEHrjNuHOFY/GXeXTGeo7FJzLvSU0COWHKlCk0bNiQ6OhoFi9ezPLly6lYsaLVYSnl0hxpEUzEtl7xNwDGmO0i0taRNzfGrARWZnhsWBb7tnHkPXOL2LOX6DlzA2cuJvFZv8aE+pe0OqQ87WqRuFq1atGjRw/Gjx9P6dKaeJXKDo4kgnzGmOgMl+OnZrWzO4g5c4keM9ZzPjGZBWGNqe9XwuqQ8qyLFy/y9ttv4+npyUcffUSrVq1o1aqV1WEplac4MlgcIyKNASMiHiLyArDfyXHlWlGnLtJ9+p8kXEnhi/53ahJwotWrV1O7dm0mTJjAlStXtEicUk7iSCIYDAzFtkzlCWyzewY7M6jcKuJkAt2n/0licipfDGhCbV9vq0PKk86dO0f//v25++678fT0ZM2aNUycOFGLxCnlJI5cUHYS24wft7b/xAV6ztwAGL4c2JSa5b2sDinPOnHiBF9++SWvvfYa7777LoULF7Y6JKXytBsmAnt9oX+1yY0xA50SUS504MQFHpuxnnz5hEUD7qRaWU0C2e3qh//zzz9PzZo1iYqK0sFgpXKII11D/westt/WAmVxs8VpRny/BwMsHqhJILsZY/j8888JDg7m1Vdf5cCBAwCaBJTKQTdMBMaYxelu84GHsFUTdQt/HTvP7wdOEdYigCpl7rA6nDzl8OHD3H///fTu3ZuaNWuybds2qlevbnVYSrmdW6k1FABUzu5AcqtZvx+icH4PejXJvcXuXNHVInEnT55k4sSJDBkyRIvEKWURR8YIzvL3GEE+4AyQ5SIzecmJ84ms2H6Exxr7UbxIAavDyRMiIyOpXLkynp6ezJw5k6pVq+Lv7291WEq5tet2DYltvl5doIz9VsIYU8UY81VOBGe1+euiSEkz9GseYHUoLi8lJYUxY8YQHBzM5MmTAWjXrp0mAaVygeu2CIwxRkSWG2Ma5lRAucWlpBQWbjhMh+Dy+JcuanU4Lm3btm2EhYWxZcsWunbtSrdu3awOSSmVjiOzhjaKSAOnR5LLLAmPJf5yMgNaaWvgdnz66ac0atSII0eOsHTpUpYtW0aFCrpqm1K5SZYtAhHxtFcQbQEMEJGDwEVsC84YY0yeTQ6paYbZfxyivl9xGlbWYnK34mqRuDp16tCrVy/GjRtHyZL6s1QqN7pe19BGoAHwYA7Fkmv8suc4h89c4vWOgVaH4nISEhJ46623yJ8/Px9//LEWiVPKBVyva0gAjDEHM7vlUHyWmLEmkkolC9NB1xy+KT///DO1atVi0qRJJCcna5E4pVzE9VoEZURkaFZPGmPGOSEey22OPsuWw+d4t3MwHvm0yJkjzp49y9ChQ5k3bx41a9ZkzZo1tGjRwuqwlFIOul6LwAO4A9vawpnd8qRZv0dSrJAn3UMr3XhnBcDJkydZunQpb7zxBtu2bdMkoJSLuV6L4JgxZkSORZILHD59iVW7j/NU66oULXgrF127j+PHj7No0SJefPHFa0XiSpUqZXVYSqlbcMMxAncyZ+0hPPIJfZv5Wx1KrmWMYf78+QQHB/PGG29cKxKnSUAp13W9RNAux6LIBeIvJfNVeAyd61akXLFCVoeTK0VFRXHvvffSt29fgoODtUicUnlElv0fxpgzORmI1RZujOZSUir9W1SxOpRcKSUlhbZt23Lq1CkmT57MoEGDyJfPkesRlVK5nXaEA0kpacxbG0XL6qUJrljM6nBylYiICAICAvD09GTOnDlUqVKFypXdpvisUm5Bv9IBK7Yf5eSFK/Rvqa2Bq5KTkxk1ahQhISHXisS1bdtWk4BSeZDbtwiMMcz6PZKa5bxoVV1XxQLYsmULYWFhbNu2jW7duvHoo49aHZJSyoncvkWwNuI0e49fIKxlALaq2+5t4sSJNG7cmOPHj7Ns2TK++uorypUrZ3VYSikncvtEsOXwWQAeqFvR4kisdbUcRP369XniiSfYs2cPXbt2tTgqpVROcPuuoavye7hnTrxw4QJvvPEGBQsWZOzYsbRs2ZKWLVtaHZZSKge556efAuCnn36iVq1aTJkyBWOMFolTyk1pInBDp0+fpk+fPnTs2JGiRYuydu1axo0bp2MkSrkpTQRu6PTp0yxfvpx33nmHrVu30rRpU6tDUkpZyKmJQETuFZF9IhIhIq9n8vxQEdkjIjtEZLWI6CR1Jzl27Bgff/wxxhhq1KhBdHQ0I0aMoGDBglaHppSymNMSgYh4AJOBjkAw8JiIBGfYbSsQaoypAywFPnRWPO7KGMOcOXMICgrinXfeISIiAoASJUpYHJlSKrdwZougMRBhjIk0xiQBXwJd0u9gjPmvMeaS/e56wNeJ8bidQ4cO0b59e8LCwqhbty7bt2/XInFKqX9x5vRRHyAm3f1YoMl19g8DfszsCREZCAwE8PPzy6748rSUlBTuuusuTp8+zdSpUxk4cKAWiVNKZcqZiSCzKSiZzk8UkceBUKB1Zs8bY2YAMwBCQ0N1juN1HDhwgCpVquDp6cncuXOpWrUqlSrpamtKqaw58ytiLJD+E8gXOJpxJxG5G3gLeMAYc8WJ8eRpycnJjBw5klq1avHpp58C0KZNG00CSqkbcmaLYBNQXUQCgCNAD6Bn+h1EpD4wHbjXGHPSibHkaeHh4YSFhbFjxw569OjBY489ZnVISikX4rQWgTEmBXgGWAX8BXxljNktIiNE5AH7bh8BdwBLRGSbiKxwVjx51SeffEKTJk04deoU3377LYsWLaJs2bJWh6WUciFOrTVkjFkJrMzw2LB023c78/h5mTEGESE0NJSwsDA+/PBDihcvbnVYSikXpEXnXMz58+d57bXXKFSoEOPHj6d58+Y0b97c6rCUUi5M5xO6kJUrVxISEsKMGTPw9PTUInFKqWyhicAFnDp1iscff5z7778fb29v1q1bx0cffaRF4pRS2UITgQs4e/Ys3333He+++y5btmyhSZPrXZenlFI3R8cIcqkjR46wcOFCXnnlFapXr050dLQOBiulnEJbBLmMMYaZM2cSHBzM8OHDOXjwIIAmAaWU02giyEUOHjxIu3btGDhwIA0aNGDHjh1Uq1bN6rCUUnmcdg3lEikpKbRr144zZ84wffp0+vfvr0XilFI5wu0TQUpqmqXH37dvH1WrVsXT05P58+dTtWpVfH21GrdSKue4/VfO3UfPU6VMUTzy5exUzKSkJN577z1q167N5MmTAWjdurUmAaVUjnPrFkFamiE8+iz3hpTP0eNu3LiRsLAwdu3aRc+ePenVq1eOHl8ppdJz6xbBgZMJxF9OplFAyRw75oQJE2jatOm1awMWLlxI6dKlc+z4SimVkVsngo1RZwBo5O/89XuvloNo3LgxAwYMYPfu3XTq1Mnpx1VKqRtx666hTYfOUNarIH4lizjtGPHx8bz66qsULlyYCRMm0KxZM5o1a+a04yml1M1y6xZBeNQZGgWUdFrNnu+++47g4GBmzZpFwYIFtUicUipXcttEEHv2EkfjE2lUOfu7heLi4ujZsycPPPAApUqVYv369YwZM0aLxCmlciW3TQThUWcBnDJQHB8fz8qVK3nvvfcIDw+nUaNG2X4MpZTKLm47RrAx6gxeBT0JLF8sW94vJiaGzz//nNdff51q1aoRHR2Nt7d3try3Uko5k9u2CDYdOkODyiVu+0KytLQ0pk2bRkhICCNHjrxWJE6TgFLKVbhlIjh7MYkDJxNofJvdQgcOHOCuu+5i8ODBNG7cmJ07d2qROKWUy3HLrqHwaNv4QOhtDBSnpKRwzz33cO7cOWbPns2TTz6pg8FKKZfknokg6gwFPPJRt9LN1/j/66+/qF69Op6enixYsICqVatSsWJFJ0SplFI5wy27hjZGnaG2rzeF8ns4/JorV67w7rvvUqdOHT799FMAWrZsqUlAKeXy3K5FcDkplZ2x8fRvWcXh16xfv56wsDD27NlD79696d27txMjVEqpnOV2LYJtMedISTM0DnBsfGDs2LE0a9aMCxcusHLlSj777DNKlSrl5CiVUirnuF0i2BR1BhFo6Hf9GUNpabYFa5o2bcqgQYPYtWsXHTt2zIkQlVIqR7ld19CmqDPULOeFd5H8mT5/7tw5XnrpJYoUKcKkSZO0SJxSKs9zqxZBSmoaW6LP0sg/89bAN998Q3BwMPPnz8fLy0uLxCml3IJbJYK/jl3gYlIqoRnWHzh58iTdu3ena9eulCtXjo0bNzJq1Ci9LkAp5RbcKhFssi9Ek/GK4vPnz/PLL7/wn//8h40bN9KgQQMrwlNKKUu41RjBpqgz+BQvTAXvwhw+fJgFCxbw5ptvUq1aNQ4fPoyXl5fVISqlVI5zaotARO4VkX0iEiEir2fyfEERWWx/foOI+DsrFmMMm6LO0Mi/BFOmTCEkJIRRo0ZdKxKnSUAp5a6clghExAOYDHQEgoHHRCQ4w25hwFljTDVgPDDGWfFEnb7EqYQk1nw9h6effpqmTZuye/duLRKnlHJ7zmwRNAYijDGRxpgk4EugS4Z9ugDz7dtLgXbipBHa9QfjAIgO/5W5c+eyatUq/P39nXEopZRyKc4cI/ABYtLdjwWaZLWPMSZFROKBUsCp9DuJyEBgIICfn98tBVPqjkI0LOfJxLW/4KP1gZRS6hpnJoLMvtlnnJjvyD4YY2YAMwBCQ0NvaXJ/+5DytA8pfysvVUqpPM2ZXUOxQKV0932Bo1ntIyKegDdwxokxKaWUysCZiWATUF1EAkSkANADWJFhnxVAH/v2I8CvRi/nVUqpHOW0riF7n/8zwCrAA5hjjNktIiOAcGPMCmA2sEBEIrC1BHo4Kx6llFKZc+oFZcaYlcDKDI8NS7edCHRzZgxKKaWuz61KTCillPo3TQRKKeXmNBEopZSb00SglFJuTlxttqaIxAHRt/jy0mS4atkN6Dm7Bz1n93A751zZGFMmsydcLhHcDhEJN8aEWh1HTtJzdg96zu7BWeesXUNKKeXmNBEopZSbc7dEMMPqACyg5+we9Jzdg1PO2a3GCJRSSv2bu7UIlFJKZaCJQCml3FyeTAQicq+I7BORCBF5PZPnC4rIYvvzG0TEP+ejzF4OnPNQEdkjIjtEZLWIVLYizux0o3NOt98jImJExOWnGjpyziLS3f673i0iX+R0jNnNgb9tPxH5r4hstf9932dFnNlFROaIyEkR2ZXF8yIiE+0/jx0i0uC2D2qMyVM3bCWvDwJVgALAdiA4wz5DgGn27R7AYqvjzoFzbgsUsW8Pdodztu/nBawB1gOhVsedA7/n6sBWoIT9flmr486Bc54BDLZvBwNRVsd9m+fcCmgA7Mri+fuAH7Gt8HgnsOF2j5kXWwSNgQhjTKQxJgn4EuiSYZ8uwHz79lKgnYhktmymq7jhORtj/muMuWS/ux7binGuzJHfM8D7wIdAYk4G5ySOnPMAYLIx5iyAMeZkDseY3Rw5ZwMUs2978++VEF2KMWYN11+psQvwmbFZDxQXkQq3c8y8mAh8gJh092Ptj2W6jzEmBYgHSuVIdM7hyDmnF4btG4Uru+E5i0h9oJIx5vucDMyJHPk91wBqiMhaEVkvIvfmWHTO4cg5DwceF5FYbOufPJszoVnmZv+/35BTF6axSGbf7DPOkXVkH1fi8PmIyONAKNDaqRE533XPWUTyAeOBvjkVUA5w5Pfsia17qA22Vt/vIlLLGHPOybE5iyPn/BgwzxgzVkSaYlv1sJYxJs354Vki2z+/8mKLIBaolO6+L/9uKl7bR0Q8sTUnr9cUy+0cOWdE5G7gLeABY8yVHIrNWW50zl5ALeA3EYnC1pe6wsUHjB392/7WGJNsjDkE7MOWGFyVI+ccBnwFYIz5EyiErThbXuXQ//ebkRcTwSaguogEiEgBbIPBKzLsswLoY99+BPjV2EdhXNQNz9neTTIdWxJw9X5juME5G2PijTGljTH+xhh/bOMiDxhjwq0JN1s48rf9DbaJAYhIaWxdRZE5GmX2cuScDwPtAEQkCFsiiMvRKHPWCuAJ++yhO4F4Y8yx23nDPNc1ZIxJEZFngFXYZhzMMcbsFpERQLgxZgUwG1vzMQJbS6CHdRHfPgfP+SPgDmCJfVz8sDHmAcuCvk0OnnOe4uA5rwLai8geIBV4xRhz2rqob4+D5/wSMFNEXsTWRdLXlb/YicgibF17pe3jHu8C+QGMMdOwjYPcB0QAl4Anb/uYLvzzUkoplQ3yYteQUkqpm6CJQCml3JwmAqWUcnOaCJRSys1pIlBKKTeniUDlWiKSKiLb0t38r7Ovf1bVGnOaiISKyET7dhsRaZbuuUEi8kQOxlLP1atxKufLc9cRqDzlsjGmntVB3Cz7RWtXL1xrAyQA6+zPTcvu44mIp71mVmbqYSspsjK7j6vyDm0RKJdi/+b/u4hssd+aZbJPiIhstLcidohIdfvjj6d7fLqIeGTy2igRGWPfb6OIVLM/Xlls6zhcXc/Bz/54NxHZJSLbRWSN/bE2IvK9vQUzCHjRfsyWIjJcRF4WkSAR2ZjhvHbYtxuKyP9EZLOIrMqssqSIzBORcSLyX2CMiDQWkXViq8m/TkRq2q/EHQE8aj/+oyJSVGz17jfZ982sYqtyN1bX3tab3rK6Ybsydpv9ttz+WBGgkH27OrarSwH8sddvByYBvezbBYDCQBDwHZDf/vgU4IlMjhkFvGXffgL43r79HdDHvt0P+Ma+vRPwsW8Xt//bJt3rhgMvp3v/a/ft51XFvv0a8Da2K0jXAWXsjz+K7WrajHHOA74HPOz3iwGe9u27ga/t232BT9O9bhTw+NV4gf1AUat/13qz9qZdQyo3y6xrKD/wqYjUw5YoamTyuj+Bt0TEF1hmjDkgIu2AhsAme4mNwkBWNZcWpft3vH27KfCQfXsBtjUOANYC80TkK2DZzZwctkJp3YHR2D7wHwVqYiuW94s9Tg8gqzoyS4wxqfZtb2C+vfVjsJckyER74AERedl+vxDgB/x1k7GrPEQTgXI1LwIngLrYujb/teCMMeYLEdkA3A+sEpH+2Er3zjfGvOHAMUwW2//axxgzSESa2I+1zZ6gHLUYW+2nZba3MgdEpDaw2xjT1IHXX0y3/T7wX2NMV3uX1G9ZvEaAh40x+24iTpXH6RiBcjXewDFjqzXfG9s35n8QkSpApDFmIrZKjXWA1cAjIlLWvk9JyXrd5kfT/funfXsdfxcn7AX8YX+fqsaYDcaYYcAp/lkeGOACtpLY/2KMOYitVfMOtqQAtrLRZcRWVx8RyS8iIVnEmZ43cMS+3fc6x18FPCv25obYqtIqN6eJQLmaKUAfEVmPrVvoYib7PArsEpFtQCC2Zf32YOuD/9k+KPsLkNXyfgXtLYrnsbVAAJ4DnrS/trf9OYCPRGSnferqGmxr6qb3HdD16mBxJsdaDDzO3/X0k7CVRh8jItuxjSP8a0A8Ex8CH4jIWv6ZHP8LBF8dLMbWcsgP7LDH/L4D763yOK0+qlQ6YlvEJtQYc8rqWJTKKdoiUEopN6ctAqWUcnPaIlBKKTeniUAppdycJgKllHJzmgiUUsrNaSJQSik39//epALMRCGgRwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred1 = skl_bag_cls.predict(X_test)\n",
    "y_prob1 = skl_bag_cls.predict_proba(X_test)[:,1]\n",
    "\n",
    "rs.report_matrix(y_test, y_pred1, y_prob1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Out-of-bag Score: 0.662093\n"
     ]
    }
   ],
   "source": [
    "p(\"Out-of-bag Score: {0:.6f}\".format(skl_bag_cls.oob_score_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Out-of-Bag score\n",
    "\n",
    "* Original sklearn bagging method: Out-of-bag Score: 0.662093\n",
    "\n",
    "* Sequential bootstrap bagging method: Out-of-bag Score: 0.655232\n",
    "\n",
    "Comparing the original sklearn bagging ensemble using decision tree classifier, our OOB score is lower. \n",
    "\n",
    "This is correct, since the original algo from sklearn will always inflate their OOB score leading to a more bias outcome.\n",
    "\n",
    "If you are not sure why, kindly refer to [AFML 4.1](https://github.com/boyboi86/AFML/blob/master/AFML%204.1.ipynb).\n",
    "\n",
    "There was a comparison to why Kfold accuracy was lower compared to OOB (due to sklearn bootstrap method)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}