XDMoD-Data-Machine-Learning-Example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# XDMoD Data Analytics Framework — Machine Learning Example\n",
    "\n",
    "Notebook v2.0 (2024-09-27)\n",
    "\n",
    "Compatible with XDMoD Data Analytics Framework v≥1.0.0 and <2.0.0\n",
    "\n",
    "© 2023–2024 University at Buffalo Center for Computational Research\n",
    "\n",
    "See the [xdmod-notebooks](https://github.com/ubccr/xdmod-notebooks) repository for licensing information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Introduction\n",
    "The XDMoD Data Analytics Framework provides API access to the data in XDMoD via the [`xdmod_data` Python module](https://pypi.org/project/xdmod-data). This notebook provides an example showing how to use the `get_raw_data()` method to obtain and process raw job performance data, which is contained in the `SUPREMM` realm in XDMoD, and use the data to train a machine learning model to classify applications based on their performance characteristics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install/upgrade the required modules\n",
    "Run the code below to install/upgrade the modules needed to run this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "! {sys.executable} -m pip install --upgrade 'xdmod-data>=1.0.0,<2.0.0' python-dotenv tabulate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If running that code caused a new version of Plotly to be installed/upgraded, you may need to refresh your browser window for plots to appear correctly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Configure notebook formatting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Exceptions\n",
    "Run the code below to simplify how Python exceptions are displayed in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "def exception_handler(exception_type, exception, traceback):\n",
    "    print(\"%s: %s\" % (exception_type.__name__, exception), file=sys.stderr)\n",
    "get_ipython()._showtraceback = exception_handler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tables\n",
    "Run the code below to set up for displaying Pandas DataFrames as Markdown tables in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from IPython.display import display, Markdown\n",
    "def display_df_md_table(df):\n",
    "    return display(Markdown(df.replace('\\n', '<br/>', regex=True).to_markdown()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plots\n",
    "Run the code below to set up the external Plotly library to make plots using a custom XDMoD theme."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import plotly.express as px\n",
    "import plotly.graph_objects as go\n",
    "import plotly.io as pio\n",
    "import xdmod_data.themes\n",
    "pio.templates.default = \"timeseries\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create an environment file\n",
    "The `xdmod-data.env` file will store your XDMoD API token.\n",
    "\n",
    "Run the code below to create the file in your home directory (if it does not already exist) and allow only you to read and write to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "from os.path import expanduser\n",
    "xdmod_data_env_path = Path(expanduser('~/xdmod-data.env'))\n",
    "try:\n",
    "    with open(xdmod_data_env_path):\n",
    "        pass\n",
    "except FileNotFoundError:\n",
    "    with open(xdmod_data_env_path, 'w') as xdmod_data_env_file:\n",
    "        xdmod_data_env_file.write('XDMOD_API_TOKEN=')\n",
    "    xdmod_data_env_path.chmod(0o600)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Obtain an API token\n",
    "Follow [these instructions](https://github.com/ubccr/xdmod-data#api-token-access) to obtain an API token."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Store your API token in the environment file\n",
    "Open the `xdmod-data.env` file and paste your token after `XDMOD_API_TOKEN=`.\n",
    "\n",
    "Save the file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Load your XDMoD API token into the environment\n",
    "Run the code below to load the contents of the `xdmod-data.env` file into the environment. It will print `True` if it successfully loaded the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv\n",
    "load_dotenv(xdmod_data_env_path, override=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize the XDMoD Data Warehouse\n",
    "Run the code below to prepare for getting data from the XDMoD data warehouse at the given URL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xdmod_data.warehouse import DataWarehouse\n",
    "dw = DataWarehouse('https://xdmod.access-ci.org')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Get the raw data\n",
    "\n",
    "Use the `get_raw_data()` method to request raw data from XDMoD and load them into a Pandas DataFrame. For example, get two months of job performance data for the ACCESS-allocated Bridges-2 resource. Each of the parameters of the `get_raw_data()` method will be explained later in this notebook. Use `with` to create a runtime context; this is also explained later in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    data = dw.get_raw_data(\n",
    "        duration=('2023-01-01', '2023-02-28'),\n",
    "        realm='SUPREMM',\n",
    "        fields=(\n",
    "            'Application',\n",
    "            'CPU User',\n",
    "            'CPU User cov',\n",
    "            'Wall Time',\n",
    "            'Total memory used',\n",
    "            'Net Ib0 Rx',\n",
    "            'Net Ib0 Tx',\n",
    "            'Memory Used Cov',\n",
    "            'Net Ib0 Rx Cov',\n",
    "            'Net Ib0 Tx Cov',\n",
    "        ),\n",
    "        filters={\n",
    "            'Resource': [\n",
    "                'Bridges 2 RM',\n",
    "            ],\n",
    "        },\n",
    "        show_progress=True,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspect the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare data for training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get a list of the most used applications in the data set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "application_counts = data['Application'].value_counts()\n",
    "display(application_counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looking at the list of applications, it should be noted:\n",
    "\n",
    "* \"NA\" means the application name was not collected.\n",
    "* \"uncategorized\" means the application name was collected but is not in the list of applications known to XDMoD.\n",
    "* \"PROPRIETARY\" means the application is one whose software license does not allow performance data about the application to be reported in conjunction with the application's name.\n",
    "* \"python\" and \"r\" could consist of multiple different codes running in the Python and R programming languages, rather than specific application packages.\n",
    "\n",
    "With that in mind, only include jobs running the top eight individual application packages from the list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "applications = ('orca', 'lammps', 'q-espresso', 'gromacs', 'specfem2d', 'namd', 'gdal', 'qmcchem')\n",
    "data = data[data['Application'].isin(applications)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One thing of which to be careful is that there may be some records that have `<NA>` values, indicating a particular record was not collected.\n",
    "\n",
    "Filter out the jobs for which any value is `<NA>`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = data.dropna()\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert the values from all the columns except \"Application\" from strings to floating point values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conversions = {}\n",
    "for column in data.columns.drop('Application').values:\n",
    "    conversions[column] = float\n",
    "data = data.astype(conversions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use [scikit-learn](https://scikit-learn.org/) to split the jobs into a training set and test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X = data[[\n",
    "    'CPU User',\n",
    "    'CPU User cov',\n",
    "    'Wall Time',\n",
    "    'Total memory used',\n",
    "    'Net Ib0 Rx',\n",
    "    'Net Ib0 Tx',\n",
    "    'Memory Used Cov',\n",
    "    'Net Ib0 Rx Cov',\n",
    "    'Net Ib0 Tx Cov',\n",
    "]]\n",
    "y = data['Application']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.10)\n",
    "display(Markdown(\"Size of the test subset is **\" + str(len(X_test)) + \"**.\"))\n",
    "display(Markdown(\"Size of the training subset is **\" + str(len(X_train)) + \"**.\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate a Random Forest Classification model\n",
    "\n",
    "Using the tools from `scikit-learn`, build a Random Forest Classification model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "classifier = RandomForestClassifier(oob_score=True)\n",
    "classifier.fit(X_train, y_train)\n",
    "print('oob score:', classifier.oob_score_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute feature importances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "importances = pd.Series(\n",
    "    classifier.feature_importances_,\n",
    "    index=pd.Index(\n",
    "        X_train.columns,\n",
    "        name='Feature Name',\n",
    "    ),\n",
    "    name='Importance',\n",
    ").sort_values(ascending=False)\n",
    "display_df_md_table(importances)\n",
    "plot = px.bar(importances, y='Importance')\n",
    "plot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create confusion matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n",
    "matrix = confusion_matrix(y_test, classifier.predict(X_test), labels=applications)\n",
    "cmd = ConfusionMatrixDisplay(matrix, display_labels=applications)\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams.update(plt.rcParamsDefault)\n",
    "plt.rcParams.update({\n",
    "    'font.size': 16,\n",
    "    'figure.figsize': (7, 7),\n",
    "    'axes.labelpad': 20,\n",
    "})\n",
    "cmd.plot(\n",
    "    colorbar=False,\n",
    "    xticks_rotation='vertical',\n",
    "    values_format='.0f',\n",
    "    cmap='Blues',\n",
    ")\n",
    "# Export the figure as PDF\n",
    "#plt.savefig('confusion_matrix.pdf', bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Details of the `get_raw_data()` method\n",
    "Now that you have seen an example of using the `get_raw_data()` method, read below for more details on how it works."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wrap data warehouse calls in a runtime context\n",
    "XDMoD data is accessed over a network connection, which involves establishing connections and creating temporary resources. To ensure these connections and resources are cleaned up properly in spite of any runtime errors, you should call data warehouse methods within a **runtime context** by using Python's `with` statement to wrap the execution of XDMoD queries, store the result, and execute any long running calculations outside of the runtime context, as in the template below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    # XDMoD queries would go here\n",
    "    pass\n",
    "# Data processing would go here\n",
    "pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parameters\n",
    "The `get_raw_data()` method has a number of parameters explained in detail below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Duration\n",
    "The **duration** provides the time constraints of the data to be fetched from the XDMoD data warehouse.\n",
    "\n",
    "As already seen, you can specify the duration as start and end dates. These are both inclusive, so if you only want one day of data, use the same date for start and end:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    data = dw.get_raw_data(\n",
    "        duration=('2023-05-01', '2023-05-01'),\n",
    "        realm='SUPREMM'\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can instead specify the duration using a special string value; a list of the valid values can be obtained by calling the `get_durations()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    durations = dw.get_durations()\n",
    "display(durations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Realm\n",
    "A **realm** is a category of data in the XDMoD data warehouse. You can use the `describe_raw_realms()` method to get a DataFrame containing the list of realms for which raw data is available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    raw_realms = dw.describe_raw_realms()\n",
    "display_df_md_table(raw_realms)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fields\n",
    "A **field** is a measurement for which raw data exists in a given realm. You can use the `describe_raw_fields(realm)` method to get a DataFrame containing the list of valid fields in the given realm. The realm must be passed in as a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    raw_fields = dw.describe_raw_fields('SUPREMM')\n",
    "display_df_md_table(raw_fields)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filters\n",
    "**Filters** allow you to include only data that have certain values for given **dimensions**, which are groupings of data. You can use the `describe_dimensions(realm)` method to get a DataFrame containing the list of valid dimensions in the given realm. The realm must be passed in as a string and can be either the ID or the label of the realm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    dimensions = dw.describe_dimensions('SUPREMM')\n",
    "display_df_md_table(dimensions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use the `get_filter_values(realm, dimension)` method to get a DataFrame containing the list of valid filter values for the given dimension in the given realm. The realm and dimension must be passed in as strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    filter_values = dw.get_filter_values('SUPREMM', 'Resource') # 'resource' also works\n",
    "display_df_md_table(filter_values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For methods in the API that take filters as arguments, you must specify the filters as a dictionary in which the keys are dimensions (labels or IDs) and the values are string filter values (labels or IDs) or sequences of string filter values. For example, to return only data for which the field of science is Materials Engineering and the resource is either Bridges-2 RM or TACC Stampede2:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with dw:\n",
    "    data = dw.get_raw_data(\n",
    "        duration=('2021-05-01', '2021-05-01'),\n",
    "        realm='SUPREMM',\n",
    "        filters={\n",
    "            'Field of Science': 'Materials Engineering', # 'fieldofscience': '177' also works\n",
    "            'Resource': ( # 'resource' also works\n",
    "                'Bridges 2 RM', # '2900' also works\n",
    "                'STAMPEDE2 TACC', # '2825' also works\n",
    "            ),\n",
    "        },\n",
    "        show_progress=True\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show progress\n",
    "Set the `show_progress` parameter to `True` to periodically print how many rows have been gotten so far. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Examples\n",
    "For additional examples, please see the [xdmod-notebooks repository](https://github.com/ubccr/xdmod-notebooks)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "![XDMoD](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAF0AAAAgCAYAAABwzXTcAAAKT2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AUkSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXXPues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgABeNMLCADATZvAMByH/w/qQplcAYCEAcB0kThLCIAUAEB6jkKmAEBGAYCdmCZTAKAEAGDLY2LjAFAtAGAnf+bTAICd+Jl7AQBblCEVAaCRACATZYhEAGg7AKzPVopFAFgwABRmS8Q5ANgtADBJV2ZIALC3AMDOEAuyAAgMADBRiIUpAAR7AGDIIyN4AISZABRG8lc88SuuEOcqAAB4mbI8uSQ5RYFbCC1xB1dXLh4ozkkXKxQ2YQJhmkAuwnmZGTKBNA/g88wAAKCRFRHgg/P9eM4Ors7ONo62Dl8t6r8G/yJiYuP+5c+rcEAAAOF0ftH+LC+zGoA7BoBt/qIl7gRoXgugdfeLZrIPQLUAoOnaV/Nw+H48PEWhkLnZ2eXk5NhKxEJbYcpXff5nwl/AV/1s+X48/Pf14L7iJIEyXYFHBPjgwsz0TKUcz5IJhGLc5o9H/LcL//wd0yLESWK5WCoU41EScY5EmozzMqUiiUKSKcUl0v9k4t8s+wM+3zUAsGo+AXuRLahdYwP2SycQWHTA4vcAAPK7b8HUKAgDgGiD4c93/+8//UegJQCAZkmScQAAXkQkLlTKsz/HCAAARKCBKrBBG/TBGCzABhzBBdzBC/xgNoRCJMTCQhBCCmSAHHJgKayCQiiGzbAdKmAv1EAdNMBRaIaTcA4uwlW4Dj1wD/phCJ7BKLyBCQRByAgTYSHaiAFiilgjjggXmYX4IcFIBBKLJCDJiBRRIkuRNUgxUopUIFVIHfI9cgI5h1xGupE7yAAygvyGvEcxlIGyUT3UDLVDuag3GoRGogvQZHQxmo8WoJvQcrQaPYw2oefQq2gP2o8+Q8cwwOgYBzPEbDAuxsNCsTgsCZNjy7EirAyrxhqwVqwDu4n1Y8+xdwQSgUXACTYEd0IgYR5BSFhMWE7YSKggHCQ0EdoJNwkDhFHCJyKTqEu0JroR+cQYYjIxh1hILCPWEo8TLxB7iEPENyQSiUMyJ7mQAkmxpFTSEtJG0m5SI+ksqZs0SBojk8naZGuyBzmULCAryIXkneTD5DPkG+Qh8lsKnWJAcaT4U+IoUspqShnlEOU05QZlmDJBVaOaUt2ooVQRNY9aQq2htlKvUYeoEzR1mjnNgxZJS6WtopXTGmgXaPdpr+h0uhHdlR5Ol9BX0svpR+iX6AP0dwwNhhWDx4hnKBmbGAcYZxl3GK+YTKYZ04sZx1QwNzHrmOeZD5lvVVgqtip8FZHKCpVKlSaVGyovVKmqpqreqgtV81XLVI+pXlN9rkZVM1PjqQnUlqtVqp1Q61MbU2epO6iHqmeob1Q/pH5Z/YkGWcNMw09DpFGgsV/jvMYgC2MZs3gsIWsNq4Z1gTXEJrHN2Xx2KruY/R27iz2qqaE5QzNKM1ezUvOUZj8H45hx+Jx0TgnnKKeX836K3hTvKeIpG6Y0TLkxZVxrqpaXllirSKtRq0frvTau7aedpr1Fu1n7gQ5Bx0onXCdHZ4/OBZ3nU9lT3acKpxZNPTr1ri6qa6UbobtEd79up+6Ynr5egJ5Mb6feeb3n+hx9L/1U/W36p/VHDFgGswwkBtsMzhg8xTVxbzwdL8fb8VFDXcNAQ6VhlWGX4YSRudE8o9VGjUYPjGnGXOMk423GbcajJgYmISZLTepN7ppSTbmmKaY7TDtMx83MzaLN1pk1mz0x1zLnm+eb15vft2BaeFostqi2uGVJsuRaplnutrxuhVo5WaVYVVpds0atna0l1rutu6cRp7lOk06rntZnw7Dxtsm2qbcZsOXYBtuutm22fWFnYhdnt8Wuw+6TvZN9un2N/T0HDYfZDqsdWh1+c7RyFDpWOt6azpzuP33F9JbpL2dYzxDP2DPjthPLKcRpnVOb00dnF2e5c4PziIuJS4LLLpc+Lpsbxt3IveRKdPVxXeF60vWdm7Obwu2o26/uNu5p7ofcn8w0nymeWTNz0MPIQ+BR5dE/C5+VMGvfrH5PQ0+BZ7XnIy9jL5FXrdewt6V3qvdh7xc+9j5yn+M+4zw33jLeWV/MN8C3yLfLT8Nvnl+F30N/I/9k/3r/0QCngCUBZwOJgUGBWwL7+Hp8Ib+OPzrbZfay2e1BjKC5QRVBj4KtguXBrSFoyOyQrSH355jOkc5pDoVQfujW0Adh5mGLw34MJ4WHhVeGP45wiFga0TGXNXfR3ENz30T6RJZE3ptnMU85ry1KNSo+qi5qPNo3ujS6P8YuZlnM1VidWElsSxw5LiquNm5svt/87fOH4p3iC+N7F5gvyF1weaHOwvSFpxapLhIsOpZATIhOOJTwQRAqqBaMJfITdyWOCnnCHcJnIi/RNtGI2ENcKh5O8kgqTXqS7JG8NXkkxTOlLOW5hCepkLxMDUzdmzqeFpp2IG0yPTq9MYOSkZBxQqohTZO2Z+pn5mZ2y6xlhbL+xW6Lty8elQfJa7OQrAVZLQq2QqboVFoo1yoHsmdlV2a/zYnKOZarnivN7cyzytuQN5zvn//tEsIS4ZK2pYZLVy0dWOa9rGo5sjxxedsK4xUFK4ZWBqw8uIq2Km3VT6vtV5eufr0mek1rgV7ByoLBtQFr6wtVCuWFfevc1+1dT1gvWd+1YfqGnRs+FYmKrhTbF5cVf9go3HjlG4dvyr+Z3JS0qavEuWTPZtJm6ebeLZ5bDpaql+aXDm4N2dq0Dd9WtO319kXbL5fNKNu7g7ZDuaO/PLi8ZafJzs07P1SkVPRU+lQ27tLdtWHX+G7R7ht7vPY07NXbW7z3/T7JvttVAVVN1WbVZftJ+7P3P66Jqun4lvttXa1ObXHtxwPSA/0HIw6217nU1R3SPVRSj9Yr60cOxx++/p3vdy0NNg1VjZzG4iNwRHnk6fcJ3/ceDTradox7rOEH0x92HWcdL2pCmvKaRptTmvtbYlu6T8w+0dbq3nr8R9sfD5w0PFl5SvNUyWna6YLTk2fyz4ydlZ19fi753GDborZ752PO32oPb++6EHTh0kX/i+c7vDvOXPK4dPKy2+UTV7hXmq86X23qdOo8/pPTT8e7nLuarrlca7nuer21e2b36RueN87d9L158Rb/1tWeOT3dvfN6b/fF9/XfFt1+cif9zsu72Xcn7q28T7xf9EDtQdlD3YfVP1v+3Njv3H9qwHeg89HcR/cGhYPP/pH1jw9DBY+Zj8uGDYbrnjg+OTniP3L96fynQ89kzyaeF/6i/suuFxYvfvjV69fO0ZjRoZfyl5O/bXyl/erA6xmv28bCxh6+yXgzMV70VvvtwXfcdx3vo98PT+R8IH8o/2j5sfVT0Kf7kxmTk/8EA5jz/GMzLdsAAAAGYktHRAD/AP8A/6C9p5MAAAAJcEhZcwAACxMAAAsTAQCanBgAAAAHdElNRQfaCR0QJxALRnmrAAAWJUlEQVRo3u16aXRVVbbut3Z7mjQngYRAGkIgiCChEYFCgiBRChBBryJQBUJZWkQuitcWRUFQaURKS1FUsApBrxEQEJGy1FKgiCIEEjpD36QjhIScnJxzsvdea8334yQB6oJC1Rvv/bjOMTLGyV7dXN9e85vN2sAv8v9ftm7detVj3njjjase8+GqT/7XYswu9fDw4cPXrl69ekdRUZHQdR2MRbp5vd6C7OzsX48bN85u6vv5559D0zTlhx070stKS01d1wGA6gL1LDktHUREmqpCUVUoigKPx0O6YdQriurouub4Yn3WmLtGhX5KyTeXbY9+bMaueNWjwuvSEKzn6Ns7Tn656u6S194twMP3X9/cl4iUNl3/3JYLkkSEhgYJn8+Qf3y+W/ndI3uIfxWoASM+iD16IuiLitIpOkpDVbWFjhmxZBoEX4wmO2XGOc/818Aaxhj/4u+HMOTmjpedS7vUw8zMzB/Xrl276PDhw88Gg0EADIrCcO7cuUGnT5+eBODt9evXIyoqCoMHD8aKFSvGVFZWrhBCKEQAY0Cgvr75ZRFjYIyBKQpsxyEpyVZUtcFxnIBlWTVvvPPezqSEln+8647b9320+hOMuevOi/Q5fqr2ibCNGYbGwB0JzoEDh0IgIi9j7KIXtmtP2ZCqGudzTVMAAFIyhOsFP1PV0B3A/n8V9HCDnHWuDtPqghxqlQAXhMK9ATAGaCrkP74L1K397KM9R09ULGmf3vq/f2ou5VIPN2/eDMuyns/IyPgRADRNhaaq0A0Du3fvXlJZWdli5MiRGDx4MIhIO3To0Hu2ZSmapkNVVaiqBikFhJAQnENwASEEpBAQQjAuuCk4jxVCpAguutqW9buSsoq976348PF/BhwATp+xYkkCQgAOJxAYyk4GsXTlrpH/3HfV+oN/4AEOzgHOASkBbkuq8dtR/w4lCElk2UC4AagPEYUbCDV1AtV+jsoarpRW2bEFe/3Z7Tuv+XDR4u/mExFGjPvwykG/6aabMGbMGNG+fftZPp8PjDFSVBWqopKqKpg3b/4iImIAMHv27HVVZ8+aum6QpqnQdJU0TYWUBMEdcCEgBSfBBQkhIQQH5xyy0SSEEAxEAAj+usCCt5e9/0EwHFa/+mbLedCrGqLA/okJVQVrPjvR+cJHRKSu3lh+PcyLDZgD2r4fa1z/Dugkz1OxkMQSW6iyXaoevibDLaO8CniIM81UofoMev6lvU+s+XTfkxs+HIfjJ89eGehNMnbs2I87dOiwWdM0pmkaqZrKDMNEQ0N43Pz581NeeOGFX505c+YWQ9dJ01Sm6ToZusFAFAAQ4EJETrqUjAvONFUZzaTsER/tHXa6vGRZZXmZxRQlYgVSAkQIN4THfbLus/tyBg1o1qOq2vFerClBNVRUnOY3v//xXhUAivZXYe3nB7pWlDXEa4YCIroAMGJ19VK/3D7r6urx6pJtbN3Gfaxp3NKVhRc7P3Z+Qjpn4fWX+qw48sNEz4Ft49Xaw/ddN2xIqwAPccbAWECAxk3On0dEndu1bXllnA4A+fn56NevHwYPHjyqtLT0ZCgcjlEYg6IoAKAFg8GV9fX1uq7rRsQKFCiqyizLQkpq6l2nSspe59yJBiHC7ZqGs9XVRX9a9PIhAIUANj04OfeVH2uq8zp27tIVIuLjFEXBudrat0+eKlndNi21hoiUzL4fxEFhF/l/xgDHdvr+7qGNyP+hDN26JOCOe9e1dyR5Iu0RA2IM4ATU1NrRTaP3HKhCVucEAMCzc7/9TfaIT3sHwzLNMFX7xVeLDm/6+uC3Qwdf89XFr/liS7Mc7gKAHbtLwRjbT0Sp3bP/u3LfsaCpKIzZAY7R93/+FBFNYP9kpZc96f369cOmTZvg8/lqdV2fxwDSdB2qqsEwDNQFAtlgrK+u69A1DZqmQVEUeD2eRVu3bNkmSLgF5xCCQwhO3HHQvVuWu2n+3/72Xry55K0fuR2asH9PUUjTdZJSQEpJtu3g75u3zgGAwn2lpsORoKoKASApCdR4HI+WhpVVy+7q0693cmQzzLydS2o83NR8OoUE0pLjkpvWzuqcACJqMfyeNZVz5h9Yuae4/qHjJeFRxUcCo3cW1T89dPQ3X475/aeriEgtK6+7JOpN5/6GHik4dy4EAP4Wie63hB05PFq0jqPFgeyColNRV0UvQ4cORUZGBl5++eVFnPMShTHoukaapsEwDGYYBlN1HZqmweVyU52/rtLlcr3UKqmVQRLgjfzNOWecO+jRLavZslauXA4AWL58ReGZitJddX4/k5IgpWSMAf66wND9B4o9hfsqDSZlHGNgnEuWme62or0qAYBVz7E5/8SLTXP6/aGRAIMQxDq0ddm+aA1EBCIgGLLTAMCV8hIK9lXrN9768Q8bN1Umal6dSBJM4zyYmqnSJxvK77rv4b8tT24T04g5XRanuDgPGGPo2yP6MBPnacgftNKXrjxgXhXoEVCWgzFm3XLLLcP8fj90XWdaI9C6psHQNBiGSefO1bDrrusy5dlnn60GAEkSwolELpwLcM5hWdYl1+jYsePq0xXlICkhhYCUgoQQUTt2FcaVlAU1ABFqcAjXdHCfTU/2nGg82viu4Gx/ACjcW9n2+8KaWMYAIkJ2n8R8lysCIkmgdYK7HQA0lD6NoqKS9/J/qM7QonTilmQ9sqLsJfP79p31aNbELh29jIiYUFW8t/Twb7ZuP/YQAAhBP+tsU1M9lXQBlZwLcuworFGuGvR7770Xq1atwtixY/cTsFhKQjOl6Do0TYeiMMYYW5ebm7vm9ddfB8BAUsDhHFzw5rCRpLzkGi7T3BWsD0BIASEFpJBMSqmfq611nzhVp3EuvBekcsEGh76VkqCaCm0vqNV27j7Va8t3Jx+qr3HAGCOXqSAxIWa2x62xJl6vOBPo0hjhxDz5QsFvmUsjImIJCTp2fTWuzYQxXbc/91jf5cnJnt+BGrkkSqfxD26b1OiN2c9hldI6JnChw7VsidNnLHbVoAOA2+1ucl+xLpcJXdMagddhGDpMlwuWZQ0AgKlTp0IBICWB8wjgXET+JF36tFi2Q5JkUxwfAV8IJoRgDZZUScI8H0Ww4C03JRRKS0TcKSe8u3Jf9savTg5HxMmzFtGakzMwvkgK3hxphsO8DQB88EnRuKpSC6oaaeiQ6q3+1ZDl1cWHarA5vwwT7m53WAezI/mJAjss0jd88WMCmPKzR33Ar5JtNFtEhNb4JSzkZ0GfN28ebrvtNsycOXOoEPy3brcbmqZHQNd1aJoBQ9epQ2Zm/NSpU/8CAExlEXrhDjh3IsA7zkVh3EVxNOfdDMNsjOkFhBAkhXCiozzhkrKQGrYbLUQSWiW4aiaO6fhdYzRD0BSUlFs53xfUJSumQlISVEM7PLh/p5rWrTycJAEM8AecWADY9OWprlAvsBtVKb5/Qld06hiPm/olo1dWYik0ZjeqymwhY9Z8dsyjKPhZ0AsKTxtQWXOEpWkMMVEaXTXoTz31FL7Lz0/Kz8/Py8rqBlXTSNd16IYB0zSh6xo0XWdej5fS2rYd+8i0ad05F5BCwnGcRj6PJEl0GRM9cuTIXb64eHCHQwoOKQQjokCv7p2r2qe3aGE51OzlOrSLq7y+W9udXTp7Le5IpmoMBXtqc0IN0qsoDLJB4LrOnv0A4IsxTjftOBSOvLhdRVUuGFqz0wwEeX1peaBZl5gYj2CKAjRi3GAJZfO2MrUxVP5JOXayNg4XJFE+r4aeWb6rA33JkiUAgLnz5s0eMOCmaFVToWs6M0wDAHMC9QGYpkmGbkDVVJbQMsGIi49flpCQaEspKUIvEeAF55Dy/Ppz584FADz+2GPXub1RvV0uF0UyWCLLthEbE73u+uv7WNHRahfLks1ZqBTOKQB46L7OmxDmYIxRVbVjSEmMGpHMvbfLZwDgcaunGSKcXlVtgYj0Fi08Ao16MAbERutG546+Zr1OlNQoEYuMYKdrCq5pH0eXs9IL5YfCc8kX9KL4aKPmzhHpzhWD/s0332Dy5Mn4+OOPe7ZqlXR/VHQUVFWFbkRoJTEx4e5TJ08WGYbBNF2DoRswDIPS2rbtqWnaDClEmDtOxJlyB9xx4HK5mj3p9OnTUVpyIvrIsRMfprVt52kIh5mQEpw7TFNVTJl8/wwA8EWbPTmn5ri4y7WJRwGgY4eEWZ4WBogIihJJhgSXrFsPH0YM6fQ+AERHucubQKgPE6qrazvkZLcpg3W+2GhbImX3vtrz/zuyJTlSa/IF0aYafOgPWQ1CyMuCXVVdDwAoL+MjFDPCL9wW0L3a9tEjsgJXDPqgQYNARPpHH330dZcuXaCqKgzDIEM34Ha5tiS2StxIRI9WVJSLqKhoMkwDuq6z+Lh4Sk5JfcrhPNNxGvmccxAIGzduzJgwcVLmlClTe/4hN/fhRx59qqZV69ZdhRDUGOUQdxwkJiTc11Q9PF0VuCZi6QTDo8LQtUIAGNQ/rahzh6j6SKgUqd2AEwb3T9retIdg2C6NZIMMVoPE5vyTmVPv7/5XGKw5wXIcnvHSjIHN+/70i5PtSEojUqEkIl0pGzq4YzUksQvSU5i60vwWWsYL1zvvF/3mH9vPDFQi2hALc/bGvF55jDF5RaB/8803ICJMGD9hbq8bbvC53C4yDAONNIKUlJTHhtxyK3/33Xe/rqw8/XU4HGKmaZJhmmCMsfT0dPS/8Ualvr6+OYJxHAd79uz9qCHc8OM5f21BfTD0alR0jMY5J845E1xQuCHM4uN8K2779bA/r1qztrE0i7SmYx7tUdA2xXOsSU9dN76IMEEEWJdXhaYbTze1m6ZZzlhjsy1QF5Q9WsTHbh89KrVahAQDgD3FAXXB6/lvNo3Zt7/uSR4hcJJ+m90+pE0+Y8xmKjvPHF4Ni/98eGD3gSs/7ZK9YlOHGzbseHzmjpX+kCQwBuG32TPPdN/ap2fG8iuqMq5fvx6DBg3CokWL0tokt3k4MTGRVFVlhm5A0zTWOinpT7feeuuOb7+NVAFfe+1Pd5SVlnLDMJmh6zBdJjRNw/Dhw5GWmoKGcAhNJ15IoTrcUR2HkxSNDpY7TAgOx7EdX0zsq89Mf3LCW++8RXf/xx0AgIrKYFJT3BflVdGpY3JNk65pKdpX4OcdRVK8YTt2aP9Xm0sAADW14dJm/8cYfthZ0RUAJo69rn+LhEgKqmgqPfH07twOvZdTu15/ob99XdlLURhxW7A+A1rai+fnTIq4k/MHXXNr9O22s60L99WP2F8c+vXx0tB1QYtIUcC4I9iQnNYnX3i6/4C/bz12ZaXdkSNHgohcFRUVO9tlZGiqojLDMGAYBjwez8mExFZPAsDAgQPwxRdfgDEWSkxMmOg4NkyXiwzDhGEYMEwT9/3+92QYBhzbjkQyTiSEdGybCSFARELX9KMu03wtMaFln7kvzn7kyenPYEB2/2Z9KiqtBDgSwpZoyjs2fnUcADBtctZu6RBxS4KHBQSUvU9N6+XPuSm1iZ+riBO4LYCwQH1YtgWAYTnpxfeMysht4VPBQ4KpUQaOnLBw/JQNxaUSD3J2baa3+sXpfbs16eE4UhchAR7i4JZkIAACgCBIS4LXOyyphY57787Iy/tgQB8AuDk748pvjtasWeNOT2831G26HCgMqqqCgSG+RfzpPr1vaGjqN2TIEGzZsgUtW7ZcfejgwadM0+jCmEKaFrnIaNUqiT0zY8bSF1544V3D0JmmaoiJiaFbbr01VFVVdW7wzYMqu3fvzgFg7vwFAID5c1/E/LnndUlPdR3yxTANjCG1TVTtiUJgeE47AMCverXbPuaB9TvLKuoThCNxc3bK2lYJ8c36tUnUSwf1jz0mieDYUvG6lWIAeO2d3Xj4gR5LthecXjNrwbbNW7ZXJTcyCgydsXvGZ+S/+fKQX+cMvKAAeEPigWuvcU4IQdxfZynhMGct493CEVLGRmnBPj0TC6fl3jidMVb50OTSqyvW5+Xl/WT755s2XfL5rl27rvnu++/v21mwa+Lu3UUT9+7dO3Hf/gMTS0rLOv3UfAteXnjZtjvvXfM/nj3x/N8vrIX+j/ZHnvkSAPDB6oOXnPP+aZ9Fakp5xT+Lxd++PQUAePjpv14xflvyj/3fu8EeNWrU//Nb85kzZ0au4Fat+sl+Xbt2bf49YsSIZiv8V+Xtt99u/h2pJf28LFy48N/7GuBy8sADDzzxzjvvLJgzZ07qkSNH7ly+fPlr48ePn2qaJtd1PSErK+u1goKCSZZlmYqiuHNycvI2bNiQFh0d3b6hoUEfNmzYke3bt++srKwcnZeX9/qkSZMe4ZG7UvWee+5559SpU122bdt2raIovFOnTmXPPfdc83caCxYsiDp69Og0t9uthMNh5c4773xr5cqVrVNSUm6cO3fu4lGjRv1BCPHZhg0byprG3H777RNdLleKYRhIT0//pGfPnke3bt36QDgcFoyx6Lfeemv+zJkzc2fNmvU2gOjx48dPNk1z4bJly8TMmTNHtWzZMjh16tQvX3nlleQDBw7cs2zZskW33XbbEFVVi/v162fu3r17rKIook2bNscWLlz44ZXiqFwN6KFQaP7KlSvfNAzjLwBme71eXdO05+Lj4++JjY39MBwO+5cuXfqq1+t9XNf1zZWVlQcNw/h9VFRUJyK6oby8fI6UMk1RlBd79+4dzzmf//77789KTk42Dx06pDc0NPxnMBh8NhwOPyqlHHrh2sXFxR2klJ2ys7MX2bbdpqioKMc0zetramr+OGPGjGWGYTwTCoUu+u7BcZwZ11577RdpaWlrAoHAX++44w5mGMbzcXFxdw0fPnzd2rVrVdu2H549e/ahxYsXz2GMzTtw4IBBREYoFHqluLh4TeO+OwshXpkyZcp2n893m6Io7Q8ePHiz1+utzczM/COAOVeD41WBXl9fv7WwsDDT7/cfk1IqHo+HhBDe0tJSp6Ki4v6kpKQWjeVTnXNur1u3Tkop3Zqm1fp8vmFut3uhYRguIYRr0qRJYUVR9AkTJuSePXv27lAo1Nrr9R5u27ZtWefOna3jx49nX7i21+ttsG27x6effporpewTGxvr55zHqqr6nM/nq4qNjU11HCd48b0m08rLy8efPXt2msfjKfj+++81IYSrrKxM5OXljR81apRMTk72JyQkLK+qqhI1NTXOlClTlFmzZu0tKys7WVdXt+/xxx//Kjo62uX3+3cJId51uVz/aVmWjImJ4UQ0rKys7ImrZYyrAl0IMWzs2LFDKysrc4kotaqqihNRGyIaLYRYIIRoyqfTGGPfu1wuCCHGjBs3bqZpmqklJSXr3W53vhAiMTc3N6woSpwQIo9zfn1mZuZ2v9//XFlZ2YBDhw71TUpKuuHCtZOSkg4CyOacLyOiQd26ddsEYHEoFHr10UcfnQ4g3jCMgos2pyhZwWBwVjAYnJ6Wljb68OHDQcuy2gghRgcCgUWMMbJtO+fBBx+ck5OT85hlWYkjR44MVVRU9FUUZYiiKAODweCdPp9vkxBi4JIlS5bquh5n2/Y/EhMTl9u2PTYcDr9mGEbWL98G/iK/yC/yi/zvlP8DXh8lEQYro5IAAAAASUVORK5CYII=) XDMoD Data Analytics Framework"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}