ipython-notebook/API Sample/Multivariate API Demo Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Endpoints"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multivariate Anomaly Detection Endpoint\n",
    "\n",
    "`multi-ad-ppe.ppe.cognitiveservices.azure.com/anomalydetector/v1.1-preview.1`\n",
    "\n",
    "Multivariate Anomaly Detection APIs\n",
    "- List models `[GET] /multivariate/models`\n",
    "- Train a model `[POST] /multivariate/models`\n",
    "- Get a model `[GET] /multivariate/models/{model_id}`\n",
    "- Delete a model `[DELETE] /multivariate/models/{model_id}`\n",
    "- Detect anomalies `[POST] /multivariate/models/{model_id}/detect`\n",
    "- Detect anomalies with the synchronous API `[POST] /multivariate/models/{model_id}/last/detect`\n",
    "- Get detection result `[GET] /multivariate/results/{result_id}`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ENDPOINT = \"[Your endpoint of Anomaly Detector resource]/anomalydetector/v1.1-preview.1\"\n",
    "HEADERS = {\n",
    "    \"Ocp-Apim-Subscription-Key\": \"[Your key of Anomaly Detector resource.]\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "API_MODEL = \"{endpoint}/multivariate/models\"\n",
    "API_MODEL_STATUS = \"{endpoint}/multivariate/models/{model_id}\"\n",
    "API_MODEL_INFERENCE = \"{endpoint}/multivariate/models/{model_id}/detect\"\n",
    "API_MODEL_LAST_INFERENCE = \"{endpoint}/multivariate/models/{model_id}/last/detect\"\n",
    "API_RESULTS = \"{endpoint}/multivariate/results/{result_id}\"\n",
    "API_DELETE = \"{endpoint}/multivariate/models/{model_id}\"\n",
    "SOURCE_BLOB_SAS = \"[The SAS URL token that generated from your dataset in Azure Storage Account.]\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List Models\n",
    "```http\n",
    "[GET] https://{endpoint}/multivariate/models?$skip=0&$top=5\n",
    "```\n",
    "#### Sample\n",
    " - Request\n",
    "```json\n",
    "header = {\"Content-Type\": \"application/json\", \"Ocp-Apim-Subscription-Key\": \"subscription_key\"}\n",
    "```\n",
    " - Response\n",
    "```json\n",
    "{\n",
    "  \"models\": [\n",
    "    {\n",
    "      \"modelId\": \"45aad126-aafd-11ea-b8fb-d89ef3400c5f\",\n",
    "      \"createdTime\": \"2020-06-30T00:00:00Z\",\n",
    "      \"lastUpdatedTime\": \"2020-06-30T00:00:00Z\",\n",
    "      \"status\": \"READY\",\n",
    "      \"displayName\": \"DevOps-MultiAD\",\n",
    "      \"variablesCount\": 18\n",
    "    }\n",
    "  ],\n",
    "  \"currentCount\": 1,\n",
    "  \"maxCount\": 20,\n",
    "  \"nextLink\": \"\"\n",
    "}\n",
    "```\n",
    " - Error Response\n",
    "```json\n",
    "{\n",
    "    \"code\": \"string\",\n",
    "    \"message\": \"string\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = requests.get(API_MODEL.format(endpoint=ENDPOINT), headers=HEADERS)\n",
    "assert res.status_code == 200, f\"Error occured. Error message: {res.content}\"\n",
    "print(res.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train Model  \n",
    "```HTTP\n",
    "[POST] https://{endpoint}/multivariate/models\n",
    "```\n",
    "- #### Request\n",
    "  - **slidingWindow** An optional field, indicates how many history points will be used to determine the anomaly score of one subsequent point.\n",
    "  - **alignPolicy** An optional field, since those multivariate need to be aligned in the same timestamp before starting the detection.\n",
    "  - **alignMode**, An optional field, indicates how we merge different variables into the same time-range which is required by the model\n",
    "  ```json\n",
    "  {Inner, Outer}\n",
    "  ```\n",
    "  - **fillNAMethod**, optional field, indicates how missed valus will be filled with. Can not be set to *NotFill*, when alignMode is *Outer*\n",
    "  ```json\n",
    "  {Previous, Subsequent, Linear, Zero, Fix, NotFill}\n",
    "  ```\n",
    "  - **paddingValue**, optional field, only be usefull if fillNAMethod is set to *Fix*. \n",
    "  - **source**, required field, source path contain the training zip file.  \n",
    "  ```bash\n",
    "  Source file link to the input variables, each variable will be a csv with two columns, the first column will be timestamp, the second column will be value.Besides these variable csv files, an extra meta.json can be included in th zip file if you would like to rename a variable.By default, the file name of the variable will be used as the variable name. For example https://mvadsampledata.blob.core.windows.net/sample/sample_data_20_3000.zip\n",
    "  ```\n",
    "  - **startTime**, require field, it means start time of data you want to use for training, should be %Y-%m-%dT%H:%M:%SZ \n",
    "  format.\n",
    "  - **endTime**, require field, it means end time of data you want to use for training, should be %Y-%m-%dT%H:%M:%SZ format.\n",
    "  - **displayName**, optional field, it means model name.\n",
    "  \n",
    "- #### Response\n",
    "  - model location in headers\n",
    "\n",
    "- #### Sample\n",
    "  - Request\n",
    "  ```json\n",
    "header ={\"Content-Type\": \"application/json\", \"Ocp-Apim-Subscription-Key\": \"subscription_key\"}\n",
    "request=\n",
    "{\n",
    "    \"slidingWindow\": 200,\n",
    "    \"alignPolicy\": {\n",
    "        \"alignMode\": \"Outer\",\n",
    "        \"fillNAMethod\": \"Linear\", \n",
    "        \"paddingValue\": 0\n",
    "    },\n",
    "    \"source\": SOURCE_BLOB_SAS,\n",
    "    \"startTime\": \"2021-01-01T00:00:00Z\", \n",
    "    \"endTime\": \"2021-01-02T12:00:00Z\", \n",
    "    \"displayName\": \"SampleRequest\"\n",
    "}\n",
    "  ```\n",
    "  - Response\n",
    "  ```json\n",
    "  Location:\t{endpoint}/multivariate/models/927e2af8-a657-11ea-8cae-0242ac110002 # GUID\n",
    "  ```\n",
    "  - Error Response\n",
    " ```json\n",
    " {\n",
    "   \"code\": \"string\",\n",
    "   \"message\": \"string\"\n",
    " } \n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SLIDING_WINDOW = 200\n",
    "data = {\n",
    "    'slidingWindow': SLIDING_WINDOW,\n",
    "    'alignPolicy': {\n",
    "        'alignMode': 'Outer',\n",
    "        'fillNAMethod': 'Linear', \n",
    "        'paddingValue': 0\n",
    "    },\n",
    "    'source': SOURCE_BLOB_SAS,\n",
    "    'startTime': '2021-01-01T00:00:00Z', \n",
    "    'endTime': '2021-01-02T12:00:00Z', \n",
    "    'displayName': 'SampleRequest'\n",
    "}\n",
    "\n",
    "res = requests.post(API_MODEL.format(endpoint=ENDPOINT), data=json.dumps(data), headers=HEADERS)\n",
    "assert res.status_code == 201, f\"Error occured. Error message: {res.content}\"\n",
    "print(res.content)\n",
    "location = res.headers['Location']\n",
    "print(location)\n",
    "model_id = location[location.rindex('/')+1:]\n",
    "print(model_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Multivaraite Model Status by modelid\n",
    "```http\n",
    "[GET] https://{endpoint}/multivariate/models/{model_id}\n",
    "```\n",
    "#### Sample\n",
    " - Request\n",
    "```json\n",
    "header ={\"Content-Type\": \"application/json\", \"Ocp-Apim-Subscription-Key\": \"subscription_key\"}\n",
    "```\n",
    " - Response\n",
    " \n",
    "```json\n",
    "{\n",
    "  \"modelId\": \"45aad126-aafd-11ea-b8fb-d89ef3400c5f\",\n",
    "  \"createdTime\": \"2020-06-30T00:00:00Z\",\n",
    "  \"lastUpdatedTime\": \"2020-06-30T00:00:00Z\",\n",
    "  \"modelInfo\": {\n",
    "    \"slidingWindow\": 20,\n",
    "    \"alignPolicy\": {\n",
    "      \"alignMode\": \"Outer\",\n",
    "      \"fillNAMethod\": \"Linear\",\n",
    "      \"paddingValue\": 0\n",
    "    },\n",
    "    \"source\": SOURCE_BLOB_SAS,\n",
    "    \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "    \"endTime\": \"2019-04-02T00:00:00Z\",\n",
    "    \"displayName\": \"Devops-MultiAD\",\n",
    "    \"status\": \"READY\",\n",
    "    \"errors\": [],\n",
    "    \"diagnosticsInfo\": {\n",
    "      \"modelState\": {\n",
    "        \"epochIds\": [\n",
    "          10,\n",
    "          20,\n",
    "          30,\n",
    "          40,\n",
    "          50,\n",
    "          60,\n",
    "          70,\n",
    "          80,\n",
    "          90,\n",
    "          100\n",
    "        ],\n",
    "        \"trainLosses\": [\n",
    "          0.6291328072547913,\n",
    "          0.1671326905488968,\n",
    "          0.12354248017072678,\n",
    "          0.10259664058685303,\n",
    "          0.09584927558898926,\n",
    "          0.09069952368736267,\n",
    "          0.08686016499996185,\n",
    "          0.08603022992610931,\n",
    "          0.08287354558706284,\n",
    "          0.08235538005828857\n",
    "        ],\n",
    "        \"validationLosses\": [\n",
    "          1.9232804775238037,\n",
    "          1.0645641088485718,\n",
    "          0.6031560301780701,\n",
    "          0.5302737951278687,\n",
    "          0.46980252861976624,\n",
    "          0.4395163357257843,\n",
    "          0.41829314827919006,\n",
    "          0.40579143166542053,\n",
    "          0.405649870634079,\n",
    "          0.38492488861083984\n",
    "        ],\n",
    "        \"latenciesInSeconds\": [\n",
    "          0.3398594856262207,\n",
    "          0.3659665584564209,\n",
    "          0.37360644340515137,\n",
    "          0.35134077072143555,\n",
    "          0.33703041076660156,\n",
    "          0.31876277923583984,\n",
    "          0.32833099365234375,\n",
    "          0.3503587245941162,\n",
    "          0.3080024719238281,\n",
    "          0.3327946662902832\n",
    "        ]\n",
    "      },\n",
    "      \"variableStates\": [\n",
    "        {\n",
    "          \"variable\": \"ad_input\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"ad_ontimer_output\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"ingestion\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"data_in_speed\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"cpu\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"ad_series_init\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"flink_last_ckpt_duration\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"data_out_speed\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        },\n",
    "        {\n",
    "          \"variable\": \"ad_output\",\n",
    "          \"filledNARatio\": 0,\n",
    "          \"effectiveCount\": 1441,\n",
    "          \"startTime\": \"2019-04-01T00:00:00Z\",\n",
    "          \"endTime\": \"2019-04-02T00:00:00Z\"\n",
    "        }\n",
    "      ]\n",
    "    }\n",
    "  }\n",
    "}\n",
    "  \n",
    "```\n",
    " - Error Response\n",
    " ```json\n",
    "{\n",
    "\"code\": \"string\",\n",
    "\"message\": \"string\"\n",
    "}\n",
    " ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = requests.get(API_MODEL_STATUS.format(endpoint=ENDPOINT, model_id = model_id), headers=HEADERS)\n",
    "assert res.status_code == 200, f\"Error occured. Error message: {res.content}\"\n",
    "res_content = json.loads(res.content)\n",
    "print(json.dumps(res_content))\n",
    "print(res_content['modelInfo']['status'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Detection with the trained model\n",
    "```http\n",
    "[POST] https://{endpoint}/multivariate/models/{model_id}/detect\n",
    "```\n",
    "inference api will return an resultid, you can get result by get inference result api.\n",
    "\n",
    "- #### Request\n",
    "   - **source**, required field, source path contain the training zip file.\n",
    "```json\n",
    " Source file link of the input variables, each variable will be a csv with two columns, the first column will be timestamp, the second column will be value.Besides these variable csv files, an extra meta.json can be included in th zip file if you would like to rename a variable.Be default, the file name of the variable will be used as the variable name.\",example: \"https://multiadsample.blob.core.windows.net/data/sample_data_2_1000.zip?sp=rl&st=2020-12-04T06:03:47Z&se=2022-12-05T06:03:00Z&sv=2019-12-12&sr=b&sig=AZTbvZ7fcp3MdqGY%2FvGHJXJjUgjS4DneCGl7U5omq5c%3D\"\n",
    "```\n",
    "   - **startTime**, a require field, it means start time of data you want to use to inference, should be %Y-%m-%dT%H:%M:%SZ format. \n",
    "   - **endTime**, a require field, it means end time of data you want to use to inference, should be %Y-%m-%dT%H:%M:%SZ format.  \n",
    "\n",
    "- #### Response\n",
    "  - result location in headers\n",
    "  \n",
    "- #### Sample  \n",
    "- Request\n",
    "\n",
    "```json\n",
    "header ={\"Content-Type\": \"application/json\", \"Ocp-Apim-Subscription-Key\": \"subscription_key\"}\n",
    "request={\n",
    "  \"source\": SOURCE_BLOB_SAS,\n",
    "  \"startTime\": \"2020-01-01T00:00:00Z\",\n",
    "  \"endTime\": \"2020-02-01T00:00:00Z\"\n",
    "}\n",
    "```\n",
    "- Response\n",
    "```json\n",
    "Location:    {endpoint}/multivariate/results/927e2af8-a657-11ea-8cae-0242ac110002 # GUID\n",
    " ```\n",
    "- Error Response\n",
    " ```json\n",
    "{\n",
    "\"code\": \"string\",\n",
    "\"message\": \"string\"\n",
    "}\n",
    " ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = {\n",
    "    'source': SOURCE_BLOB_SAS,\n",
    "    'startTime': '2021-01-02T12:00:00Z', \n",
    "    'endTime': '2021-01-03T00:00:00Z', \n",
    "}\n",
    "\n",
    "res = requests.post(API_MODEL_INFERENCE.format(endpoint=ENDPOINT, model_id=model_id), \n",
    "                    data=json.dumps(data), headers=HEADERS)\n",
    "assert res.status_code == 201, f\"Error occured. Error message: {res.content}\"\n",
    "print(res.content)\n",
    "result_id = res.headers['location'].split(\"/\")[-1]\n",
    "print(f\"result id = {result_id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Detection Result\n",
    "```http\n",
    "[GET] https://{endpoint}/multivariate/results/{result_id}\n",
    "```\n",
    "- #### Sample\n",
    "- Request\n",
    "```json\n",
    "header ={\"Content-Type\": \"application/json\", \"Ocp-Apim-Subscription-Key\": \"subscription_key\"}\n",
    "```\n",
    "- Response\n",
    "```json\n",
    "Response={\n",
    "        \"resultId\": \"663884e6-b117-11ea-b3de-0242ac130004\",\n",
    "        \"summary\": {\n",
    "          \"status\": \"READY\",\n",
    "          \"errors\": [],\n",
    "          \"variableStates\": [\n",
    "            {\n",
    "              \"variable\": \"variable_1\",\n",
    "              \"filledNARatio\": 0,\n",
    "              \"effectiveCount\": 30,\n",
    "              \"startTime\": \"2021-01-01T00:00:00Z\",\n",
    "              \"endTime\": \"2021-01-01T00:29:00Z\"\n",
    "            },\n",
    "            {\n",
    "              \"variable\": \"variable_2\",\n",
    "              \"filledNARatio\": 0,\n",
    "              \"effectiveCount\": 30,\n",
    "              \"startTime\": \"2021-01-01T00:00:00Z\",\n",
    "              \"endTime\": \"2021-01-01T00:29:00Z\"\n",
    "            },\n",
    "            {\n",
    "              \"variable\": \"variable_3\",\n",
    "              \"filledNARatio\": 0,\n",
    "              \"effectiveCount\": 30,\n",
    "              \"startTime\": \"2021-01-01T00:00:00Z\",\n",
    "              \"endTime\": \"2021-01-01T00:29:00Z\"\n",
    "            }\n",
    "          ],\n",
    "          \"setupInfo\": {\n",
    "            \"source\": SOURCE_BLOB_SAS,\n",
    "            \"startTime\": \"2021-01-01T00:00:00Z\",\n",
    "            \"endTime\": \"2021-01-01T00:29:00Z\"\n",
    "          }\n",
    "        },\n",
    "        \"results\": [\n",
    "          {\n",
    "            \"timestamp\": \"2021-01-01T00:28:00Z\",\n",
    "            \"value\": {\n",
    "              \"isAnomaly\": false,\n",
    "              \"severity\": 0,\n",
    "              \"score\": 0.6928471326828003\n",
    "            },\n",
    "            \"errors\": []\n",
    "          },\n",
    "          {\n",
    "            \"timestamp\": \"2021-01-01T00:29:00Z\",\n",
    "            \"value\": {\n",
    "              \"isAnomaly\": true,\n",
    "              \"severity\": 0.5337404608726501,\n",
    "              \"score\": 0.9171165823936462,\n",
    "              \"interpretation\": [\n",
    "                {\n",
    "                  \"variable\": \"variable_2\",\n",
    "                  \"contributionScore\": 0.5371576215,\n",
    "                  \"correlationChanges\": {\n",
    "                    \"changedVariables\": [\n",
    "                      \"variable_1\",\n",
    "                      \"variable_3\"\n",
    "                    ],\n",
    "                    \"changedValues\": [\n",
    "                      0.1741322,\n",
    "                      0.1093203\n",
    "                    ]\n",
    "                  }\n",
    "                },\n",
    "                {\n",
    "                  \"variable\": \"variable_3\",\n",
    "                  \"contributionScore\": 0.3324159383,\n",
    "                  \"correlationChanges\": {\n",
    "                    \"changedVariables\": [\n",
    "                      \"variable_2\"\n",
    "                    ],\n",
    "                    \"changedValues\": [\n",
    "                      0.1229392\n",
    "                    ]\n",
    "                  }\n",
    "                },\n",
    "                {\n",
    "                  \"variable\": \"variable_1\",\n",
    "                  \"contributionScore\": 0.1304264402,\n",
    "                  \"correlationChanges\": {\n",
    "                    \"changedVariables\": [],\n",
    "                    \"changedValues\": []\n",
    "                  }\n",
    "                }\n",
    "              ]\n",
    "            }\n",
    "          }\n",
    "        ]\n",
    "      }\n",
    "```\n",
    " - Error Response\n",
    " ```json\n",
    "{\n",
    "\"code\": \"string\",\n",
    "\"message\": \"string\"\n",
    "}\n",
    " ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = requests.get(API_RESULTS.format(endpoint=ENDPOINT, result_id=result_id), headers=HEADERS)\n",
    "assert res.status_code == 200, f\"Error occured. Error message: {res.content}\"\n",
    "print(res.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Detection with the synchronized API\n",
    "```http\n",
    "[POST] https://{endpoint}/multivariate/models/{modelId}/last/detect\n",
    "```\n",
    "Detect multivariate anomaly with the trained model Id and the multivariate time-series data in request body. This API returns the anomalies in the given data.\n",
    "\n",
    "- #### Request\n",
    "   - **variables**, required field, containing time-series values and timestamps\n",
    "   - **detectingPoints**, required field, containing how many timestamps to detect (valid range 1-10 inclusive). \n",
    "\n",
    "- #### Response\n",
    "  - detection results\n",
    "  \n",
    "- #### Sample  \n",
    "- Request\n",
    "\n",
    "```json\n",
    "header ={\"Content-Type\": \"application/json\", \"Ocp-Apim-Subscription-Key\": \"subscription_key\"}\n",
    "request={\n",
    "  \"variables\": [\n",
    "    {\n",
    "      \"name\": \"Variable_1\",\n",
    "      \"timestamps\": [\n",
    "        \"2021-01-01T00:00:00Z\", \"2021-01-01T00:01:00Z\", \"2021-01-01T00:02:00Z\", \"2021-01-01T00:03:00Z\", \"2021-01-01T00:04:00Z\",\n",
    "        \"2021-01-01T00:05:00Z\", \"2021-01-01T00:06:00Z\", \"2021-01-01T00:07:00Z\", \"2021-01-01T00:08:00Z\", \"2021-01-01T00:09:00Z\",\n",
    "        \"2021-01-01T00:10:00Z\", \"2021-01-01T00:11:00Z\", \"2021-01-01T00:12:00Z\", \"2021-01-01T00:13:00Z\", \"2021-01-01T00:14:00Z\",\n",
    "        \"2021-01-01T00:15:00Z\", \"2021-01-01T00:16:00Z\", \"2021-01-01T00:17:00Z\", \"2021-01-01T00:18:00Z\", \"2021-01-01T00:19:00Z\",\n",
    "        \"2021-01-01T00:20:00Z\", \"2021-01-01T00:21:00Z\", \"2021-01-01T00:22:00Z\", \"2021-01-01T00:23:00Z\", \"2021-01-01T00:24:00Z\",\n",
    "        \"2021-01-01T00:25:00Z\", \"2021-01-01T00:26:00Z\", \"2021-01-01T00:27:00Z\", \"2021-01-01T00:28:00Z\", \"2021-01-01T00:29:00Z\"\n",
    "      ],\n",
    "      \"values\": [\n",
    "        0.4551378545933972, 0.7388603950488748, 0.201088255984052, 0.7462812245891899, 0.07308128850401663,\n",
    "        0.33090474587393537, 0.7544925268153315, 0.987506336316328, 0.6665932993421468, 0.6308351543168672,\n",
    "        0.08083310161466228, 0.8414415588668442, 0.514583545640453, 0.0954489875193526, 0.7786793231920507,\n",
    "        0.41646133667960994, 0.030176187583339287, 0.3474214937189324, 0.508530173413991, 0.42451199127255046,\n",
    "        0.2115944222725208, 0.24733519545833516, 0.8791022110982156, 0.9479621899884665, 0.26702703121252136,\n",
    "        0.6954503497669413, 0.1235728391488995, 0.8214915473050647, 0.11813002444192677, 0.8579045951076123\n",
    "      ]\n",
    "    },\n",
    "    {\n",
    "      \"name\": \"Variable_2\",\n",
    "      \"timestamps\": [\n",
    "        \"2021-01-01T00:00:00Z\", \"2021-01-01T00:01:00Z\", \"2021-01-01T00:02:00Z\", \"2021-01-01T00:03:00Z\", \"2021-01-01T00:04:00Z\",\n",
    "        \"2021-01-01T00:05:00Z\", \"2021-01-01T00:06:00Z\", \"2021-01-01T00:07:00Z\", \"2021-01-01T00:08:00Z\", \"2021-01-01T00:09:00Z\",\n",
    "        \"2021-01-01T00:10:00Z\", \"2021-01-01T00:11:00Z\", \"2021-01-01T00:12:00Z\", \"2021-01-01T00:13:00Z\", \"2021-01-01T00:14:00Z\",\n",
    "        \"2021-01-01T00:15:00Z\", \"2021-01-01T00:16:00Z\", \"2021-01-01T00:17:00Z\", \"2021-01-01T00:18:00Z\", \"2021-01-01T00:19:00Z\",\n",
    "        \"2021-01-01T00:20:00Z\", \"2021-01-01T00:21:00Z\", \"2021-01-01T00:22:00Z\", \"2021-01-01T00:23:00Z\", \"2021-01-01T00:24:00Z\",\n",
    "        \"2021-01-01T00:25:00Z\", \"2021-01-01T00:26:00Z\", \"2021-01-01T00:27:00Z\", \"2021-01-01T00:28:00Z\", \"2021-01-01T00:29:00Z\"\n",
    "      ],\n",
    "      \"values\": [\n",
    "        0.9617871613964145, 0.24903311574778408, 0.4920561254118613, 0.9895601049618598, 0.9171759283128094,\n",
    "        0.5754204711105273, 0.1811033296265634, 0.8852311981742577, 0.9543231904644779, 0.7088012446094262,\n",
    "        0.7843572237149014, 0.7664787010700046, 0.3699552325387093, 0.504519908266789, 0.5848930929950164,\n",
    "        0.7628913396089576, 0.8148405868900065, 0.08540458873739332, 0.03481976727525682, 0.21275099339467762,\n",
    "        0.9836175579199806, 0.9321441483364282, 0.038466608085469534, 0.1723138437622782, 0.8626383410218382,\n",
    "        0.35053229974224254, 0.631141662835182, 0.0730352607990088, 0.08886179043386, 0.7488606040971179\n",
    "      ]\n",
    "    },\n",
    "    {\n",
    "      \"name\": \"Variable_3\",\n",
    "      \"timestamps\": [\n",
    "        \"2021-01-01T00:00:00Z\", \"2021-01-01T00:01:00Z\", \"2021-01-01T00:02:00Z\", \"2021-01-01T00:03:00Z\", \"2021-01-01T00:04:00Z\",\n",
    "        \"2021-01-01T00:05:00Z\", \"2021-01-01T00:06:00Z\", \"2021-01-01T00:07:00Z\", \"2021-01-01T00:08:00Z\", \"2021-01-01T00:09:00Z\",\n",
    "        \"2021-01-01T00:10:00Z\", \"2021-01-01T00:11:00Z\", \"2021-01-01T00:12:00Z\", \"2021-01-01T00:13:00Z\", \"2021-01-01T00:14:00Z\",\n",
    "        \"2021-01-01T00:15:00Z\", \"2021-01-01T00:16:00Z\", \"2021-01-01T00:17:00Z\", \"2021-01-01T00:18:00Z\", \"2021-01-01T00:19:00Z\",\n",
    "        \"2021-01-01T00:20:00Z\", \"2021-01-01T00:21:00Z\", \"2021-01-01T00:22:00Z\", \"2021-01-01T00:23:00Z\", \"2021-01-01T00:24:00Z\",\n",
    "        \"2021-01-01T00:25:00Z\", \"2021-01-01T00:26:00Z\", \"2021-01-01T00:27:00Z\", \"2021-01-01T00:28:00Z\", \"2021-01-01T00:29:00Z\"\n",
    "      ],\n",
    "      \"values\": [\n",
    "        0.4030756879437628, 0.15526889968448554, 0.36352226408981103, 0.6051200637229004, 0.8516795018476276,\n",
    "        0.2645605735279929, 0.6810875830037345, 0.9165894221681316, 0.700783245230424, 0.5624155469940331,\n",
    "        0.6277289685127893, 0.15992056539730204, 0.6020964482827594, 0.35937967753105915, 0.8731686034848609,\n",
    "        0.20301549117588935, 0.029261872151168933, 0.6261499548828445, 0.45850782028563386, 0.8275006940083313,\n",
    "        0.032760268834037376, 0.4485202784055029, 0.8915691008748384, 0.891669051517807, 0.9469979353323046,\n",
    "        0.115293087370132, 0.08818772518459506, 0.7426286620589166, 0.32372247468990756, 0.936268139507417\n",
    "      ]\n",
    "    }\n",
    "  ],\n",
    "  \"detectingPoints\": 2\n",
    "}\n",
    "```\n",
    "- Response\n",
    "```json\n",
    "{\n",
    "  \"variableStates\": [\n",
    "    {\n",
    "      \"variable\": \"variable_1\",\n",
    "      \"filledNARatio\": 0,\n",
    "      \"effectiveCount\": 30,\n",
    "      \"startTime\": \"2021-01-01T00:00:00Z\",\n",
    "      \"endTime\": \"2021-01-01T00:29:00Z\"\n",
    "    },\n",
    "    {\n",
    "      \"variable\": \"variable_2\",\n",
    "      \"filledNARatio\": 0,\n",
    "      \"effectiveCount\": 30,\n",
    "      \"startTime\": \"2021-01-01T00:00:00Z\",\n",
    "      \"endTime\": \"2021-01-01T00:29:00Z\"\n",
    "    },\n",
    "    {\n",
    "      \"variable\": \"variable_3\",\n",
    "      \"filledNARatio\": 0,\n",
    "      \"effectiveCount\": 30,\n",
    "      \"startTime\": \"2021-01-01T00:00:00Z\",\n",
    "      \"endTime\": \"2021-01-01T00:29:00Z\"\n",
    "    }\n",
    "  ],\n",
    "  \"results\": [\n",
    "    {\n",
    "      \"timestamp\": \"2021-01-01T00:28:00Z\",\n",
    "      \"value\": {\n",
    "        \"isAnomaly\": false,\n",
    "        \"severity\": 0,\n",
    "        \"score\": 0.6928471326828003\n",
    "      },\n",
    "      \"errors\": []\n",
    "    },\n",
    "    {\n",
    "      \"timestamp\": \"2021-01-01T00:29:00Z\",\n",
    "      \"value\": {\n",
    "        \"isAnomaly\": true,\n",
    "        \"severity\": 0.5337404608726501,\n",
    "        \"score\": 0.9171165823936462,\n",
    "        \"interpretation\": [\n",
    "          {\n",
    "            \"variable\": \"variable_2\",\n",
    "            \"contributionScore\": 0.5371576215,\n",
    "            \"correlationChanges\": {\n",
    "              \"changedVariables\": [\n",
    "                \"variable_1\",\n",
    "                \"variable_3\"\n",
    "              ],\n",
    "              \"changedValues\": [\n",
    "                0.1741322,\n",
    "                0.1093203\n",
    "              ]\n",
    "            }\n",
    "          },\n",
    "          {\n",
    "            \"variable\": \"variable_3\",\n",
    "            \"contributionScore\": 0.3324159383,\n",
    "            \"correlationChanges\": {\n",
    "              \"changedVariables\": [\n",
    "                \"variable_2\"\n",
    "              ],\n",
    "              \"changedValues\": [\n",
    "                0.1229392\n",
    "              ]\n",
    "            }\n",
    "          },\n",
    "          {\n",
    "            \"variable\": \"variable_1\",\n",
    "            \"contributionScore\": 0.1304264402,\n",
    "            \"correlationChanges\": {\n",
    "              \"changedVariables\": [],\n",
    "              \"changedValues\": []\n",
    "            }\n",
    "          }\n",
    "        ]\n",
    "      },\n",
    "      \"errors\": []\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "```\n",
    "- Error Response\n",
    " ```json\n",
    "{\n",
    "\"code\": \"string\",\n",
    "\"message\": \"string\"\n",
    "}\n",
    " ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "with open(\"multivariate_sample.json\", \"r\") as f:\n",
    "    sample_input = json.load(f)\n",
    "\n",
    "url = API_MODEL_LAST_INFERENCE.format(model_id=model_id)\n",
    "payload = {\n",
    "    \"variables\": sample_input,\n",
    "    \"detectingPoints\": 10\n",
    "}\n",
    "\n",
    "res = requests.post(url, headers=HEADERS, data=json.dumps(payload))\n",
    "print(res.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize Results\n",
    "Demo code to draw results. Additional python package is required."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bokeh.io import output_file, show, output_notebook, save\n",
    "from bokeh.layouts import gridplot\n",
    "from bokeh.plotting import figure\n",
    "from matplotlib import pyplot\n",
    "from bokeh.models import ColumnDataSource, HoverTool\n",
    "from bokeh.palettes import Dark2_5 as palette\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os\n",
    "import itertools  \n",
    "import shutil\n",
    "import uuid\n",
    "import zipfile\n",
    "from urllib.request import urlretrieve\n",
    "%matplotlib inline\n",
    "output_notebook()\n",
    "\n",
    "def unzip_file(zip_src, dst_dir):\n",
    "    r = zipfile.is_zipfile(zip_src)\n",
    "    if r:\n",
    "        fz = zipfile.ZipFile(zip_src, 'r')\n",
    "        print(fz)\n",
    "        for file in fz.namelist():\n",
    "            fz.extract(file, dst_dir)\n",
    "    else:\n",
    "        print('This is not zip')\n",
    "        \n",
    "def load_data(local_data_path, start, end):\n",
    "    cwd = os.getcwd()\n",
    "    new_dir = os.path.join(cwd, str(uuid.uuid1()))\n",
    "    shutil.rmtree(new_dir, ignore_errors=True)\n",
    "    os.mkdir(new_dir)\n",
    "    unzip_file(local_data_path, new_dir)\n",
    "    files = os.listdir(new_dir)\n",
    "    frames = []\n",
    "    for file in files:\n",
    "        if file[-4:] != '.csv':\n",
    "            continue\n",
    "        frame = pd.read_csv('{}//{}'.format(new_dir, file))\n",
    "        var = file[:file.find('.csv')]\n",
    "        frame = frame.rename(columns={'value': var})\n",
    "        frame = frame[frame['timestamp'] >= start]\n",
    "        frame = frame[frame['timestamp'] <= end]\n",
    "        frame['timestamp'] = pd.to_datetime(frame['timestamp'])\n",
    "        frame.set_index(['timestamp'], inplace=True)\n",
    "        frames.append(frame)\n",
    "    shutil.rmtree(new_dir, ignore_errors=True)\n",
    "    return frames\n",
    "\n",
    "\n",
    "def plot_lines_multi(x, y, p, color, name, t_str=\"hover,save,pan,box_zoom,reset,wheel_zoom\", t_loc='above'):\n",
    "    '''...\n",
    "    '''\n",
    "    p.line(x, y, color=color, legend_label=name)\n",
    "\n",
    "def draw(data_source, local_data_path, result_id, sensitivity, start, end):\n",
    "    urlretrieve(data_source, local_data_path)\n",
    "    print(local_data_path, result_id, sensitivity, start, end)\n",
    "    series = load_data(local_data_path, start, end)\n",
    "    p_list = []\n",
    "    colors = itertools.cycle(palette)\n",
    "    # p_value = figure(background_fill_color=\"#fafafa\", x_axis_type=\"datetime\")\n",
    "    for var, color in zip(series, colors):\n",
    "        name = var.columns.values[0]\n",
    "        p_value = figure(background_fill_color=\"#fafafa\", x_axis_type=\"datetime\")\n",
    "        plot_lines_multi(var.index, var[name], p_value, color, name)\n",
    "        p_list.append(p_value)\n",
    "    header = HEADERS\n",
    "    raw_result = json.loads(requests.get(API_RESULTS.format(endpoint=ENDPOINT, result_id=result_id), headers=header).content)\n",
    "    if raw_result['summary']['status'] != 'READY':\n",
    "        print(\"result not ready\")\n",
    "        return\n",
    "    filter_item = list(filter(lambda x: 'value' in x and 'isAnomaly' in x['value'] and x['timestamp'] >= start and x['timestamp'] <= end, raw_result['results']))\n",
    "    timestamps = [item['timestamp'] for item in filter_item]\n",
    "    isAnomaly = [item['value']['isAnomaly'] for item in filter_item]\n",
    "    RawScore = [item['value']['score'] for item in filter_item]\n",
    "    Severity = [item['value']['severity'] for item in filter_item]\n",
    "    result = pd.DataFrame({'Timestamp': timestamps, 'isAnomaly': isAnomaly, 'RawScore': RawScore, 'Severity': Severity})\n",
    "    result.loc[(result.Severity <= (1 - sensitivity)) & (result.isAnomaly == True), 'isAnomaly'] = False\n",
    "    result['Timestamp'] = pd.to_datetime(result['Timestamp'])\n",
    "    result.set_index(['Timestamp'], inplace=True)\n",
    "    result = result.reindex(['isAnomaly', 'RawScore', 'Severity'], axis=1)\n",
    "    colors = ['red', 'blue', 'black']\n",
    "    for col, color in zip(result.columns, colors):\n",
    "        p = figure(background_fill_color=\"#fafafa\", x_axis_type=\"datetime\")\n",
    "        p.line(result.index, result[col], color=color, alpha=0.8, legend_label=col)\n",
    "        p_list.append(p)\n",
    "    grid = gridplot([[x] for x in p_list], sizing_mode='scale_width', plot_height=50)\n",
    "    show(grid)\n",
    "    result = result.sort_values(by=['RawScore'], ascending=False)\n",
    "    top_anomaly = list(result[result.isAnomaly].index.strftime('%Y-%m-%dT%H:%M:%SZ'))\n",
    "    if len(top_anomaly) > 0: \n",
    "        print(\"Top Anomaly Timestamp is : {0}\".format(top_anomaly[0]))\n",
    "        return series, raw_result, top_anomaly[0]\n",
    "    else:\n",
    "        print(\"No anomaly detected. Sensitivity may be too low.\")\n",
    "        return series, raw_result, None\n",
    "\n",
    "def show_contribution(series, raw_result, anomaly_timestamp):\n",
    "    anomaly_result = [x for x in raw_result['results'] if 'interpretation' in x['value'] and x['timestamp'] == top_anomaly][0]\n",
    "    contributors = [x['variable'] for x in anomaly_result['value']['interpretation']]\n",
    "    series_index = pd.DataFrame({'index': list(range(0, len(series))), 'name': [x.columns[0] for x in series]})\n",
    "    series_index = series_index.set_index('name')\n",
    "    sorted_series = [series[i][series[i].index <= top_anomaly].tail(SLIDING_WINDOW) for i in series_index.reindex(contributors)['index'].values]\n",
    "    p_list = []\n",
    "    colors = itertools.cycle(palette)\n",
    "    for var, color in zip(sorted_series, colors):\n",
    "        name = var.columns.values[0]\n",
    "        p_value = figure(background_fill_color=\"#fafafa\", x_axis_type=\"datetime\")\n",
    "        plot_lines_multi(var.index, var[name], p_value, color, name)\n",
    "        p_list.append(p_value)\n",
    "    grid = gridplot([[x] for x in p_list], sizing_mode='scale_width', plot_height=50)\n",
    "    show(grid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_source = SOURCE_BLOB_SAS\n",
    "local_data_path = \"sample_data_5_3000.zip\"\n",
    "sensitivity = 0.7\n",
    "start_date = \"2021-01-02T12:00:00Z\"\n",
    "end_date = \"2021-01-03T00:00:00Z\"\n",
    "series, raw_result, top_anomaly = draw(data_source, local_data_path, result_id, sensitivity, start_date, end_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if top_anomaly:\n",
    "    show_contribution(series, raw_result, top_anomaly)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Delete model by modelid\n",
    "```http\n",
    "[DELETE] https://{endpoint}/multivariate/models/{model_id}\n",
    "```\n",
    "#### Sample\n",
    " - Request\n",
    "```json\n",
    "header ={\"Content-Type\": \"application/json\", \"Ocp-Apim-Subscription-Key\": \"subscription_key\"}\n",
    "```\n",
    " - Response\n",
    "```json\n",
    "response={}\n",
    "```\n",
    " - Error Response\n",
    " ```json\n",
    "{\n",
    "\"code\": \"string\",\n",
    "\"message\": \"string\"\n",
    "}\n",
    " ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = requests.delete(API_DELETE.format(endpoint=ENDPOINT, model_id=model_id), headers=HEADERS)\n",
    "assert res.status_code == 204, f\"Error occured. Error message: {res.content}\"\n",
    "print(res.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Model list has been updated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = requests.get(API_MODEL.format(endpoint=ENDPOINT), headers=HEADERS)\n",
    "assert res.status_code == 200, f\"Error occured. Error message: {res.content}\"\n",
    "print(res.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.12 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]"
  },
  "vscode": {
   "interpreter": {
    "hash": "aa60471fcc78aee1d161dfae290e862a10214dc7d6ca1ceb34988aca5ae05936"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}