how-to-use-azureml/deployment/deploy-with-controlled-rollout/deploy-aks-with-controlled-rollout.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
        "\n",
        "Licensed under the MIT License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/deployment/production-deploy-to-aks/production-deploy-to-aks.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Deploy models to Azure Kubernetes Service (AKS) using controlled roll out\n",
        "This notebook will show you how to deploy mulitple AKS webservices with the same scoring endpoint and how to roll out your models in a controlled manner by configuring % of scoring traffic going to each webservice. If you are using a Notebook VM, you are all set. Otherwise, go through the [configuration notebook](../../../configuration.ipynb) to install the Azure Machine Learning Python SDK and create an Azure ML Workspace."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Check for latest version\n",
        "import azureml.core\n",
        "print(azureml.core.VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Initialize workspace\n",
        "Create a [Workspace](https://docs.microsoft.com/python/api/azureml-core/azureml.core.workspace%28class%29?view=azure-ml-py) object from your persisted configuration."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.workspace import Workspace\n",
        "\n",
        "ws = Workspace.from_config()\n",
        "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register the model\n",
        "Register a file or folder as a model by calling [Model.register()](https://docs.microsoft.com/python/api/azureml-core/azureml.core.model.model?view=azure-ml-py#register-workspace--model-path--model-name--tags-none--properties-none--description-none--datasets-none--model-framework-none--model-framework-version-none--child-paths-none-).\n",
        "In addition to the content of the model file itself, your registered model will also store model metadata -- model description, tags, and framework information -- that will be useful when managing and deploying models in your workspace. Using tags, for instance, you can categorize your models and apply filters when listing models in your workspace."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Model\n",
        "\n",
        "model = Model.register(workspace=ws,\n",
        "                       model_name='sklearn_regression_model.pkl',                # Name of the registered model in your workspace.\n",
        "                       model_path='./sklearn_regression_model.pkl',  # Local file to upload and register as a model.\n",
        "                       model_framework=Model.Framework.SCIKITLEARN,  # Framework used to create the model.\n",
        "                       model_framework_version='0.19.1',             # Version of scikit-learn used to create the model.\n",
        "                       description='Ridge regression model to predict diabetes progression.',\n",
        "                       tags={'area': 'diabetes', 'type': 'regression'})\n",
        "\n",
        "print('Name:', model.name)\n",
        "print('Version:', model.version)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register an environment (for all models)\n",
        "\n",
        "If you control over how your model is run, or if it has special runtime requirements, you can specify your own environment and scoring method.\n",
        "\n",
        "Specify the model's runtime environment by creating an [Environment](https://docs.microsoft.com/python/api/azureml-core/azureml.core.environment%28class%29?view=azure-ml-py) object and providing the [CondaDependencies](https://docs.microsoft.com/python/api/azureml-core/azureml.core.conda_dependencies.condadependencies?view=azure-ml-py) needed by your model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Environment\n",
        "from azureml.core.conda_dependencies import CondaDependencies\n",
        "\n",
        "environment=Environment('my-sklearn-environment')\n",
        "environment.python.conda_dependencies = CondaDependencies.create(conda_packages=[\n",
        "    'pip==20.2.4'],\n",
        "    pip_packages=[\n",
        "    'azureml-defaults',\n",
        "    'inference-schema[numpy-support]',\n",
        "    'numpy==1.23',\n",
        "    'scikit-learn==0.22.1',\n",
        "    'scipy'\n",
        "])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "When using a custom environment, you must also provide Python code for initializing and running your model. An example script is included with this notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "with open('score.py') as f:\n",
        "    print(f.read())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create the InferenceConfig\n",
        "Create the inference configuration to reference your environment and entry script during deployment"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.model import InferenceConfig\n",
        "\n",
        "inference_config = InferenceConfig(entry_script='score.py', \n",
        "                                   source_directory='.',\n",
        "                                   environment=environment)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Provision the AKS Cluster\n",
        "If you already have an AKS cluster attached to this workspace, skip the step below and provide the name of the cluster.\n",
        "\n",
        "> Note that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.compute import AksCompute\n",
        "from azureml.core.compute import ComputeTarget\n",
        "# Use the default configuration (can also provide parameters to customize)\n",
        "prov_config = AksCompute.provisioning_configuration()\n",
        "\n",
        "aks_name = 'my-aks' \n",
        "# Create the cluster\n",
        "aks_target = ComputeTarget.create(workspace = ws, \n",
        "                                  name = aks_name, \n",
        "                                  provisioning_configuration = prov_config) \n",
        "aks_target.wait_for_completion(show_output=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create an Endpoint and add a version (AKS service)\n",
        "This creates a new endpoint and adds a version behind it. By default the first version added is the default version. You can specify the traffic percentile a version takes behind an endpoint. \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# deploying the model and create a new endpoint\n",
        "from azureml.core.webservice import AksEndpoint\n",
        "# from azureml.core.compute import ComputeTarget\n",
        "\n",
        "#select a created compute\n",
        "compute = ComputeTarget(ws, 'my-aks')\n",
        "namespace_name=\"endpointnamespace\"\n",
        "# define the endpoint name\n",
        "endpoint_name = \"myendpoint1\"\n",
        "# define the service name\n",
        "version_name= \"versiona\"\n",
        "\n",
        "endpoint_deployment_config = AksEndpoint.deploy_configuration(tags = {'modelVersion':'firstversion', 'department':'finance'}, \n",
        "                                                              description = \"my first version\", namespace = namespace_name, \n",
        "                                                              version_name = version_name, traffic_percentile = 40)\n",
        "\n",
        "endpoint = Model.deploy(ws, endpoint_name, [model], inference_config, endpoint_deployment_config, compute)\n",
        "endpoint.wait_for_deployment(True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "endpoint.get_logs()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Add another version of the service to an existing endpoint\n",
        "This adds another version behind an existing endpoint. You can specify the traffic percentile the new version takes. If no traffic_percentile is specified then it defaults to 0. All the unspecified traffic percentile (in this example 50) across all versions goes to default version."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Adding a new version to an existing Endpoint.\n",
        "version_name_add=\"versionb\" \n",
        "\n",
        "endpoint.create_version(version_name = version_name_add, inference_config=inference_config, models=[model], tags = {'modelVersion':'secondversion', 'department':'finance'}, \n",
        "                        description = \"my second version\", traffic_percentile = 10)\n",
        "endpoint.wait_for_deployment(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Update an existing version in an endpoint\n",
        "There are two types of versions: control and treatment. An endpoint contains one or more treatment versions but only one control version. This categorization helps compare the different versions against the defined control version."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "endpoint.update_version(version_name=endpoint.versions[version_name_add].name, description=\"my second version update\", traffic_percentile=40, is_default=True, is_control_version_type=True)\n",
        "endpoint.wait_for_deployment(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Test the web service using run method\n",
        "Test the web sevice by passing in data. Run() method retrieves API keys behind the scenes to make sure that call is authenticated."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Scoring on endpoint\n",
        "import json\n",
        "test_sample = json.dumps({'data': [\n",
        "    [1,2,3,4,5,6,7,8,9,10], \n",
        "    [10,9,8,7,6,5,4,3,2,1]\n",
        "]})\n",
        "\n",
        "test_sample_encoded = bytes(test_sample, encoding='utf8')\n",
        "prediction = endpoint.run(input_data=test_sample_encoded)\n",
        "print(prediction)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Delete Resources"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# deleting a version in an endpoint\n",
        "endpoint.delete_version(version_name=version_name)\n",
        "endpoint.wait_for_deployment(True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# deleting an endpoint, this will delete all versions in the endpoint and the endpoint itself\n",
        "endpoint.delete()"
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "shipatel"
      }
    ],
    "category": "deployment",
    "compute": [
      "None"
    ],
    "datasets": [
      "Diabetes"
    ],
    "deployment": [
      "Azure Kubernetes Service"
    ],
    "exclude_from_index": false,
    "framework": [
      "Scikit-learn"
    ],
    "friendly_name": "Deploy models to AKS using controlled roll out",
    "index_order": 3,
    "kernelspec": {
      "display_name": "Python 3.8 - AzureML",
      "language": "python",
      "name": "python38-azureml"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.0"
    },
    "star_tag": [
      "featured"
    ],
    "tags": [
      "None"
    ],
    "task": "Deploy a model with Azure Machine Learning"
  },
  "nbformat": 4,
  "nbformat_minor": 2
}