Importing MLClient leads to error "ImportError: cannot import name '_T' from 'marshmallow.fields' "

[OKazanskyi]

• Package Name: azure-ai-ml
• Package Version: 1.23.0
• Operating System: ML Studio, Compute Standard_A2_v2
• Python Version: Python3.8 - AzureML

Describe the bug
When I run command from azure.ai.ml import MLClient I get an error ImportError: cannot import name '_T' from 'marshmallow.fields' (/anaconda/envs/azureml_py38/lib/python3.10/site-packages/marshmallow/fields.py)

To Reproduce
Steps to reproduce the behavior:

1. Run the Lab 10 https://github.com/MicrosoftLearning/mslearn-azure-ml.git
   I tried remove resource group and install it back, restart the compute and reinstall the library. Nothing helped

Expected behavior
Library MLClient should be successfully loaded

Screenshots
NA

Additional context
The full error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[4], line 1
----> 1 from azure.ai.ml import MLClient

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/__init__.py:11
      8 from typing import Any, Optional
     10 # from .entities._builders.parallel_func import parallel
---> 11 from azure.ai.ml.entities._inputs_outputs import Input, Output
     13 from ._ml_client import MLClient
     14 from ._utils._logger_utils import initialize_logger_info

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/entities/__init__.py:18
     15 from azure.ai.ml._restclient.v2022_10_01.models import CreatedByType
     16 from azure.ai.ml._restclient.v2022_10_01_preview.models import UsageUnit
---> 18 from ._assets._artifacts._package.base_environment_source import BaseEnvironment
     19 from ._assets._artifacts._package.inferencing_server import (
     20     AzureMLBatchInferencingServer,
     21     AzureMLOnlineInferencingServer,
   (...)
     24     TritonInferencingServer,
     25 )
     26 from ._assets._artifacts._package.model_configuration import ModelConfiguration

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/entities/_assets/__init__.py:9
      5 __path__ = __import__("pkgutil").extend_path(__path__, __name__)
      8 from ._artifacts.artifact import Artifact
----> 9 from ._artifacts.code import Code
     10 from ._artifacts.data import Data
     11 from ._artifacts.index import Index

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/entities/_assets/_artifacts/code.py:10
      7 from typing import Any, Dict, Optional, Union
      9 from azure.ai.ml._restclient.v2022_05_01.models import CodeVersionData, CodeVersionDetails
---> 10 from azure.ai.ml._schema import CodeAssetSchema
     11 from azure.ai.ml._utils._arm_id_utils import AMLVersionedArmId
     12 from azure.ai.ml._utils._asset_utils import IgnoreFile, get_content_hash, get_content_hash_version, get_ignore_file

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/_schema/__init__.py:7
      1 # ---------------------------------------------------------
      2 # Copyright (c) Microsoft Corporation. All rights reserved.
      3 # ---------------------------------------------------------
      5 __path__ = __import__("pkgutil").extend_path(__path__, __name__)
----> 7 from ._data_import import DataImportSchema
      8 from ._sweep import SweepJobSchema
      9 from .assets.code_asset import AnonymousCodeAssetSchema, CodeAssetSchema

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/_schema/_data_import/__init__.py:7
      1 # ---------------------------------------------------------
      2 # Copyright (c) Microsoft Corporation. All rights reserved.
      3 # ---------------------------------------------------------
      5 __path__ = __import__("pkgutil").extend_path(__path__, __name__)
----> 7 from .data_import import DataImportSchema
      9 __all__ = ["DataImportSchema"]

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/_schema/_data_import/data_import.py:9
      1 # ---------------------------------------------------------
      2 # Copyright (c) Microsoft Corporation. All rights reserved.
      3 # ---------------------------------------------------------
      4 
      5 # pylint: disable=unused-argument
      7 from marshmallow import post_load
----> 9 from azure.ai.ml._schema.core.fields import NestedField
     10 from azure.ai.ml._schema.job.input_output_entry import DatabaseSchema, FileSystemSchema
     11 from azure.ai.ml._utils._experimental import experimental

File /anaconda/envs/azureml_py38/lib/python3.10/site-packages/azure/ai/ml/_schema/core/fields.py:19
     17 from marshmallow import RAISE, fields
     18 from marshmallow.exceptions import ValidationError
---> 19 from marshmallow.fields import _T, Field, Nested
     20 from marshmallow.utils import FieldInstanceResolutionError, from_iso_datetime, resolve_field_instance
     22 from ..._utils._arm_id_utils import AMLVersionedArmId, is_ARM_id_for_resource, parse_name_label, parse_name_version

ImportError: cannot import name '_T' from 'marshmallow.fields' (/anaconda/envs/azureml_py38/lib/python3.10/site-packages/marshmallow/fields.py)

[O-EAI]

Getting same thing here, this is the issue: microsoft/promptflow#3902

[iamrk04]

This is actively being fixed. Here is the PR covering the fix. We just need to wait for the release of azure-ai-ml==1.23.1.

If you need to solve this immediately, try installing an older version of this package, For example, run the following command in the env: pip install azure-ai-ml==1.22.4. This should fix the issue.

Thanks!

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/azure-ml-sdk @azureml-github.

[OKazanskyi]

I got the same error after uninstalling and installing azure-ai-ml==1.22.4

[apsonawane]

Still getting the same error after installing azure-ai-ml==1.22.4
Installing marshmallow==3.23.2
Installing az extension ml version 2.32.4

[iamrk04]

Hi @OKazanskyi ,

the new version of azure-ai-ml, with the fix, has been released. Please run the following to upgrade it: pip install azure-ai-ml --upgrade. This helped resolve the issue at my end.

Thanks!

[lambdakris]

FYI, this is affecting the azure-ai-evaluation package as well.

[xavimf87]

It worked with azure-ai-ml==1.23.1

[ncclementi]

@OKazanskyi any chance that this changes make it into a release in that can be installed with conda?

Looks like the last conda release was 2 months ago see https://anaconda.org/Microsoft/azure-ai-ml and the PR is broken in the feedstock, conda-forge/azure-ai-ml-feedstock#29 any chance someone can take a look?