[Bug]: AttributeError: 'AzureOpenAIEmbedding' object has no attribute 'predict'

[CarlosECastilloM]

Bug Description

This is my code:

1. Load Document

from llama_index.core import SimpleDirectoryReader
doc = SimpleDirectoryReader(input_files=["MetaGPT.pdf"]).load_data()

2. Split Document in Chunks (splitting into sentences)

from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(chunk_size=1024)
nodes = splitter.get_nodes_from_documents(doc)

3. LLM Model and Embedding Model Settings (optional step)

from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding

from dotenv import load_dotenv
from openai import AzureOpenAI

load_dotenv()

client = AzureOpenAI(
api_key = AZURE_OPENAI_API_KEY,
azure_endpoint = AZURE_OPENAI_ENDPOINT,
api_version = OPENAI_API_VERSION
)

client_embed = AzureOpenAIEmbedding(
model = AZURE_EMBEDDING_MODEL,
api_key= AZURE_OPENAI_API_KEY,
api_version = OPENAI_API_VERSION,
azure_endpoint = AZURE_OPENAI_ENDPOINT
)

PART II: CREATING 2 INDEXES: SUMMARY INDEX AND A VECTOR INDEX

4. Creating a Summary and a Vector Index

from llama_index.core import SummaryIndex
from llama_index.core import VectorStoreIndex

summary_index = SummaryIndex(nodes)

vector_index = VectorStoreIndex(
nodes = nodes,
embed_model = client_embed,
show_progress = True

)

5. Defining the Query Tools

summary_query_engine = summary_index.as_query_engine(
response_mode="tree_summarize",
use_async=True,
)

vector_query_engine = vector_index.as_query_engine()

6. Establishing the Query Tools

from llama_index.core.tools import QueryEngineTool

summary_tool = QueryEngineTool.from_defaults(
query_engine = summary_query_engine,
description = (
"Useful for summarization questions related to MetaGPT"
),
)

vector_tool = QueryEngineTool.from_defaults(
query_engine=vector_query_engine,
description=(
"Useful for retrieving specific context from the MetaGPT paper."
),
)

7. Establish a Router from different type of Selectors

from llama_index.core.query_engine.router_query_engine import RouterQueryEngine
from llama_index.core.selectors import LLMSingleSelector

engine = LLMSingleSelector.from_defaults(llm = client_embed)

query_engine = RouterQueryEngine(

selector = engine,
query_engine_tools=[
    summary_tool,
    vector_tool,
],
verbose=True

)

8. Making queries from the document

response = query_engine.query("What is the summary of the document?")
print(str(response))

Version

0.12.22

Steps to Reproduce

I wonder if you could help me to solve the error I get.

Relevant Logs/Tracbacks

response = query_engine.query("What is the summary of the document?")
print(str(response))
Traceback (most recent call last):

  Cell In[336], line 1
    response = query_engine.query("What is the summary of the document?")

  File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
    result = func(*args, **kwargs)

  File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\base\base_query_engine.py:52 in query
    query_result = self._query(str_or_query_bundle)

  File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
    result = func(*args, **kwargs)

  File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\query_engine\router_query_engine.py:163 in _query
    result = self._selector.select(self._metadatas, query_bundle)

  File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\base\base_selector.py:88 in select
    return self._select(choices=metadatas, query=query_bundle)

  File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
    result = func(*args, **kwargs)

  File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\selectors\llm_selectors.py:106 in _select
    prediction = self._llm.predict(

  File ~\AppData\Local\anaconda3\Lib\site-packages\pydantic\main.py:828 in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')

AttributeError: 'AzureOpenAIEmbedding' object has no attribute 'predict'

[dosubot]

Hey @CarlosECastilloM! I'm here to help you with any bugs, questions, or contributions you have. Let's dive into this issue together!

The AzureOpenAIEmbedding class does not have a predict method, which is why you're encountering the AttributeError. This class is intended for handling embeddings, not predictions. For prediction tasks, you should use the AzureOpenAI, SyncAzureOpenAI, or AsyncAzureOpenAI classes instead.

Here's how you can modify your code to use the correct class for predictions:

1. Replace the client_embed initialization with the AzureOpenAI class for prediction tasks:

from llama_index.llms.azure_openai import AzureOpenAI

client = AzureOpenAI(
    engine="AZURE_AZURE_OPENAI_DEPLOYMENT_NAME",
    model="YOUR_AZURE_OPENAI_COMPLETION_MODEL_NAME",
    api_key=AZURE_OPENAI_API_KEY,
    azure_endpoint=AZURE_OPENAI_ENDPOINT,
    api_version=OPENAI_API_VERSION,
)

2. Use this client instance for tasks that require predictions, such as initializing the LLMSingleSelector.

This should resolve the AttributeError by ensuring that you are using the appropriate class for prediction tasks ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[CarlosECastilloM]

Thanks for your prompt response. Now I got the following error:

response = query_engine.query("What is the summary of the document?")
print(str(response))
Traceback (most recent call last):

Cell In[369], line 16
response = query_engine.query("What is the summary of the document?")

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
result = func(*args, **kwargs)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\base\base_query_engine.py:52 in query
query_result = self._query(str_or_query_bundle)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
result = func(*args, **kwargs)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\query_engine\router_query_engine.py:163 in _query
result = self._selector.select(self._metadatas, query_bundle)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\base\base_selector.py:88 in select
return self._select(choices=metadatas, query=query_bundle)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
result = func(*args, **kwargs)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\selectors\llm_selectors.py:106 in _select
prediction = self._llm.predict(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
result = func(*args, **kwargs)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\llms\llm.py:603 in predict
if self.metadata.is_chat_model:

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\llms\openai\base.py:360 in metadata
context_window=openai_modelname_to_contextsize(self._get_model_name()),

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\llms\openai\utils.py:246 in openai_modelname_to_contextsize
raise ValueError(

ValueError: Unknown model 'text-embedding-3-large'. Please provide a valid OpenAI model name in: o1, o1-2024-12-17, o1-preview, o1-preview-2024-09-12, o1-mini, o1-mini-2024-09-12, o3-mini, o3-mini-2025-01-31, gpt-4, gpt-4-32k, gpt-4-1106-preview, gpt-4-0125-preview, gpt-4-turbo-preview, gpt-4-vision-preview, gpt-4-1106-vision-preview, gpt-4-turbo-2024-04-09, gpt-4-turbo, gpt-4o, gpt-4o-audio-preview, gpt-4o-audio-preview-2024-12-17, gpt-4o-audio-preview-2024-10-01, gpt-4o-mini-audio-preview, gpt-4o-mini-audio-preview-2024-12-17, gpt-4o-2024-05-13, gpt-4o-2024-08-06, gpt-4o-2024-11-20, gpt-4.5-preview, gpt-4.5-preview-2025-02-27, chatgpt-4o-latest, gpt-4o-mini, gpt-4o-mini-2024-07-18, gpt-4-0613, gpt-4-32k-0613, gpt-4-0314, gpt-4-32k-0314, gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-3.5-turbo-0125, gpt-3.5-turbo-1106, gpt-3.5-turbo-0613, gpt-3.5-turbo-16k-0613, gpt-3.5-turbo-0301, text-davinci-003, text-davinci-002, gpt-3.5-turbo-instruct, text-ada-001, text-babbage-001, text-curie-001, ada, babbage, curie, davinci, gpt-35-turbo-16k, gpt-35-turbo, gpt-35-turbo-0125, gpt-35-turbo-1106, gpt-35-turbo-0613, gpt-35-turbo-16k-0613

[CarlosECastilloM]

Forgot to mention: @dosu

[dosubot]

The error you're encountering is due to using an invalid model name, 'text-embedding-3-large'. The openai_modelname_to_contextsize function is raising a ValueError because it doesn't recognize this model name.

To resolve this, you need to use a valid model name. For the AzureOpenAIEmbedding class, the valid model name is "text-embedding-ada-002" ^[1]. Make sure to update your code to use this model name or any other valid model name from the list provided in the error message.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[CarlosECastilloM]

Thanks @dosu.
When doing what you suggested I get the following error, coming from the following part of the code:

vector_index = VectorStoreIndex(
nodes = nodes,
embed_model = client_embed,
show_progress = True

)

Generating embeddings: 0%| | 0/34 [00:00<?, ?it/s]

Error:

Traceback (most recent call last):

Cell In[386], line 1
vector_index = VectorStoreIndex(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\indices\vector_store\base.py:76 in init
super().init(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\indices\base.py:77 in init
index_struct = self.build_index_from_nodes(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\indices\vector_store\base.py:310 in build_index_from_nodes
return self._build_index_from_nodes(content_nodes, **insert_kwargs)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\indices\vector_store\base.py:279 in _build_index_from_nodes
self._add_nodes_to_index(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\indices\vector_store\base.py:232 in _add_nodes_to_index
nodes_batch = self._get_node_with_embedding(nodes_batch, show_progress)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\indices\vector_store\base.py:139 in _get_node_with_embedding
id_to_embed_map = embed_nodes(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\indices\utils.py:160 in embed_nodes
new_embeddings = embed_model.get_text_embedding_batch(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py:322 in wrapper
result = func(*args, **kwargs)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\core\base\embeddings\base.py:335 in get_text_embedding_batch
embeddings = self._get_text_embeddings(cur_batch)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\embeddings\openai\base.py:465 in _get_text_embeddings
return _retryable_get_embeddings()

File ~\AppData\Local\anaconda3\Lib\site-packages\tenacity_init_.py:289 in wrapped_f
return self(f, *args, **kw)

File ~\AppData\Local\anaconda3\Lib\site-packages\tenacity_init_.py:379 in call
do = self.iter(retry_state=retry_state)

File ~\AppData\Local\anaconda3\Lib\site-packages\tenacity_init_.py:314 in iter
return fut.result()

File ~\AppData\Local\anaconda3\Lib\concurrent\futures_base.py:449 in result
return self.__get_result()

File ~\AppData\Local\anaconda3\Lib\concurrent\futures_base.py:401 in __get_result
raise self._exception

File ~\AppData\Local\anaconda3\Lib\site-packages\tenacity_init_.py:382 in call
result = fn(*args, **kwargs)

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\embeddings\openai\base.py:458 in _retryable_get_embeddings
return get_embeddings(

File ~\AppData\Local\anaconda3\Lib\site-packages\llama_index\embeddings\openai\base.py:169 in get_embeddings
data = client.embeddings.create(input=list_of_text, model=engine, **kwargs).data

File ~\AppData\Local\anaconda3\Lib\site-packages\openai\resources\embeddings.py:125 in create
return self._post(

File ~\AppData\Local\anaconda3\Lib\site-packages\openai_base_client.py:1283 in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))

File ~\AppData\Local\anaconda3\Lib\site-packages\openai_base_client.py:960 in request
return self._request(

File ~\AppData\Local\anaconda3\Lib\site-packages\openai_base_client.py:1064 in _request
raise self._make_status_error_from_response(err.response) from None

NotFoundError: Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}

Please help me fix it.

[dosubot]

The "NotFoundError: Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist'}}" error indicates that the specified deployment does not exist in your Azure OpenAI resource. Here are the steps to resolve this issue:

1. Verify Deployment Name: Ensure that the azure_deployment parameter is correctly set to the name of an existing deployment in your Azure OpenAI resource. This can be set directly in your code or through environment variables.

2. Environment Variables: If you are using environment variables, make sure they are correctly set. The relevant environment variable for the deployment is likely AZURE_OPENAI_DEPLOYMENT.

3. Check Azure Resource: Log into your Azure portal and verify that the deployment name you are using matches exactly with the one configured in your Azure OpenAI resource.

4. Code Example:

   azure_embedding = AzureOpenAIEmbedding(
       azure_endpoint="https://YOUR_RESOURCE_NAME.openai.azure.com/",
       azure_deployment="YOUR_DEPLOYMENT_NAME",
       api_key="YOUR_API_KEY",
       api_version="YOUR_API_VERSION"
   )

Ensure that all these parameters are correctly configured to match your Azure setup. If the deployment name is incorrect or does not exist, you will encounter the DeploymentNotFound error ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}