[Feature Request]: Support perplexity.ai

[sinwoobang]

Is your feature request related to a problem? Please describe.

Yes, when interacting with perplexity.ai, I encountered a BadRequestError with the following message:

BadRequestError: Error code: 400 - {'error': {'message': 'After the (optional) system message(s), user and assistant roles should be alternating.', 'type': 'invalid_message', 'code': 400}}

This error indicates that perplexity.ai requires a strict alternating sequence of user and assistant roles in the messages.

Describe the solution you'd like

To support seamless integration with perplexity.ai, the solution should enforce the strict alternating sequence rule for user and assistant roles in the messages. This will ensure that the messages are structured in a way that is compatible with perplexity.ai's requirements, preventing the BadRequestError from occurring.

Additional context

To reproduce the issue, you can create multiple agents with a group chat agent and designate one of the agents to be operated by perplexity.ai. When attempting to interact with this agent, you will encounter the BadRequestError if the message sequence does not adhere to the strict alternating user/assistant role pattern required by perplexity.ai.

By implementing the proposed solution and enforcing the alternating role sequence, the integration with perplexity.ai should become more robust and error-free. This will enhance the overall user experience and allow for seamless communication between Autogen and perplexity.ai.

[ekzhu]

Thanks for creating this issue @sinwoobang. Could you post a sample code snippet for how to use perplexity ai to help others get started? Also, since you might be the one with the most knowledge about perplexity ai here, would you like to look into this? It could be a configuration issue. We can add some documentation to: https://microsoft.github.io/autogen/docs/topics/non-openai-models/about-using-nonopenai-models

[sinwoobang]

@ekzhu The Perplexity AI API Spec is compatible with Open AI Client. It works if you change BASE_URL of the Open AI Python SDK to https://api.perplexity.ai. Therefore, It shall work without extra configuration.

Here is my snippet https://gist.github.com/sinwoobang/33feda60c4ed36388a944e4177d3d957. Please ask Pele vs Maradona vs Messi, especially Messi with real-time data.. While the agents communicate with each other, it emits the error.

[sinwoobang]

@ekzhu https://microsoft.github.io/autogen/blog/2024/01/26/Custom-Models#step-1-create-the-custom-model-client-class I got a reference link on Discord. One folk mentioned that the custom model client may resolve this issue. What do you think about it? The sequence of request messages is in control of the model client?

[sinwoobang]

I found "messages" in params, it looks possible.

[edanweis]

@sinwoobang any updates? did you manage to re-sequence the messages for this custom model? I'm having trouble calling the OpenAICient from the custom model class.

[sinwoobang]

@edanweis I did. What part are you struggling with?

[edanweis]

@edanweis I did. What part are you struggling with?
Thanks @sinwoobang I managed with this

class PerplexityAIClient:
    def __init__(self, config: Dict[str, Any]):
        self._config = config
        self.model = config["model"]
        self.base_url = config["base_url"]
        self.api_key = config["api_key"]
        self._client = OpenAIWrapper(
            model=self.model,
            api_key=self.api_key,
            base_url=self.base_url
        )
        self._last_tooluse_status = {}

    def create(self, params: Dict[str, Any]) -> Completion:
        messages = params["messages"]

        # Ensure the sequence of user and assistant messages is alternating
        for i, message in enumerate(messages):
            if message['role'] != 'system':
                # If it's the last message or an odd-indexed message, set role to 'user'
                message['role'] = "user" if i == len(messages) - 1 or i % 2 != 0 else "assistant"

        # Remove all messages except the first and last
        params["messages"] = [messages[0], messages[-1]]

        # Copy params and modify for client request
        params = params.copy()
        params["stream"] = False
        params.pop("model_client_cls")
        params["max_tokens"] = params.get("max_tokens", 4096)

        # Make request to client
        response = self._client.create(**params)

        # Remove message retrieval function if it exists
        if hasattr(response, 'message_retrieval_function'):
            delattr(response, 'message_retrieval_function')

        return response

[sinwoobang]

@edanweis I did. What part are you struggling with?
Thanks @sinwoobang I managed with this

class PerplexityAIClient:
    def __init__(self, config: Dict[str, Any]):
        self._config = config
        self.model = config["model"]
        self.base_url = config["base_url"]
        self.api_key = config["api_key"]
        self._client = OpenAIWrapper(
            model=self.model,
            api_key=self.api_key,
            base_url=self.base_url
        )
        self._last_tooluse_status = {}

    def create(self, params: Dict[str, Any]) -> Completion:
        messages = params["messages"]

        # Ensure the sequence of user and assistant messages is alternating
        for i, message in enumerate(messages):
            if message['role'] != 'system':
                # If it's the last message or an odd-indexed message, set role to 'user'
                message['role'] = "user" if i == len(messages) - 1 or i % 2 != 0 else "assistant"

        # Remove all messages except the first and last
        params["messages"] = [messages[0], messages[-1]]

        # Copy params and modify for client request
        params = params.copy()
        params["stream"] = False
        params.pop("model_client_cls")
        params["max_tokens"] = params.get("max_tokens", 4096)

        # Make request to client
        response = self._client.create(**params)

        # Remove message retrieval function if it exists
        if hasattr(response, 'message_retrieval_function'):
            delattr(response, 'message_retrieval_function')

        return response

May I review the code? I would remove the part below since you have params["messages"] = [messages[0], messages[-1]]. You can revise messages[-1] then. What do you think? @edanweis

    for i, message in enumerate(messages):
        if message['role'] != 'system':
            # If it's the last message or an odd-indexed message, set role to 'user'
            message['role'] = "user" if i == len(messages) - 1 or i % 2 != 0 else "assistant"

[edanweis]

@sinwoobang Yes of course, nice catch! Thanks. I realised that perplexity online models are effectively search engines for which a message history doesn't make much sense. Perhaps this should be a function call.

[sinwoobang]

@edanweis Exactly. I am still deciding too. Perplexity is somewhere in the middle between a function call and a LLM service.

[autogptfreak]

Hey folks, what would you recommend to me as someone encountering this same error with Perplexity.ai as my LLM?
I tried your custom client class but am running into registration issues...
Is there an easy way to get Perplexity to work with Autogen?

[autogptfreak]

It seems registering a custom ModelClient is a challenge in itself, so I've created an issue for that: #3502