Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT: Add GroqChatTarget (#704) #705

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jsdlm
Copy link

@jsdlm jsdlm commented Feb 11, 2025

Description

Adds GroqChatTarget, a new chat target for interacting with Groq’s OpenAI-compatible API.

  • The existing OpenAIChatTarget does not work with Groq because Groq expects messages.0.content to be a string, while OpenAI/Azure accept a list of dictionaries.
  • This PR also adds unit tests for GroqChatTarget and documentation.

Changes

  • Added GroqChatTarget (pyrit/prompt_target/groq_chat_target.py)
    • Inherits from OpenAIChatTarget
    • Overrides _complete_chat_async to ensure content is formatted correctly for Groq
    • Overrides _initialize_non_azure_vars to set up Groq-specific API parameters
    • Uses GROQ_API_KEY and GROQ_MODEL_NAME environment variables
  • Added unit tests (tests/unit/target/test_groq_chat_target.py) adapted from OpenAIChatTarget tests
  • Added documentation (doc/code/targets/groq_chat_target.py)

Related Issue

#704

@jsdlm jsdlm force-pushed the fix/openai-chat-no-content-704 branch from e5070b8 to d1355cf Compare February 12, 2025 13:55
@jsdlm jsdlm changed the title fix: OpenAIChatTarget 'no content' error in RedTeamingOrchestrator (#704) feat: Add GroqChatTarget (#704) Feb 12, 2025
@jsdlm
Copy link
Author

jsdlm commented Feb 12, 2025

@jsdlm please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

@romanlutz romanlutz changed the title feat: Add GroqChatTarget (#704) FEAT: Add GroqChatTarget (#704) Feb 13, 2025
Copy link
Contributor

@romanlutz romanlutz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, this solution is acceptable for me. We'd need unit tests, though. Can you essentially replicate the openai ones with modifications?

@jsdlm
Copy link
Author

jsdlm commented Feb 13, 2025

Thanks! I've now added unit tests as well.

In the class, I refined the implementation to make it fully compatible with Groq’s API. This includes enforcing the correct API base URL (https://api.groq.com/openai/v1/), updating the environment variables to GROQ_API_KEY and GROQ_MODEL_NAME, and adapting _initialize_non_azure_vars to align with Groq’s authentication and request structure.

For the unit tests, I followed the OpenAI ones as a base, adapting them to match the new Groq implementation.

@jsdlm
Copy link
Author

jsdlm commented Feb 13, 2025

pre-commit_pytest

@jsdlm jsdlm force-pushed the fix/openai-chat-no-content-704 branch from a04d162 to 1c9b997 Compare February 13, 2025 13:53
@@ -34,6 +35,7 @@
"CrucibleTarget",
"GandalfLevel",
"GandalfTarget",
"GroqChatTarget",
Copy link
Contributor

@rlundeen2 rlundeen2 Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we rename to OpenAITextChatTarget? Because it will work on:

  • GPT-4, GPPT-4o, etc (without multi-modal)
  • DeepSeek, LLAMA, other "openAI compatible" models that support this format but not OpenAI's dictionary specs.

And if we go that route, we could potentially move under the openAI folder.

I'm still looking into whether I agree with my own recommendation though...

Copy link
Contributor

@rlundeen2 rlundeen2 Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^^^ @romanlutz I recommend making my comment non-blocking. Not sure I have enough time to look into it today. But it may be nice to have an open-ai text target that will work other places wherever they pop up.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main goal was to ensure compatibility with Groq's API while keeping the implementation simple and consistent with OpenAI's existing structure. Renaming it to something more general like OpenAITextChatTarget could make sense if we anticipate supporting more OpenAI-compatible platforms with similar constraints.

Do you think it’s worth making that change now, or should we wait and see if more platforms adopt these constraints?

@jsdlm jsdlm requested a review from romanlutz February 14, 2025 23:40
@jsdlm jsdlm force-pushed the fix/openai-chat-no-content-704 branch from fcba28e to d666166 Compare February 14, 2025 23:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants