Skip to content

Commit

Permalink
Add notes on api key and modifying specifications in AGS (#5466)
Browse files Browse the repository at this point in the history
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

Add clear notes on how to specify api key and modifying specifications
in AGS.
Add diagrams explaining how to switch between visual builder and JSON
mode

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->


![teambuilder](https://github.com/user-attachments/assets/9eede334-7f60-4c87-bec6-cf41839ba231)


## Related issue number

<!-- For example: "Closes #1234" -->

@nour-bouzid 

## Checks

- [ ] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
  • Loading branch information
victordibia authored Feb 9, 2025
1 parent 9a028ac commit 340a8e8
Show file tree
Hide file tree
Showing 7 changed files with 112 additions and 49 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,52 @@ myst:

# Usage

AutoGen Studio provides a Team Builder interface where developers can define multiple components and behaviors. Users can create teams, add agents to teams, attach tools and models to agents, and define team termination conditions.
After defining a team, users can test it in the Playground view to accomplish various tasks through direct interaction.
AutoGen Studio (AGS) provides a Team Builder interface where developers can define multiple components and behaviors. Users can create teams, add agents to teams, attach tools and models to agents, and define team termination conditions.
After defining a team, users can test directly in the team builder view or attach it to a session for use in the Playground view.

> See a video tutorial on AutoGen Studio v0.4 (02/25) - [https://youtu.be/oum6EI7wohM](https://youtu.be/oum6EI7wohM)
[![A Friendly Introduction to AutoGen Studio v0.4](https://img.youtube.com/vi/oum6EI7wohM/maxresdefault.jpg)](https://www.youtube.com/watch?v=oum6EI7wohM)

## Setting Up an API Key

Most of your agents will require an API key. You can set up an environment variable `OPENAI_API_KEY` (assuming you are using OpenAI models) and AutoGen will automatically use this for any OpenAI model clients you specify for your agents or teams. Alternatively you can specify the api key as part of the team or agent configuration.

See the section below on how to build an agent team either using the visual builder or by directly editing the JSON configuration.

## Building an Agent Team

<br/>

AutoGen Studio integrates closely with all component abstractions provided by AutoGen AgentChat, including {py:class}`~autogen_agentchat.teams`, {py:class}`~autogen_agentchat.agents`, {py:class}`~autogen_core.models`, {py:class}`~autogen_core.tools`, and termination {py:class}`~autogen_agentchat.conditions`.

The Team Builder view in AGS provides a visual team builder that allows users to define components through either drag-and-drop functionality or by editing a JSON configuration of the team directly.

### Using the Visual Builder

The visual builder is enabled by default and allows users to drag-and-drop components from the provided Component library to the Team Builder canvas. The team builder canvas represents a team and consists of a main team node and a set of a connected agent nodes. It includes a Component Library that has a selection of components that can be added to the team or agent nodes in the canvas.

![Team Builder](teambuilder.jpg)

The core supported behaviours include:

- Create a new team. This can be done by clicking on the "New Team" button in the Team Builder view or by selecting any of the existing default teams that ship with the default AGS Gallery. Once you do this, a new team node and agent node(s) will be created in the canvas.
- Drag and drop components from the library to the team or agent nodes in the canvas.
- Teams: drag in agents and termination conditions to the team node (there are specific drop zones for these components)
- Agents: drag in models and tools to the agent node (there are specific drop zones for these components)
- Editing Team/Agent Nodes: Click on the edit icon (top right) of the node to view and edit its properties. This pops up a panel that allows you to edit the fields of the node. In some cases you will need to scroll down and click into specific sections e.g., for an agent with a model client, you will need to click into the model client section to edit the model client properties. Once done with editing, click on the save button to save the changes.

### Using the JSON Editor

![JSON Editor](jsoneditor.jpg)

AGS also lets you directly modify the JSON configuration of the team. This can be done by toggling the visual builder mode off. Once you do this, you will see the JSON configuration of the team. You can then edit the JSON configuration directly.

> Did you know that you define your agents in Python, export them to JSON and then paste them in the JSON editor? The section below shows how to accomplish this.
## Declarative Specification of Componenents

AutoGen Studio is built on the declarative specification behaviors of AutoGen AgentChat. This allows users to define teams, agents, models, tools, and termination conditions in python and then dump them into a JSON file for use in AutoGen Studio.
AutoGen Studio is built on the declarative specification behaviors of AutoGen AgentChat. This allows users to define teams, agents, models, tools, and termination conditions in Python and then dump them into a JSON file for use in AutoGen Studio.

Here's an example of an agent team and how it is converted to a JSON file:

Expand Down Expand Up @@ -98,30 +134,15 @@ print(config.model_dump_json())
}
```

This example shows a team with a single agent, using the `RoundRobinGroupChat` type and a `TextMentionTermination` condition.

## Building an Agent Team

<br/>

AutoGen Studio integrates closely with all component abstractions provided by AutoGen AgentChat, including {py:class}`~autogen_agentchat.teams`, {py:class}`~autogen_agentchat.agents`, {py:class}`~autogen_core.models`, {py:class}`~autogen_core.tools`, and termination {py:class}`~autogen_agentchat.conditions`.

The Team Builder interface allows users to define components through either declarative specification or drag-and-drop functionality:

Team Builder Operations:
This example shows a team with a single agent, using the `RoundRobinGroupChat` type and a `TextMentionTermination` condition. You will also notice that the model client is an `OpenAIChatCompletionClient` model client where only the model name is specified. In this case, the API key is assumed to be set as an environment variable `OPENAI_API_KEY`. You can also specify the API key as part of the model client configuration.

- Create a new team
- Edit Team JSON directly (toggle visual builder mode off) or
- Use the visual builder, drag-and-drop components from the library:
- Teams: Add agents and termination conditions
- Agents: Add models and tools
- Save team configurations
To understand the full configuration of an model clients, you can refer to the [AutoGen Model Clients documentation](https://microsoft.github.io/autogen/dev/user-guide/core-user-guide/components/model-clients.html).

Note: For each node in the visual builder, you can click on the edit icon (top right) to view and edit the JSON configuration.
Note that you can similarly define your model client in Python and call `dump_component()` on it to get the JSON configuration and use it to update the model client section of your team or agent configuration.

## Gallery - Sharing and Reusing Components

A Gallery is a collection of components - teams, agents, models, tools, and terminations - that can be shared and reused across projects.
AGS provides a Gallery view, where a gallery is a collection of components - teams, agents, models, tools, and terminations - that can be shared and reused across projects.

Users can create a local gallery or import a gallery (from a URL, a JSON file import or simply by copying and pasting the JSON). At any given time, users can select any of the current Gallery items as a **default gallery**. This **default gallery** will be used to populate the Team Builder sidebar with components.

Expand Down Expand Up @@ -155,7 +176,7 @@ Team configurations can be integrated into Python applications using the `TeamMa
from autogenstudio.teammanager import TeamManager

tm = TeamManager()
result_stream = tm.run(task="What is the weather in New York?", team_config="team.json") # or wm.run_stream(..)
result_stream = tm.run(task="What is the weather in New York?", team_config="team.json") # or tm.run_stream(..)
```

To export team configurations, use the export button in Team Builder to generate a JSON file for Python application use.
22 changes: 18 additions & 4 deletions python/packages/autogen-studio/autogenstudio/gallery/builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from autogen_core.models import ModelInfo
from autogen_ext.agents.web_surfer import MultimodalWebSurfer
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.models.openai._openai_client import AzureOpenAIChatCompletionClient

from autogenstudio.datamodel import Gallery, GalleryComponents, GalleryItems, GalleryMetadata

Expand Down Expand Up @@ -142,8 +143,7 @@ def create_default_gallery() -> Gallery:

# Create base model client
base_model = OpenAIChatCompletionClient(model="gpt-4o-mini")
builder.add_model(base_model.dump_component())

builder.add_model(base_model.dump_component(), label="OpenAI GPT-4o Mini", description="OpenAI GPT-4o-mini")
# Create Mistral vllm model
mistral_vllm_model = OpenAIChatCompletionClient(
model="TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
Expand All @@ -152,8 +152,22 @@ def create_default_gallery() -> Gallery:
)
builder.add_model(
mistral_vllm_model.dump_component(),
label="Mistral-7B vllm",
description="Example on how to use the OpenAIChatCopletionClient with local models (Ollama, vllm etc).",
label="Mistral-7B Local",
description="Local Mistral-7B model client for instruction-based generation (Ollama, LMStudio).",
)

# create an azure mode
az_model_client = AzureOpenAIChatCompletionClient(
azure_deployment="{your-azure-deployment}",
model="gpt-4o-mini",
api_version="2024-06-01",
azure_endpoint="https://{your-custom-endpoint}.openai.azure.com/",
api_key="sk-...", # For key-based authentication.
)
builder.add_model(
az_model_client.dump_component(),
label="AzureOpenAI GPT-4o-mini",
description="GPT-4o Mini Azure OpenAI model client.",
)

builder.add_tool(
Expand Down
Loading

0 comments on commit 340a8e8

Please sign in to comment.