Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Agentic memory #5227

Open
wants to merge 82 commits into
base: main
Choose a base branch
from
Open

Agentic memory #5227

wants to merge 82 commits into from

Conversation

rickyloynd-microsoft
Copy link
Contributor

@rickyloynd-microsoft rickyloynd-microsoft commented Jan 28, 2025

Adds a baseline implementation of agentic memory to the autogen-ext package.

For technical details, see the Agentic Memory README.

To see it in action, view the page logs generated while running the code samples.

This PR is a draft pending a few remaining items in-progress:

  • Unit tests
  • Full documentation
  • Lint & format fixes
  • Potential usage of autogen-core/memory

Make memory optional.
Filter out insights with negative scores.
Refactor memory paths.
Enrich page logging.
Seed messages with random int for variability.
Save sessions as yaml for readability.
Eval simplifications.
from .page_logger import PageLogger


class AgentWrapper:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implement the autogen_agentchat.base.TaskRunner protocol. It has two methods, run and run_stream.

Instead of subclassing this, we can pass in an autogen_agentchat.base.TaskRunner to the constructor and use that as the inner runner.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might have to rethink this for component config, but for now it's important to make sure the concepts are aligned rather than diverging.

raise AssertionError("Invalid base agent")

self.logger.leave_function()
return response, work_history
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we implement the run method, the response should be the last message, and the work history should be the full message history.

from .page_logger import PageLogger


class AgenticMemoryController:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should try to subclass autogen_agentchat.agents.BaseChatAgent to make sure wee can play well with the rest of the framework.

from .agentic_memory_controller import AgenticMemoryController


class Apprentice:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between Apprentice and AgentController. It seems to me they are basically the same except the former has memory hidden as part of its implementation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We really need some standard logging here. The presentation layer of the logs cannot be the log itself -- they must be decoupled. Otherwise the logs cannot be consumed and used by observability tools.

- add_task_with_solution: Adds a task-insight pair to the memory bank, to be retrieved together later.
- get_relevant_insights: Returns any insights from the memory bank that appear sufficiently relevant to the given
"""
def __init__(self, settings: Dict, reset: bool, logger: PageLogger) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than using dictionary settings, we should either flatten the settings in the constructor, or use a config class that is a Pydantic basemodel for validation and serializable configs. See existing example in autogen_agentchat.agents.AssistantAgent. If this class (and others in this PR) implements the ComponentConfig, you can easily load the configurations from a file to create an object of the class.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look at the serializable configuration example in the latest release note: https://github.com/microsoft/autogen/releases/tag/v0.4.4. We already support this type of configuration you are doing here, so I suggest we keep this aligned with the rest of the framework.

@ekzhu
Copy link
Collaborator

ekzhu commented Jan 29, 2025

I haven't gone deep into the implementation logics, but I believe from just API level, we need to work on it to make sure it aligns with the rest of the framework. E.g., serializable configuration is already supported, and we should be using that. TaskRunner protocol is already there, and we should be using that. BaseChatAgent already provides an agent abstraction, and we should be using that as well.

Copy link
Member

@jackgerrits jackgerrits left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is simply too large. Please propose your changes progressively and iteratively using a separate sequence of PRs. We cannot effectively review these changes as is.

@ekzhu
Copy link
Collaborator

ekzhu commented Jan 30, 2025

@jackgerrits we will be refactoring this in the current branch first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants