Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task-Centric Memory #5227

Open
wants to merge 132 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
132 commits
Select commit Hold shift + click to select a range
442a9d8
initial checkin
rickyloynd-microsoft Nov 29, 2024
f8584cd
support for extensive evaluations
rickyloynd-microsoft Dec 2, 2024
607e7ff
Enhance retrieval with task generalization and insight validation
rickyloynd-microsoft Dec 4, 2024
b045636
Support TRAPI client.
rickyloynd-microsoft Dec 9, 2024
63b28d7
Restoring earlier results, and general cleanup.
rickyloynd-microsoft Dec 24, 2024
b921d83
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Dec 24, 2024
9dfb074
Modify imports after merge from main.
rickyloynd-microsoft Dec 24, 2024
93a5ca4
Log model and token counts.
rickyloynd-microsoft Dec 26, 2024
2cb9344
Only instantiate the client once.
rickyloynd-microsoft Dec 26, 2024
878f458
Fix bug that was duplicating insights across trials.
rickyloynd-microsoft Dec 26, 2024
21562f1
Add the Grader class.
rickyloynd-microsoft Dec 27, 2024
3a40b30
Adjustments for comparison tests.
rickyloynd-microsoft Dec 28, 2024
8622c5e
Test generalization over multiple tasks.
rickyloynd-microsoft Dec 30, 2024
20b26c1
Add teachability and a test for it.
rickyloynd-microsoft Dec 31, 2024
9d47227
Learning from demonstration, in-progress.
rickyloynd-microsoft Jan 1, 2025
52d4e00
In memory retrieval, validate insights separately rather than together.
rickyloynd-microsoft Jan 1, 2025
6b15777
Finish learning from demonstration.
rickyloynd-microsoft Jan 2, 2025
a18674c
Added RecordableChatCompletionClient as a guardrail during refactoring.
rickyloynd-microsoft Jan 3, 2025
52e213e
Ran 3 evals with session recording and replay.
rickyloynd-microsoft Jan 5, 2025
a440b0a
Add results to recorded sessions, including session length.
rickyloynd-microsoft Jan 5, 2025
cab51f1
Use yaml file for eval settings.
rickyloynd-microsoft Jan 7, 2025
d91e58c
Simplify paths and other settings.
rickyloynd-microsoft Jan 7, 2025
f1d7a2f
Renamed the memory classes.
rickyloynd-microsoft Jan 7, 2025
17d4c42
Apprentice.
rickyloynd-microsoft Jan 8, 2025
19654e8
Moved test into the evaluator, and removed eval.py's other util funct…
rickyloynd-microsoft Jan 8, 2025
7aa20c1
renaming
rickyloynd-microsoft Jan 8, 2025
83a7ddc
Rerouted calls to AgenticMemoryController through FastLearner.
rickyloynd-microsoft Jan 9, 2025
3047c1c
Replace task_assignment_callback with AgentWrapper.
rickyloynd-microsoft Jan 9, 2025
1f20b79
Segregate files into subfolders, eval framework vs. implementation, etc.
rickyloynd-microsoft Jan 10, 2025
de4c12b
Rename FastLearner subclass to Apprentice, and import it only as spec…
rickyloynd-microsoft Jan 10, 2025
a9d6108
Refactoring, preparatory to removing eval_framework from the branch a…
rickyloynd-microsoft Jan 11, 2025
d67e2cc
Remove the outdated final_format_instructions parameter.
rickyloynd-microsoft Jan 11, 2025
6470fd8
Move tasks into yaml files.
rickyloynd-microsoft Jan 12, 2025
b025199
Move client support to a subdir.
rickyloynd-microsoft Jan 12, 2025
4f9267c
Move evaluations to a separate dir.
rickyloynd-microsoft Jan 12, 2025
db34844
single line
rickyloynd-microsoft Jan 14, 2025
c780852
Add baseline evaluation for the no-memory case.
rickyloynd-microsoft Jan 16, 2025
fa688f7
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Jan 17, 2025
43bda2f
Support o1 models
rickyloynd-microsoft Jan 18, 2025
be081b3
simplification of client creation code
rickyloynd-microsoft Jan 18, 2025
29d1494
simplify folder structure
rickyloynd-microsoft Jan 18, 2025
8e9a550
Move task data strings out of the eval functions.
rickyloynd-microsoft Jan 20, 2025
b3fe084
simplify page_log
rickyloynd-microsoft Jan 21, 2025
077615f
simplify page_log
rickyloynd-microsoft Jan 21, 2025
8847168
simplify page_log
rickyloynd-microsoft Jan 21, 2025
4091ab3
conventional logging terminology
rickyloynd-microsoft Jan 22, 2025
3865cff
control logger enabling
rickyloynd-microsoft Jan 22, 2025
6c73674
add logging to string map
rickyloynd-microsoft Jan 22, 2025
db5e07b
simplify logging
rickyloynd-microsoft Jan 22, 2025
07cb3f0
simplify logging
rickyloynd-microsoft Jan 22, 2025
e88bd69
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Jan 22, 2025
9b3f77d
merge from main
rickyloynd-microsoft Jan 23, 2025
a0dee67
Changes made by poe check.
rickyloynd-microsoft Jan 23, 2025
7e359e9
docstrings etc.
rickyloynd-microsoft Jan 23, 2025
9466ea8
docstrings etc.
rickyloynd-microsoft Jan 24, 2025
4ec9bff
docstrings etc.
rickyloynd-microsoft Jan 24, 2025
76c16f9
docstrings etc.
rickyloynd-microsoft Jan 24, 2025
a8cd0d7
docstrings etc.
rickyloynd-microsoft Jan 24, 2025
ed7fae1
docstrings etc.
rickyloynd-microsoft Jan 25, 2025
93de858
docstrings etc.
rickyloynd-microsoft Jan 25, 2025
1a309f9
docstrings etc.
rickyloynd-microsoft Jan 25, 2025
8993aa1
docstrings etc.
rickyloynd-microsoft Jan 25, 2025
fa60d5a
Simplify naming
rickyloynd-microsoft Jan 25, 2025
882d578
Simplify tests
rickyloynd-microsoft Jan 26, 2025
00cbb8c
standardize logging levels
rickyloynd-microsoft Jan 27, 2025
88294d2
Remove Evaluator class
rickyloynd-microsoft Jan 27, 2025
7d0ed63
sample code
rickyloynd-microsoft Jan 27, 2025
5b3876f
readme
rickyloynd-microsoft Jan 28, 2025
21220d4
readme fixes
rickyloynd-microsoft Jan 28, 2025
232ed0f
samples readme
rickyloynd-microsoft Jan 28, 2025
87ee27b
readme files
rickyloynd-microsoft Jan 28, 2025
b21d140
readme files
rickyloynd-microsoft Jan 28, 2025
1e88eb6
remove ame
rickyloynd-microsoft Jan 28, 2025
a3addc1
readme
rickyloynd-microsoft Jan 28, 2025
c6ffa43
comment out api_key lines
rickyloynd-microsoft Jan 28, 2025
8f66612
Optional disabling of prefix caching (to decorrelate repeated runs)
rickyloynd-microsoft Jan 28, 2025
491964f
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Jan 28, 2025
2ed08ae
Remove unnecessary instantiation of Grader
rickyloynd-microsoft Jan 29, 2025
f879487
Updated image using git-lfs
rickyloynd-microsoft Jan 30, 2025
60f8ad3
Merge branch 'agentic_memory' of github.com:microsoft/autogen into ag…
rickyloynd-microsoft Jan 30, 2025
ed0a4a6
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Jan 30, 2025
f0eceef
installation fixes
rickyloynd-microsoft Jan 30, 2025
70db202
Refactor to remove AgentWrapper, and use AssistantAgent as a TaskRunn…
rickyloynd-microsoft Jan 31, 2025
5e4ad48
uv fixes
rickyloynd-microsoft Jan 31, 2025
b6c59ae
uv fixes
rickyloynd-microsoft Jan 31, 2025
bef7e5d
uv fixes
rickyloynd-microsoft Feb 1, 2025
1fb5ee4
uv fixes
rickyloynd-microsoft Feb 1, 2025
1d7f4eb
uv fixes
rickyloynd-microsoft Feb 1, 2025
516e689
uv fixes
rickyloynd-microsoft Feb 1, 2025
ffe719a
uv fixes
rickyloynd-microsoft Feb 1, 2025
ba14e78
uv fixes
rickyloynd-microsoft Feb 3, 2025
2eb817e
uv fixes
rickyloynd-microsoft Feb 3, 2025
95b1276
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 3, 2025
6633169
uv fixes
rickyloynd-microsoft Feb 4, 2025
880df13
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 4, 2025
b4ea0ce
uv fixes
rickyloynd-microsoft Feb 5, 2025
53da266
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 5, 2025
6a04851
Add line to autogenstudio section of uv.lock
rickyloynd-microsoft Feb 5, 2025
ad514eb
poe check fixes
rickyloynd-microsoft Feb 5, 2025
18ae4dc
uv
rickyloynd-microsoft Feb 5, 2025
7be995d
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 5, 2025
0e01720
hash output for detecting log changes
rickyloynd-microsoft Feb 6, 2025
0416e8d
Merge branch 'main' of github.com:microsoft/autogen
rickyloynd-microsoft Feb 6, 2025
0298591
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 6, 2025
0bc0500
Make logger and config args optional
rickyloynd-microsoft Feb 7, 2025
66029cf
terminology: settings -> configs
rickyloynd-microsoft Feb 7, 2025
d9ad986
Merge branch 'main' of github.com:microsoft/autogen
rickyloynd-microsoft Feb 7, 2025
2b2cbdb
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 7, 2025
566709b
Simplify the API in preparation for Webby
rickyloynd-microsoft Feb 11, 2025
2ef5e4a
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 11, 2025
9e7d245
uv.lock
rickyloynd-microsoft Feb 11, 2025
1ce4cd9
Restore version for autogenstudio
rickyloynd-microsoft Feb 11, 2025
494d81e
Add a retrieval sample, which accesses the memory controller directly…
rickyloynd-microsoft Feb 12, 2025
54b0faa
Change of terminology: Agentic Memory -> Task-Centric Memory
rickyloynd-microsoft Feb 12, 2025
0b9f042
Move support files into utils subdir.
rickyloynd-microsoft Feb 12, 2025
a720863
Update readme files per reviewer feedback.
rickyloynd-microsoft Feb 14, 2025
b0e72a7
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 14, 2025
dba5b55
Get API Reference documentation to build correctly.
rickyloynd-microsoft Feb 15, 2025
01d8b9d
Add code example provided by @ekzhu
rickyloynd-microsoft Feb 15, 2025
193466b
Added installation instructions and a code snippet to the docstring.
rickyloynd-microsoft Feb 15, 2025
39d460a
Code format fix in the docstring
rickyloynd-microsoft Feb 15, 2025
8f9d066
Use TypedDicts in the nested-config pattern to minimize code changes …
rickyloynd-microsoft Feb 17, 2025
94eab06
Merge branch 'refs/heads/main' into agentic_memory
rickyloynd-microsoft Feb 17, 2025
f892c18
Add clarifying diagrams.
rickyloynd-microsoft Feb 18, 2025
a15bfd1
fix image sizes
rickyloynd-microsoft Feb 18, 2025
3ea8011
Merge branch 'main' into agentic_memory
rickyloynd-microsoft Feb 19, 2025
58ecd7e
Add file docstrings to sample code.
rickyloynd-microsoft Feb 20, 2025
00e27e1
Merge branch 'agentic_memory' of github.com:microsoft/autogen into ag…
rickyloynd-microsoft Feb 20, 2025
4466eee
uv sync --all-extras
rickyloynd-microsoft Feb 20, 2025
64dc3c0
restore previous uv.lock
rickyloynd-microsoft Feb 20, 2025
e15d0eb
changes for webby
rickyloynd-microsoft Feb 21, 2025
af362f6
experimental
rickyloynd-microsoft Feb 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix bug that was duplicating insights across trials.
rickyloynd-microsoft committed Dec 26, 2024
commit 878f458d31cefa3586685a6c31c8c428d74fb305
Original file line number Diff line number Diff line change
@@ -52,18 +52,19 @@ async def test_on_task(self, task: str, expected_answer: str, task_assignment_ca

for trial in range(num_trials):
page.add_lines("----- TRIAL {} -----\n".format(trial + 1), flush=True)
task_plus_insights = task

# Try to retrieve any relevant memories from the DB.
filtered_insights = await self.retrieve_relevant_insights(task)
if len(filtered_insights) > 0:
page.add_lines("Relevant insights were retrieved from memory.\n", flush=True)
memory_section = self.format_memory_section(filtered_insights)
if len(memory_section) > 0:
task = task + '\n\n' + memory_section
task_plus_insights = task + '\n\n' + memory_section

# Attempt to solve the task.
page.add_lines("Try to solve the task.\n", flush=True)
response, _ = await task_assignment_callback(task, self.client, self.page_log)
response, _ = await task_assignment_callback(task_plus_insights, self.client, self.page_log)

response_is_correct = (response.lower() == expected_answer.lower())
if response_is_correct: