Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Separate doc filtration and reasoning LLMs #1

Open
rawwerks opened this issue Feb 3, 2025 · 2 comments · May be fixed by #2
Open

Separate doc filtration and reasoning LLMs #1

rawwerks opened this issue Feb 3, 2025 · 2 comments · May be fixed by #2

Comments

@rawwerks
Copy link

rawwerks commented Feb 3, 2025

I think it would be pretty powerful to separate this into two separate LLMs. Something like minimax or Gemini 2.0 can do the document pipeline (huge context window and very cheap), then only those documents (or perhaps summarizations of those documents) go to a smart/reasoning LLM for the final answer
@homanp
Copy link
Collaborator

homanp commented Feb 3, 2025

you can pass in which model you want in theory. And than route this to some reasoning model for chat or agentic purposes.

@homanp
Copy link
Collaborator

homanp commented Feb 3, 2025

Feel free to open a PR if I misunderstood you!

jasonkneen referenced this issue in jasonkneen/reag Feb 3, 2025
@jasonkneen
Separate doc filtration and reasoning LLMs
0d6b159 
Fixes #1

Separate the document filtration and reasoning LLMs into two distinct models.

* **Python Changes:**
  - Add a new parameter `filtration_model` to the `ReagClient` constructor in `python/src/reag/client.py`.
  - Update the `query` method to use the `filtration_model` for document filtration and the `model` for generating the final answer.
  - Update `python/README.md` to reflect the changes in the `ReagClient` class and the new two-step process.

* **TypeScript Changes:**
  - Add a new parameter `filtrationModel` to the `ReagClient` constructor in `typescript/src/client.ts`.
  - Update the `query` method to use the `filtrationModel` for document filtration and the `model` for generating the final answer.
  - Update `typescript/README.md` to reflect the changes in the `ReagClient` class and the new two-step process.

---

For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/superagent-ai/reag/issues/1?shareId=XXXX-XXXX-XXXX-XXXX).
@jasonkneen jasonkneen linked a pull request Feb 3, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Separate doc filtration and reasoning LLMs jasonkneen/reag
2 participants
@homanp @rawwerks