Improve prompt for question decomposition + show more docs #3960
+635
−104
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
joachim-danswer
changed the title
evan-danswer
approved these changes
Feb 11, 2025
| question=question, | ||
| history=history, | ||
| entity_term_extraction_str=entity_term_extraction_str, | ||
| base_answer=base_answer, | ||
| answered_sub_questions="\n - ".join(addressed_question_list), | ||
| # answered_sub_questions="\n - ".join(addressed_question_list), |
There was a problem hiding this comment.
delete commented code
| question=question, | ||
| history=history, | ||
| entity_term_extraction_str=entity_term_extraction_str, | ||
| base_answer=base_answer, | ||
| answered_sub_questions="\n - ".join(addressed_question_list), | ||
| # answered_sub_questions="\n - ".join(addressed_question_list), | ||
| answered_subquestions_with_answers="\n\n---\n\n".join( |
There was a problem hiding this comment.
probably want a string constant here so it's easy to change across the board later
| """ | ||
| # Start with relevant docs or fallback to original question docs | ||
| streaming_documents = ( | ||
| relevant_docs if relevant_docs else original_question_docs[:15] |
There was a problem hiding this comment.
the 15 should be a constant defined somewhere
- getting docs directly from subquestion results vs relying on langgraph aggregation - simplify subquestion citation lists for answer generation - deduping of individual Inference lists
evan-danswer
force-pushed
the
agent-search-feature-decomp-prompt-change-v2
branch
from
a55b95f to
7fa453a
Compare
yuhongsun96
reviewed
Feb 13, 2025
| @@ -72,8 +72,14 @@ def create_refined_sub_questions( | |||
|
|
|||
| initial_question_answers = state.sub_question_results | |||
|
|
|||
| addressed_question_list = [ | |||
| x.question for x in initial_question_answers if x.verified_high_quality | |||
| # addressed_question_list = [ | |||
|
rolled into #3994 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The goal is A) to change the prompts to create better sub-questions, and B) to show also some of the context docs even if they were not cited.
For A) At the core, two main changes were made:
The decomposition prompt is explicitly instructed not to generate follow-up questions to the user. All
sub-questions need to be search-type questions
The refinement sub-questions should have the ability to see the initial sub-answers to possibly refer to the
results. Foe this, the prompts for both the initial and the refinement a sub-question generation need to be
changed. (The initial prompt needs to know that it could be valuable to generate for example a question that gives us the competitors to a company A, so that then the refined subquestion could reference the competitors by name for
searches.)
Also, the node generating the LLM request to create the refined sub-questions had to change.
For B)
How Has This Been Tested?
Locally, for some individual questions. No examples yet for sub-questions whose answers are used in refinement questions.
(More testing will follow.)
Backporting (check the box to trigger backport action)
Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.