the production process of the prompt set in detail

[robot1lyj]

Thank you for your work. I would like to do task planning for an embodied scenario. However, when referring to the prompt file of Blocksworld and making the prompt file for my own environment, I'm not quite clear about the process. Could you please explain the production process of this prompt set?
Are all the prompts derived from the production of the LLM planning project?

[Ber666]

Hi,

There are a few different types of prompts used for blocksworld. Taking this as an example:

• "world_update_[ACTION]": It's manually crafted. The goal is to instruct the LLM to predict the world state change.
• "self-eval": a prompt for LLM to evaluate its last action, as a part of the reward model.
• "example_pool": It includes some examples from LLM planning project (We did some preprocessing and save all the intermediate states). Together with intro field, this will be used to compose a few-shot prompt for LLM to predict the next action. (we measure the likelihood as a reward in the RAP framework) There are two more details:
  • For each test case, we sample from the example pool, to get a more robust evaluation results (this might not be necessary in your use case).
  • We will adapt the prompt based on how many steps has been taken (Please see Appendix E: Adaptive Prompting in our paper)

I think it's very likely that you don't need to follow such a complicated pipeline to build prompts for your application. Feel free to follow up with any questions if it still confuses you.