diff --git a/js/state.json b/js/state.json index 5531922..3a1348d 100644 --- a/js/state.json +++ b/js/state.json @@ -89,7 +89,7 @@ "readme_content": "# Library: Basic Text Tools\n\n## Description\n\nThis library provides basic tools for text manipulation and analysis.\n\n---\n\n## Custom Function Definitions\n\n### `format_text`\n\n#### Description\n\nFormats text according to specified options.\n\n#### Parameters\n\n- `input`: (string, required) The text to format.\n- `style`: (string, optional, default=\"plain\") The desired style (`plain`, `bold`, `italic`, `markdown`).\n- `uppercase`: (boolean, optional, default=false) Whether to uppercase the text.\n\n#### Example\n\n```example\n# Define:\n - function: format_text\n - input: \"my text\"\n - style: bold\n - uppercase: true\n```\n\n### `summarize_text`\n\n#### Description\n\nSummarizes the given text.\n\n#### Parameters\n\n- `input`: (string, required) The text to summarize.\n- `length`: (integer, optional, default=100) The desired length of the summary (in words).\n\n#### Example\n\n```example\n# Define:\n- function: summarize_text\n- input: \"A very long text that needs to be summarized\"\n- length: 50\n```\n\n### `extract_keywords`\n\n#### Description\n\nExtracts keywords from the given text.\n\n#### Parameters\n\n- `input`: (string, required) The input text to analyze.\n- `max_keywords`: (integer, optional, default=5) The maximum number of keywords to extract.\n- `stop_words`: (string, optional) A comma separated list of stop words to ignore.\n\n#### Example\n\n```example\n# Define:\n- function: extract_keywords\n- input: \"This is a sample text for keyword extraction.\"\n- max_keywords: 3\n- stop_words: \"is, a, the, for\"\n```\n\n---\n\n\n# READMEs Programming System\r\n\r\nA self-contained programming system using README.md as executable documentation, powered by LLM-driven navigation and execution.\r\n\r\n## System Configuration\r\n```yaml\r\nversion: 1.2\r\nmetadata:\r\n standalone: true\r\n web_compatible: true\r\n llm_driven: true\r\n \r\ncontext:\r\n enabled: true\r\n warmhole_links: true\r\n state_tracking: true\r\n llm_control: true\r\n \r\nvariables:\r\n readme_content: \"\"\r\n previous_output: \"\"\r\n system_state: \"\"\r\n llm_context: {}\r\n warmhole_history: []\r\n```\r\n\r\n## LLM Optimization Features\r\n\r\n### Context Management\r\nThe LLM maintains and updates system context including:\r\n- Current warmhole state\r\n- Execution history\r\n- System goals and constraints\r\n- Previous outputs and results\r\n\r\n### Smart Navigation\r\nThe LLM automatically:\r\n- Decides which warmholes to navigate\r\n- Determines optimal execution paths\r\n- Creates or modifies warmholes as needed\r\n- Manages state transfers between warmholes\r\n\r\n### Autonomous Execution\r\nThe LLM:\r\n- Interprets user intent\r\n- Selects appropriate functions to execute\r\n- Provides input parameters\r\n- Validates outputs and handles errors\r\n- Documents changes and decisions\r\n\r\n## Function Examples\r\n\r\n### Natural Language Processing\r\n```markdown\r\n# Function: process_intent\r\n- description: \"Process user intent and determine execution path\"\r\n- input: message: string\r\n- output: plan: object\r\n- template: llm_processor\r\n- llm_context: true\r\n```\r\n\r\n### LLM Processor Template\r\n```markdown\r\n# Template: llm_processor\r\n- input_placeholder: \"{{message}}\"\r\n- transform: |\r\n return llm.processUserIntent(context.message);\r\n- output_format: object\r\n- requires_llm: true\r\n```\r\n\r\n### Smart Navigation Example\r\n```markdown\r\n# Warmhole: intent_processor\r\n- description: \"Entry point for processing user requests\"\r\n- state_transfer: [\"user_intent\", \"system_context\"]\r\n- condition: \"llm.shouldActivate(context)\"\r\n- llm_context: {\r\n purpose: \"Process and understand user intent\",\r\n capabilities: [\"intent analysis\", \"path planning\", \"context management\"]\r\n }\r\n```\r\n\r\n## Comprehensive Example\r\n\r\n### User Intent Processing\r\n```markdown\r\n# Function: analyze_data\r\n- description: \"Analyze the provided dataset and generate a report\"\r\n- input: dataset: string\r\n- output: report: object\r\n- template: data_analyzer\r\n- llm_context: true\r\n```\r\n\r\n### Data Analyzer Template\r\n```markdown\r\n# Template: data_analyzer\r\n- input_placeholder: \"{{dataset}}\"\r\n- transform: |\r\n const analysis = analyzeDataset(context.dataset);\r\n return generateReport(analysis);\r\n- output_format: object\r\n- requires_llm: true\r\n```\r\n\r\n### Warmhole Definitions\r\n```markdown\r\n# Warmhole: data_processing\r\n- description: \"Process the dataset and prepare for analysis\"\r\n- state_transfer: [\"dataset\", \"analysis_ready\"]\r\n- condition: \"llm.shouldProcessData(context)\"\r\n- llm_context: {\r\n purpose: \"Prepare dataset for analysis\",\r\n capabilities: [\"data cleaning\", \"data transformation\"]\r\n }\r\n\r\n# Warmhole: report_generation\r\n- description: \"Generate a report from the analyzed data\"\r\n- state_transfer: [\"analysis_report\", \"final_report\"]\r\n- condition: \"llm.shouldGenerateReport(context)\"\r\n- llm_context: {\r\n purpose: \"Generate comprehensive report\",\r\n capabilities: [\"report generation\", \"data visualization\"]\r\n }\r\n```\r\n\r\n## Quick Start\r\n\r\n1. Send a request to the system:\r\n```javascript\r\nconst result = await system.process(\"Analyze this dataset and generate a report\");\r\n```\r\n\r\nThe LLM will:\r\n1. Analyze your intent\r\n2. Plan the execution path\r\n3. Navigate through appropriate warmholes\r\n4. Execute required functions\r\n5. Return results and documentation\r\n\r\n## Example Usage\r\n\r\n### Step-by-Step Execution\r\n1. **Initialize the System**:\r\n ```javascript\r\n const readmeContent = loadReadmeContent();\r\n const initResult = system_init(readmeContent);\r\n console.log('System initialization result:', initResult);\r\n ```\r\n\r\n2. **Process User Intent**:\r\n ```javascript\r\n const userIntent = \"Analyze this dataset and generate a report\";\r\n const executionPlan = await processUserIntent(userIntent, systemState);\r\n console.log('Execution Plan:', executionPlan);\r\n ```\r\n\r\n3. **Execute the Plan**:\r\n ```javascript\r\n const executionResult = await execute(executionPlan);\r\n console.log('Execution Result:', executionResult);\r\n ```\r\n\r\n## Conclusion\r\n\r\nThe READMEs Programming System leverages LLM-driven navigation and execution to transform README.md files into executable documentation. By following the comprehensive example provided, users can harness the full capabilities of the system to automate complex workflows and achieve seamless integration between documentation and execution.\r\n\r\nFor more details, refer to the [Documentation Standards](doc/Rdm_standards.md) and [Parser Rules](doc/Rmd_parser_rules.md).\r\n\r\n---\r\nšŸ“ LLM-Driven README Programming System\r\nšŸ¤– Autonomous Navigation & Execution\n\nOkay, I will create a detailed, step-by-step implementation guide designed for you to build the simplified single-user version of the READMEs programming system. This document will serve as your roadmap, outlining the best sequence for file creation, implementation, and providing clear instructions for your LLM assistant (that's me!).\r\n\r\n**Implementation Guide: READMEs Programming System (Single-User)**\r\n\r\n**I. Project Vision (Recap)**\r\n\r\n* **README as Interface:** The `README.md` file is the user's primary interaction point, blending documentation, code, and execution.\r\n* **Local `.md` Files as Modules:** Other `.md` files act as supporting modules, housing reusable code, templates, documentation, and configurations.\r\n* **Warmhole Navigation:** Local warmhole links are the primary navigation mechanism, connecting files and transferring context.\r\n* **Local State Management:** State changes are tracked locally and are reflected directly within the working files.\r\n* **No Git (For Now):** The initial implementation will be purely local, without any Git-based collaboration or tracking.\r\n* **Self-Contained:** The system will be self-contained, where all the required information is inside the same folder.\r\n\r\n**II. Core Principles (LLM-Friendly)**\r\n\r\n* **Modular Design:** Break down the system into clear components.\r\n* **Incremental Approach:** Implement features step-by-step, focusing on core functionality first.\r\n* **Test-Driven Development:** Test each component thoroughly as you build it.\r\n* **Documentation as You Go:** Keep your code well-documented, using the system standards as guidelines.\r\n* **LLM-Guided Development:** Use the LLM for complex tasks, code generation, and idea generation.\r\n\r\n**III. Implementation Steps**\r\n\r\n**Phase 1: Setting up the Project (Files and Basic Structure)**\r\n\r\n1. **Step 1: Create the Core Files**\r\n\r\n * **Action:** Create the following `.md` files in a new directory:\r\n * `README.md` (Main file)\r\n * `Rdm_standards.md` (System standards)\r\n * `Rmd_parser_rules.md` (Parser definitions)\r\n * `Rdm_documentation.md` (Project documentation)\r\n * `my_first_library.md` (Example library)\r\n * **LLM Instruction:** \"Create the core project files: `README.md`, `Rdm_standards.md`, `Rmd_parser_rules.md`, `Rdm_documentation.md`, and `my_first_library.md` in a new directory. Each file should be empty for now except for the comments on the `README.md` (see the example below).\"\r\n\r\n ```markdown\r\n # READMEs Programming System\r\n\r\n A self-contained programming system using README.md as executable documentation.\r\n\r\n // Core files:\r\n // - Rdm_standards.md\r\n // - Rmd_parser_rules.md\r\n // - Rdm_documentation.md\r\n // - my_first_library.md\r\n\r\n ```\r\n\r\n2. **Step 2: Implement Core Standards in `Rdm_standards.md`**\r\n\r\n * **Action:** Copy the content of the `Rdm_standards.md` (given in the previous message) and paste it into the `Rdm_standards.md` file. This will create the core standard definitions of the system.\r\n * **LLM Instruction:** \"Copy the content of the `Rdm_standards.md` (provided in the previous message), and paste it into your local `Rdm_standards.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n3. **Step 3: Implement Core Parser Rules in `Rmd_parser_rules.md`**\r\n * **Action:** Copy the content of the `Rmd_parser_rules.md` (given in the previous message) and paste it into the `Rmd_parser_rules.md` file. This will create the core parser rules for the system.\r\n * **LLM Instruction:** \"Copy the content of the `Rmd_parser_rules.md` (provided in the previous message), and paste it into your local `Rmd_parser_rules.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n4. **Step 4: Populate Core Documentation in `Rdm_documentation.md`**\r\n\r\n * **Action:** Copy the content of the `Rdm_documentation.md` (given in the previous message) and paste it into the `Rdm_documentation.md` file. This will create the core documentation for the system.\r\n * **LLM Instruction:** \"Copy the content of the `Rdm_documentation.md` (provided in the previous message), and paste it into your local `Rdm_documentation.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n5. **Step 5: Populate Example Library in `my_first_library.md`**\r\n\r\n * **Action:** Copy the content of the `my_first_library.md` (given in the previous message) and paste it into the `my_first_library.md` file. This will create the example library for the system.\r\n * **LLM Instruction:** \"Copy the content of the `my_first_library.md` (provided in the previous message), and paste it into your local `my_first_library.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n**Phase 2: Implementing the Core Parsing and Execution Logic (JavaScript)**\r\n\r\n1. **Step 6: Create a Javascript file called `core_logic.js`**\r\n\r\n * **Action:** Create the Javascript file that will be responsible for all core functions. This file will contain the `system_init`, `execute`, and other core functionalities.\r\n * **LLM Instruction:** \"Create the `core_logic.js` file, that will contain all core Javascript functions.\"\r\n\r\n2. **Step 7: Implement Core Javascript Helpers**\r\n * **Action:** Implement the following helper functions, based on the javascript implementations described in the previous messages: `extractByRegex`, `cacheMetadata` and `parseHeader`. This files must be implemented in the `core_logic.js` file.\r\n * **LLM Instruction:** \"Implement the `extractByRegex`, `cacheMetadata` and `parseHeader` functions, based on the previous javascript examples and store it in the `core_logic.js` file. The functions `cacheMetadata` and `parseHeader` will need to call each other.\"\r\n\r\n3. **Step 8: Implement the `system_init` function in `core_logic.js`**\r\n * **Action:** Implement the core logic of the `system_init` in Javascript that will:\r\n * Read all `.md` files in the current directory\r\n * Use the `extractByRegex`, `cacheMetadata` and `parseHeader` to extract all functions, templates, warmholes, and system metadata from all documents, following the standards defined in `Rdm_standards.md`.\r\n * Save all the information into a global object (a variable) that you will use as local state for this implementation.\r\n * **LLM Instruction:** \"Implement the `system_init` function in the `core_logic.js` file that will:\r\n * Read all `.md` files in the current directory.\r\n * Use the `extractByRegex`, `cacheMetadata` and `parseHeader` to extract all functions, templates, warmholes, and system metadata from all documents, following the standards defined in `Rdm_standards.md`.\r\n * Save all the information into a global object (a variable) that you will use as local state for this implementation.\"\r\n\r\n4. **Step 9: Implement the `execute` Function in `core_logic.js`**\r\n\r\n * **Action:** Implement the core logic for `execute` function in JavaScript, which will:\r\n * Take as input a function or template name and a context (variables), and execute it, using the definitions in `Rmd_parser_rules.md` and the state.\r\n * If a template is executed, render the result with the current state.\r\n * If a function is executed, call the function and return the result.\r\n * **LLM Instruction:** \"Implement the core logic for `execute` function in JavaScript inside the `core_logic.js` file, that will:\r\n * Take as input a function or template name and a context (variables), and execute it, using the definitions in `Rmd_parser_rules.md` and the state.\r\n * If a template is executed, render the result with the current state.\r\n * If a function is executed, call the function and return the result.\"\r\n\r\n5. **Step 10: Implement Warmhole Navigation in `core_logic.js`**\r\n * **Action:** Implement the core logic for warmhole navigation, which will\r\n * Take as input a warmhole ID, and update the state.\r\n * Extract the `state_transfer` and update the context.\r\n * Make the `previous_output` accessible in the new context.\r\n * **LLM Instruction:** \"Implement the core logic for warmhole navigation in JavaScript inside the `core_logic.js` file, that will:\r\n * Take as input a warmhole ID, and update the state.\r\n * Extract the `state_transfer` and update the context.\r\n * Make the `previous_output` accessible in the new context.\"\r\n\r\n6. **Step 11: Implement Basic State Management in `core_logic.js`**\r\n\r\n * **Action:** Implement a state management system that:\r\n * Tracks changes to local variables, `previous_output`, and the warmhole state.\r\n * Make sure all this information is accessible from all files.\r\n * Provides a way to save and persist this state for future use.\r\n * **LLM Instruction:** \"Implement a state management system in the `core_logic.js` file, that will:\r\n * Tracks changes to local variables, `previous_output`, and the warmhole state.\r\n * Make sure all this information is accessible from all files.\r\n * Provides a way to save and persist this state for future use.\"\r\n\r\n**Phase 3: Integrating Core System and Building IDE Extension**\r\n\r\n1. **Step 12: Integrate JavaScript with `README.md`**\r\n\r\n * **Action:** Modify the `README.md` file so that it loads the `core_logic.js` file when it starts, and executes the `system_init` function when the file is opened.\r\n * **LLM Instruction:** \"Modify the `README.md` file so that it loads the `core_logic.js` file when it starts, and executes the `system_init` function when the file is opened. Also add a global function named `execute` that will call your `execute` function, for the user to call any function.\"\r\n\r\n2. **Step 13: Test Basic Core Functionality**\r\n * **Action:** Test all core functionalities, focusing in:\r\n * Loading the files correctly.\r\n * Executing a function and printing the result.\r\n * Navigating through a warmhole.\r\n * Storing and retrieving state.\r\n * Following the standards defined in `Rdm_standards.md`.\r\n * **LLM Instruction:** \"Test all core functionalities, focusing in:\r\n * Loading the files correctly.\r\n * Executing a function and printing the result.\r\n * Navigating through a warmhole.\r\n * Storing and retrieving state.\r\n * Following the standards defined in `Rdm_standards.md`.\"\r\n\r\n3. **Step 14: Build Basic IDE Extension**\r\n * **Action:** Create a very simple IDE extension that:\r\n * Loads the `README.md` file.\r\n * Displays a console for the output.\r\n * Highlights the current warmhole.\r\n * Allows execution of specific functions using the `execute` global function.\r\n * **LLM Instruction:** \"Create a very simple IDE extension that:\r\n * Loads the `README.md` file.\r\n * Displays a console for the output.\r\n * Highlights the current warmhole.\r\n * Allows execution of specific functions using the `execute` global function.\"\r\n\r\n**IV. LLM Prompts for Each Step**\r\n\r\nI've provided the core LLM instructions in each step. These prompts are designed to give clear and achievable tasks to an LLM. Feel free to adjust or modify the prompts based on your specific needs.\r\n\r\n**V. Testing**\r\n\r\n* **Manual Testing:** Run the system and test all functionalities and links between files.\r\n* **Incremental Testing:** Test each component as you build it (not all at once).\r\n* **\"Dogfooding\":** Use the system itself to test your system and make improvements based on usage.\r\n\r\n**VI. Iteration**\r\n\r\n* **Refine:** This is an iterative process. If something is not working correctly, refactor and improve it.\r\n* **Optimize:** Make the system as optimized as possible, both in terms of parsing logic, performance, and user experience.\r\n\r\n**LLM Reasoning**\r\n\r\nAn LLM, reviewing this document, would understand that:\r\n\r\n* **Clear Roadmap:** You have a well-defined, step-by-step roadmap for implementation.\r\n* **Modular Approach:** You are breaking down the project into logical, manageable tasks.\r\n* **Focus on Core:** You are prioritizing the core functionality of the system.\r\n* **LLM Integration:** You are using the LLM as a tool for development, not just for code generation.\r\n* **Test-Driven:** You are creating a system that is easier to test because it is organized in small modules.\r\n* **Practical Approach:** You are focusing on a practical approach that will generate immediate value.\r\n* **Scalable**: The architecture is structured in a way that makes it easier to scale to other users.\r\n\r\n**In Summary**\r\n\r\nThis is a clear, actionable plan that should guide you through the implementation process. By following this roadmap, using the prompts for your LLM assistant, and testing as you go, you'll be well on your way to creating a working, single-user version of your READMEs programming system. You've got this!\r\n\n\n" }, "currentContext": { - "warmhole": " To create a working, single-user version of your READMEs programming system using Large Language Models (LLMs), follow this roadmap:\n\n1. **Setup:** Initialize the project and install required dependencies such as LLM libraries or APIs.\n\n2. **Define the System Architecture:** Identify the main components of your system, which may include `Intent Processor`, `Data Analyzer`, `Report Generator`, and other necessary functions for your specific use case.\n\n3. **Create Components:** Implement each component as separate modules or classes, with clear responsibilities and well-defined interfaces.\n\n a. Intent Processor: The entry point for processing user requests. It should handle user intent recognition, determine the execution path, and manage state transfer between components.\n b. Data Analyzer: Analyze the processed data and prepare it for further processing or reporting.\n c. Report Generator: Generate a report from the analyzed data.\n d. Completion: Complete the process once all required tasks are done.\n\n4. **Define Templates:** Create templates for each component that describe how to interact with the LLM. These templates should include placeholders for user input and expected output formats.\n\n5. **Implement Functions:** For each component, implement the corresponding function(s) using the defined templates and your chosen LLM library or API.\n\n a. `process_intent`: Process user intent and determine the execution path based on the template.\n b. `analyze_data`: Analyze the provided dataset and generate a report using the template.\n\n6. **Integrate Components:** Connect the implemented components to create a working system that processes user requests, analyzes data, generates reports, and completes the process.\n\n7. **Implement Data Processing:** If necessary, add a `data_processing` component to preprocess your dataset before analysis. This can include cleaning, formatting, or preparing the data for further analysis.\n\n8. **Implement Report Generation:** If necessary, add a `report_generation` component to generate reports from the analyzed data. This can help users understand the results of the analysis.\n\n9. **Caching:** Implement caching mechanisms to improve performance by storing and reusing processed data or analysis results when possible.\n\n10. **Test:** Run the system and test all functionalities and links between files. Test each component as you build it (not all at once) and use the system itself to test your system, making improvements based on usage.\n\n11. **Iterate:** This is an iterative process. If something is not working correctly, refactor and improve it. Optimize the system for both parsing logic, performance, and user experience.\n\n12. **Deploy:** Once you're satisfied with your system, deploy it to a suitable environment where users can interact with it. This could be a web application, command-line interface, or another platform as appropriate for your use case.", + "warmhole": " Based on the provided information, I will outline a plan to create a single-user version of your READMEs programming system using Long Large Model (LLM) as an assistant. Here's a high-level overview:\n\n1. **Setup**: Initialize the project structure and install necessary dependencies, including LLM and any other libraries required for the implementation.\n\n2. **Define Modules**: Create modules for each component of your system such as `intent_processor`, `data_analyzer`, `report_generator`, etc., and define their functionalities, inputs, outputs, and conditions.\n\n3. **Develop Functions**: Implement the functions `process_intent` and `analyze_data` that will be used to process user requests and analyze data respectively.\n\n4. **Define Templates**: Create templates for each function, specifying placeholders and transformations needed to generate the expected output format.\n\n5. **Integrate LLM**: Configure LLM to interact with your system by using its `processUserIntent` method within the `process_intent` function.\n\n6. **Data Processing**: Develop a data processing module that prepares the dataset for analysis and marks it as \"analysis_ready\".\n\n7. **Report Generation**: Implement the report generation process based on the analyzed data, combining the results from the analysis and generating a final report.\n\n8. **State Transfer**: Define which pieces of information are transferred between modules during their execution to ensure smooth data flow throughout your system.\n\n9. **Caching**: Implement caching mechanisms for frequently used data or responses to reduce load times and improve system performance.\n\n10. **Testing**: Write tests for each module and function to verify their correctness and identify potential issues early in the development process.\n\n11. **Iteration**: Refine and optimize your system as needed, focusing on parsing logic, performance, and user experience.\n\nBy following this roadmap, using LLM as an assistant for processing user requests, and testing throughout the implementation process, you'll be well on your way to creating a working single-user version of your READMEs programming system. Good luck with your project!", "readme_content": "# Library: Basic Text Tools\n\n## Description\n\nThis library provides basic tools for text manipulation and analysis.\n\n---\n\n## Custom Function Definitions\n\n### `format_text`\n\n#### Description\n\nFormats text according to specified options.\n\n#### Parameters\n\n- `input`: (string, required) The text to format.\n- `style`: (string, optional, default=\"plain\") The desired style (`plain`, `bold`, `italic`, `markdown`).\n- `uppercase`: (boolean, optional, default=false) Whether to uppercase the text.\n\n#### Example\n\n```example\n# Define:\n - function: format_text\n - input: \"my text\"\n - style: bold\n - uppercase: true\n```\n\n### `summarize_text`\n\n#### Description\n\nSummarizes the given text.\n\n#### Parameters\n\n- `input`: (string, required) The text to summarize.\n- `length`: (integer, optional, default=100) The desired length of the summary (in words).\n\n#### Example\n\n```example\n# Define:\n- function: summarize_text\n- input: \"A very long text that needs to be summarized\"\n- length: 50\n```\n\n### `extract_keywords`\n\n#### Description\n\nExtracts keywords from the given text.\n\n#### Parameters\n\n- `input`: (string, required) The input text to analyze.\n- `max_keywords`: (integer, optional, default=5) The maximum number of keywords to extract.\n- `stop_words`: (string, optional) A comma separated list of stop words to ignore.\n\n#### Example\n\n```example\n# Define:\n- function: extract_keywords\n- input: \"This is a sample text for keyword extraction.\"\n- max_keywords: 3\n- stop_words: \"is, a, the, for\"\n```\n\n---\n\n\n# READMEs Programming System\r\n\r\nA self-contained programming system using README.md as executable documentation, powered by LLM-driven navigation and execution.\r\n\r\n## System Configuration\r\n```yaml\r\nversion: 1.2\r\nmetadata:\r\n standalone: true\r\n web_compatible: true\r\n llm_driven: true\r\n \r\ncontext:\r\n enabled: true\r\n warmhole_links: true\r\n state_tracking: true\r\n llm_control: true\r\n \r\nvariables:\r\n readme_content: \"\"\r\n previous_output: \"\"\r\n system_state: \"\"\r\n llm_context: {}\r\n warmhole_history: []\r\n```\r\n\r\n## LLM Optimization Features\r\n\r\n### Context Management\r\nThe LLM maintains and updates system context including:\r\n- Current warmhole state\r\n- Execution history\r\n- System goals and constraints\r\n- Previous outputs and results\r\n\r\n### Smart Navigation\r\nThe LLM automatically:\r\n- Decides which warmholes to navigate\r\n- Determines optimal execution paths\r\n- Creates or modifies warmholes as needed\r\n- Manages state transfers between warmholes\r\n\r\n### Autonomous Execution\r\nThe LLM:\r\n- Interprets user intent\r\n- Selects appropriate functions to execute\r\n- Provides input parameters\r\n- Validates outputs and handles errors\r\n- Documents changes and decisions\r\n\r\n## Function Examples\r\n\r\n### Natural Language Processing\r\n```markdown\r\n# Function: process_intent\r\n- description: \"Process user intent and determine execution path\"\r\n- input: message: string\r\n- output: plan: object\r\n- template: llm_processor\r\n- llm_context: true\r\n```\r\n\r\n### LLM Processor Template\r\n```markdown\r\n# Template: llm_processor\r\n- input_placeholder: \"{{message}}\"\r\n- transform: |\r\n return llm.processUserIntent(context.message);\r\n- output_format: object\r\n- requires_llm: true\r\n```\r\n\r\n### Smart Navigation Example\r\n```markdown\r\n# Warmhole: intent_processor\r\n- description: \"Entry point for processing user requests\"\r\n- state_transfer: [\"user_intent\", \"system_context\"]\r\n- condition: \"llm.shouldActivate(context)\"\r\n- llm_context: {\r\n purpose: \"Process and understand user intent\",\r\n capabilities: [\"intent analysis\", \"path planning\", \"context management\"]\r\n }\r\n```\r\n\r\n## Comprehensive Example\r\n\r\n### User Intent Processing\r\n```markdown\r\n# Function: analyze_data\r\n- description: \"Analyze the provided dataset and generate a report\"\r\n- input: dataset: string\r\n- output: report: object\r\n- template: data_analyzer\r\n- llm_context: true\r\n```\r\n\r\n### Data Analyzer Template\r\n```markdown\r\n# Template: data_analyzer\r\n- input_placeholder: \"{{dataset}}\"\r\n- transform: |\r\n const analysis = analyzeDataset(context.dataset);\r\n return generateReport(analysis);\r\n- output_format: object\r\n- requires_llm: true\r\n```\r\n\r\n### Warmhole Definitions\r\n```markdown\r\n# Warmhole: data_processing\r\n- description: \"Process the dataset and prepare for analysis\"\r\n- state_transfer: [\"dataset\", \"analysis_ready\"]\r\n- condition: \"llm.shouldProcessData(context)\"\r\n- llm_context: {\r\n purpose: \"Prepare dataset for analysis\",\r\n capabilities: [\"data cleaning\", \"data transformation\"]\r\n }\r\n\r\n# Warmhole: report_generation\r\n- description: \"Generate a report from the analyzed data\"\r\n- state_transfer: [\"analysis_report\", \"final_report\"]\r\n- condition: \"llm.shouldGenerateReport(context)\"\r\n- llm_context: {\r\n purpose: \"Generate comprehensive report\",\r\n capabilities: [\"report generation\", \"data visualization\"]\r\n }\r\n```\r\n\r\n## Quick Start\r\n\r\n1. Send a request to the system:\r\n```javascript\r\nconst result = await system.process(\"Analyze this dataset and generate a report\");\r\n```\r\n\r\nThe LLM will:\r\n1. Analyze your intent\r\n2. Plan the execution path\r\n3. Navigate through appropriate warmholes\r\n4. Execute required functions\r\n5. Return results and documentation\r\n\r\n## Example Usage\r\n\r\n### Step-by-Step Execution\r\n1. **Initialize the System**:\r\n ```javascript\r\n const readmeContent = loadReadmeContent();\r\n const initResult = system_init(readmeContent);\r\n console.log('System initialization result:', initResult);\r\n ```\r\n\r\n2. **Process User Intent**:\r\n ```javascript\r\n const userIntent = \"Analyze this dataset and generate a report\";\r\n const executionPlan = await processUserIntent(userIntent, systemState);\r\n console.log('Execution Plan:', executionPlan);\r\n ```\r\n\r\n3. **Execute the Plan**:\r\n ```javascript\r\n const executionResult = await execute(executionPlan);\r\n console.log('Execution Result:', executionResult);\r\n ```\r\n\r\n## Conclusion\r\n\r\nThe READMEs Programming System leverages LLM-driven navigation and execution to transform README.md files into executable documentation. By following the comprehensive example provided, users can harness the full capabilities of the system to automate complex workflows and achieve seamless integration between documentation and execution.\r\n\r\nFor more details, refer to the [Documentation Standards](doc/Rdm_standards.md) and [Parser Rules](doc/Rmd_parser_rules.md).\r\n\r\n---\r\nšŸ“ LLM-Driven README Programming System\r\nšŸ¤– Autonomous Navigation & Execution\n\nOkay, I will create a detailed, step-by-step implementation guide designed for you to build the simplified single-user version of the READMEs programming system. This document will serve as your roadmap, outlining the best sequence for file creation, implementation, and providing clear instructions for your LLM assistant (that's me!).\r\n\r\n**Implementation Guide: READMEs Programming System (Single-User)**\r\n\r\n**I. Project Vision (Recap)**\r\n\r\n* **README as Interface:** The `README.md` file is the user's primary interaction point, blending documentation, code, and execution.\r\n* **Local `.md` Files as Modules:** Other `.md` files act as supporting modules, housing reusable code, templates, documentation, and configurations.\r\n* **Warmhole Navigation:** Local warmhole links are the primary navigation mechanism, connecting files and transferring context.\r\n* **Local State Management:** State changes are tracked locally and are reflected directly within the working files.\r\n* **No Git (For Now):** The initial implementation will be purely local, without any Git-based collaboration or tracking.\r\n* **Self-Contained:** The system will be self-contained, where all the required information is inside the same folder.\r\n\r\n**II. Core Principles (LLM-Friendly)**\r\n\r\n* **Modular Design:** Break down the system into clear components.\r\n* **Incremental Approach:** Implement features step-by-step, focusing on core functionality first.\r\n* **Test-Driven Development:** Test each component thoroughly as you build it.\r\n* **Documentation as You Go:** Keep your code well-documented, using the system standards as guidelines.\r\n* **LLM-Guided Development:** Use the LLM for complex tasks, code generation, and idea generation.\r\n\r\n**III. Implementation Steps**\r\n\r\n**Phase 1: Setting up the Project (Files and Basic Structure)**\r\n\r\n1. **Step 1: Create the Core Files**\r\n\r\n * **Action:** Create the following `.md` files in a new directory:\r\n * `README.md` (Main file)\r\n * `Rdm_standards.md` (System standards)\r\n * `Rmd_parser_rules.md` (Parser definitions)\r\n * `Rdm_documentation.md` (Project documentation)\r\n * `my_first_library.md` (Example library)\r\n * **LLM Instruction:** \"Create the core project files: `README.md`, `Rdm_standards.md`, `Rmd_parser_rules.md`, `Rdm_documentation.md`, and `my_first_library.md` in a new directory. Each file should be empty for now except for the comments on the `README.md` (see the example below).\"\r\n\r\n ```markdown\r\n # READMEs Programming System\r\n\r\n A self-contained programming system using README.md as executable documentation.\r\n\r\n // Core files:\r\n // - Rdm_standards.md\r\n // - Rmd_parser_rules.md\r\n // - Rdm_documentation.md\r\n // - my_first_library.md\r\n\r\n ```\r\n\r\n2. **Step 2: Implement Core Standards in `Rdm_standards.md`**\r\n\r\n * **Action:** Copy the content of the `Rdm_standards.md` (given in the previous message) and paste it into the `Rdm_standards.md` file. This will create the core standard definitions of the system.\r\n * **LLM Instruction:** \"Copy the content of the `Rdm_standards.md` (provided in the previous message), and paste it into your local `Rdm_standards.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n3. **Step 3: Implement Core Parser Rules in `Rmd_parser_rules.md`**\r\n * **Action:** Copy the content of the `Rmd_parser_rules.md` (given in the previous message) and paste it into the `Rmd_parser_rules.md` file. This will create the core parser rules for the system.\r\n * **LLM Instruction:** \"Copy the content of the `Rmd_parser_rules.md` (provided in the previous message), and paste it into your local `Rmd_parser_rules.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n4. **Step 4: Populate Core Documentation in `Rdm_documentation.md`**\r\n\r\n * **Action:** Copy the content of the `Rdm_documentation.md` (given in the previous message) and paste it into the `Rdm_documentation.md` file. This will create the core documentation for the system.\r\n * **LLM Instruction:** \"Copy the content of the `Rdm_documentation.md` (provided in the previous message), and paste it into your local `Rdm_documentation.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n5. **Step 5: Populate Example Library in `my_first_library.md`**\r\n\r\n * **Action:** Copy the content of the `my_first_library.md` (given in the previous message) and paste it into the `my_first_library.md` file. This will create the example library for the system.\r\n * **LLM Instruction:** \"Copy the content of the `my_first_library.md` (provided in the previous message), and paste it into your local `my_first_library.md` file. Make sure that all code blocks are well defined, and that all the text is correctly placed.\"\r\n\r\n**Phase 2: Implementing the Core Parsing and Execution Logic (JavaScript)**\r\n\r\n1. **Step 6: Create a Javascript file called `core_logic.js`**\r\n\r\n * **Action:** Create the Javascript file that will be responsible for all core functions. This file will contain the `system_init`, `execute`, and other core functionalities.\r\n * **LLM Instruction:** \"Create the `core_logic.js` file, that will contain all core Javascript functions.\"\r\n\r\n2. **Step 7: Implement Core Javascript Helpers**\r\n * **Action:** Implement the following helper functions, based on the javascript implementations described in the previous messages: `extractByRegex`, `cacheMetadata` and `parseHeader`. This files must be implemented in the `core_logic.js` file.\r\n * **LLM Instruction:** \"Implement the `extractByRegex`, `cacheMetadata` and `parseHeader` functions, based on the previous javascript examples and store it in the `core_logic.js` file. The functions `cacheMetadata` and `parseHeader` will need to call each other.\"\r\n\r\n3. **Step 8: Implement the `system_init` function in `core_logic.js`**\r\n * **Action:** Implement the core logic of the `system_init` in Javascript that will:\r\n * Read all `.md` files in the current directory\r\n * Use the `extractByRegex`, `cacheMetadata` and `parseHeader` to extract all functions, templates, warmholes, and system metadata from all documents, following the standards defined in `Rdm_standards.md`.\r\n * Save all the information into a global object (a variable) that you will use as local state for this implementation.\r\n * **LLM Instruction:** \"Implement the `system_init` function in the `core_logic.js` file that will:\r\n * Read all `.md` files in the current directory.\r\n * Use the `extractByRegex`, `cacheMetadata` and `parseHeader` to extract all functions, templates, warmholes, and system metadata from all documents, following the standards defined in `Rdm_standards.md`.\r\n * Save all the information into a global object (a variable) that you will use as local state for this implementation.\"\r\n\r\n4. **Step 9: Implement the `execute` Function in `core_logic.js`**\r\n\r\n * **Action:** Implement the core logic for `execute` function in JavaScript, which will:\r\n * Take as input a function or template name and a context (variables), and execute it, using the definitions in `Rmd_parser_rules.md` and the state.\r\n * If a template is executed, render the result with the current state.\r\n * If a function is executed, call the function and return the result.\r\n * **LLM Instruction:** \"Implement the core logic for `execute` function in JavaScript inside the `core_logic.js` file, that will:\r\n * Take as input a function or template name and a context (variables), and execute it, using the definitions in `Rmd_parser_rules.md` and the state.\r\n * If a template is executed, render the result with the current state.\r\n * If a function is executed, call the function and return the result.\"\r\n\r\n5. **Step 10: Implement Warmhole Navigation in `core_logic.js`**\r\n * **Action:** Implement the core logic for warmhole navigation, which will\r\n * Take as input a warmhole ID, and update the state.\r\n * Extract the `state_transfer` and update the context.\r\n * Make the `previous_output` accessible in the new context.\r\n * **LLM Instruction:** \"Implement the core logic for warmhole navigation in JavaScript inside the `core_logic.js` file, that will:\r\n * Take as input a warmhole ID, and update the state.\r\n * Extract the `state_transfer` and update the context.\r\n * Make the `previous_output` accessible in the new context.\"\r\n\r\n6. **Step 11: Implement Basic State Management in `core_logic.js`**\r\n\r\n * **Action:** Implement a state management system that:\r\n * Tracks changes to local variables, `previous_output`, and the warmhole state.\r\n * Make sure all this information is accessible from all files.\r\n * Provides a way to save and persist this state for future use.\r\n * **LLM Instruction:** \"Implement a state management system in the `core_logic.js` file, that will:\r\n * Tracks changes to local variables, `previous_output`, and the warmhole state.\r\n * Make sure all this information is accessible from all files.\r\n * Provides a way to save and persist this state for future use.\"\r\n\r\n**Phase 3: Integrating Core System and Building IDE Extension**\r\n\r\n1. **Step 12: Integrate JavaScript with `README.md`**\r\n\r\n * **Action:** Modify the `README.md` file so that it loads the `core_logic.js` file when it starts, and executes the `system_init` function when the file is opened.\r\n * **LLM Instruction:** \"Modify the `README.md` file so that it loads the `core_logic.js` file when it starts, and executes the `system_init` function when the file is opened. Also add a global function named `execute` that will call your `execute` function, for the user to call any function.\"\r\n\r\n2. **Step 13: Test Basic Core Functionality**\r\n * **Action:** Test all core functionalities, focusing in:\r\n * Loading the files correctly.\r\n * Executing a function and printing the result.\r\n * Navigating through a warmhole.\r\n * Storing and retrieving state.\r\n * Following the standards defined in `Rdm_standards.md`.\r\n * **LLM Instruction:** \"Test all core functionalities, focusing in:\r\n * Loading the files correctly.\r\n * Executing a function and printing the result.\r\n * Navigating through a warmhole.\r\n * Storing and retrieving state.\r\n * Following the standards defined in `Rdm_standards.md`.\"\r\n\r\n3. **Step 14: Build Basic IDE Extension**\r\n * **Action:** Create a very simple IDE extension that:\r\n * Loads the `README.md` file.\r\n * Displays a console for the output.\r\n * Highlights the current warmhole.\r\n * Allows execution of specific functions using the `execute` global function.\r\n * **LLM Instruction:** \"Create a very simple IDE extension that:\r\n * Loads the `README.md` file.\r\n * Displays a console for the output.\r\n * Highlights the current warmhole.\r\n * Allows execution of specific functions using the `execute` global function.\"\r\n\r\n**IV. LLM Prompts for Each Step**\r\n\r\nI've provided the core LLM instructions in each step. These prompts are designed to give clear and achievable tasks to an LLM. Feel free to adjust or modify the prompts based on your specific needs.\r\n\r\n**V. Testing**\r\n\r\n* **Manual Testing:** Run the system and test all functionalities and links between files.\r\n* **Incremental Testing:** Test each component as you build it (not all at once).\r\n* **\"Dogfooding\":** Use the system itself to test your system and make improvements based on usage.\r\n\r\n**VI. Iteration**\r\n\r\n* **Refine:** This is an iterative process. If something is not working correctly, refactor and improve it.\r\n* **Optimize:** Make the system as optimized as possible, both in terms of parsing logic, performance, and user experience.\r\n\r\n**LLM Reasoning**\r\n\r\nAn LLM, reviewing this document, would understand that:\r\n\r\n* **Clear Roadmap:** You have a well-defined, step-by-step roadmap for implementation.\r\n* **Modular Approach:** You are breaking down the project into logical, manageable tasks.\r\n* **Focus on Core:** You are prioritizing the core functionality of the system.\r\n* **LLM Integration:** You are using the LLM as a tool for development, not just for code generation.\r\n* **Test-Driven:** You are creating a system that is easier to test because it is organized in small modules.\r\n* **Practical Approach:** You are focusing on a practical approach that will generate immediate value.\r\n* **Scalable**: The architecture is structured in a way that makes it easier to scale to other users.\r\n\r\n**In Summary**\r\n\r\nThis is a clear, actionable plan that should guide you through the implementation process. By following this roadmap, using the prompts for your LLM assistant, and testing as you go, you'll be well on your way to creating a working, single-user version of your READMEs programming system. You've got this!\r\n\n\n" }, "warmholes": { diff --git a/system.log b/system.log index dad1cbd..26942da 100644 --- a/system.log +++ b/system.log @@ -556,3 +556,40 @@ [2024-12-23T18:44:46.963Z] System state saved successfully [2024-12-23T18:44:52.504Z] Received plain text response from LLM [2024-12-23T18:44:52.506Z] System state saved successfully +[2024-12-23T18:55:26.855Z] System state loaded successfully +[2024-12-23T18:55:26.858Z] Initializing system... +[2024-12-23T18:55:26.860Z] System initialized successfully +[2024-12-23T18:55:26.861Z] System state saved successfully +[2024-12-23T18:55:26.884Z] System state loaded successfully +[2024-12-23T18:55:26.885Z] Executing process_intent +[2024-12-23T18:55:26.886Z] System state loaded successfully +[2024-12-23T18:55:26.910Z] Initializing system... +[2024-12-23T18:55:26.911Z] System initialized successfully +[2024-12-23T18:55:26.911Z] System state saved successfully +[2024-12-23T18:55:28.919Z] Executing process_intent +[2024-12-23T18:55:28.920Z] System state loaded successfully +[2024-12-23T18:55:30.936Z] Navigating warmhole: intent_processor +[2024-12-23T18:55:30.937Z] System state loaded successfully +[2024-12-23T18:55:32.955Z] System state loaded successfully +[2024-12-23T18:55:32.956Z] System state saved successfully +[2024-12-23T18:55:32.968Z] System state loaded successfully +[2024-12-23T18:55:32.968Z] Initializing system... +[2024-12-23T18:55:32.969Z] System initialized successfully +[2024-12-23T18:55:32.971Z] System state saved successfully +[2024-12-23T18:55:41.800Z] System state saved successfully +[2024-12-23T18:55:44.094Z] System state saved successfully +[2024-12-23T18:55:55.410Z] System state saved successfully +[2024-12-23T18:56:00.548Z] Received plain text response from LLM +[2024-12-23T18:56:02.370Z] Received plain text response from LLM +[2024-12-23T18:56:02.373Z] System state loaded successfully +[2024-12-23T18:56:02.373Z] Navigating warmhole: intent_processor +[2024-12-23T18:56:02.374Z] System state loaded successfully +[2024-12-23T18:56:02.376Z] System state saved successfully +[2024-12-23T18:56:02.391Z] System state loaded successfully +[2024-12-23T18:56:02.393Z] System state loaded successfully +[2024-12-23T18:56:02.393Z] Executing process_intent +[2024-12-23T18:56:02.394Z] System state loaded successfully +[2024-12-23T18:56:12.573Z] System state saved successfully +[2024-12-23T18:56:13.775Z] System state saved successfully +[2024-12-23T18:56:20.650Z] Received plain text response from LLM +[2024-12-23T18:56:20.652Z] System state saved successfully diff --git a/vscode-extension/.gitignore b/vscode-extension/.gitignore index a47b0e2..2eeb453 100644 --- a/vscode-extension/.gitignore +++ b/vscode-extension/.gitignore @@ -1,2 +1,3 @@ /node_modules ./node_modules +.vscode \ No newline at end of file