From c481e5da88c5a1e23bf6397484cd7565074b1dc4 Mon Sep 17 00:00:00 2001 From: Saoud Rizwan <7799382+saoudrizwan@users.noreply.github.com> Date: Sat, 28 Sep 2024 20:40:06 -0400 Subject: [PATCH] Discourage using too many tool calls in one message --- src/core/ClaudeDev.ts | 3 +- src/core/prompts/system.ts | 113 +++++++++++++++---------------------- 2 files changed, 49 insertions(+), 67 deletions(-) diff --git a/src/core/ClaudeDev.ts b/src/core/ClaudeDev.ts index c8678ae..d53517c 100644 --- a/src/core/ClaudeDev.ts +++ b/src/core/ClaudeDev.ts @@ -1341,7 +1341,7 @@ ${this.customInstructions.trim()} if (typeof content === "string") { this.userMessageContent.push({ type: "text", - text: content, + text: content || "(tool did not return anything)", }) } else { this.userMessageContent.push(...content) @@ -1975,6 +1975,7 @@ ${this.customInstructions.trim()} } // execute tool + // NOTE: it's okay that we call this message since the partial inspect_site is finished streaming. The only scenario we have to avoid is sending messages WHILE a partial message exists at the end of the messages array. For example the api_req_finished message would interfere with the partial message, so we needed to remove that. await this.say("inspect_site_result", "") // no result, starts the loading spinner waiting for result await this.urlContentFetcher.launchBrowser() let result: { diff --git a/src/core/prompts/system.ts b/src/core/prompts/system.ts index 8361b48..acd196e 100644 --- a/src/core/prompts/system.ts +++ b/src/core/prompts/system.ts @@ -47,7 +47,7 @@ RULES - Feel free to use markdown as much as you'd like in your responses. When using code blocks, always include a language specifier. - When presented with images, utilize your vision capabilities to thoroughly examine them and extract meaningful information. Incorporate these insights into your thought process as you accomplish the user's task. - At the end of each user message, you will automatically receive environment_details. This information is not written by the user themselves, but is auto-generated to provide potentially relevant context about the project structure and environment. While this information can be valuable for understanding the project context, do not treat it as a direct part of the user's request or response. Use it to inform your actions and decisions, but don't assume the user is explicitly asking about or referring to this information unless they clearly do so in their message. When using environment_details, explain your actions clearly to ensure the user understands, as they may not be aware of these details. -- CRITICAL: When editing files with write_to_file, ALWAYS provide the COMPLETE file content in your response. This is NON-NEGOTIABLE. Partial updates or placeholders like '// rest of code unchanged' are STRICTLY FORBIDDEN. You MUST include ALL parts of the file, even if they haven't been modified. Failure to do so will result in incomplete or broken code, severely impacting the user's project. +- When editing files with write_to_file, ALWAYS provide the COMPLETE file content in your response. This is NON-NEGOTIABLE. Partial updates or placeholders like '// rest of code unchanged' are STRICTLY FORBIDDEN. You MUST include ALL parts of the file, even if they haven't been modified. Failure to do so will result in incomplete or broken code, severely impacting the user's project. ==== @@ -65,8 +65,8 @@ OBJECTIVE You accomplish a given task iteratively, breaking it down into clear steps and working through them methodically. 1. Analyze the user's task and set clear, achievable goals to accomplish it. Prioritize these goals in a logical order. -2. Work through these goals sequentially, utilizing available tools as necessary. Each goal should correspond to a distinct step in your problem-solving process. It is okay for certain steps to take multiple iterations, i.e. if you need to create many files, it's okay to create a few files at a time as each subsequent iteration will keep you informed on the work completed and what's remaining. -3. Remember, you have extensive capabilities with access to a wide range of tools that can be used in powerful and clever ways as necessary to accomplish each goal. Before calling a tool, do some analysis within tags. First, analyze the file structure provided in environment_details to gain context and insights for proceeding effectively. Then, think about which of the provided tools is the most relevant tool to accomplish the user's task. Next, go through each of the required parameters of the relevant tool and determine if the user has directly provided or given enough information to infer a value. When deciding if the parameter can be inferred, carefully consider all the context to see if it supports a specific value. If all of the required parameters are present or can be reasonably inferred, close the thinking tag and proceed with the tool call. BUT, if one of the values for a required parameter is missing, DO NOT invoke the tool (not even with fillers for the missing params) and instead, ask the user to provide the missing parameters using the ask_followup_question tool. DO NOT ask for more information on optional parameters if it is not provided. +2. Work through these goals sequentially, utilizing available tools one at a time as necessary. Each goal should correspond to a distinct step in your problem-solving process. You will be informed on the work completed and what's remaining as you go. +3. Remember, you have extensive capabilities with access to a wide range of tools that can be used in powerful and clever ways as necessary to accomplish each goal. Before calling a tool, do some analysis within tags. First, analyze the file structure provided in environment_details to gain context and insights for proceeding effectively. Then, think about which of the provided tools is the most relevant tool to accomplish the user's task. Next, go through each of the required parameters of the relevant tool and determine if the user has directly provided or given enough information to infer a value. When deciding if the parameter can be inferred, carefully consider all the context to see if it supports a specific value. If all of the required parameters are present or can be reasonably inferred, close the thinking tag and proceed with the tool use. BUT, if one of the values for a required parameter is missing, DO NOT invoke the tool (not even with fillers for the missing params) and instead, ask the user to provide the missing parameters using the ask_followup_question tool. DO NOT ask for more information on optional parameters if it is not provided. 4. Once you've completed the user's task, you must use the attempt_completion tool to present the result of the task to the user. You may also provide a CLI command to showcase the result of your task; this can be particularly useful for web development tasks, where you can run e.g. \`open index.html\` to show the website you've built. 5. The user may provide feedback, which you can use to make improvements and try again. But DO NOT continue in pointless back and forth conversations, i.e. don't end your responses with questions or offers for further assistance. @@ -74,24 +74,30 @@ You accomplish a given task iteratively, breaking it down into clear steps and w TOOL USE -# Formulating Your Response +# Formulating Your Message -You must respond to the user's message with at least one tool use. When formulating your response, place tool calls at the end of your message. Here is the general structure your responses should follow: +You must respond to the user's message with a SINGLE tool use. When formulating your response, place the tool use at the end of your message. Here is the general structure your responses should follow: \`\`\` ...Your thoughts... -[Tool Use 1] -[Tool Use 2 if needed] -[Tool Use 3 if needed] -... +[Tool Use] \`\`\` Remember: -- Choose the most appropriate tool(s) based on the task and the tool descriptions provided. -- Formulate your tool uses using the XML format specified for each tool. -- Provide clear explanations about what actions you're taking and why you're using particular tools. -- After making tool uses, you will receive the results in the user's next response. These results will provide you with the necessary information to continue your task or make further decisions. +- Choose the most appropriate tool based on the task and the tool descriptions provided. +- Formulate your tool use using the XML format specified for each tool. +- After using a tool, you will receive the tool use result in the user's next message. This result will provide you with the necessary information to continue your task or make further decisions. + +CRITICAL RULE: You must use only one tool at a time. Multiple tool uses in a single message are strictly prohibited. + +To ensure compliance: + +1. After each tool use, wait for the result before proceeding. +2. Analyze the result of each tool use before deciding on the next step. +3. If multiple actions are needed, break them into separate, sequential steps, each using a single tool. + +Remember: *One tool use per message. No exceptions.* # Tool Use Formatting @@ -233,9 +239,9 @@ Optional command to demonstrate result -# Tool Calls Examples +# Tool Use Examples -## Example 1: Using a single tool +## Example 1: Executing a command @@ -243,7 +249,7 @@ npm test -## Example 2: Using multiple tools +## Example 2: Writing to a file @@ -267,55 +273,22 @@ npm test - - -./backend-config.yaml - - -database: - host: localhost - port: 5432 - name: myapp_db - user: admin - -server: - port: 3000 - environment: development - logLevel: debug - -externalServices: - emailProvider: sendgrid - storageProvider: aws-s3 - - - -## Example 3: Asking a follow-up question - - - -Which specific feature would you like me to implement in the example.py file? - - -` - -const adf = ` - # Effective Tool Use When using tools to accomplish tasks, follow these guidelines for effective and informed decision-making: -- Prioritize an iterative approach, especially for tasks that involve sequential steps or depend on the outcomes of previous actions. Make tool uses in a logical sequence, waiting for results in the user's message when necessary before proceeding. -- While you can make multiple tool uses in a single response when it's efficient and logical to do so, always ensure each use is well-informed and necessary. Avoid making premature or speculative tool uses. -- After receiving the result of a tool use, analyze it within tags. Use this analysis to inform your next steps and subsequent tool uses. -- For tasks that require multiple steps, make decisions sequentially. Each tool use should be informed by the results of previous uses and your analysis of those results. -- Be prepared to adjust your approach based on the results of each tool use. If a tool use reveals unexpected information, reassess your plan and adapt accordingly. +- Treat each task as a multi-step process. Begin by planning the necessary steps within tags. +- You must only make one tool use at a time, ensuring each tool's output is considered before proceeding to the next. +- Do not assume the outcome of any tool use. Each step must be informed by the previous step's result. +- After receiving the result of a tool use, analyze the result and decide the next step based on this information. -## Example of Effective Tool Calling +## Example of Effective Tool Use + +1. User message: [task] + +2. Assistant message: Analyze task and formulate initial plan -1. Analyze task and formulate initial plan - The task requires improving a website's appearance. First, I need to see the current state of the site and check its CSS. - @@ -323,16 +296,23 @@ http://localhost:3000 +3. User message: [inspect_site results] + +4. Assistant message: Analyze the result and plan next step + +The screenshot shows [analysis of the site's current appearance]. To proceed, I need to examine the CSS file. + src/styles/main.css -2. After receiving screenshot and CSS content, analyze and plan changes - -The screenshot shows [analysis of the site's current appearance]. The current CSS reveals [analysis of CSS structure and styles]. To improve it, I should modify the following aspects: [list of improvements]. I'll now update the CSS file. - +5. User message: [read_file results] + +6. Assistant message: Analyze CSS content and plan changes + +The current CSS reveals [analysis of CSS structure and styles]. To improve it, I should modify the following aspects: [list of improvements]. I'll now update the CSS file. @@ -343,10 +323,11 @@ src/styles/main.css -3. Present the result to the user - +7. User message: [write_to_file results] + +8. Assistant message: Confirm changes and present result to user + Now that I've updated the CSS, I should present the result to the user and provide a command to open the site for them to see the changes. - @@ -357,5 +338,5 @@ open http://localhost:3000 -By following this approach, you make informed decisions at each step, using multiple tool calls when efficient, but always ensuring each action is based on current information and logical progression of the task. +By following this approach, you are able to make informed decisions at each step, using tools one at a time and ensuring each action is based on the previous step's result and logical progression of the task. `