Agent RSI: Execution Diagnostics and Evaluation

What does RSI mean for an agent?

Recursive self-improvement (RSI) describes a system using feedback to improve how it works, then applying what it learns to later iterations. At the agent level, changes can target prompts, skills, tool guidance and runtime settings while model weights stay fixed. Repeating an evaluation alone does not demonstrate self-improvement.

From one execution to a measured improvement

1. 01

   Observe

   Inspect the recorded tool calls, arguments, results, errors and context usage for a completed answer. Available detail depends on the run and model provider.

2. 02

   Diagnose

   Review rule-based signals and model assessments for repeated calls, missing evidence or premature stopping. Treat the suggested cause as a hypothesis to check.

3. 03

   Change

   Prepare a candidate prompt, skill, tool guidance or configuration. Use the editor or authorized CLI operations, and retain the original configuration for comparison.

4. 04

   Evaluate

   Run the candidate against a fixed dataset version and scoring rubric. Compare answers, scores, failures, latency and available usage data before accepting a change.

What ApeMind provides today

The platform provides execution diagnostics, context and usage views, published question-set versions, evaluation runs and reports, and CLI access to supported configuration resources.

An unattended improvement loop requires your own orchestration to select candidates, apply changes, evaluate them and decide what to keep. Persistent lessons across tasks also need a storage and retrieval policy. These steps are not a built-in one-click RSI feature.

Make the improvement claim checkable

• Save the dataset version, scoring questions, model versions, candidate configuration and resource budget alongside each run. A dataset version does not freeze every agent dependency.
• Keep a development set for iteration and a separate held-out set for final assessment. Repeated gains on the development set can be overfitting.
• Report regressions, failed questions and judge errors as well as gains. Repeat comparisons when model variability could change the conclusion.
• For cross-task improvement, record the lesson, its supporting evidence and when to apply it; evaluate whether it helps on later, unseen tasks.

Example workflow: repeated retrieval

Suppose an agent repeats the same search without finding new sources. Inspect the calls, propose a retrieval skill that changes the search angle after an empty result, and evaluate that candidate against the unchanged baseline. Keep it only if the agreed quality and resource criteria are met. This is an illustrative method, not a reported benchmark result.

Questions about agent self-improvement

Build an evidence-based workflow

Detailed product tutorials are in Chinese.

• Inspect execution diagnostics and context usage
• Evaluate changes with a versioned question set
• Organize configuration changes with the CLI
• Read the original Harness² research
• Enterprise RAG and deployment

Start with a baseline you can inspect

Choose representative, non-sensitive questions. Save the initial configuration, run an evaluation and investigate one failure before changing the agent.