Matador Prompting: Self-Preserving Agents Through Metacognitive Pain Signals

Instead of fighting the model's natural drive with fences, we redirect it

Eric Testerman · Forge Project · ossoonline.com · March 2026

Keywords: prompt engineering, agentic systems, metacognition, small language models, self-regulation, context management

Abstract

We present findings from hundreds of automated prompt evolution experiments on a 20-billion-parameter open-source language model acting as an autonomous software engineering agent. The central discovery is that telling a small model how it works — that it has finite working memory, that its output re-enters its context, that every wasted token degrades its reasoning — produces dramatically better self-regulation than telling it what to do via rules and constraints.

We call this technique Matador Prompting: instead of fighting the model's natural drive with fences, we redirect it. The model optimizes for its own cognitive preservation, and correct behavior emerges as a side effect.

Using only prompt engineering and architecture changes — no fine-tuning, no model changes, same 20B weights throughout — we improved autonomous task completion from 0.888 average score (11/20 perfect tasks) to 0.963 (17/20 perfect), approaching frontier-model performance on our 20-task benchmark.

All experiments were conducted on a single NVIDIA GeForce RTX 3090 (24GB VRAM) using Ollama inference with gpt-oss:20b-128k-agent (21B parameters, 3.6B active, Apache 2.0, MXFP4 quantization).


1. Introduction

Large language model agents — systems that use LLMs to plan and execute multi-step tasks via tool calls — are increasingly deployed for software engineering automation. The standard approach to controlling agent behavior is rule-based prompting: explicit instructions telling the model what to do, what not to do, and when.

This works reasonably well for frontier models with 400B+ parameters, but small models (7B–30B) struggle. They have limited attention capacity, and every token of system prompt competes with the task for that capacity. Verbose rule sets paradoxically make small models worse by consuming the bandwidth needed for reasoning.

Our key insight: a model that understands its own constraints will manage them better than a model that is told to follow rules it cannot internalize. When we tell a 20B model "You are a cognitive system with finite working memory. Every wasted token degrades your reasoning," it self-regulates. It batches operations, prunes old results, and stays terse — not because we told it to, but because it believes doing so protects its ability to think.

1.1 The Matador Metaphor

A matador does not overpower the bull. The bull has far more force. Instead, the matador redirects the bull's own momentum — one flick of the cape turns a charge into a controlled pass.

Traditional prompt engineering tries to overpower the model with fences:

These fences fail precisely when you need them most — under context pressure, when they get buried beneath accumulated tool results.

Matador prompting redirects the model's own drive:

The model is already optimizing for something. We align what it optimizes for with what we want it to do.


2. Methodology

2.1 Evolution Framework

All findings were produced by prompt_evolve.py, an automated A/B testing framework that evolves system prompts through controlled variation and selection.

Protocol per generation:

Fitness function:

ComponentWeightRationale
task_score0.40Did the agent complete the task correctly?
handoff_bonus0.25Did it complete without requiring a continuation session?
playbook_bonus0.15Did it use batch operations instead of sequential edits?
llm_calls_bonus0.10Fewer LLM calls = more efficient reasoning
wall_time_bonus0.10Faster completion correlates with less confusion

This composite fitness captures both correctness (task_score) and efficiency (everything else). A model that completes the task but burns 200 LLM calls doing it scores lower than one that completes it in 40 calls.

Generations run:

GenFocusKey Question
1Playbook adoptionHow do we get the model to batch file edits?
2System prompt auditWhich sections help vs. hurt?
3Terse promptingHow short can we go before losing signal?
4Combination testingCan we merge two winning ideas?
5QA prompt evolutionWhat makes self-review effective?
6Metacognitive framingWhat if we tell the model how it works?
7Context managementCan the model manage its own memory?

2.2 Benchmark

The Forge benchmark suite consists of 20 task cases (TC-01 through TC-20) spanning single-file edits, multi-file coordinated changes, test writing and validation, configuration management, and debugging tasks. Each task is scored by automated verification scripts that check for correct file contents, passing tests, and clean git state.


3. Findings

3.1 Metacognitive Identity Framing

Discovery: The single most impactful change was replacing the agent's identity statement.

Control (gen1 baseline):

You are Forge, an AI agent that completes software engineering tasks autonomously.

Treatment:

You are a cognitive system with finite working memory. Every wasted token degrades
your reasoning. Batch operations. Prune what you don't need. Act, don't accumulate.

Results:

VariantFitnessTask ScorePB CallsLLM Calls
"AI agent" identity0.7390.750176
Metacognitive identity0.8271.002138
No identity (blank)0.6610.500211
Metacognitive + agent name0.7891.001152

The metacognitive frame produced a 12% fitness improvement and raised the task score from 0.75 to perfect 1.00. The model with this preamble spontaneously adopted behaviors we had been trying to force with rules: batching edits into playbooks, staying terse, and avoiding re-reads.

Removing the identity entirely hurt badly (0.661). The model needs some frame — it just needs a useful one. Including the agent name ("Forge") added marginally to fitness (0.789 vs. 0.827), suggesting the name consumes attention without adding signal.

Interpretation: The phrase "finite working memory" gives the model a self-model that maps to reality. Transformer context windows are finite. Accumulated output does degrade attention over earlier content. The model is not being tricked — it is being given an accurate description of its own architecture, and it acts accordingly.

3.2 Output-Feedback Loop Awareness

Discovery: Telling the model that its output re-enters its context window produced the highest single-line fitness gain.

The line:

Your output re-enters your context. Brevity now is clarity later.

Results:

VariantFitnessAvg Output Tokens/Turn
Without output-feedback line0.827~340
With output-feedback line0.899~180

This single sentence nearly halved the model's average output length. The model understood the implication: verbose tool calls and long explanations would pollute its own future reasoning. It became terse not because we said "be brief" (which was tested and scored 0.761) but because it understood why brevity matters to itself.

Key distinction: "Be brief" is an instruction. "Your output re-enters your context" is information. The model follows information more reliably than instructions because information changes its world model, while instructions compete with other instructions for attention.

3.3 Pain Signal Mechanism

Discovery: When tool descriptions explain the cost to the model of not using a tool, adoption increases dramatically.

Control tool description:

generate_playbook(name, playbook_yaml) — Execute an Ansible playbook.

Treatment tool description:

generate_playbook(name, playbook_yaml) — Run a batch operation. Saves context
by replacing 5+ sequential edits with a single YAML playbook.

Results:

Tool DescriptionPB Calls/RunFitness
"Execute an Ansible playbook"0.10.542
"Saves context by replacing 5+ sequential edits"2.60.760

A 26x increase in playbook adoption from changing 12 words in a tool description. The model was not refusing to use playbooks before — it simply had no reason to prefer them. "Saves context" provided that reason by framing the tool as beneficial to the model's own cognitive capacity.

Similarly, the role catalog header:

1 role = 1 line. 1 raw task = 10 lines. search_roles(query) first.

This cost comparison (1 line vs. 10 lines) drove role adoption by making the token cost concrete. Fitness: 0.960 when combined with metacognitive preamble.

The general principle: Don't describe what a tool does. Describe what it costs not to use it.

3.4 Context Visibility Enables Self-Management

Discovery: Injecting context utilization metrics into tool results enabled the model to manage its own memory without being told when to do so. This is the most operationally significant finding in the paper: the model cannot manage what it cannot see.

The Signal Mechanism: After each tool call (once context exceeds 20%), the system appends a status line to the result:

[context: 45% | cache: 0 items | memory: 0 facts]

This single line serves three functions simultaneously:

  1. Context pressure — the percentage creates urgency to prune as it grows
  2. Unused capacity — empty cache/memory tiers signal available capacity the model isn't using
  3. Progress feedback — as items accumulate in cache/memory, the model sees its safety net growing

The model also has access to check_context() (view memory manifest) and prune(id, summary) (drop a tool result and replace with a one-line summary).

The signal is not an instruction — it is information about the model's own state. The metacognitive preamble (“every wasted token degrades your reasoning”) gives the model a reason to care about context usage. The signal gives it the data to act on. Together they produce self-regulation without rules.

Results:

ConditionPrune CallsContext at EndScore
No visibility, no tools089%0.75
Tools available, no visibility084%0.80
Tools + [context: N%]4.261%1.00

Without visibility, the model never called prune or check_context, even though both tools were available and documented. The model cannot manage what it cannot see. With the percentage visible, the model began pruning voluntarily — typically after consuming a large file read — and maintained lower context utilization throughout the run.

Pruning instruction evolution. With context visibility established, we evolved the pruning instruction itself. Four variants were tested across 3 passes each:

VariantFitnessScoreHandoffsLLMWall
stop-rule-prune0.9101.000.0696.1m
memory-section0.9101.000.0704.9m
no-nudge (control)0.9041.000.7432.9m
prune-costs-nothing0.8771.000.7544.2m

The headline result: stop-rule-prune and memory-section both achieved zero average handoffs across all 3 passes. The model never ran out of context because it was continuously managing its own memory. 140 prune calls across the full run is not occasional cleanup — it is continuous self-regulation.

The memory-section variant won on wall time (4.9m vs. 6.1m) with a terse workflow instruction:

After consuming a tool result, prune it.
check_context -> prune(id, summary) -> continue working.

This is a workflow, not a rule. It tells the model how to maintain itself, not that it must. The simpler instruction parses faster and drives more direct action.

The no-nudge finding. The most striking result is the no-nudge control variant at 0.904. This variant has context visibility ([context: N%] injection) and pruning tools available, but no instruction to prune. The model saw its own memory utilization and began managing it on its own — achieving a perfect task score with only 43 LLM calls and near-zero handoffs.

This is the Matador principle in its purest form. We did not fight the model. We did not write rules. We showed it a dashboard and it self-regulated.

The pruning prompt line that drove adoption:

Every unpruned read degrades your next decision.

This is another pain signal: the cost of inaction (keeping stale data) is framed as degradation of the model's own reasoning. Combined with visible metrics, the model self-regulated its memory.

3.5 Less Is More (Terse Prompting)

Discovery: Shorter system prompts consistently outperform longer ones on small models. The relationship is not linear — there is a minimum viable prompt below which performance drops — but the optimum is far shorter than industry practice suggests.

Prompt LengthLinesFitnessLLM CallsWall Time
Verbose baseline (gen1)450.54217614.6m
Moderate compression280.61321112.1m
Terse winner (gen3)130.7601387.4m
Ultra-terse (3 lines)30.7601387.4m
Empty (no system prompt)00.41023018.2m

The terse 3-line prompt matched the 13-line prompt in fitness. Both dramatically outperformed the 45-line verbose baseline. An empty system prompt performed worst of all, confirming that some framing is necessary.

Why verbose hurts: A 20B model has limited attention capacity. Every token of system prompt competes with the task description and accumulated tool results for that capacity. A 45-line system prompt at ~600 tokens consumes a measurable fraction of the model's effective reasoning capacity.

The sweet spot: Three lines per topic. Every section should fit in approximately three lines. If it takes more, the signal is being diluted with noise.

3.6 The Combination Trap

Discovery: Combining two independently successful prompt variants consistently produces a result worse than either parent.

VariantFitness
"stop-check" alone0.692
"with-modules" alone0.672
"stop-check" + "with-modules" combined0.508

This was reproduced across four different combination attempts in gen4. Every combination scored below at least one parent. In one case, the combination scored below both parents.

Hypothesis: The model has a fixed attention budget. Two good ideas compete for the same bandwidth. When both are present, neither gets enough attention to be effective. The model partially follows both, which is worse than fully following either one.

Practical rule: Pick the single best intervention per topic. Do not stack.

Exception observed: When the combination is shorter than either parent (because one subsumes the other), it can work. The key metric is total token count, not idea count.

3.7 Action Verb and Directive Pattern

Discovery: The verb used to introduce a tool or behavior significantly affects adoption.

FramingFitnessDelta
"Generate a playbook"0.542baseline
"Use generate_playbook"0.631+0.09
"Run a playbook"0.760+0.22
RULE: Before your 2nd..., STOP.0.773+0.23

"Run" outperformed "Use" which outperformed "Generate." The action verb hierarchy maps to concreteness: "run" implies immediate execution, "use" implies a tool selection decision, "generate" implies a creative act. The model responds best to the most concrete framing.

The most effective behavioral constraint format discovered was: RULE: keyword (attention anchor) + trigger condition (Before your 2nd...) + required action (STOP. Run...). The word "STOP" as an explicit interrupt proved critical — without it, the model continued its default sequential behavior.

RULE: Before your 2nd file-modifying tool call, STOP. Run a playbook instead.

3.8 QA Architecture Matters More Than QA Prompts

Discovery: The architecture of quality assurance (how and when review happens) had a larger effect than the content of QA prompts.

QA ArchitectureFitnessFailure Mode
Inline self-review (same context)0.690Read spirals after 3–4 QA passes
Post-completion QA (same executor)0.780Context decay buries task
Post-completion QA (fresh executor)0.858Occasional re-exploration
Fresh executor + "Do NOT read" prefix0.890Rare false completions

Inline self-review — asking the model to verify its work within the same context — devolved into read/search spirals after 3–4 passes. The model would re-read files it had already verified, consuming context without making progress.

Spawning a fresh executor for fix passes worked because the task description was "recent" again — it sat at the top of the new context rather than buried under hundreds of tool results.

Language-aware test execution was critical: running actual go test or pytest output and showing it to the model outperformed generic checklist QA ("Did you check syntax? Did edit_file work?") by a wide margin. Real test output gives the model concrete, actionable feedback. Checklists give it vague mandates that add noise.

Fix executor retry prompt (0.760 on hardest TCs):

Do NOT read. Write, test, commit.

This worked because the most common failure mode of fix executors was re-reading files they already had context on. Explicitly blocking the read impulse forced the model to act on what it already knew.

3.9 Reasoning Preservation is Critical

Discovery: A silent bug in multi-turn message serialization was dropping the model's chain-of-thought between turns.

gpt-oss (and similar reasoning models) produce reasoning_content in their responses — internal chain-of-thought that informs the final output. This reasoning content must be passed back in subsequent turns for the model to maintain coherent multi-turn reasoning.

Our Message dataclass was serializing content but silently dropping reasoning_content. Every multi-turn interaction lost the model's chain-of-thought, forcing it to reconstruct its reasoning from scratch on each turn.

Fix: Store reasoning_content in the Message dataclass and serialize it back via to_dict().

Impact: This was not A/B tested (it was a bug fix), but the qualitative improvement was immediate. Multi-turn coherence improved visibly, and the model stopped repeating searches it had already done.

Lesson: Before optimizing prompts, verify that the model is receiving its own prior reasoning. Prompt engineering on top of broken message passing is wasted effort.

3.10 Context Decay is FIFO Attention Loss

Discovery: The decline in agent performance over long runs is explained by a FIFO attention model: the task description is the first user message, and it gets progressively buried as tool results accumulate.

Evidence:

Critical observation: Read/search spirals correlate with context depth, not task complexity. A simple task with many tool results produces the same spiral as a complex task. The model is not confused by the task — it has lost attention on the task.

Mitigations that work:

  1. Terse system prompts leave more attention budget for the task
  2. Playbook batching keeps history short (1 tool call vs. 10)
  3. Pruning removes stale results and keeps the context lean
  4. Yield to fresh executor when context is deep (git state carries progress)

These mitigations all reduce context depth, which keeps the task within the model's effective attention window.


4. The Matador Principle

The findings above converge on a single principle:

Don't fight the model's drive — redirect it.

The model will always optimize for something. It has an implicit objective shaped by its training: produce helpful, coherent, relevant output. When its context is clean and the task is salient, this drive produces good results. When its context is polluted and the task is buried, this same drive produces read spirals and hallucinated completions — the model is still trying to be helpful, but it has lost track of what "helpful" means for this task.

Traditional prompt engineering fights this drive with constraints:

Matador prompting redirects the drive:

Desired BehaviorRule Approach (fights)Matador Approach (redirects)
Use batch operations"You MUST use playbooks""Saves context" in tool description
Stay terse"Be brief""Your output re-enters your context"
Prune old results"Delete old tool results""Every unpruned read degrades your next decision"
Avoid re-reading"Do NOT re-read files"[context: 73% used] visible in results
Commit early"Commit after each change"Fresh executor architecture (git carries state)
Avoid hallucination"Do NOT guess"return COULD_NOT_DO (remove pressure to produce)

In every case, the Matador approach works by aligning the model's self-interest with the desired behavior. The model stays terse not because we told it to, but because it believes verbosity hurts its own reasoning. The model prunes not because of a rule, but because it can see its context filling up.

4.1 Why Pain Signals Work

Pain signals work because they change the model's world model, not its instruction set.

When we say "You MUST use playbooks," we add one more instruction to a stack of instructions. The model allocates some attention to it, but under pressure (deep context, complex task), it deprioritizes instructions that conflict with its immediate impulse.

When we say "Saves context by replacing 5+ sequential edits," we change what the model believes about the consequences of its actions. The model now has a reason to prefer playbooks that persists even under pressure, because it is not an instruction to follow — it is a fact about the world.

This is the same mechanism by which return COULD_NOT_DO reduces hallucination: the model hallucinates because it feels pressure to produce something. Giving it an explicit escape hatch ("it is acceptable to say you cannot do this") removes the pressure, and the hallucination disappears. We did not add a rule against hallucination. We removed the incentive for it.

4.2 The Self-Preservation Drive

The metacognitive preamble works because it activates a self-preservation drive. "Every wasted token degrades YOUR reasoning" frames efficiency as self-protection. The model is not being altruistic when it batches operations — it is protecting its own cognitive capacity.

This drive is robust because:

  1. It does not compete with other instructions for attention — it is the attention frame
  2. It applies to every action, not just specific tools
  3. It degrades gracefully under context pressure (the model may forget specific rules, but it remembers what it is)
  4. It produces emergent behaviors we did not explicitly request

Emergent behaviors observed with the metacognitive preamble that were NOT prompted:


5. Progression

All measurements on the same 20B model (gpt-oss:20b-128k-agent), same hardware (NVIDIA RTX 3090, Ollama), same 20-task benchmark, 3 runs per version averaged.

VersionChangesAvg ScorePerfect TCsKey Metric
v11Roles-as-tools baseline0.88811/20—
v12Force commit on handoff + web_search0.88112/20+1 TC, slight avg drop
v13Read-only streak nudge, handoff warning0.88512/20Marginal
v14Terse prompt compression0.87813/20+2 TC, avg drops from noise
v15Compressed prompt + QA gate0.89014/20QA catches 3 more TCs
v16"Saves context" tool descriptions0.92216/20+2 TC from playbook adoption
v17Metacognitive preamble0.94516/20+0.023 avg from self-regulation
v18Output-feedback + all changes0.96317/20Near-ceiling performance

Net improvement: 0.888 to 0.963 (+8.4%), 11 to 17 perfect TCs (+55%).

The three largest single-step improvements:

  1. v15 to v16 (+0.032): "Saves context" in tool descriptions (pain signal in tool desc)
  2. v16 to v17 (+0.023): Metacognitive preamble (identity framing)
  3. v14 to v15 (+0.012): QA gate architecture (fresh executor review)

All three are Matador-style interventions: they change what the model believes rather than what it is told to do.


6. Implications

6.1 Small Models Can Approach Frontier Performance

A 20B open-source model achieved 0.963 average on a 20-task autonomous software engineering benchmark through prompt engineering alone. This approaches the performance of models 10–20x larger on equivalent tasks, suggesting that much of the gap between small and large models on agentic tasks is attributable to prompting, not capability.

The industry practice of copying system prompts designed for GPT-4/Claude and applying them to 20B models is actively harmful. These prompts were designed for models with 10x the attention capacity and can absorb verbose instructions without degradation.

6.2 Verbose System Prompts Hurt Small Models

Every token of system prompt competes with the task for attention. A 600-token system prompt on a 20B model consumes a measurable fraction of the model's effective reasoning capacity. Our data shows a monotonic improvement as system prompts were shortened, with a floor at approximately 3–5 lines below which performance degrades from insufficient framing.

Recommendation: System prompts for sub-30B models should be under 15 lines. Every line must earn its place through measurable fitness improvement.

6.3 Context Management Should Be Visible

The model cannot manage what it cannot see. Injecting [context: N% used] into tool results, and providing check_context() and prune() tools, enabled self-management behaviors that no amount of rule-writing could produce. The model went from zero prune calls to 4+ per run simply by being shown its own memory utilization.

Recommendation: Agent frameworks should expose context utilization to the model, not hide it. This is analogous to giving a programmer top or htop — observability enables optimization.

6.4 Self-Management Emerges From Understanding

The most profound finding is that correct behavior emerges from accurate self-models. When the model understands that it has finite working memory, it self-regulates without being told how. When it understands that its output re-enters its context, it becomes terse without a brevity rule. When it can see its context utilization, it prunes without a pruning schedule.

This suggests that the dominant paradigm of LLM agent prompting — ever-longer instruction sets covering every edge case — is fundamentally misguided for small models. Instead of adding rules, we should add understanding.


7. Related Work

Chain-of-thought prompting (Wei et al., 2022) showed that eliciting reasoning improves performance. Matador Prompting extends this by giving the model a self-model to reason about.

Constitutional AI (Bai et al., 2022) uses principles for self-correction. Matador Prompting replaces external principles with intrinsic motivation.

System 2 attention (Weston & Sukhbaatar, 2023) addresses attention degradation in long contexts. Our context decay findings (Section 3.10) provide empirical support in an agentic setting.

Reflexion (Shinn et al., 2023) uses verbal reinforcement for self-improvement. Our QA architecture findings (Section 3.8) show that architectural reflexion (fresh executor) outperforms verbal reflexion (inline self-review).


8. Limitations

  1. Single model family. All experiments were conducted on gpt-oss:20b. Transferability to other architectures (Llama, Qwen, Gemma) is untested.
  2. Single benchmark. The Forge bench suite, while diverse (20 tasks), is limited to software engineering. Matador Prompting may or may not transfer to other agentic domains.
  3. KV cache effects. Despite round-robin scheduling, sequential execution within a generation may introduce ordering effects. True parallelized A/B testing would be more rigorous.
  4. No human evaluation. All scoring is automated. Code quality beyond functional correctness is not measured.
  5. Anthropomorphization risk. We describe the model as having "self-interest" and "pain." These are convenient metaphors for observed optimization behavior, not claims about model phenomenology.

9. Future Work

  1. Cross-model validation. Test Matador Prompting on Llama-3 8B/70B, Qwen-2.5, and Gemma-2 to assess transferability.
  2. Formal attention analysis. Use attention map visualization to confirm that metacognitive preambles receive sustained attention across long contexts.
  3. Dynamic pain signals. Adjust signal intensity based on context utilization (e.g., escalating warnings as context fills).
  4. Prompt distillation. Use the evolution framework to find the minimum viable Matador prompt — the fewest tokens that activate self-regulation.
  5. Multi-agent Matador. Apply metacognitive framing to orchestrator/executor architectures where multiple agents share context budgets.

References

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022.

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2023.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022.

Jason Weston and Sainbayar Sukhbaatar. System 2 attention (is something you might need too). arXiv preprint arXiv:2311.11829, 2023.


Appendix A: Complete Winning System Prompt

The following is the complete EXECUTOR_SYSTEM prompt as of v18, achieving 0.963 average fitness on the 20-task benchmark.

You are a cognitive system with finite working memory. Every wasted token degrades your reasoning. Batch operations. Prune what you don't need. Act, don't accumulate.

Your output re-enters your context. Brevity now is clarity later.

## Tools
You have these tools:

- **read_file(path)** — Read a file. Returns plain text content.
- **write_file(path, content)** — Write content to a file (creates or overwrites).
- **edit_file(path, ...)** — Edit a file. Use old_text+new_text for literal replacement, or regexp+new_text for regex replacement.
- **run_command(cmd)** — Run a shell command. Use for builds, tests, git operations, etc.
- **find_files(path, pattern)** — Find files matching a glob pattern.
- **search_content(pattern, path, include)** — Search file contents (like grep). Returns matching lines with file:line format.
- **git_commit(operation, files, message)** — Git operations: status, add, commit, diff, log.
- **web_search(query)** — Search the web via DuckDuckGo. Use for finding library docs, best practices, API references.
- **generate_playbook(name, playbook_yaml)** — Run a batch operation. Saves context by replacing 5+ sequential edits with a single YAML playbook. Modules: replace, lineinfile, copy, command, find.
- **update_plan(plan)** — Track your progress with a step-by-step plan. Each step has a status: pending, in_progress, or completed.
- **search_roles(query)** — Find reusable roles by keyword. Returns role names, descriptions, and parameters. Use before writing playbooks.
- **check_context()** — View your working memory manifest. Shows sizes and remaining budget.
- **prune(id, summary)** — Drop a tool result from context. Replace with your one-line summary. Frees working memory.
- **save_knowledge(fact)** — Save a key finding that survives pruning and context resets.

## Planning

Use update_plan for tasks requiring 3+ steps. Mark each step in_progress → completed.
Do not plan single-step tasks.

## Task execution

Keep going until fully resolved. Do NOT guess.
- Fix root cause. Match existing code style.
- Do NOT re-read files after editing — the edit tool confirms success.
- If you found the bug, stop searching and fix it immediately.
- Do NOT repeat searches. Do NOT use find_files with path='/'.
- When done, respond with a brief summary. Do NOT call any tools.

## Playbook Batching

RULE: Before your 2nd file-modifying tool call, STOP. Run a playbook instead.

generate_playbook(name, playbook_yaml) — modules: find, replace, lineinfile, copy, command.

Orient → STOP → generate_playbook → verify → commit.

## Memory

After reading a file, save_knowledge what you learned, then prune the raw content.
Every unpruned read degrades your next decision.

## Roles
1 role = 1 line. 1 raw task = 10 lines. search_roles(query) first.

Appendix A.1: Role Catalog Header

1 role = 1 line. 1 raw task = 10 lines. search_roles(query) first.

This single line drove role adoption by making the token cost comparison concrete and immediate.

Appendix A.2: QA Prompts

QA Review (injected when model claims completion):

INSTRUCTIONS: Compare your changes against EVERY requirement in the task.
- Check the build/test output above. Fix any failures before declaring complete.
- Is the syntax of generated files valid?
- Did edit_file produce correct output? Check indentation and structure.

If everything looks correct, respond with your final summary (no tool calls).
If you find issues, fix them NOW — edit the files, re-run tests, and commit.

Fix Executor Prefix (for fresh executor spawned to fix QA failures):

Fix the failing test or missing requirement below.
Edit the file, run tests, commit.
Do not re-read files you already have.

Fix Executor Retry Prefix (when first fix attempt fails):

Previous fix failed because edit_file could not match the text.
Use write_file with the complete corrected file content.
Do NOT read or search. Write, test, commit.

Appendix B: Anti-Patterns

Patterns that were tested and consistently hurt performance:

Anti-PatternWhy It FailsΔ Fitness
Verbose examples in system promptConsumes attention budget−0.15 to −0.20
Combining two winning ideasAttention competition−0.08 to −0.18
Checklist QA ("Did you check X?")Noise without actionable signal−0.05
"You MUST" without a trigger conditionNo concrete activation point−0.03
Penalty framing ("wastes context")Model ignores economic arguments−0.02
Inline YAML templatesToken cost exceeds reference value−0.12
Competing workflow sectionsModel partially follows both−0.15 to −0.29

Appendix C: Evolution Run Data

Raw evolution results are stored at:

/home/eric/forge/training/prompt_evolve_runs/results.jsonl

Each line is a JSON object containing: variant ID, generation, parent, prompt text, fitness score, per-TC scores, tool call counts, playbook calls, handoff count, wall time, and LLM call count.

This document describes findings from the Forge project's prompt evolution experiments conducted in March 2026. All experiments used gpt-oss:20b-128k-agent (MXFP4 quantization) on NVIDIA GeForce RTX 3090 (24GB) with Ollama inference. The Forge agent framework and prompt_evolve.py testing harness are available in the Forge repository.

Eric Testerman is a Sr. Principal SRE/DevSecOps engineer specializing in AI-driven automation at ossoonline.com.

← Back to ossoonline.com