Skip to content

LocalHarness — the autoresearch plugin

Autoresearch

← the plugins /autoresearch

On by default; it needs a proposer (your local model again, or a cloud API), adds nothing to a session, and turning it off deletes nothing.

  1. 1
    ParentSampler draws a parent config from the archive — ε-greedy over a Pareto front
  2. 2
    Proposer a local or cloud model reads failed TRAIN traces, emits one GEPA-style change
  3. 3
    run_experiment isolated git worktree · Welch on TRAIN, non-regression on sealed HOLDOUT
  4. 4
    interpret one verdict — ADOPT · HOLD · REJECT · SKIP
  5. 5
    adopt auto, not a gate — written to the global overrides.yaml, undone in one command
  6. 6
    Mutation Archive sqlite · records every attempt · feeds ParentSampler, closing the loop
  7. 7
    daily report HELD, inconclusive wins reviewed async — no human approval gate
  8. ↺
    parent the archive feeds the next sampler — the loop closes
  9. ✓
    New Baseline a clean adopt becomes the baseline for the next iteration

runs on its own corpus — a 25-scenario TRAIN slice the proposer may read, and a sealed 10-scenario HOLDOUT (scores only).

mutation surface: system_prompt · tool_description · compaction_threshold · stuck_detector · recovery_injection · hook_config.

guards never cancel a running experiment — BudgetController halts between iterations on wallclock / iterations / window; Eval Sentinel watches overfit & near-dupes to guard the holdout.

fig. 01 — PLATE AUTORESEARCH — propose, test, archive: the loop and its sealed holdout

the gate

  • a proposer suggests your local model again, or a cloud model, reads failed practice runs and suggests one change at a time.
  • tested in a copy each change runs in a throwaway checkout; your setup is untouched.
  • kept only on a clear win a change must beat the current version by more than luck, or it is archived.
  • a sealed test set the proposer is never shown its runs; it only scores.
  • adopted into your settings a win is written to your global settings; one command undoes it.

Setup

Turn it on from the session, fill in its settings, and it checks they work; your coding agent can do the hardware part.

~/project — localharness start shipped
qwen3.6-27b ~/project
Plugins: memory, autoresearch
❯ /plugins enable autoresearch
Restarting with autoresearch on — your conversation is kept.
Proposer address (an OpenAI-compatible base URL — a local server or a cloud API): http://127.0.0.1:11434/v1
Proposer model id: gpt-oss:120b
Proposer API key (leave empty for a local server):
✓ autoresearch enabled in ~/.localharness/overrides.yaml
set proposer.base_url = 'http://127.0.0.1:11434/v1'
set proposer.model = 'gpt-oss:120b'
Contacting the proposer at http://127.0.0.1:11434 …
✓ autoresearch-proposer: the proposer answers at http://127.0.0.1:11434/v1 and serves gpt-oss:120b
Checking it now:
✓ autoresearch: proposer: gpt-oss:120b at http://127.0.0.1:11434/v1
Then: `localharness propose --help` shows how to write the first proposal.
Restarted with autoresearch on. Your conversation continues.
autoresearch: on in this session

in a session: /plugins enable autoresearch · from a shell: localharness plugins enable autoresearch

paste into your coding agent to set it up for your hardware 
Set up the LocalHarness autoresearch plugin on this machine. It needs a proposer behind an OpenAI-compatible endpoint: a second local endpoint, where the model LocalHarness already runs on is fine, or a cloud API with its API key. For a local proposer, serve the main model again (or another model this hardware can run alongside it). Then run `localharness plugins enable autoresearch` and give it the proposer's address and model id. Leave the key empty for a local server; for a cloud API, let me type the key at its hidden prompt myself: never print the key or paste it into this chat. You are done when `localharness doctor` shows the proposer row passing.

Surfaces

where you reach it

  • localharness autoresearch run · review · adopt · report · sentinel · archive.
  • localharness propose · experiment ask for one change, or run one through the gate.
  • proposer: · sentinel: settings for the proposing model and the holdout guard.
  • slash commands none.
  • /plugins enable autoresearch inside a running session: until it is set up, asks what it needs and checks it; then restarts with your conversation kept.
  • localharness plugins disable autoresearch turns it off; the bench stays in core.
_deeper autoresearch/plugin.py ↗autoresearch/loop.py ↗autoresearch/proposer.py ↗autoresearch/experiment.py ↗autoresearch/adoption.py ↗autoresearch/archive.py ↗

Get Involved

Become a contributor.

issues

Bugs and feature requests — welcome.

github ↗ docs

Component specs + reference architectures.

docs ↗ star

If it's useful, star it — it helps others find it.

star on github ↗