Evolution & recalibration

The strategy-creation loop writes ideas and builds strategies. Evolution decides what happens to those strategies next, and keeps the survivors honest over time. This page is for operators running the autonomous loop.

Three mechanisms do the work:

• The testing cycle — the autonomous lifecycle that walks a strategy from quick screen through paper toward live, and retires it when it decays.
• The diversity guard — steering that stops the agents from building the same strategy family over and over, or re-mining families that keep failing.
• Regime-aware recalibration — a background re-tuner that adjusts parameters when the market regime shifts, but only when a backtest proves the change is better.

The testing cycle

Evolution drives the lifecycle described in the pipeline, autonomously. Each run of the testing cycle (forven-testing-cycle, every 60 minutes on the Balanced preset) advances eligible strategies, together with the gauntlet workflow's own step loop:

1. quick_screen — the first filter. Terminal failures (a duplicate of an active strategy, an overfit reject, too few trades with an undefined Sharpe) archive the strategy with no retry. A strategy that can't be fairly tested — no trades, too few to judge, a data-quality hold — is archived as untestable instead, which isn't a merit verdict.
2. gauntlet — the readiness steps (timeframe sweep, optimization, apply best params, confirmation backtest, validation suite). When every test has a saved verdict, the door into paper runs. By default that is the research door, which records the selection-bias checks instead of blocking on them. (Under Strict paper, the door also submits the one-shot held-back test if it is missing.) See the gauntlet.
3. paper — on passing the door the strategy is promoted (automatically, by default) to paper trading: simulated execution against the live feed, no real capital. When the paper pool is full, it waits in the paper queue instead.
4. live — after the paper soak, forven-paper-graduation checks the strict paper → live gate: by default a sequential test on the forward trades, the breadth test, the checks the door recorded and, last, the one-shot held-back test. A survivor is never taken live by evolution: going live needs your typed GO LIVE and a per-asset notional ceiling. Live is unsupported in the open-source build.
5. retirement — a strategy that stops earning its place is archived. The paper pool's hourly job retires paper strategies whose forward record shows them losing; the decay kill-switch retires live ones.

The testing cycle is one of the generation jobs: it's paused in Semi and Manual, while the gauntlet step loop keeps running in Semi. See autonomy modes.

Promotion gates

A strategy only advances when it clears the gate for its stage. With the Default stance and the research door, the gauntlet → paper gate looks for at least 20 confirmation trades, a non-negative return, drawdown within 30%, an out-of-sample profit factor of at least 1.0 and Sharpe above 0, no loss under 2× costs, and a Monte Carlo tail drawdown within 40%. The robustness score (30), the walk-forward and parameter-jitter results, the deflated Sharpe (0.90) and the held-back test are recorded there and enforced at paper → live, together with a sequential test on the forward trades and the breadth test. Strict paper enforces the recorded checks before paper instead. These describe the gate's shape, not a performance forecast; promotion gates has the full logic.

Pipeline throughput

How much the testing cycle processes per run is tunable, because a deep backlog can otherwise starve the scheduler:

In drain mode a cycle processes every queued backtest within the time budget and defers the rest. With drain mode off, a cycle returns after one fixed batch. Adaptive mode sizes each cycle to clear the backlog by pipeline_target_clear_hours.

The paper pool tick

Paper is a managed pool: at most 40 strategies at once by default, 3 per market and 10 per family, with a queue for the rest (see the paper pool). An hourly job, forven-paper-pool-tick, keeps it moving. Each pass it:

1. Retires paper strategies whose forward record shows them losing — clearly losing over 15+ trades, a 15% forward drawdown, 60 days without a trade, or a failed sequential test — archiving each as paper_retired:<why>. Live strategies are never touched.
2. Admits the best-ranked strategies from the paper queue into the freed slots.
3. Starts one breadth test: for the paper strategy longest in paper that needs one, then for the queue, best first.
4. Freezes the coin list of any paper strategy whose breadth test now reads General, making it a multi-coin paper strategy.

The sweep that archives stale or low-quality strategies in the quick screen and gauntlet never archives a strategy waiting in the paper queue, and its low-quality Sharpe bar (0.3) applies only under Strict paper.

Retirement and the learning loop

When a strategy reaches a terminal state — archived, retired or live_graduated — Forven looks up the quant skills cited in the runs that built it and nudges their confidence: about +3% on a positive outcome, −5% on a negative one. A strategy that decays out of live slightly lowers confidence in the patterns that produced it. The closure is idempotent, so a re-run never double-counts.

Untestable archives are the exception: they skip the post-mortem and the negative signal, because nothing was learned about merit.

The diversity guard

Left alone, an idea generator will happily keep building the same winning family until the lab is a monoculture, and keep re-mining families that never work. The diversity guard is a section added to the agent's context on every strategy-creation run.

It reads three things:

• Saturation. It looks at the most recent 80 strategies and flags any family above 35% of them as overrepresented ("hard" at 55%), telling the agent to prefer families outside the saturated set.
• Dead regions. Over the last 90 days, a family tried at least eight times where none made money in paper is called out — "Do not propose more of these without a structurally different mechanism."
• What's working. Families with strategies that made money in paper are listed, so the agent knows where survivors came from.

A failure taxonomy rides alongside it: the most common clusters of gate rejections over the last 30 days (family × gate × reason × regime), so the agent designs away from regions the pipeline has already rejected. For example:

# STRATEGY DIVERSITY GUARD
Recent strategy memory is family-skewed. Treat saturated families as
overrepresented prior art, not inspiration.
- RSI / oscillator momentum: 31/80 recent strategies (39%).
- Prefer families outside the saturated set: RSI / oscillator momentum.

The guard doesn't delete or block anything; it only biases what the next idea will be. The thresholds are built-in.

Archetype fingerprinting works at registration time. It classifies each strategy by type, regime class, indicator family and risk profile, then rejects near-duplicates: an exact fingerprint match is always a duplicate, and same-type strategies are flagged at roughly 85% fingerprint similarity with parameter similarity above 80%. The guard governs the mix of families; fingerprinting blocks redundant copies within one.

Regime-aware recalibration

A strategy tuned in a trending market can quietly degrade when the market goes range-bound. Recalibration (forven-recalibration, every 30 minutes) notices the shift and re-tunes — carefully. For each tracked asset it:

1. Detects the current regime and a confidence score (0–1). The regimes are TREND_UP, TREND_DOWN, RANGE_BOUND and HIGH_VOL — see market regimes.
2. Compares it with the last regime it stored for that asset. If unchanged, it skips the asset.
3. On a shift, skips any strategy whose family isn't allowed in the new regime.
4. Computes a regime-aware parameter overlay.
5. Runs two backtests over the same window — current parameters against adjusted ones — and persists the adjusted parameters only if their Sharpe is higher.
6. Logs the result to the activity feed (for example S012 on ETH: sharpe 0.45 -> 0.62) and records the new regime.

Two properties make it safe to leave running:

• Non-destructive. A parameter change is never saved unless a fresh backtest proves it beats the current one.
• It respects capital-stage locks. Parameters of strategies in paper or live are locked against background writers; the recalibrator computes and logs the overlay but refuses to save it.

What you'll see

Evolution runs in the background, so its effects surface across existing pages:

• The Forge and the strategy page — stage changes, gauntlet step status, the held-back test, and parameter updates when recalibration saves new ones.
• The strategy's activity — quick-screen results, gate outcomes and archive reasons.
• The activity feed — testing-cycle summaries and recalibration results per asset.
• Quant skills — the confidence changes that retirement produces, with version history.

Caveats (beta)

• Drain mode loops until a strategy is ready or an error occurs; a no-progress guard stops the loop with a no_progress status if a step runs but readiness doesn't advance. If a strategy is stuck, that status is your clue.
• Regime detection and recalibration are best-effort. A skipped recalibration is normal.

This is research infrastructure for stress-testing ideas, not a money machine. Results from any stage describe the past and aren't predictive. Nothing here is financial advice.

Related