Operations & system controls

Operating Forven means answering three questions, often: is it running, how much may it do on its own, and is anything broken. There is no single "ops console" page. The controls live where you already are:

This page walks through the controls in the order you reach for them. Several of them touch trading and autonomy, so read what each does before you click it.

Forven is a research tool. Nothing on this page is a prediction, a recommendation, or financial advice. The controls here govern a research process; they do not create an edge, and no setting guarantees against loss.

System mode: how much may it do on its own?

The Manual / Semi / Auto switch in the control strip sets how much the system may do without you. Each change asks for confirmation.

• Manual freezes all autonomous background work: every scheduled job skips — data collection and the daily database backup included — and autonomous runs queue as paused instead of running. A banner shows how many are waiting. Anything you start yourself still runs.
• Semi — "agents will not write new ideas on their own. Ideas you submit are built into strategies and fully evaluated by the Gauntlet and lifecycle machinery." Semi sets the generation pause, so the Strategy Creation, Validation Cycle and Auto Strategy Intake jobs skip; the Gauntlet Workflow Advancer, the scanner and the rest keep running.
• Auto — "agents write new ideas and build strategies from them within the daily budget, and the pipeline evaluates and promotes them." The budget is the daily strategy-creation budget in Settings → System → Throughput.

Leaving Manual is explicit: switching modes thaws the frozen backlog; there is no back door. The full behaviour of each mode is on autonomy modes.

# Read or set the system mode (operator-gated when an operator key is set)
Invoke-RestMethod -Uri "http://127.0.0.1:8003/api/system/mode"
Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:8003/api/system/mode" `
  -ContentType "application/json" -Body '{"mode": "semi_auto"}'

Halts and the emergency stop

• Trading Allowed / Trading Halted in the control strip is the one-glance answer to "may new live positions open?" — it links to risk. The halt reason shows in a banner underneath.
• Reset Halt appears while trading is halted. It clears the system pause, the kill-switch, and the daily-loss halt together, after confirmation. Understand why it halted first. It does not clear startup exchange recovery — the halt a keyless install shows — which lifts once Hyperliquid credentials are saved and the exchange check passes. See troubleshooting.
• Emergency Halt closes every open live position at market and halts trading. It asks for confirmation. Paper sessions are not touched.

System pause and generation pause (API)

Two older switches remain available over the API for scripting. They are not in the UI.

• System pause — a binary trading halt. POST /api/system/stop sets it; POST /api/system/start clears it. While paused, no new live positions open. It does not freeze research jobs — that is what the system mode is for — and, like every real-capital halt, it never blocks paper. Pausing never rewrites the system mode.
• Generation pause — freezes the three generation jobs: forven-strategy-creation (Strategy Creation), forven-testing-cycle (Validation Cycle) and forven-auto-intake (Auto Strategy Intake). The scanner, the gauntlet workflow advancer and execution keep running. POST /api/system/generation/pause and /resume. This is the same switch as the Semi/Auto choice: pausing generation puts the system in Semi, and resuming it puts it in Auto. In Manual, resuming is refused — change the mode instead.

# Halt new live trading (operator-gated; requires FORVEN_OPERATOR_KEY if set)
Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:8003/api/system/stop"
Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:8003/api/system/start"

A pause does not close positions. Stopping the system, changing the mode, or stopping the backend never exits anything already open. To flatten live exposure, close positions manually or use Emergency Halt. On a default install execution is paper, so there is no real-money exposure — but build the habit now.

Execution mode

The paper/live chip in the control strip is display-only:

• Mode: PAPER — no strategy or bot is trading real money.
• Live ×N (red, links to Live Trades) — N live-stage strategies and live-armed bots can send real orders. It replaces the Mode chip whenever N is above zero.

Next to it, TESTNET or MAINNET names the Hyperliquid network. The in-app and API switch (POST /api/execution-mode) accepts only paper; live is reachable only by deliberate out-of-band settings and is unsupported. See execution modes.

Processes, autopilot, and the scheduler

The Processes tab on the Pipeline page shows what is running right now:

• Active processes — backtests, gauntlet runs, and scans in flight, with progress.
• Autopilot — running, paused, or stopped; active workers against the worker pool; queued jobs; dead letters (jobs that exhausted their retries); health; and the last tick error.
• Recent completions.

Scheduled jobs, including routines, are on Agents → Schedules: every built-in job grouped as Trading, Strategy pipeline, Agents & learning, Market data and Maintenance, with filters for Failing and Off. Click a job's cadence to change it, or use its switch to turn it off. The full job catalog — and enabling, disabling, or re-timing a job — is on the scheduler page (PATCH /api/scheduler/{job_id}, or the CLI scheduler enable|disable|run). Job cadence and worker counts are mostly set for you by the throughput preset in Settings → System → Throughput.

Stale locks recover themselves — usually. If a job hangs, its lock is held in the database. The scheduler force-recovers locks older than the stale threshold, but it will not recover while the worker thread is still alive, to prevent duplicate execution. A job stuck "running" with a live thread usually means a slow external call (a model or the exchange). A watchdog also restarts a live scheduler that has provably stalled. See troubleshooting.

Health and readiness

• Diagnostics runs the health checks — scheduler, workers, providers, and more — with Run checks now, plus the 24-hour cost rollup, MCP server status, and Resumable Runs left over from a previous session.
• GET /api/health is the machine-readable summary (and reports whether the mainnet gate is armed, and for how long).
• The soak report (GET /api/system/soak-report) is a longer-horizon readiness readout — daemon, risk, approvals, tasks, and clustered rate-limit bursts — as ok / warn / critical. Use it to answer "has the lab been stable lately?" before leaving it unattended in Auto.
• Runtime diagnostics show thread health and where work is running: GET /api/system/runtime/threads and /api/system/runtime/health-profile.

Data freshness is judged by one SLA, with an allowance that depends on who reads each series — live, paper, gauntlet, research universe or nothing. Data → Health shows it; the detail lives on health monitoring.

Notifications and the attention system

Forven surfaces what needs you in several layers: sidebar badges (Approvals, Diagnostics, Data, Live Trades, Paper Trades and Bot Factory — Paper Trades is off by default), pop-ups for important events, the dashboard's Needs attention panel, and the Diagnostics inbox. Every alert flows through one routing path with a Pop-up and a Discord switch per event (Settings → Notifications), deduplicated with a cooldown, and a critical alert is never suppressed by a matching non-critical one.

From a notification you can acknowledge it (POST /api/notifications/{id}/acknowledge, or /acknowledge-all), resend it (/resend), or hand it to the Full-Stack Engineer agent as a repair task (/repair-task). The detail lives on notifications.

Steps: a daily ops sweep

1. Look at the control strip: Daemon ACTIVE, WS Live, Trading Allowed, and the system mode you intend — Manual for a hands-on session, Auto only when you mean to leave the lab unattended.
2. Confirm Mode: PAPER and TESTNET. If you see Live ×N or MAINNET, real keys and out-of-band settings are in play at your own risk — stop and confirm that is intentional.
3. Open Pipeline → Processes: workers active, no growing dead-letter count, no persistent tick error. Then Agents → Schedules: nothing under Failing.
4. Open Diagnostics: health checks green, and the 24-hour cost in line with what you expect.
5. Clear the sidebar badges — approvals waiting on you first.
6. Before leaving it unattended, glance at the soak report for slow-building instability.

What you'll see

On a healthy lab: the control strip reads the mode and state you set, the autopilot shows workers active and no dead letters, every scheduler job has a future next run, Diagnostics is green, and the badges are clear or hold only items you have read. When something is wrong, it shows here first — a halt banner, a dead-letter count, a red health check, or a critical notification that refuses to be deduped away.

Caveats

• Paper is the supported mode. Live-only readings stay empty on a supported install.
• Pause is not a flatten. Nothing on this page closes positions except Emergency Halt.
• Leaving Manual requires a mode change. Resuming generation alone does nothing in Manual.
• A held scheduler lock can be legitimate. A "running" job may be waiting on a slow external call.

Related

• Autonomy modes — what Manual, Semi, and Auto each enable in depth.
• Scheduler & jobs — the job catalog and cadence control.
• Diagnostics — health checks, costs, and resumable runs.
• Health monitoring — how component states and data-stream SLAs are computed.