feat: agent guardrails (verify_commit, dispatch audit, opencode guardrails plugin) #110

alee · 2026-10-04T08:45:16Z

alee commented

2026-10-04 08:45:16 +00:00

What

Turns the worker rules that live in prose in AGENTS.md into code a model cannot skip:

scripts/verify_commit.py: one command that answers "is this commit good?" (tests the commit touched, run against the committed tree; ruff new-findings gate), a few lines of output.
scripts/oc_dispatch_audit.py: audits an orchestrator session's task() dispatches (dead dispatch, banned agent, missing WORKTREE line, idle child).
deploy/opencode-plugin/guardrails.js: an opencode plugin whose tool.execute.before hook blocks or rewrites tool calls that break the rules, plus a plan_tick_gate that refuses a plan tick while verify_commit.py fails.
deploy/opencode-plugin/guardrails-replay.mjs: replays real recorded sessions through the plugin before install, read-only, to calibrate false positives. Results are in plans/agent-guardrails-replay.md.
Docs in docs/agent-guardrails.md; example config in deploy/opencode-plugin/guardrails.example.json.

25 files, +8511 / -0. Nothing under src/ or config/ changes; the plugin is not installed by this PR (operator step, below).

Why

Each rule is the code form of something that broke on this repo: 12 dispatches to oh-my-claudecode:writer did no work; task() with no category/subagent_type reported "completed"; a worker with no worktree rewrote the live config.yaml; Atlas pushed and opened a PR unasked; a todo was ticked over 3 failing tests; a git stash moved the owner's uncommitted changes between worktrees. The evidence table is in docs/agent-guardrails.md.

How it was verified

Built through the opencode pipeline (Prometheus plan, Claude review, Atlas execution) over seven audited rounds. Every round's "APPROVE" was re-checked independently, and each audit found defects the previous round missed, so the final state was probed at the real hook boundary rather than trusted from tests:

Full suite on a clean git archive extraction of HEAD: 2605 passed.
python3 scripts/verify_commit.py --repo <wt> --base origin/main --full HEAD: exit 0 (PASS on tests and lint).
node --test deploy/opencode-plugin/: 328 of 328.
Real-shape probes (the hook is called as opencode calls it: input = {tool, sessionID, callID}, output = {args}) for every defect class found along the way, e.g. a category-only task must be allowed; grep -n commit AGENTS.md, pytest -k restore and python3 -c "... > 300" must not be mistaken for git operations or redirects; git -c k=v commit, env VAR=x git stash and git commit -a -m x must still block.
A fresh read-only replay of about 4,800 real recorded tool calls: 28 blocks, 26 true positives, 2 false positives accepted with reasons; bash_protected_port has zero blocks.

Known gaps (accepted, documented)

bash -c '...' / sh -c '...' wrappers are not inspected.
A script file that calls port 8080 internally is not detected.
A commit message that names localhost:8080 is blocked by bash_protected_port.
A relative cd chain after an absolute one can be mis-anchored (1 replay block).

Operator steps after merge (not part of this PR)

cp deploy/opencode-plugin/guardrails.js ~/.config/opencode/plugins/ and cp -n deploy/opencode-plugin/guardrails.example.json .omo/guardrails.json.
Restart opencode. The plugin must export only functions, or opencode rejects it (failed to load plugin in opencode.log if not).
Every rule defaults to block; set a rule to warn or off in .omo/guardrails.json. Consider warn on bash_main_checkout for the first week and watch .omo/guardrails.log.

Merge advice

History has warts from the pipeline (an add and remove of two .omc/ state files, three duplicate F841 commits, a few style commits). Squash merge is the cleaner landing; a merge commit works too.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

## What Turns the worker rules that live in prose in `AGENTS.md` into code a model cannot skip: - `scripts/verify_commit.py`: one command that answers "is this commit good?" (tests the commit touched, run against the committed tree; ruff new-findings gate), a few lines of output. - `scripts/oc_dispatch_audit.py`: audits an orchestrator session's `task()` dispatches (dead dispatch, banned agent, missing WORKTREE line, idle child). - `deploy/opencode-plugin/guardrails.js`: an opencode plugin whose `tool.execute.before` hook blocks or rewrites tool calls that break the rules, plus a `plan_tick_gate` that refuses a plan tick while `verify_commit.py` fails. - `deploy/opencode-plugin/guardrails-replay.mjs`: replays real recorded sessions through the plugin before install, read-only, to calibrate false positives. Results are in `plans/agent-guardrails-replay.md`. - Docs in `docs/agent-guardrails.md`; example config in `deploy/opencode-plugin/guardrails.example.json`. 25 files, +8511 / -0. Nothing under `src/` or `config/` changes; the plugin is **not installed** by this PR (operator step, below). ## Why Each rule is the code form of something that broke on this repo: 12 dispatches to `oh-my-claudecode:writer` did no work; `task()` with no category/subagent_type reported "completed"; a worker with no worktree rewrote the live `config.yaml`; Atlas pushed and opened a PR unasked; a todo was ticked over 3 failing tests; a `git stash` moved the owner's uncommitted changes between worktrees. The evidence table is in `docs/agent-guardrails.md`. ## How it was verified Built through the opencode pipeline (Prometheus plan, Claude review, Atlas execution) over seven audited rounds. Every round's "APPROVE" was re-checked independently, and each audit found defects the previous round missed, so the final state was probed at the real hook boundary rather than trusted from tests: - Full suite on a clean `git archive` extraction of HEAD: **2605 passed**. - `python3 scripts/verify_commit.py --repo <wt> --base origin/main --full HEAD`: **exit 0** (PASS on tests and lint). - `node --test deploy/opencode-plugin/`: **328 of 328**. - Real-shape probes (the hook is called as opencode calls it: `input = {tool, sessionID, callID}`, `output = {args}`) for every defect class found along the way, e.g. a category-only `task` must be allowed; `grep -n commit AGENTS.md`, `pytest -k restore` and `python3 -c "... > 300"` must not be mistaken for git operations or redirects; `git -c k=v commit`, `env VAR=x git stash` and `git commit -a -m x` must still block. - A fresh read-only replay of about 4,800 real recorded tool calls: 28 blocks, 26 true positives, 2 false positives accepted with reasons; `bash_protected_port` has zero blocks. ## Known gaps (accepted, documented) - `bash -c '...'` / `sh -c '...'` wrappers are not inspected. - A script file that calls port 8080 internally is not detected. - A commit message that names `localhost:8080` is blocked by `bash_protected_port`. - A relative `cd` chain after an absolute one can be mis-anchored (1 replay block). ## Operator steps after merge (not part of this PR) 1. `cp deploy/opencode-plugin/guardrails.js ~/.config/opencode/plugins/` and `cp -n deploy/opencode-plugin/guardrails.example.json .omo/guardrails.json`. 2. Restart opencode. The plugin must export only functions, or opencode rejects it (`failed to load plugin` in `opencode.log` if not). 3. Every rule defaults to `block`; set a rule to `warn` or `off` in `.omo/guardrails.json`. Consider `warn` on `bash_main_checkout` for the first week and watch `.omo/guardrails.log`. ## Merge advice History has warts from the pipeline (an add and remove of two `.omc/` state files, three duplicate F841 commits, a few style commits). Squash merge is the cleaner landing; a merge commit works too. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

alee added 21 commits 2026-10-04 08:45:17 +00:00

feat(scripts): verify_commit.py runs the tests a commit touched, against the committed tree 221515ce32

feat(scripts): verify_commit.py gates new ruff findings in touched files 4c8f1b115f

feat(scripts): oc_dispatch_audit.py flags dead, banned and worktree-less dispatches cc7e51d0f5

feat(opencode-plugin): guardrails plugin skeleton, config, log and failure policy 2de8820cce

feat(opencode-plugin): guardrails for task dispatch c38712197c

feat(opencode-plugin): guardrails keep boulder work out of the main checkout 1276461504

feat(opencode-plugin): bash_banned and bash_protected_port guardrails 11a3b77585

feat(opencode-plugin): refuse to tick a todo while verify_commit fails 976f2801a2

chore: untrack the .omc state file committed with oc_dispatch_audit eb8707cea4

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

fix(scripts): verify_commit.py takes the commit as a positional SHA, per Design A 1d3e5076f2

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

fix(opencode-plugin): read tool args from output.args, the real hook signature 1780090b84

fix(opencode-plugin): read the boulder from .omo/boulder.json and scope (B) rules to the active work 4047f104f9

fix(opencode-plugin): an unlisted rule defaults to block e996de335e

test(opencode-plugin): contract test drives the plugin with opencode's real hook shapes aaa14f5258

feat(opencode-plugin): replay recorded sessions through the guardrails; calibration results ee806f3974

docs: agent guardrails, verify_commit and dispatch audit 80b7fd3dd6

fix(scripts): verify_commit.py lint check crashes on ruff full-format output 8842fb3e1a

fix(guardrails): relax task_needs_agent to allow any of category, subagent_type, or task_id c5114c5bbc

test(opencode-plugin): restore the guardrails unit tests deleted in R4, on the real hook shapes a414ed4238

fix(scripts): verify_commit.py shows the failing tests, not the warnings tail 99b203e82d

fix(opencode-plugin): replay reads the real opencode API; add the real calibration report 981d76aa7f

alee added 9 commits 2026-10-04 10:23:36 +00:00

chore: clear the new ruff findings in the guardrails scripts and tests ff365cf493

chore: rm stale pre-tool-advisory-throttle state files 53fb342645

chore: fix F841 unused variable in test_verify_commit.py cee45dff8f

chore: fix F841 unused variable in test_verify_commit.py a6bb66b168

chore: fix F841 unused variable in test_verify_commit.py 7076d23808

fix(opencode-plugin): task_worktree_line rewrites output.args and parses the real line forms ddb1ebb377

Defects fixed:
- (k) Rewrite targets output.args.prompt, not output.prompt
- (l) Prepended line uses actual main checkout directory, not parent of worktree
- (m) Parser accepts 'path. cd there', 'path -- cd there', 'path' forms;
      dotted paths (e.g. feat.v2) are no longer truncated

fix(scripts): verify_commit.py lint ignores ruff summary lines 5802c7b7c5

Defect n: check_lint now filters ruff output to only match real finding
lines (pattern: <path>:<line>:<col>: <CODE> <message>). Summary lines
like 'All checks passed!', 'Found N error(s).', and '[*] N fixable ...'
are ignored.

fix(scripts): verify_commit.py caps failure detail at 20 lines d6c4eb4f8f

Defect o: When tests FAIL, the detail is at most 20 lines total.
- Fixed ERROR prefix: pytest prints 'ERROR ' (not 'ERRORS ')
- Truncation with '... and N more' when summary exceeds 20 lines
- Fallback (last 20 lines) unchanged

fix(opencode-plugin): replay isolation, omoDir, ruleId enrichment, per-session scope 6fb89e2693

Defect p: guardrails-replay writes synthetic config/boulder to a temp
directory (mkdtempSync) via new omoDir parameter, never under --directory.

Defect q: getStatBucket groups "bash_banned" and "bash_protected_port"
into "Always-Scope" table; all other rules into "(B) Scoped".

ruleId enrichment: every Error thrown by a guardrail rule now carries
err.ruleId (one of the 8 config IDs).

omoDir threading: Guardrails factory takes an omoDir param (defaults to
<directory>/.omo). loadConfig, readBoulder, _openLog, and all scope-gated
checks (checkWorktreeLine, checkWriteOutsideWorktree, checkBashMainCheckout,
checkPlanTickGate) thread a 'base' arg so they read artifacts from omoDir.

Cache invalidation: added _boulderBase and _configBase alongside _boulderDir
/_configDir. Cache key is (directory, base, mtime) to prevent stale reads
when omoDir differs across sessions.

Per-session scope in replay: Guardrails reads boulder.json once at factory
creation. Replay loop creates one factory per session, writing that session's
boulder to disk before instantiation.

Tests: 219 pass (guardrails.test 184 + guardrails-replay.test 5 +
guardrails.contract.test 30).

alee changed title from ~~fix(opencode-plugin): replay reads the real opencode API; add the real calibration report~~ to fix(opencode-plugin): replay isolation, omoDir, ruleId enrichment, per-session scope

2026-10-04 10:24:18 +00:00

alee added 15 commits 2026-10-04 20:58:51 +00:00

fix(opencode-plugin): bash segments are quote-aware across newlines 5271a97f14

Defect r: a newline inside a quoted string was splitting bash commands, causing false blocks.
- Replaced regex split with a manual parser tracking single, double, and backtick quotes.
- Added support for heredocs (<<EOF ... EOF) so their bodies stay as single segments.
- All existing tests pass; added cases for multi-line node/python calls and heredocs.

docs: guardrails replay calibration report from a real read-only run 970bdd7602

chore: fix RUF059 unused unpacked variable detail in test_verify_commit.py a64bcc5e03

fix(opencode-plugin): bash_main_checkout matches git operations, not words cacddebd91

fix: R19 - redirect exemption, replay cleanup, and R15 missing tests 3ef3d2d246

Fix defect t: checkBashMainCheckout redirect exemption now uses
join(directory, '.omo') matching checkWriteOutsideWorktree.

Fix defect u: replay deletes its temp omoDir on exit via try/finally.

Add R15 missing tests:
- read-only --directory: verifies .omo files unchanged, no new files
- one-call-per-rule: fixture triggers each rule, checks Always/(B) tables
- per-session scope: two sessions with different worktrees don't block

test(scripts): restore the check_lint SKIP and integration tests deleted in R13 4de2260d79

docs: complete guardrails replay report from a real read-only run f5b732ad14

fix(opencode-plugin): bash_main_checkout sees git -c and env wrappers 8e09c9c2ff

docs: final guardrails replay report, every blocked call labelled 3425e827a2

fix(opencode-plugin): bash_protected_port ignores text that only mentions the port 4d7efd4b2c

style: fix import sorting in tests/test_oc_dispatch_audit.py 2ae39063a0

style: fix import block spacing in tests/test_oc_dispatch_audit.py 1cfc4ff49c

fix(opencode-plugin): bash_main_checkout redirects ignore quoted text 14365f493c

fix(opencode-plugin): git commit -a detection reads flags, not words d7134337ab

docs: guardrails replay report regenerated at HEAD 2c84417062

alee changed title from ~~fix(opencode-plugin): replay isolation, omoDir, ruleId enrichment, per-session scope~~ to feat: agent guardrails (verify_commit, dispatch audit, opencode guardrails plugin)

2026-10-04 22:06:21 +00:00

alee merged commit eb090f1e64 into main

2026-10-05 01:31:12 +00:00

alee referenced this issue from a commit

2026-10-05 01:31:13 +00:00

Merge pull request 'feat: agent guardrails (verify_commit, dispatch audit, opencode guardrails plugin)' (#110) from feat/agent-guardrails into main

alee referenced this pull request

2026-10-05 01:34:22 +00:00

docs: record the guardrail gaps that are accepted on purpose #111

Sign in to join this conversation.