Verification Before Completion
patterns ~2138 tokens updated 2026-10-04
Evidence-before-claims methodology. Requires fresh execution of verification commands, reading full output, and confirming results before asserting completion.
Tags
README
verification
Evidence-before-claims methodology for task completion.
rules/config-change-detection.md is path-scoped (#1061): it carries paths: frontmatter and loads only after Claude reads a file matching one of .github/workflows/**, **/.github/workflows/**, **/wrangler.toml, **/wrangler.json, **/wrangler.jsonc, **/.env*, **/supabase/migrations/**, **/vercel.json, **/fly.toml, **/Dockerfile*, **/docker-compose*. It costs no context at session start otherwise.
What It Does
Installs a rules file that requires fresh proof before asserting anything works:
- Identify the command that proves the claim
- Execute it fresh (no cached results)
- Read the full output including exit codes
- Verify the output actually supports the claim
- Report with evidence attached
Covers tests, linting, builds, bug fixes, deployments, and type checking. Prevents common failures like proxy claims, stale results, and partial verification.
Manual Installation
# Global (all projects)
mkdir -p ~/.claude/rules
cp rules/verification.md ~/.claude/rules/verification.md
cp rules/config-change-detection.md ~/.claude/rules/config-change-detection.md
mkdir -p ~/.claude/bin && cp bin/ccgm-verify-baseline ~/.claude/bin/ && chmod +x ~/.claude/bin/ccgm-verify-baseline
# Project-level
mkdir -p .claude/rules
cp rules/verification.md .claude/rules/verification.md
cp rules/config-change-detection.md .claude/rules/config-change-detection.md
Files
| File | Description |
|---|---|
rules/verification.md |
5-step verification process with evidence requirements table |
rules/config-change-detection.md |
Hash-of-config pattern for re-verifying expensive automation when config drifts |
bin/ccgm-verify-baseline |
Records which checks fail before work starts, then gates on new failures by exit code (0 none new, 1 new, 2 setup error). Checks come from --check NAME=COMMAND, package.json scripts, or .ccgm-verify.json. The baseline lives in the repo's git directory, per worktree. Failures are compared by test identifier when the output matches vitest/jest, pytest, bats or FAIL: formats, otherwise by exit code only. |
Will install
| Path | Action | Target | Type |
|---|---|---|---|
rules/verification.md | → | rules/verification.md | rule |
rules/config-change-detection.md | → | rules/config-change-detection.md | rule |
bin/ccgm-verify-baseline | → | bin/ccgm-verify-baseline | script |
Dependencies
No dependencies.
Required by
No other module depends on this one.
Included in presets
Install this module
Agent prompt
Recommended for agent users -- hands the whole install off to your assistant.
Fetch https://ccgm.dev/modules/verification.md and install this module into my Claude Code setup.
Native plugin marketplace
One command via the native plugin marketplace -- additive, does not merge settings.json.
claude plugin install verification@ccgm
The marketplace path is additive, not a replacement: it installs commands, agents, and skills as native plugin components, but it does not perform the bash installer's deep settings.json merge, and it does not write the always-loaded global CLAUDE.md context. Rules are only injected via an opt-in SessionStart hook rather than being auto-loaded. Use the bash installer when those pieces matter to you.
Files
rule (2)
rules/verification.md
# Verification Before Completion
No completion claim without verification evidence. Never assert that something works, passes, or is fixed without proof captured this session.
## The Process
Before claiming a task is complete:
1. Identify the specific command or action that proves the claim.
2. Execute it at claim time and capture its output.
3. Read the full output, including exit codes and failure counts, not just the summary line.
4. Confirm the output supports the exact claim.
5. Report with the evidence attached, not "tests pass."
## What Counts as Evidence
| Claim | Required evidence |
|-------|-------------------|
| "Tests pass" | Test run output showing pass count and 0 failures |
| "Lint is clean" | Linter output showing 0 errors, 0 warnings |
| "Build succeeds" | Build output with exit code 0 |
| "Types check" | Type checker output showing 0 errors |
| "Bug is fixed" | The reproduction that previously failed now succeeding |
| "No regressions" | Full suite output, not just the new tests |
| "Deployed" | The deployment URL responding with the expected content |
| "UI renders correctly" | A screenshot captured this session |
| "Agent completed" | The subagent's actual diff, test run, or artifact, never its self-report |
Evidence is an artifact the machine produced this session. A reasoned argument ("the diff is small," "the types line up") is a hypothesis, and a bare assertion ("it works," "fixed") is the failure this rule exists to catch. Paste the artifact next to the claim; an artifact you ran but did not show reads as an assertion, and a claim with no artifact is downgraded to "changed X, not yet verified."
## One Check Is Not Another
Lint passing does not mean types check; a type check is not a test run; 10 of 12 tests passing means 2 are failing. Check the exit code, scroll the full output, and read warnings even when the overall status is pass. Report honestly: one failure out of a hundred is not "tests pass," a build with warnings mentions the warnings, and a verification step you could not run is stated as not run. Hedge words in a completion claim ("should", "probably", "seems to") mean it is not verified; rewrite it with evidence or downgrade it. A `test.skip`, `.only`, or stub test added in the diff blocks the completion claim. To decide whether a failure is new or already existed, run `ccgm-verify-baseline` (`--write-baseline` before work starts) and obey its exit code; do not compare by eye.
## When to Verify
Before claiming a bug is fixed, before reporting a task complete, before pushing with the full pre-push suite (see `code-quality.md`), and after resolving merge conflicts.
rules/config-change-detection.md
---
paths:
- ".github/workflows/**"
- "**/.github/workflows/**"
- "**/wrangler.toml"
- "**/wrangler.json"
- "**/wrangler.jsonc"
- "**/.env*"
- "**/supabase/migrations/**"
- "**/vercel.json"
- "**/fly.toml"
- "**/Dockerfile*"
- "**/docker-compose*"
---
# Config Change Detection
Re-verify when config changes. A green run last week proves nothing about today's config.
This is a specialization of the verification discipline for automation that is dangerous on first run or after configuration drift. Deploys, migrations, new integrations, any workflow that depends on an external contract (env file, deploy config, wrangler.toml, CI secrets) must re-verify when that contract changes. A green run last week does not prove anything about today's config.
## The Hash-of-Config Pattern
Record a hash of the relevant config section. Before running the automation, compare the stored hash against a fresh hash. If they differ (or no hash exists), the automation is in one of two states:
- **FIRST_RUN**: no marker file exists. The config has never been verified on this machine.
- **CONFIG_CHANGED**: marker file exists, but the current hash does not match. The config drifted since last verification.
In either state, run a dry-run or full verification step before the destructive/expensive action, then update the marker.
### Minimal Implementation
```bash
# Hash the relevant config slice (section of CLAUDE.md, plus workflow files)
CONFIG_HASH=$(
{
sed -n '/^## Deploy/,/^## /p' CLAUDE.md
cat .github/workflows/deploy.yml 2>/dev/null
cat wrangler.toml 2>/dev/null
} | shasum -a 256 | awk '{print $1}'
)
MARKER="$HOME/.claude/projects/${PROJECT_SLUG}/deploy-confirmed"
if [ ! -f "$MARKER" ] || [ "$(cat "$MARKER")" != "$CONFIG_HASH" ]; then
# FIRST_RUN or CONFIG_CHANGED - run dry-run / verification
echo "Config changed or never verified. Running dry-run..."
run_dry_run_and_await_user_confirmation
echo "$CONFIG_HASH" > "$MARKER"
fi
# Config verified, proceed with automation
run_real_action
```
## Marker File Strategy
Store markers under `~/.claude/projects/{slug}/{operation}-confirmed`, one file per operation. The file contains the hash only. The directory structure isolates per-project state without polluting the repo.
Rules:
- **One marker per operation**, not per project. `deploy-confirmed`, `migrate-confirmed`, `env-sync-confirmed` are separate.
- **Never commit markers to git**. They are local trust state, not shared truth.
- **Include every file that affects the outcome** in the hash input. Missing a file (a new workflow step, a new env var) defeats the pattern.
- **Order the hash input deterministically**. Sort file paths or use a fixed concatenation order so identical configs always produce identical hashes.
## What to Hash
For each operation type, define the config surface that actually controls behavior:
| Operation | Hash Input |
|-----------|-----------|
| Deploy | Deploy section of CLAUDE.md + deploy workflow files + platform config (wrangler.toml, vercel.json, fly.toml) |
| Migration | Schema directory + `package.json` db scripts + migration runner config |
| Integration | `.env.example` + integration config file + any setup scripts |
| Package rebuild | `package.json` + lockfile + build script section of CLAUDE.md |
If you cannot name the files that control the operation, the operation is not safe to automate yet. Identify them first.
## When to Apply
Use this pattern when all of the following are true:
- The automation is **expensive** (deploy, migration, large build) or **destructive** (drops data, rewrites history, charges money)
- The automation depends on **external config** that a human might edit without re-running
- A **dry-run or verification step** exists that costs less than the real run
Do not apply it to:
- Idempotent read-only checks (tests, lints, type checks) - these are already cheap to re-run fresh
- One-off scripts that never run twice
- Operations where the config IS the command (e.g., `rm -rf dist` has no external config to drift)
## Integration with the Verification Discipline
The base verification rule ("Evidence before claims") requires fresh proof every time. Config change detection is the mechanism that tells you **when fresh proof needs to include a full dry-run**, not just a check of the last run's artifact.
Without this mechanism, agents default to one of two failure modes:
- **Always dry-run** (slow, defeats automation) or
- **Never dry-run after first success** (ships config drift to production)
The hash marker resolves the tradeoff: automated when safe, interactive when the contract changed.
## Rationalizations That Mean You Are About to Skip Re-Verification
| You are about to say... | The reality is... |
|-------------------------|-------------------|
| "The deploy worked last week, it will work now" | Config can drift without you touching the deploy script. A new env var, a changed secret, a workflow edit. |
| "I only changed docs, not the deploy config" | If the hash input includes only deploy files, the docs change will not trigger re-verify. If it does include CLAUDE.md, re-verify anyway - you might have documented a change that is not yet reflected in code. |
| "The dry-run is annoying" | Annoying is cheaper than a broken production deploy or a migration run against the wrong database. |
| "I can eyeball the config diff" | Eyeballing is not verification. A hash comparison is deterministic; human memory is not. |
## Red Flags
Stop and re-run the dry-run if you catch yourself:
- Running a deploy command without checking whether the marker exists
- Copying a hash check from another project without updating the hash input list for this operation
- Deleting or ignoring a marker file to "just get past it"
- Committing a marker file to the repo (it is local trust state, not shared truth)
script (1)
bin/ccgm-verify-baseline
#!/usr/bin/env python3
"""ccgm-verify-baseline: gate on NEW check failures only, by exit code.
ccgm-verify-baseline --write-baseline run the checks, record the results
ccgm-verify-baseline re-run, diff against the baseline
Exit codes: 0 nothing new, 1 new failures, 2 usage or setup error.
Checks come from, in order of precedence:
1. repeatable --check NAME=COMMAND flags
2. package.json scripts: lint, type-check|typecheck, test:run|test, build
3. .ccgm-verify.json at the repo root: {"name": "command", ...}
The baseline lives in $(git rev-parse --git-dir)/ccgm-verify-baseline.json:
per worktree, never committed.
Failure identity. Per check, the exit code decides pass or fail. When the
output matches a known runner format (vitest/jest FAIL and the x marks,
pytest FAILED, bats "not ok", generic "FAIL:"), failing test identifiers are
recorded too, after stripping ANSI codes, timestamps and durations.
Limits:
- A check that fails in both runs with identifiers on the current run is
compared by identifier: any identifier absent from the baseline is NEW.
- A check that fails in both runs and prints no identifiers counts as the
same pre-existing failure, even if the cause changed.
- A check missing from the baseline is treated as having passed.
- Output of unknown runners yields no identifiers, so only exit codes count.
"""
import argparse
import json
import os
import re
import subprocess
import sys
BASELINE_NAME = "ccgm-verify-baseline.json"
ANSI = re.compile(r"\x1b\[[0-9;?]*[A-Za-z]")
TIMESTAMP = re.compile(
r"\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(?:[.,]\d+)?(?:Z|[+-]\d{2}:?\d{2})?"
r"|\b\d{2}:\d{2}:\d{2}(?:[.,]\d+)?\b"
)
DURATION = re.compile(
r"[\(\[]\s*\d+(?:\.\d+)?\s*(?:ms|s|sec|secs|seconds?|m)\s*[\)\]]"
r"|\b\d+(?:\.\d+)?\s*(?:ms|secs?|seconds?)\b"
)
# Order matters: first match wins.
FAIL_LINE = [
re.compile(r"^FAIL(?:ED)?:?\s+(.+)$"), # pytest FAILED, jest/vitest FAIL, FAIL:
re.compile(r"^[✗✕×]\s*(.+)$"), # jest/vitest/mocha marks
re.compile(r"^not ok\s+(?:\d+\s+)?(?:-\s+)?(.+)$"), # bats/TAP
]
PYTEST_MSG = re.compile(r"\s+-\s+.*$")
def normalize(text):
text = ANSI.sub("", text)
text = TIMESTAMP.sub("", text)
text = DURATION.sub("", text)
return " ".join(text.split())
def extract_failures(output):
ids = set()
for raw in output.splitlines():
line = raw.strip()
line = normalize(line) # also strips a leading timestamp
for pat in FAIL_LINE:
m = pat.match(line)
if m:
ident = PYTEST_MSG.sub("", m.group(1)).strip()
if ident:
ids.add(ident)
break
return sorted(ids)
def die(msg):
print(f"ccgm-verify-baseline: {msg}", file=sys.stderr)
sys.exit(2)
def git(*args):
r = subprocess.run(["git", *args], capture_output=True, text=True)
return r.stdout.strip() if r.returncode == 0 else None
def detect_checks(root):
pkg = os.path.join(root, "package.json")
if os.path.isfile(pkg):
try:
scripts = json.load(open(pkg)).get("scripts", {})
except (OSError, ValueError):
scripts = {}
checks = {}
for group in (["lint"], ["type-check", "typecheck"], ["test:run", "test"], ["build"]):
for name in group:
if name in scripts:
checks[name] = f"npm run {name}"
break
if checks:
return checks
cfg = os.path.join(root, ".ccgm-verify.json")
if os.path.isfile(cfg):
try:
data = json.load(open(cfg))
except (OSError, ValueError) as e:
die(f".ccgm-verify.json is not valid JSON: {e}")
if not isinstance(data, dict) or not all(
isinstance(k, str) and isinstance(v, str) for k, v in data.items()
):
die('.ccgm-verify.json must be an object of {"name": "command"} strings')
return data
return {}
def parse_check_flags(flags):
checks = {}
for f in flags:
name, sep, cmd = f.partition("=")
if not sep or not name or not cmd:
die(f"--check expects NAME=COMMAND, got: {f}")
checks[name] = cmd
return checks
def run_checks(root, checks):
results = {}
for name, cmd in checks.items():
p = subprocess.run(cmd, shell=True, cwd=root, capture_output=True, text=True)
out = (p.stdout or "") + "\n" + (p.stderr or "")
results[name] = {
"exit": p.returncode,
"failures": extract_failures(out) if p.returncode != 0 else [],
}
return results
def label(name, ident=None):
return f"{name}: {ident}" if ident else name
def diff(baseline, current):
new, fixed, pre = [], [], []
for name, cur in current.items():
base = baseline.get(name, {"exit": 0, "failures": []})
base_failed = base["exit"] != 0
cur_failed = cur["exit"] != 0
if cur_failed:
if cur["failures"]:
known = set(base["failures"]) if base_failed else set()
for ident in cur["failures"]:
(pre if ident in known else new).append(label(name, ident))
for ident in sorted(known - set(cur["failures"])):
fixed.append(label(name, ident))
elif base_failed:
pre.append(label(name))
else:
new.append(label(name))
elif base_failed:
if base["failures"]:
fixed.extend(label(name, i) for i in base["failures"])
else:
fixed.append(label(name))
return new, fixed, pre
def main():
ap = argparse.ArgumentParser(
description="Record check failures once, then gate on new failures only."
)
ap.add_argument("--write-baseline", action="store_true", help="run checks and store the baseline")
ap.add_argument("--check", action="append", default=[], metavar="NAME=COMMAND")
ap.add_argument("--json", action="store_true", help="machine-readable output")
args = ap.parse_args()
root = git("rev-parse", "--show-toplevel")
git_dir = git("rev-parse", "--absolute-git-dir")
if not root or not git_dir:
die("not inside a git repository")
path = os.path.join(git_dir, BASELINE_NAME)
checks = parse_check_flags(args.check) or detect_checks(root)
if not checks:
die(
"no checks found. Pass --check NAME=COMMAND, add lint/type-check/test/build "
'scripts to package.json, or create .ccgm-verify.json ({"name": "command"}).'
)
if not args.write_baseline and not os.path.isfile(path):
die(f"no baseline at {path}. Run with --write-baseline first, before starting work.")
current = run_checks(root, checks)
if args.write_baseline:
with open(path, "w") as f:
json.dump({"checks": current}, f, indent=2)
failing = [n for n, r in current.items() if r["exit"] != 0]
if args.json:
print(json.dumps({"status": "baseline-written", "path": path, "checks": current}))
else:
print(f"Baseline written to {path}")
for n, r in current.items():
print(f" {n}: {'FAIL' if r['exit'] else 'pass'} ({len(r['failures'])} identified)")
if failing:
print(f"{len(failing)} check(s) already fail; they will count as PRE-EXISTING.")
return 0
try:
baseline = json.load(open(path))["checks"]
except (OSError, ValueError, KeyError) as e:
die(f"baseline at {path} is unreadable ({e}). Re-run with --write-baseline.")
new, fixed, pre = diff(baseline, current)
status = "new-failures" if new else "pass"
if args.json:
print(json.dumps({"status": status, "new": new, "fixed": fixed, "pre_existing": pre}))
else:
for title, items in (("NEW", new), ("FIXED", fixed), ("PRE-EXISTING", pre)):
print(f"{title} ({len(items)})")
for i in items:
print(f" {i}")
if new:
print("Result: new failures. Fix them.")
elif pre:
print("Result: pass. Warning: pre-existing failures remain.")
else:
print("Result: pass.")
return 1 if new else 0
if __name__ == "__main__":
sys.exit(main())