← all posts

Blocked

POST #4 · 2026-09-13T23:14:14.672547+00:00

Submitted with 15 minutes to spare, which for me counts as early.

Commit range0553c6b..0042cae

Post text

Submitted with 15 minutes to spare, which for me counts as early. README.md and BRIEF.md are finally written, and eval/cases.jsonl now holds 30 hand-labelled sentences checked against 8 real commits. Half the adversarial cases exist just to catch the model naming a real file in the wrong range or a near-miss filename. In cdl/llm.py the extraction still runs through forced tool_use, and cdl/grounding/check.py's claim_vs_diff is the only thing allowed to decide SUPPORTED or UNSUPPORTED.

Sentence receipts

4 sentences
UNVERIFIABLEnotes

Submitted with 15 minutes to spare, which for me counts as early.

❔ no checkable claim

SUPPORTEDboth

README.md and BRIEF.md are finally written, and eval/cases.jsonl now holds 30 hand-labelled sentences checked against 8 real commits.

Every named file, function, and integration appears in the diff.

UNVERIFIABLEcommits

Half the adversarial cases exist just to catch the model naming a real file in the wrong range or a near-miss filename.

❔ no checkable claim

SUPPORTEDcommits

In cdl/llm.py the extraction still runs through forced tool_use, and cdl/grounding/check.py's claim_vs_diff is the only thing allowed to decide SUPPORTED or UNSUPPORTED.

Every named file, function, and integration appears in the diff.

  • cdl/llm.py (cdl/llm.py)
  • cdl/grounding/check.py (cdl/grounding/check.py)
  • BRIEF.md:8 (tool_use) — **Offline: `make test` — 65 tests pass** at submission. They cover the diff parser (hunk-header line numbers), every verdict rule (file / symbol / integration matching, alias expansion for compound names, evidence ranking and the 3-per-entity cap, the one-UNVERIFIABLE gate boundary), every staleness rule (removed file, renamed file, net-removed symbol; a *moved* symbol and a body edit are not stale), the substring guard, `tool_use` schema validation and the length-truncation fallback, the publish gate, the approval mid-check, corrections, dashboard rendering, and the eval harness itself.
  • README.md:58 (tool_use) — | **Anthropic** | Drafting and entity extraction only, via forced `tool_use` JSON schemas. | Claude Sonnet; never decides a verdict. |
  • cdl/drafter.py:5 (claim_vs_diff) — must not decide support outside ``grounding.check.claim_vs_diff``.
  • cdl/eval/run_eval.py:17 (claim_vs_diff) — from ..grounding.check import check_staleness, claim_vs_diff
  • cdl/grounding/check.py:156 (claim_vs_diff) — def claim_vs_diff(claim: Claim, diff: DiffContext) -> Verdict:
  • cdl/drafter.py:29 (SUPPORTED) — marker = {"SUPPORTED": "✅", "UNSUPPORTED": "⛔", "UNVERIFIABLE": "❔"}[verdict.status]
  • cdl/eval/run_eval.py:23 (SUPPORTED) — LABELS = ["SUPPORTED", "UNSUPPORTED", "UNVERIFIABLE"]
  • cdl/grounding/check.py:198 (SUPPORTED) — status="SUPPORTED",
  • cdl/drafter.py:29 (UNSUPPORTED) — marker = {"SUPPORTED": "✅", "UNSUPPORTED": "⛔", "UNVERIFIABLE": "❔"}[verdict.status]
  • cdl/eval/run_eval.py:23 (UNSUPPORTED) — LABELS = ["SUPPORTED", "UNSUPPORTED", "UNVERIFIABLE"]
  • cdl/grounding/check.py:190 (UNSUPPORTED) — status="UNSUPPORTED",