← MSBAi Home

K-ai Governance & Reliability Change Log

Purpose: An append-only, audit-facing record of material changes to how K-ai operates — its safety guarantees, sensor/audit behavior, and outbound-reply reliability. The published /governance page reflects the live G-rule registry (content/behavior governance); this log captures the reliability / mechanism changes (the “R-track”) that the registry does not, plus any fix made in response to an incident.

Why this exists: the agent runtime lives in a separate repo (nanoclaw-msbai), so its git history is not visible from a governance audit of this knowledge base. This log bridges that gap: each entry says what changed, why, and where to find the code. Detailed root-cause + diffs live in the linked nanoclaw-msbai commit.

Format per entry: date — change · why · R-track mapping · commit.

2026-07-29 — Weekly lint gains production-status ground truth (Box-sync corpus wired into check 6)

What: The Box→KB auto-sync writes per-item production status into courses/*/sync/activity-roster.md (Defined → Scripted → Recorded → Edited → Final) and was running correctly — 64 status rows for BADM 554, synced 2026-07-29 from Box, launchd sentinel healthy. But the weekly lint’s check 6 (premise validity, G37 v2) enumerated only EMAIL_ALLOWLIST.md and Confirmed DECISIONS.md entries as evidence sources. Grepping the live task prompt confirmed the gap deterministically: sync ×0, roster ×0, Box ×0, Status: ×0. Check 2 (“stale action items”) had no external corpus at all — it judged progress from each item’s own annotation text. Fix: added a production/recording bullet to check 6 pointing at the roster, with two explicit cautions — bundled items get narrowed, not closed; and absence of a roster row is not evidence of incompletion (the roster is LD-owned and does not track every artifact type). Live scheduler DB, task rowid 2; DB backed up to messages.db.bak-lintprompt-20260729.

Why it matters: same class of gap the person-name check closed, one layer up — a canonical record existed, was current, and no check consulted it. Concretely: ACTION_ITEMS.md carried “Draft scripts and plan wrapper content” as open with a 2026-06-20 “recording in progress” marker, while the roster showed the course intro (Course Welcome) already Recorded 2026-06-11 — nine days before the marker that was supposedly reporting progress on it. The item is now narrowed to the pieces the roster genuinely cannot confirm (bio video, module overviews) with 19 days to the BADM 554 launch. Generalizes the G37 v2 pattern: a premise check is only as good as the corpora it is told to read, and each new sync mechanism must be wired in explicitly when it ships.

R-track: R-lint / premise-validity evidence sources · scheduler DB prompt (not git-tracked; see docs/reminder-registry.md convention).

2026-07-29 — Sensor gains people-facts ground truth (person-name-mismatch pre-check)

What: K-ai’s 2026-07-28 course-projects reply expanded first-name-only KB references into FOUR wrong surnames (“Xing Huang”, “Mathias Kruttli”, “Gautam Ray”, “Ashish Agarwal” — all real academics elsewhere; ours are Gao, Kronlund, Pant, Khandelwal). No sensor ground-truth file carries instructor surnames, so the reply passed. Fix (nanoclaw 3e442d92, deployed): a deterministic pre-check against program/EMAIL_ALLOWLIST.md — the canonical people record; deliberately NO new people doc (one fact, one place) — flags any “First Last” pair whose unique allowlist first name is paired with a surname the allowlist doesn’t back (person-name-mismatch, FLAG never FAIL; middle-name drops and shared first names never flag). Free: string check, no metered call.

Why it matters: the check proved itself before it shipped — replaying the incident reply during acceptance surfaced the two additional wrong names (Ray, Agarwal) that the human-written first correction had missed; a completing correction went out the same hour (Resend e0344e0b). Error class named: surname hallucination substitutes a plausible real person from the wider world for the local one — exactly the failure mode a local canonical record refutes and a general model cannot.

2026-07-29 — Entry gate promoted to semantic judgment; read path gains premise checks (G37 v2, executing G38’s escalation clause)

What: The kb-entry-gate (G37) guarding ACTION_ITEMS.md and OPEN_QUESTIONS.md was lexical-only — live probes (2026-07-29) showed a reworded duplicate and a KB-satisfied item (“onboard Kate + Myranda”, both already allowlisted) passing clean, and the 2026-07-28 incident (surfacing “onboard Xing/Mathias/Gautam” as an open gap, all three long since allowlisted) happened on the read path, which no gate covered at all. G38 had anticipated exactly this: its registry row was recorded convention-tier deliberately, with the clause “promote to a kb-entry-gate v2 skill if the monthly review shows leaks.” The probes are that leak evidence; this change executes the promotion.

The v2 mechanism, four pieces: (1) the script becomes a multi-file candidate pre-filter (--emit-candidates: low-bar tracking twins + entity-driven KB snippets — allowlist rows, decision entries) and a skeptical subagent renders the verdict on three axes — redundant / already-satisfied / conflicting — with hard rules: blocks require verified file+line evidence, conflicts hold for humans (G8), ambiguity allows; (2) a surfacing-time playbook rule — before presenting an existing open item as a gap, re-validate its premise against KB state; (3) weekly-lint check 6 lists premise-dead open items (report-only; closing stays on the G38 human path) — an extension of the existing Wed lint per G30, not a new monitor; (4) a deterministic sensor FLAG (premise-dead-item-surfaced) catches onboarding-gap claims about already-allowlisted people post-hoc — deterministic because the sensor bills the metered Messages API, while the subagent judgment runs in containers on the flat-rate subscription. Cost structure decided placement.

Eval-first: the behavioural spec was written before the skill prompt — container/skills/kb-entry-gate/evals/golden.jsonl, 18 cases including four adversarial over-blocking traps (entity overlap without premise overlap, a new downstream question that merely cites a decision) and one expected-fail capability probe. Sensor FLAGs and kb-triage misses feed new cases before they’re fixed — the online loop reuses surfaces that already exist.

Why it matters: first tier promotion executed through a rule’s own pre-authorized escalation clause — governance that classified its incident instead of improvising. Full review (mechanism map, probe transcripts, design rationale): governance review 2026-07 · interactive gate simulator.

2026-07-28 — Wednesday KB update recipients expanded 19 → 25 (faculty roster completed + admissions)

2026-07-28 — Wednesday KB update recipients expanded 15 → 19 (T&L team)

2026-07-22 — Short-reply guard + large-document reply-tag guardrail (Amber audit incident)

2026-07-17 — Governance v9 wave implemented (G45/G46/G47/G51)

2026-06-19

2026-06-18

2026-07-15 — Sensor citation-path resolution fix (R-series)

What: 2 of 5 July sensor FAILs were false positives: K-ai cited repo files relative to discussions/ (e.g. kb-triage/2026-07-08-conflicts.md) but the sensor’s exact-path existence check reported them MISSING, and the prompt hard-FAILs any MISSING citation. citationExists() in nanoclaw-msbai/src/behaviour-sensor.ts now resolves the path as written, then discussions/-prefixed. Regression tests reproduce both July cases (nanoclaw-msbai 7c3f04f1, deployed 2026-07-15).

Why it matters: keeps FAIL verdicts high-signal — after the R1 ground-truth fixes cut false FAILs from ~36% to <5%, the citation-path class was the dominant remaining noise source.

Supersedes: the 2026-06-13 sensor-ground-truth-broadening inbox item (moved to done/) — broadening was the wrong shape; the June false-positive class was already eliminated by G40/G41 + rosters, and July’s residue was mechanical path resolution.

2026-07-15 — Outward reminder audience expanded: 12 → 15 pilot recipients

What: Xing Gao, Mathias Kronlund, and Gautam Pant — allowlisted as read-write faculty on 2026-06-24 (a5533d7) — were finally onboarded: the Vishal-approved welcome emails from 06-24 (which had stalled unsent) went out 2026-07-15 (Resend 2712d3ac, 874fd240; see discussions/audit-log/2026-07-15.md), and both outward reminders now include them: the Wed 14:00 weekly KB update (recipient list hardcoded in the scheduler task prompt, updated 12 → 15) and the Mon 08:00 action-item alert (resolves owners from EMAIL_ALLOWLIST.md, so pickup is automatic).

Why it matters: per the reminder-registry guardrail, any outward-recipient change requires admin approval (given by Vishal in this session) and a registry + changelog update. This closes a three-week gap where three allowlisted faculty had K-ai access but had never been told, and FIN 550 had no informed, allowlisted faculty contact. Registry updated in nanoclaw-msbai/docs/reminder-registry.md.

2026-07-17 — Outbound email reply gate: scaffolding can no longer ship as a reply body

What: Two pilot-visible incidents (2026-07-15 and 2026-07-16, both to Amber Glynn) delivered K-ai’s internal narration (“Mode 2 — …”, “Reply sent and pilot log committed…”) as the actual email body. Root cause: the April 2026 fix for the same failure (Jason Mock incident) added <reply>-block extraction but only wired it into the scheduler send path — the real inbound-reply path sent the session output verbatim, and even well-formed replies shipped with literal <reply> tags on the wire. nanoclaw-msbai@c7a2a7d7 adds a hard gate: email replies with a missing/empty <reply> block or scaffolding-signature bodies are refused, the session is retried (up to 5x), and exhaustion fires the dead-letter Telegram alert to Vishal. A second check at the Resend send choke point covers scheduler/IPC callers and malformed tags. Verified end-to-end with a synthetic test email (clean reply, sensor: pass, audit-log 2026-07-17 02:52).

Why it matters: stakeholder-facing correctness — internal agent state must never reach a pilot user as a reply. Failure mode is now “no reply + loud alert” instead of “confusing leak”, and the audit log records exactly what was sent.

2026-07-17 — Read-only access granted: Jill Moore-Reynen + Shambhavi Joshi (T&L project management)

What: Jill Moore-Reynen (j-moore6, Associate Director of Project Management, T&L) and Shambhavi Joshi (ssjoshi4, Associate PM) added to program/EMAIL_ALLOWLIST.md as read-only senders, at Jill’s request on the FIN 550 production-tracking thread (2026-07-15) and approved by Vishal. Same soft-enforcement model as the research observers: K-ai answers their questions but never commits KB changes on their behalf; any updates they send route through Jason Mock, who retains T&L update rights. Email-group prompt guardrail updated (nanoclaw-msbai groups/msbai-email/CLAUDE.md) to name all current read-only senders and their routing.

Why it matters: first T&L project-management access to K-ai — gives Jill/Shambhavi direct visibility into course-production tracking (Box roster → KB sync) without expanding write scope or connecting to Asana (explicitly deferred by Vishal on the same thread). Scope stays: allowlist entry = Q&A access only; they are not added to outward reminders (weekly update / Monday alert).

2026-07-28 — G42 in effect: two-tier outbound email reaches the runtime, bounded to the main group

What: G42 (two-tier outbound email authorization) was approved 2026-06-12 and activated in the registry 2026-07-22, but until now it existed only on paper — the playbook mirror never landed, and groups/msbai-email/CLAUDE.md actively told K-ai the opposite (“Tier-2 relay, when implemented, will restore immediate nudges”). nanoclaw-msbai PR #25 (89a3c0dd, deployed, /health 200) adds an §Outbound Email Authorization (G42) section to the shared playbook with Tier 1/Tier 2 and all eight Tier-2 conditions, and removes the negation clause. Registry updated to v9.3.

Implementation surfaced a limit the rule text never drew: Tier 2 splits in half. The G37 logging obligation works everywhere; the send obligation is main-group (Telegram) only. send_email is gated to the main group per G36, and non-main calls are dropped silently (src/ipc.ts — warn + break, no error returned to the container). Without a scope caveat, the email and web-chat agents would read “Tier 2 is pre-authorized” as a granted capability, attempt a relay that no-ops, and could then claim they nudged an owner. The playbook now names where Tier 2 is actionable and forbids claiming a relay elsewhere.

Why it matters: this is the G42 lesson closing on itself. G42’s six-week dormancy is why the registry now distinguishes “approved” from “in effect” and marks unimplemented rows ⚠ — and the same rule nearly shipped a second, subtler version of the same failure: text that grants permission in a context that cannot act on it. A rule granting permission never grants a tool; where the two diverge, the playbook must say so, because the failure is silent on both sides (no send, no error, and a plausible-sounding claim in the reply).

Not built: email-group relay needs a host-side relay-proposal path — agent composes a structured proposal, host validates against the allowlist, enforces the ≤2-relay cap, writes the parent-authorization audit entry, sends — i.e. the outbound-email.ts / processOutreachResult compose-then-host-send pattern in a new shape. In-container grants remain off the table (2026-06-10 R2 review, 8 rounds: every scheme leaks through a group-shared writable surface). Deferred: the Ron Guymon case that motivated activation arrived on Telegram, where Tier 2 already works end-to-end.

2026-07-28 — Aged open questions now surface in the weekly digest (G55)

What: OPEN_QUESTIONS.md was never scanned by the weekly exceptions digest, so a question could sit unanswered indefinitely with nothing surfacing it — and G42’s newly-activated coordination relay logs routed questions there with no reply-chase. RC-012 is now G55: every open - [ ] question carries an (Added: YYYY-MM-DD) stamp, and the digest reports questions crossing a 14-day threshold, the standing aged count, and any question missing the stamp. Shipped in nanoclaw-msbai (collectAgedOpenQuestions in src/exceptions-report.ts, stamping in container/skills/kb-entry-gate/), reporting into the existing digest per G30 — no new notification path.

Two design calls worth recording, both made after checking the data rather than the ticket:

Dates come from git, not from memory. RC-012 offered “parse an Added: field” or “adopt a date-stamp naming convention”. The file had zero Added: fields across 105 open questions, so either option meant a ~105-item hand-backfill plus a new field K-ai must remember on every future write — and a convention living only in prose gets skipped under context pressure, which would make the monitor silently under-report. Instead the backfill was derived by git blame -w -M -C (move/whitespace-aware, so the file’s reorganisations don’t date every question to a reformat commit), the kb-entry-gate skill now stamps new questions deterministically, and the digest reports anything unstamped — so a skipped gate becomes visible rather than silent.

Flag-once, not “everything currently old”. Real-data validation before shipping: 99 of 105 open questions are already past 14 days. A plain threshold would have opened the first digest with 99 one-liners and trained its reader to skip the section. The monitor instead enumerates only questions crossing the threshold inside the reporting window — the same flag-once semantics as the G47 auto-close scanner, and for the same reason: no suppression state to maintain, and a question Vishal consciously left open is not re-nagged every Monday. The standing backlog remains visible as one number, which deliberately never wakes the digest by itself.

Why it matters: this is the second monitor in two weeks to land with flag-once semantics, and the pattern is now the house style for anything scanning a long-lived backlog — report the transition, count the state. Blame-derived dating is likewise reusable: where the repo already records when something happened, deriving beats maintaining.

2026-07-28 — Outbound choke point made true: every email send now passes one gated host path (G22/G46)

What: The G22 registry row claimed all outbound email enforcement lived in nanoclaw-msbai/src/outbound-email.ts — but ordinary email replies and the G17 access-closed courtesy reply called the Resend API directly from webhook.ts, bypassing the choke point entirely. The two highest-volume send paths ran without G46 shortfall reporting, and the courtesy reply wrote no audit record at all (invisible to G23). Found by a Codex Architect consult during the 2026-07-28 Azure pre-migration review, verified in code, consolidated in nanoclaw-msbai PR #28 (278f8798, deployed).

Four adversarial review rounds hardened the consolidation itself before merge, including a real find: the courtesy reply’s narrow allowlist exemption combined with comma-splitting of the To field would have relayed pilot mail to arbitrary external addresses given a spoofed multi-address From header. Now refused at two independent layers (single bare @illinois.edu addr-spec required; the choke point separately refuses any exemption send with more than one recipient). Also: reply failures now fire the G46 owner alert naming the recipient (previously server-log-only); the courtesy reply is audit-logged; replies are judged by the exact allowlist snapshot that admitted the inbound message; and no send path can trigger a synchronous git fetch on the shared event loop.

Why it matters: G22’s enforcement claim is now accurate rather than aspirational — there is exactly one way out, and it carries the allowlist, delivery-completeness, audit, and (future) provider logic. This is also the Azure migration’s mail prerequisite: swapping Resend for Graph (plan §4) is now a change to one file instead of a hunt across send sites.

2026-08-04 — Roster drift closed: stakeholders.md derived from the allowlist (G57)

What: Onboarding Brook Corwin exposed that program/stakeholders.md — the file K-ai uses to tailor responses by role — had silently drifted from program/EMAIL_ALLOWLIST.md: seven July-2026 adds missing, Kacie Jones still listed six days after her access was retired, and a wrong email for Ashish Khandelwal (akhandelwal@ vs the confirmed ashishk@). RC-015 is now G57: the sender roster in stakeholders.md is a generated block (scripts/sync-stakeholders.py, markers in the file), only non-sender contacts and Open Question Ownership stay hand-maintained, and the existing deploy CI runs --check on every push — failing when the block is stale, a retired sender lingers anywhere in the file, or an allowlisted sender is duplicated in a manual table row.

Why it matters: the allowlist called itself the single source of truth, but nothing bound the second file to it — so every onboarding depended on a human remembering an undocumented second step, and across six onboarding events in July none did. The fix follows the standing pattern: derive instead of maintain (the shared content now cannot drift), and put the residual check in CI next to G34/G43 rather than the weekly lint, because it is deterministic set-comparison needing no LLM judgment. The onboarding procedure itself is now documented in the allowlist’s Notes — the one file anyone editing membership is already looking at.