← All work Case study · Directing AI agents · Verification

Referral System: directing AI agents through a production rebuild

A student-referral system used school-wide at a Norwalk elementary school. I rebuilt it in 16 phases with AI agents, using a written handoff document as their persistent rules, and checked their work at every phase.

Audience
Teachers & administrators
Platform
Google Apps Script
Process
41 commits · 16 phases
Status
Live, school-wide
Admin portal before the rebuild
Before
Admin portal after the rebuild, sample data
After
All screenshots use fictional demo data.

The problem

The original referral form was long and slow, and administrators had no clear queue for follow-up. It had to stay on Google Apps Script (district constraint) and be rebuilt without disrupting a live system that staff rely on daily.

How I directed the agents

Codex planned the phases, and Claude Code implemented them. The key artifact was an 866-line handoff document written so any fresh agent session could pick up exactly where the last left off, with the constraints stated as hard rules:

HANDOFF.md
"It must stay Apps Script… Do not propose moving it… even in passing."

Each phase had a scope, acceptance checks and a commit, which produced 41 commits from Phase 0 to go-live between late August and September 2026.

The false-pass catch

One automated check comparing old and new behavior reported:

Verifier output
WEAK PASS: agreed on all 0 sample(s)

It "passed" because there was nothing to compare. I had the check reclassify zero-sample results as INCONCLUSIVE, so an empty test could never look like a success again.

Why this matters: evaluating AI means distrusting green checkmarks. A metric that can pass vacuously is worse than no metric, and noticing that is the core of evaluator work.

Measured results

  • Required form fields cut from 11 to 6, with 22 optional fields moved behind a disclosure
  • Teacher page payload cut from 335 KB to 217 KB
  • WCAG AA contrast: a specified text color at 4.28:1 darkened to pass 4.5:1
  • Verified no horizontal overflow at 375px across all 7 views

Staff enablement

  • 794-line operator setup guide and deployment notes
  • Two PDF user guides for teachers and administrators
  • A newsletter for the principal to introduce the new system to staff

What this shows for AI training roles

  • Writing persistent instructions and constraints that keep agents on track across sessions
  • Catching vacuous or misleading evaluation results
  • Setting phase-by-phase acceptance criteria and holding AI output to them
  • Shipping to real users, then training them