Chak Sing Ho | Hans

Software Engineering Intern, Intelligent Creation (Effect House)

TikTok · San Jose, CA · May 11, 2026 - Sep 8, 2026

• Architected the control plane of an overnight agent-evaluation flywheel in which coding agents fix real production bugs: SQLite-transaction + TTL-lease task ownership with fencing, non-blocking flock admission, git-worktree isolation, and a supervisor state machine, running 32 nightly shards across 12 lanes with crash-safe recovery and no duplicate execution.
• Showed that agent-written tests overstated the pass rate about 2.4x (65.6% vs 27.5% on the original developer tests, n=854, reproduced twice), and led an adversarial audit of the grading chain that rebuilt verdicts around fail-to-pass oracles, closed-book agents, and a single verdict writer.
• Built a Runtime Verifier that judges a fix by driving the real editor on pre-fix and post-fix builds and returns PASS only on a fail-to-pass flip, using a TypeScript job state machine (RECEIVED -> PREFLIGHT -> RUNNING -> CLEANING_UP -> REPORTING) over a Python verdict kernel, schema-enforced reason codes for every non-PASS, and a crash-safe file queue serialized by non-blocking flocks.
• Wired the verifier into Lark through a bot and a zero-dependency MCP server: a developer @-mentions the bot, an agent submits the job, and verdict cards post back (12.1 s end to end on the fastest tier); added a human-gated reproduce -> approve -> external-agent fix -> judge loop that returns only a one-bit retry signal and voids any PASS from an agent session that saw the held-out test.
• Built an MCP performance agent (31 governed tools) that measures AR effects against fail-closed, same-workload baselines, attributes cost through JS profiling, frame telemetry, and hierarchical ablation, and writes fixes back transactionally (expected hash, journal, atomic replace, automatic restore).
• Flywheel · Made recovery safe with a composite identity — task id, run id, controller generation, lane id, owner token, pinned revision, worktree — compared together, so a worker resurrected from an older generation cannot write into state it no longer owns. Shared JSON is published by temp file, fsync, and atomic replace, and corrupt ledgers fail closed rather than reading as empty and silently re-running work that already landed.
• Flywheel · Designed the lock taxonomy (supervisor singleton flock, lease lock, eval admission slot, reentrant ledger lock, telemetry append/rotate lock, single distill-compiler lock) under one rule: a lock must span the entire read-decide-write, because locking only the final write still loses updates. Parallel lanes avoid contending on shared knowledge files altogether by writing content-addressed fragments that one compiler folds together.
• Flywheel · Ran the supervisor as a pure state machine over collected facts — hold, restart, repair, stop — behind a restart circuit breaker, with a hard line between a broken road that self-heals (worktree gone, process dead) and a pulled stop gate that must never auto-clear.
• Flywheel · Ran it as a multi-agent system where roles are separated by context, not just by name: a blind fixer agent that cannot see the grader, a reviewer fed only a summary of the previous attempt, a scheduler choosing the next task, and producers minting tasks and reaping dead worktrees. ~18 worker loops over roughly 6 concurrent model slots, 32 nightly shards across 12 lanes.
• Flywheel · The hardest problem was diagnostic rather than functional: twelve distinct failures — a runaway process escaping its cgroup, a service booting from a tree with no modules, timeouts reporting success as failure, one lane killing another lane's editor through a default port fallback — all surfaced as the same symptom, "the editor is flaky." Introduced a shared failure-layer vocabulary so every refusal names its own layer and no layer may report another layer's failure as its own.