How did Stephen Toub catch regressions when AI agents ported the Copilot codebase to Rust?

Stephen Toub said end-to-end tests were the key safeguard, and nearly every regression traced to behaviour those tests did not cover; he wishes he had strengthened them further before starting. About two thirds of regressions were caught on main before any pre-release, which ships three times a day to roughly 5 to 10% of users. Others appeared as GitHub issues asking whether a behaviour change was intentional, or in crash telemetry.

• Toub said Rust's compiler did not catch the lifetime and concurrency bugs introduced by moving from single-threaded TypeScript to concurrent code.
• He said that even if agents are right 99.99% of the time, a port of a million lines still leaves real bugs.

Source: our own transcription of the episode. The full conversation is on Ryan Peterman (2026-11-02). Read the whole episode summary →

▸More from this episode