Insights · Production Triage
What I look for in a production triage.
Three days is not enough time to read every line of a codebase. It is enough time to find the risks that actually sink launches — because they cluster in the same four places, every time.
Systems rarely fail where the code is hardest.
The bugs founders worry about — the clever algorithm, the model quality, the scaling story — are rarely what takes a system down. What takes systems down is the unglamorous connective tissue: a contract two services interpret differently, a migration that cannot be safely re-run, an environment variable that only exists on one machine, a deploy process that depends on someone remembering a step.
So a triage does not start by reading code top to bottom. It starts by tracing the paths where damage would be expensive: money, identity, and irreversible writes.
The four places I look first.
Contracts. Every boundary where two components exchange data — API schemas, queue messages, webhook payloads, LLM tool signatures. I look for places where the contract is assumed rather than enforced, because that is where a harmless-looking change in one service silently corrupts another.
Data flow. Where does state actually live, and what happens when a step fails halfway? Idempotency is the question I ask most in any triage: if this job runs twice, does the system end up correct, or does it double-charge, double-book, or double-count? Most prototype ETL fails this test, and nobody discovers it until production makes it fail for them.
Infrastructure. Not "is it modern" — whether it is observable and reproducible. Can the team see a failure before a customer reports it? Can they rebuild the environment from what is in the repository, or does production depend on undocumented state? For AI systems specifically: is there any evaluation harness, or do regressions in model behavior ship silently?
Release discipline. How does code get from a laptop to production, and what is the undo path? A system with no rollback plan turns every post-release defect into a forward-fix emergency. The cheapest risk reduction in most triages lives here.
Ranking by blast radius, not by count.
A triage that produces forty findings is a triage that produces zero decisions. Every risk I report is ranked by blast radius times likelihood, tied to the specific component where it lives, with a plain statement of what happens if it fires. Five well-ranked risks beat forty generic ones, because five is a number a founder can act on.
The remediation plan that follows is written so any senior engineer can execute it — bounded work items with acceptance criteria, not advice. If the highest-leverage move is "do not build the thing you were about to build," the report says so.
What a good outcome looks like.
Sometimes the outcome is a red flag caught before it reached production — an ETL that would have duplicated payment records, an auth cutover that would have locked out the oldest customers. Sometimes the outcome is documented evidence that the system is sound and the team should ship without fear. Both are worth three days, because both replace uncertainty with a decision.