Somewhere on your team right now, someone is doing something for the third time this week that they've done a hundred times before. Restarting a service. Manually promoting a config change through environments. Checking a dashboard to confirm what an alert should have told them automatically. They're not filing a ticket about it. They're not mentioning it in standup. They're just doing it, because that's what you do.
That's toil. And the reason it's so dangerous is precisely that it's invisible.
Toil Hides in the Work That "Only Takes a Few Minutes"
The SRE discipline has a specific definition for toil: manual, repetitive, automatable work that scales with service growth and produces no enduring value. The key word is enduring. Restarting a service doesn't make the next restart less likely. Manually rotating a certificate doesn't improve the rotation process. You do the work, the work disappears, and the system is exactly as fragile as it was before.
What makes toil particularly insidious is the "few minutes" framing. Engineers are remarkably good at absorbing small costs. A five-minute manual check here, a ten-minute remediation there — none of it feels worth escalating. But toil compounds. If three engineers on a six-person team are each absorbing forty-five minutes of daily toil, that's a meaningful fraction of your operational capacity going to work that produces nothing durable. The system doesn't get more reliable. The team doesn't get smarter. The toil just... continues.
Google's SRE framework suggests that toil consuming more than fifty percent of an SRE's time is a problem worth treating as a production incident — not a staffing issue, not a backlog item, but an active threat to team sustainability. Most teams I've seen don't track this at all. They track incidents. They track deployment frequency. They track MTTR. They don't track how many hours per week their engineers spend doing work that a script could do better.
The Measurement Problem Is Also a Prioritization Problem
Here's the uncomfortable part: toil is hard to measure because it's distributed across people who have no incentive to surface it. If you're the person who knows how to do the manual certificate rotation, surfacing that as toil means either committing to automate it yourself or admitting you've been doing unnecessary work. Neither feels great in a sprint planning meeting.
The result is that toil tends to accumulate in the people who are most capable of absorbing it — your senior engineers, your most experienced SREs. They're efficient enough that the toil doesn't visibly slow them down, so it never gets prioritized. Meanwhile, those same people are the ones best positioned to eliminate it permanently. The toil is consuming exactly the capacity you'd need to address the toil. That's a trap.
The teams that break out of it usually do so by making toil visible before trying to eliminate it. A simple weekly log — what did you do that you've done before, how long did it take, could it be automated — surfaces the pattern faster than any retrospective. You don't need a sophisticated observability platform for this. You need a shared doc and a norm that surfacing toil is valued, not embarrassing.
Automation Isn't the Point; Judgment Is
The instinct when you find toil is to automate it immediately. Sometimes that's right. But the more useful question is: why does this toil exist?
Manual certificate rotation exists because the automated rotation broke once and someone lost trust in it. Manual service restarts exist because the underlying memory leak was never fixed. Manual config promotion exists because the deployment pipeline doesn't have the right environment parity. The toil is often a symptom of a deeper reliability gap — and automating the symptom without addressing the gap just means your automation will fail in the same way your manual process did, except now it'll fail silently at 3am.
The best operational teams I've seen treat toil elimination as a diagnostic process. Before writing the script, they ask: what does the existence of this toil tell us about the system? Sometimes the answer is "nothing, this is just a missing automation." But often it's "this service restarts because we never fixed the memory leak we identified eight months ago," and the right fix is the memory leak, not a more reliable restart script.
Toil measurement, in that framing, becomes a form of operational intelligence. The pattern of where your toil lives tells you where your systems are fragile, where your automation has gaps, and where your team has quietly accepted unreliability as normal.
The teams that run production well aren't the ones who've eliminated all toil — that's not achievable. They're the ones who've made toil visible enough to make deliberate choices about it. Which toil do we automate? Which do we eliminate at the source? Which do we accept, consciously, because the fix costs more than the toil?
Conscious acceptance is fine. Invisible accumulation is what breaks teams.
