You drew it six months ago. Maybe it was a whiteboard session, maybe a Confluence page with boxes and arrows, maybe just a mental model you've carried since the service launched. It felt complete at the time. Someone probably took a photo of the whiteboard and called it documentation.
Then your team shipped forty features, three other teams made changes upstream, and someone quietly added a retry library that now hammers a shared cache under load. The map in your head — and almost certainly the one on Confluence — describes a system that no longer exists.
This is the dependency problem that doesn't get enough attention. We talk a lot about unknown unknowns: the hidden dependency that surfaces during an incident and surprises everyone. But there's a more insidious version: the dependency you think you know, documented and trusted, that has quietly drifted from reality. The map that lies to you with confidence is more dangerous than no map at all.
Stale Knowledge Fails at the Worst Moment
The failure mode is predictable. An incident fires at 2am. Someone pulls up the architecture diagram to trace the blast radius. They make decisions based on what that diagram says — which services are affected, which teams to page, which rollback path is safe. But the diagram is six months old, and in the intervening time, Service B started calling Service D directly instead of going through the gateway. Nobody updated the diagram. Why would they? It was working fine.
The decisions made from that stale map are now wrong. You're not paging the right teams. You're not scoping the incident correctly. You're potentially making the rollback worse because you don't know about the new direct dependency.
This is why incident retrospectives so often surface "we didn't know about X" as a contributing factor — and why the fix almost never sticks. Teams add a note to the runbook, maybe update the diagram, and then the system keeps changing and the diagram falls behind again within weeks. The process of keeping dependency knowledge current is fighting against the natural entropy of a system that's actively being developed.
The Diagram Is a Snapshot; Production Is a Stream
The fundamental mismatch is temporal. A dependency diagram is a snapshot. Production is a continuous stream of change. Every deployment, every config update, every library version bump is potentially a topology change. Treating a snapshot as authoritative in a streaming environment is a category error.
The teams that handle this best tend to stop treating dependency maps as documents to be maintained and start treating them as outputs to be generated. If your service mesh, your distributed tracing, or your traffic analysis can produce a current dependency graph on demand, you're working from reality. If you're relying on someone to remember to update a diagram after a deployment, you're working from memory — and memory degrades.
This doesn't require exotic tooling. Distributed tracing instrumentation, when it's actually deployed consistently, shows you what's calling what in production right now. Service mesh telemetry gives you real traffic flows. The gap between "what we think the architecture is" and "what the traffic actually shows" is often the most useful diagnostic a team can run — not during an incident, but as a regular practice.
What Actually Helps: Treating Discovery as Operational Work
The teams I've seen handle this well share one habit: they treat dependency discovery as ongoing operational work, not a one-time documentation exercise. Concretely, that looks like a few things.
First, they instrument for observability at service boundaries — not just for performance monitoring, but specifically so they can answer "what is calling this service right now?" during an incident without having to guess. If you can't answer that question in under two minutes during a production fire, you have a gap.
Second, they run periodic dependency audits — not as a compliance exercise, but as incident preparation. Pick a service, generate its actual dependency graph from traffic data, and compare it to what the documentation says. The delta is your risk surface. Do this before an incident, not during one.
Third, they treat "we discovered an undocumented dependency during this incident" as a severity-one finding in the postmortem — not a footnote. If your system surprised you about its own topology during a failure, that's a structural problem worth fixing, not a curiosity to note and move on from.
The dependency map in your head will always be somewhat wrong. The goal isn't a perfect map — it's building the operational habits that let you discover the gaps before they cost you at 2am. The teams that do this well aren't smarter about their systems; they're more honest about how fast their systems change, and they've built practices that keep pace with that change rather than pretending a six-month-old diagram is still telling the truth.