Latest issue
Semantic Caching Saves Tokens, Not Trust — And the Threshold Decides Which One You Get
10/27/2026
A software engineer watched his team's LLM bill triple in three weeks with no pricing change and no traffic spike. When he finally pulled the logs, the cause wasn't volume — it was repetition. Thousands of near-identical support questions, phrased ten different ways, each one hit…
Recent posts
- Your RAG Eval Dashboard Says 94% Accuracy. Your Retrieval Layer Is Still Returning the Wrong Policy.
I wrote about retrieval quality back in April (/p/your-rag-pipelines-retrieval-layer-is-lying-to-you), and the core problem hasn't changed: a RAG pipeline can look healthy on every…
A small team's production app threw a 503 from its LLM provider at 2pm on a Tuesday. Then again two days later. The engineer behind it manually failed over to a backup, updated env…
On September 4, 2026, OpenAI, Anthropic, Google, and xAI all degraded within the same few-hour window, according to reporting compiled by baeseokjae's postmortem analysis. Anthropi…
Most prompt injection writing is theoretical until someone actually counts. A team running an on-call agent did the counting: they hid 96 attack instructions inside the logs and ti…
Thursday, September 3rd: ChatGPT, Claude, and Grok all degraded within the same morning window. Pagerly's incident guide documented the pattern most teams experienced — a trickle o…
A team builds a weather agent. They estimate 500 input tokens plus 200 output tokens per query, run the math against their provider's rate card, and budget $3.60 a month. In produc…
A request comes in. It takes four seconds to return. Your GPU utilization looks fine. Your error rate is zero. Your logs show a 200. And you have absolutely no idea what just happe…
A team runs 500,000 queries a month. Their dashboards look fine. Then a customer audit reveals that 30% of answers were fabricated from fragments of the wrong document sections. Th…
A RAG-based customer support assistant shipped to hundreds of users before anyone noticed it was confidently citing billing policies that didn't exist. The post-mortem finding, per…
Most teams treat rising LLM costs as a vendor negotiation waiting to happen. They watch the per-token price, wait for a cheaper model to drop, and assume the bill will eventually c…
A RAG-based customer support assistant shipped to production and confidently cited nonexistent billing policies to hundreds of users before anyone caught it. The post-mortem findin…
Most teams add a fallback model and call it done. Then they hit a 429 at 11am on a Tuesday, watch their retry logic turn one failed request into forty, and discover that "just add…
A request comes in. The model returns something wrong. You open your logs and find: status: 200, latency: 1.4s, tokens: 847. Completely healthy. Completely useless. This is the def…
There's a specific failure mode I see constantly in production AI systems: a team sets a latency SLO, hits it consistently on their dashboards, and still gets user complaints about…
There's a specific kind of production incident that starts with a Slack message like "the extraction pipeline is returning nulls again." You dig in. The LLM returned valid JSON — s…
Most teams discover their LLM fallback strategy is broken at the worst possible moment: not during a 503 from a provider, but three steps later, when a downstream service starts be…
There's a specific kind of production incident that never pages anyone. The agent runs on schedule. Latency is flat. Costs are stable. HTTP 200s all the way down. And over six week…
A startup launched an AI research assistant. Their cost model said $0.04 per query. Their actual cost was $4.20 per session. By week three, they'd accumulated $67,000 in unexpected…
Most teams discover their LLM capacity plan is wrong at 2am, not during sprint planning. The bill looked fine. The projected monthly spend was within budget. Then a queue drained,…
