Root Cause Analysis for Forward Deployed Engineers: How to Diagnose Before You Build

When a number suddenly goes wrong in production — returns spike, sign-ups crater, latency triples — the untrained instinct is to start building a fix. A forward deployed engineer does the opposite. Before touching code, you run root cause analysis: you find the true cause of the problem, not the first suspect that looks guilty. Diagnosis comes before construction, because a fix aimed at the wrong cause is worse than no fix — it burns a sprint and leaves the real problem running.

I want to walk through that discipline using one incident: a fashion e-commerce company whose product returns exploded right before the biggest sale of its year. The numbers here are illustrative, but the method is exactly how you'd work it as an FDE — and the root cause at the end is one an engineer I trained under spent three real days chasing in his own company.

#The incident: returns triple in three days

Ahead of the festival-season rush, this retailer averaged about 200 order returns a day — background noise. Then it jumped: roughly 2,000 returns on September 12th, 2,500 on the 13th, and 3,000 on the 14th. A ten-to-fifteen-fold spike heading straight into the highest-traffic, highest-revenue window of the year.

Every returned order is a refund, a reverse-logistics cost, and a customer who may not come back. The pressure is to do something immediately. The FDE's value in that moment is refusing to confuse motion with progress.

#Separate the three problems before solving any of them

The first move is to split the panic into three distinct questions, because they have different owners and different timelines:

1. Short-term — identification: what is actually causing the spike right now?
2. Immediate — resolution: how do we stop the bleeding today, before we fully understand it?
3. Long-term — systemic fix: what change keeps this from happening again?

Teams fail here by collapsing all three into "fix the returns." You cannot resolve what you have not identified, and you should not ship a permanent system to solve a cause you haven't confirmed. Asking why before what and how — the habit that separates a developer from a forward deployed engineer — applies to broken systems just as much as to new features.

#Draw the system before you diagnose it

You can't find a root cause inside a system you can't picture, so map it first. Two flows matter here:

• The delivery cycle: an order placed on the website goes to a warehouse, out to one of several delivery partners, through a pickup warehouse and transit, to the customer — with a tracking ID threaded through the whole path.
• The refund cycle: a customer opens a chat, a rule-based bot attempts an automatic refund, and anything it can't handle escalates to a human in customer support who works it through the CRM — problem, decision, process.

Sketching these does two things at once: it exposes every point where a return could be triggered, and it shows where the evidence lives — tickets, tracking records, the CRM. Root cause analysis is only as good as the map you draw before you start.

#Bucket the symptom — RCA starts with categories

A "returns spike" is not a cause; it's an aggregate hiding several. So the first analytical move is to segregate the returns into categories. My own first cut was by failure type:

• incorrect product delivered,
• size mismatch,
• quality issue.

A fellow engineer cut it a sharper way — by where the fault actually lived:

• internal — our own website or listing;
• external — a vendor, the delivery service, or a wrong product;
• customer-facing — the buyer found it cheaper or found a better alternative.

Both are valid; the second is closer to true root cause analysis because each bucket points at an owner. This bucketing is the same reflex you use when scoping what to build — but the goal is inverted. There you categorize to decide what to build first; here you categorize to find what's broken. Either way, you can't diagnose or prioritize what you haven't separated.

#Narrow to the cause with data, not opinions

With buckets in place, you drill — the way the "5 Whys" technique drills — replacing each guess with evidence and throwing the guess away:

1. Interview upstream. The first suspect is always "did we change something?" Ask the data-engineering and backend teams whether there was a recent deploy, migration, or listing change. Here the answer was no — so you rule it out and move on, instead of letting the loudest theory win by default.
2. Analyze the tickets. Pull the return reasons from the CRM and count them. One reason dominates the chart: size. Most returns are size complaints, not quality or wrong-item.
3. Segment the dominant reason. "Size" is still too coarse. Cut it by region, by delivery partner, and by product type. The size returns aren't spread evenly — they concentrate.
4. Follow the concentration to its source. That concentrated segment traces back to products from a single vendor — one that had been onboarded only about two months earlier.

Each step discards a hypothesis and keeps the evidence. That's the whole game: you're not assembling a plausible story, you're eliminating everything that isn't the cause until one thing is left standing.

#The root cause was a tag, not a technology

Here's where the case turns instructive. The new vendor's garments carried the origin country's size chart — Chinese sizing — and were listed on an India storefront without converting the sizes. Same listing, same label "M," a different body underneath it. Customers ordered their usual size, it didn't fit, and they sent it back. The onboarding protocol that should have normalized sizing to the destination market simply hadn't been followed for that vendor.

Notice what the root cause was not. It wasn't a bug in the returns flow, not the delivery partner, not a wave of fraud, and not anything you could throw an AI at. It was a process gap printed on a physical tag. The engineer who taught me this method spent three real days arriving at exactly that answer inside his own company — three days of narrowing, not building. In my own work I've relearned the same lesson repeatedly: the first cause that looks obvious is almost never the real one, and the discipline is to keep asking "why" one level past the point where it already feels finished.

#Only now decide the fix — and whether AI belongs in it

With the cause confirmed, the three timelines finally get real answers:

• Immediate: correct or pull the offending listings and add the right size conversion — stop today's bleeding.
• Short-term: audit every recently onboarded vendor for the same gap, because a protocol skipped once is usually skipped more than once.
• Long-term: fix the system that let it through — a real vendor-verification protocol (proof of origin, size normalization, a transparency record in the database) and a better support experience, replacing the brittle rule-based bot with a multilingual, retrieval-grounded assistant that has guardrails instead of guesswork.

But — and this is the FDE part — that long-term solution may use AI, or may not. A multilingual RAG chatbot is one candidate, not a foregone conclusion. Before building it you do a cost analysis, propose the optimal approach, ship an MVP, and only then deploy. Reaching for an LLM is a decision with its own rules, and a vendor-verification protocol is mostly process, not model. The FDE's job is to recommend the optimal approach and own the outcome — not to reach for the shiniest one.

#Diagnosis is the deliverable

Anyone can ship a fix. The forward deployed engineer's edge is shipping the right fix — which means spending the unglamorous hours to find the true cause before a single line of code is written. In this incident the entire remedy was a size conversion and a tightened process; all the value lived in the three days of diagnosis that pointed there. Root cause analysis isn't the preamble to the work. For an FDE, it is the work.