In the last post I argued that debugging, not generation, is the bottleneck in hardware design. Here is what that actually means in practice.
The debug loop
Every RCA session, whether run by a human or an agent, follows the same loop:
- 1
Observe
Find the first point where behavior diverges from expectation.
- 2
Hypothesize
Propose which part of the design could have produced it.
- 3
Narrow
Trace signals backward, add probes, rerun, and rule things out.
- 4
Confirm
Prove the suspect is the cause, not just a correlated effect.
Each step sounds simple. The difficulty is in how fast the search space explodes.
Why the search space explodes
- Latency hides causality.In pipelined and multi-cycle designs, an input at one time affects an output many cycles later. The relevant evidence is spread across a long window.
- State you can't see.Registers, FIFOs, and FSMs carry history. The wrong value you observe may be the end of a long chain of earlier, individually plausible values.
- Masking.One bug can hide another. Fix the visible one and a second failure appears, which makes it easy to mistake the problem for a different one.
- Many fixes, one cause.A symptom can often be silenced in several places. Only one of them is the root cause, and the others leave the real fault in place.
- Handshakes and backpressure.Protocol-level bugs often appear only under specific stall or timing patterns that a quick test never exercises.
Diagnosis versus patching
Diagnosis
Explains the whole causal chain: where the bad value originated, how it propagated, and why it surfaced where it did.
Patch
Only makes the symptom disappear. In practice, patches that don't address the cause come back later as harder, more intermittent failures.
What this means for agents
An agent that does RCA well needs to reason causally across time and structure, form and discard hypotheses, and resist the temptation to stop at the first plausible answer.
That is a very different skill from writing code that compiles.
Why AI agents fail at root cause analysis
So agents can write RTL. Why can't they debug it?
Treating the symptom
The most common mistake is finding the first mismatching signal and editing the logic around it. The test may even pass afterward. But the fault is still in the design, now hidden behind a patch.
Recognizing syntax instead of reasoning about behavior
Models are trained on a lot of public Verilog, and public Verilog is repetitive. Common idioms and common bugs appear again and again. An agent can learn to spot a familiar bug in a familiar coding style without understanding why it is a bug. Rewrite the same logic in a different structure and the recognition fails.
This is a hypothesis, not an established result, and it is one the veriRCA benchmark is designed to test.
Memorization
If a benchmark has a few dozen designs with a handful of known bugs, a model that has seen them (or similar ones) in training can score well without any debugging skill. The score reflects recall, not reasoning.
Weak ground truth
Hand-labeled bugs are expensive to produce, and labels can be ambiguous. When a design has several plausible causes, scoring an agent's answer against a single human-chosen label is noisy. It's hard to tell a wrong diagnosis from a different but valid one.
The real bottleneck is data
All four problems point at the same issue. To measure and train RCA, you need:
- Many designs
- Many bugs
- Exact ground truth
- Enough variation to stop memorization and pattern-matching
Real-world bugs with verified root causes are rare, private, and slow to collect. Hand-writing them doesn't scale.