Have you ever stepped through -O3 code?
A jumping cursor and a variable that is optimized out are not bugs. They are what a source-level coverage verdict rests on. Where is the evidence it is right?
Set a breakpoint in an optimized build. Step.
The cursor jumps backwards. It stops twice on the same line. It skips three
lines that definitely execute. A variable you are watching reads
<optimized out>. You step into a function and land in a different one that
was inlined into it, then step again and land in the caller.
Every experienced embedded developer has had this session. The usual reaction is
to rebuild at -O0, or to give up and read the assembly.
Now hold that experience next to the following claim: this build has 98 % MC/DC coverage at source level.
The number was produced by mapping machine addresses back to source lines - through the same debug information that just told you the program was executing line 41 while it was demonstrably somewhere else.
The mapping is not one-to-one, and the compiler says so
At -O0 there is a clean correspondence between a source statement and a
contiguous run of instructions. Optimization removes it, deliberately:
- Inlining copies a function body into several callers. One source line now has instructions at many addresses, in different functions.
- Code motion hoists a computation out of a loop. The instruction now runs at a point in the program where its source line is not being executed.
- Merging and folding replace several source constructs with one instruction sequence.
- Duplication - loop unrolling, tail duplication, jump threading - produces several copies of one source line, each with its own history.
- Elimination deletes code the compiler proved unnecessary. Its source line has no instructions at all.
These are not defects. They are why the optimized build is fast, and every one of them breaks the assumption that an address maps to the source line.
What the compiler does when it cannot decide
This is the part worth sitting with, and it is documented in the compiler’s own source. When LLVM merges two instructions that carry different source locations, it has to produce a location for the result. Its documentation states the rule verbatim:
When two instructions are combined into a single instruction we also need to combine the original locations into a single location. When the locations are the same we can use either location. When they differ, we need a third location which is distinct from either. If they share a common scope, use this scope and compare the line/column pair of the locations with the common scope: if both match, keep the line and column; if only the line number matches, keep the line and set the column as 0; otherwise set line and column as 0. If they do not share a common scope the location is ambiguous and can’t be represented in a line entry. In this case, set line and column as 0 and use the scope of any location.
- LLVM,
include/llvm/IR/DebugInfoMetadata.h, documentation ofDILocation::getMergedLocation
Read the emphasized sentence again. The compiler is not making a best guess and recording it. It is stating that the question has no answer, and writing down a designated value that means no source line.
That is the honest behavior, and it is the right engineering decision. It is also fatal to a claim built on top of it: the entry exists, it is well-formed, it parses - and it asserts nothing.
The debug format provides that escape value on purpose. In the line number
table, the line register is defined like this:
An unsigned integer indicating a source line number. Lines are numbered beginning at 1. The compiler may emit the value 0 in cases where an instruction cannot be attributed to any source line.
- DWARF Debugging Information Format Version 5, section 6.2.2, table 6.3
The neighboring register is worth reading in the same breath. is_stmt is
“a boolean indicating that the current instruction is a recommended
breakpoint location” - one that is “intended to represent a line, a statement
and/or a semantically distinct subpart of a statement”. Recommended. Intended to
represent. The quotation marks around represent are the standard’s own.
This is not sloppy drafting. It is a format being honest about the fact that after optimization there is often no such thing as the line for an address.
A reproducible instance
None of this requires a large program. Take a macro that expands into two
nested if statements:
void act(int);
#define CHECK(x, y) \
if ((x) > 0) { \
if ((y) > 0) { \
act(1); \
} else { \
act(2); \
} \
}
void h(int a, int b)
{
CHECK(a, b);
}
Compiled with clang 22.1.4 for AArch64 at -O2 -g, the emitted assembly
carries its line table as .loc directives. The first comparison is
attributed to line 14, column 5 - the macro invocation, not the macro body.
The second comparison, and the conditional increment the compiler made out of
the inner if/else, carry line 0 (excerpt):
.loc 1 14 5 prologue_end // ex3-macro-provenance.c:14:5
cmp w0, #1
...
.loc 1 0 5 is_stmt 0 // ex3-macro-provenance.c:0:5
cmp w1, #1
cinc w0, w8, lt
...
.loc 1 14 5 is_stmt 1 // ex3-macro-provenance.c:14:5
Line 0 is the value the standard reserves for an instruction that “cannot be attributed to any source line” - the escape hatch quoted above, in use, in a fourteen-line program. Read the object file back with a disassembler and the whole block is annotated with line 14:
0000000000000000 <h>:
; ex3-macro-provenance.c:14
0: cmp w0, #0x1
4: b.lt 0x18 <h+0x18>
8: cmp w1, #0x1
c: mov w8, #0x1
10: cinc w0, w8, lt
; ex3-macro-provenance.c:14
14: b 0x14 <h+0x14> (relocation: act)
; ex3-macro-provenance.c:15
18: ret
Two things happened here at once. The inner decision has no line of its own;
and it has no branch of its own either - the compiler turned if (y > 0) act(1) else act(2) into a conditional increment that selects the argument.
A tool that credits line 14 when the first comparison executes has said
nothing false. It has also said nothing about the inner decision, which is the
one a reviewer would ask about.
Three consequences that show up in a report
1. Attribution to the wrong line, silently
A hoisted instruction carries the line it came from, which is not the line whose execution it now indicates. A coverage tool counting executions at that address credits a line that did not run in the sense the report implies.
Nothing about this is detectable from the data. The entry is present and plausible.
2. Coverage for a line that no longer exists
If a decision was eliminated because the compiler proved it constant, the source line remains in your listing and has no instructions behind it. Depending on the tool, it is reported as uncovered - correct but misleading, since no test could ever cover it - or quietly dropped from the denominator, which is worse.
Telling those apart is exactly the residue question that DO-178C section 6.4.4.3 asks about.
3. Column information that is not always there
The usual answer to multiple expressions on one line is DWARF column information. It helps - and it is not universal. In our own compatibility notes we recorded revisions of the same compiler family where column information is emitted for one target and not for another. A tool that relies on it inherits that variability without being able to detect it.
So where is the evidence that DWARF is correct?
This is the question the whole subject turns on, and it is almost never asked. A source-level coverage verdict on an optimized build rests on the debug information being right. Every argument for the verdict therefore contains, as an unstated premise, the claim the debug information is correct.
Ask for the evidence behind that premise. There are two places it could come from, and neither one supplies it.
The standard does not promise it
You might expect the format specification to define what a compiler must emit, so that “correct” has a meaning and a producer could be tested against it. It explicitly declines to. The section is called “Permissive Rather Than Prescriptive”:
The DWARF Standard specifies the meaning of DWARF descriptions. It does not specify in detail what a particular producer must generate for any source to object conversion. […] DWARF is permissive, allowing different producers to generate different descriptions for the same source to object conversion, and permitting different consumers to provide more or less functionality or information to the user. This may result in debugging information being larger or smaller, compilers or debuggers which are faster or slower, and more or less functional. These are described as differences in quality-of-implementation.
- DWARF Debugging Information Format Version 5, section 1.3.12
And, two paragraphs further, on the force of its own wording:
Where the text says “should”, this is advice about best practice, but is not a requirement.
Read that as a certification engineer rather than as a compiler engineer. Two compilers may describe the same translation differently, and both are conforming. Richer or poorer descriptions are not right or wrong, they are quality-of-implementation. There is therefore no conformance criterion against which a producer’s debug information could be qualified - not because nobody has written the test suite, but because the specification deliberately does not define the thing the test suite would check.
For the compiler’s code generation the situation is the opposite: the semantics of the source language define what the output must do, which is why a code generator can be qualified at all.
The one systematic study says the opposite of reliable
If the specification does not settle it, measurement might. In 2020 a team from Georgia Tech and Apple built the first framework for systematically testing compiler-generated debug information on optimized code, and published it at PLDI. Their own framing of the state of the field:
Almost all modern production software is compiled with optimization. […] However, little is known about the correctness of the debug information generated by optimizing compilers when debugging optimized code.
The difficulty is that there is no oracle: on an optimized build there is nothing to compare the answer against. They constructed one - compile the same program unoptimized, take the value observed there as ground truth, and force the two builds to be comparable by inserting an unoptimizable barrier at the location under test.
What came out of it:
| Confirmed bug reports against GCC and LLVM | 47 (21 GCC, 26 LLVM) |
| Fixed at the time of writing | 11, about 23 % |
| Affected the then-current development version | all of them |
| Oldest still-live finding | reached back to GCC 4.4, released about nine years earlier |
| Optimization levels affected | GCC across -O1 to -O3, and -Og - the flag whose purpose is a better debugging experience - was not exempt; LLVM across -O1 to -O3 |
| Second compiler, three days of testing | 2 further confirmed bugs, in the Rust compiler |
The LLVM component with the most findings was the DWARF parser itself, and one report went to the DWARF standard committee as a request to clarify wording that two implementers had read differently.
And then the sentence that matters most for anything built on top of this:
It is interesting to note that the wrong variable information in both examples is only observable in a debugger. GCC and LLVM generate the correct optimized machine executables […] It demonstrates that the state-of-the-art optimizing compilers may generate wrong debug information for optimized code.
- Li, Ding, Zhang, Italiano: Debug Information Validation for Optimized Code, PLDI 2020
The binaries were correct. The programs did the right thing. Only the description of what they were doing was wrong - and nothing except a debugger, or a purpose-built oracle, would ever notice.
What this does and does not show
Being precise about the scope, because the argument is stronger without stretching it:
- The study tests breakpoint locations and printed variable values, not coverage verdicts. It does not report a number for wrong line attribution in a coverage tool, and neither do we.
- It tests two compilers plus one, not every toolchain, and certainly not whichever one you use.
- Eleven fixes and an active response from maintainers say this is a healthy project, not a scandal. LLVM even has a dedicated routine that tries to reconstruct debug information when an instruction is deleted; the paper notes it is explicitly a best-effort function that cannot always succeed.
What it does establish is the thing the premise needed and does not have: when someone finally built an oracle and looked, findings appeared immediately, in the current versions of both major compilers, at every optimization level - in an area where a defect changes program behavior by nothing at all.
So when a tool description says the source-level mapping is reliable, the question is simply: on what evidence? The format does not define correctness, the compilers are not qualified for it, the tests do not exercise it, and the one group that measured it found bugs everywhere it looked.
Why an assessor does not accept this
Put yourself on the other side of the table. The question is not whether the tool is good. It is:
How do you know that the source-level verdict is correct?
For the compiler’s code generation, there is a clean answer: you verify the product. If code generation were wrong, the program would misbehave, and the test campaign would find it.
For the compiler’s debug information, that answer does not exist. Wrong
line tables change program behavior by exactly nothing. The binary passes every
test with a thoroughly broken .debug_line section - including the very tests
that produced the coverage report.
So the reviewer is left with three questions and no comfortable answers:
| Question | Answer for code generation | Answer for debug information |
|---|---|---|
| Is it verified by testing the product? | yes, that is the standard argument | no - it is functionally silent |
| Is there a defined correctness criterion? | yes: the program’s semantics | no: the format is permissive by design (section 1.3.12), and has a value meaning unknown |
| Does a failure announce itself? | yes: wrong behavior | no: a wrong line entry looks like a right one |
This is not an argument that debug information is useless. It is excellent for what it was built for - a debugger, where a human is in the loop and notices the jumping cursor. It is an argument about what it can carry as evidence, and the answer is: less than a certification report assumes.
What we concluded from it
Our own patent application states the problem in its background section, and it is worth quoting because it is where the design came from:
The control-flow reconstructed from the trace is at the machine level. For tasks that require a transformation to the source code level, this can be done using DWARF debug information. However, especially when optimizing compilations, it is usually not possible to make a precise assignment between the machine level and the source code level, because the compiler may have added additional code, removed code, transformed code (e.g., converted a control-flow into a data flow), or moved code.
- the background section of our published international patent application; publication details are on the patents page
The design decision that follows is a single sentence: the symbol table is authoritative, DWARF is advisory.
The symbol table is different in kind. The linker consumes it to build the binary, so it is exercised by the same process that produces the artifact under test. A symbol resolves to an address or it does not - that criterion is defined and mechanically checkable. And when an anchor symbol is missing or has collided, that is visible in the table, which makes the failure mode detected rather than silent.
Debug information is still used - to enrich source-level reporting, because it is genuinely good at that. It is simply not the thing the verdict rests on, and where the two disagree, the link-verified mapping wins.
The mechanism is described in why the mapping breaks under optimization and on the CEDARtools.Anchor page.
What to ask, concretely
If you take one question from this article into a tool evaluation, take the first one. It is the one that has no comfortable answer.
- Where is the evidence that the debug information is correct? Not that the tool reads it correctly - that the compiler wrote it correctly. Ask for the specification clause it conforms to, the qualification argument, or the measurement. All three are worth asking for precisely because none of them exists in the general case.
- At which optimization level was the source-level mapping validated?
- What does the tool do with an instruction whose debug location is line 0? Count it, drop it, or flag it - all three are defensible, and only one of them is what you want in an evidence report.
- What happens when debug information and symbol table disagree?
- How would a wrong attribution be detected? If the honest answer is “it would not be”, that is worth knowing before the review rather than during it.
Related: DWARF as a chain of trust - the qualification argument in short form · what an assessor actually asks
Sources, listed for reference and verification: the compiler
passage is quoted from LLVM’s own header
include/llvm/IR/DebugInfoMetadata.h. The specification passages are from the
DWARF Debugging Information Format Version 5,
sections 1.3.12 and 6.2.2. The study is Yuanbo Li, Shuo Ding, Qirun Zhang and
Davide Italiano, Debug Information Validation for Optimized
Code, PLDI 2020. A wrong citation in an
article about wrong information would be a poor joke, so every quotation above is
traceable to the primary document.