Menu

Searches titles, sections and text. ↑ ↓ to move, Enter to open, Esc to close. Opens with / or Ctrl+K.

    Contact

    100 % MC/DC, 40 % Evidence

    One intermediate Boolean can halve what your coverage report proves. We measured it on current Clang and GCC - every step reproducible in your browser.


    The pattern

    x = a && b && c;
    /* ... */
    y = x && d && e;

    A source-level coverage tool reports two decisions here. The first has three conditions. In the second, x is a condition - a leaf, not a sub-expression. That follows directly from the DO-178C definition of a decision as a Boolean expression built from conditions and operators, and it is what essentially every coverage tool implements.

    It is also, in the general case, the only defensible reading. Between the two statements, a, b, c or x may be reassigned, may be volatile, may be written from an interrupt handler. Folding the two statements into y = a && b && c && d && e is sound only under a clean def-use chain with no intervening write.

    But when that def-use chain is clean, the two readings are not equally strong. The gap is larger than most teams expect.

    An example that survives review

    A safety interlock for a motor stage. Written as one decision, it looks like this:

    /* drive.c */
    void actuator_step(void)
    {
        if (temp_ok && curr_ok && supp_ok && cmd_ok && !fault) {
            drive_enable();
        } else {
            drive_disable();
        }
    }

    Five conditions. MC/DC requires six test cases, and five of them are genuinely hard: each needs the other four conditions held at a specific value at the same instant. On a HIL bench, that is real work.

    Now the same logic, restructured. Nothing below is unusual, and nothing looks like an attempt to game a metric:

    /* status.h */
    static inline bool supply_healthy(void)
    {
        return temp_ok && curr_ok && supp_ok;
    }
    /* drive.c */
    void actuator_step(void)
    {
        if (supply_healthy() && cmd_ok && !fault) {
            drive_enable();
        } else {
            drive_disable();
        }
    }

    Separation of concerns: supply health is one named idea with one definition. Reuse: three other modules need the same predicate. Readability: the if now fits on one line and reads like the requirement. Every one of those justifications is sound engineering.

    No bad intent is required for this pattern to appear. That is precisely what makes it worth talking about - and a code generator can produce it just as innocently as a developer can.

    The suite that satisfies both decisions

    Writing 1 for the safe value of each condition:

    #temp_okcurr_oksupp_okcmd_ok!faulthealthyenablesatisfies
    11111111baseline (D1+D2)
    20110000temp_ok in D1
    31010000curr_ok in D1
    41100000supp_ok in D1
    50001100healthy in D2
    61110110cmd_ok in D2
    71111010fault in D2

    Seven test cases. Unique-cause MC/DC: 100 % on both decisions. Any tool that measures unique-cause MC/DC on this source form reports the same thing; the report is green.

    Now run the same seven vectors against the folded expression. It needs five independence pairs. Two are present - cmd_ok and fault. Three are missing: the suite never once contains a case in which temp_ok, curr_ok or supp_ok is the only thing standing between the actuator and enable.

    100 % becomes 40 %.

    A note on the table before going on. It writes a value into every column, but short-circuit evaluation means that not every condition is evaluated in every case. In case 2, once temp_ok is false, curr_ok and supp_ok are never evaluated; in case 5, once healthy is false, neither cmd_ok nor !fault is. What an execution actually produces is a condition-evaluation history - a sequence over {true, false, not evaluated} - rather than a complete Boolean vector. The distinction matters as soon as MC/DC variants are compared, and it matters again whenever such histories are reconstructed from an execution trace rather than read off a table.

    Both figures above are unique-cause figures. Masking MC/DC applies a different independence rule to the same histories: a case in which a condition was never evaluated because an earlier one short-circuited it is an admissible witness under masking, but not under unique cause. A coverage report should therefore name the variant it computed before its percentage is compared with anything.

    There is a second dilution hiding in the same refactoring. The helper’s own MC/DC obligation is satisfied once, aggregated across every call site. Extract the predicate into a function used by four modules and you have four decisions whose coverage is discharged by whichever caller happens to reach the required vectors - regardless of whether the effect propagates at any of the other three.

    We measured it - and then tried to hide it

    That green report is not only a statement about trace-based tools. We ran exactly this source, with exactly the seven vectors above, through two compiler-integrated coverage mechanisms - Clang 22.1 source-based MC/DC (-fcoverage-mcdc) and GCC 15.2 condition coverage (-fcondition-coverage, measured at -O0: no optimizer involved) - and then hid the relationship between the supply predicate and the actuator decision step by step, the way ordinary refactoring does.

    StepWhat changesClang source-based MC/DCGCC condition coverageTry it
    As written aboveinline supply_healthy()two decisions of three conditions each, both 100 % - the helper is a single condition of the actuator decision12 of 12 outcomesCompiler Explorer
    Named intermediatebool healthy = supply_healthy(); … if (healthy && ...)two decisions, both 100 %12 of 12Compiler Explorer
    Across a call boundarypoll_supply() writes a flag; the actuator only reads ittwo decisions, both 100 % - the producer obligation now lives in a different function12 of 12Compiler Explorer
    Status wordstatus |= ...; if ((status & ST_ALL) == ST_ALL && ...)one decision left (100 %); the three supply conditions carry no MC/DC obligation at all - they survive only as branch outcomes of the ?: operators12 of 12 - the ?: count as single conditionsCompiler Explorer

    The two strongest steps, in code:

    /* across a call boundary */
    void poll_supply(void)
    {
        supply_ok_flag = temp_ok && curr_ok && supp_ok;
    }
    ...
    if (supply_ok_flag && cmd_ok && !fault) { ... }
    /* status word */
    status = (temp_ok ? ST_TEMP : 0u)
           | (curr_ok ? ST_CURR : 0u)
           | (supp_ok ? ST_SUPP : 0u);
    ...
    if (((status & ST_ALL) == ST_ALL) && cmd_ok && !fault) { ... }

    Every step is a refactoring a reviewer would wave through. Every report stays green, at every optimization level. Neither measured mechanism follows the definition-use chain to the five-condition decision the requirement is actually about. The folded obligation does not fail - it is never raised.

    What that costs

    Suppose the temperature threshold comparison is wrong - an off-by-one, or the wrong relational operator. The tests that exercise temp_ok = 0 are cases 2 and 5. In both, the actuator is disabled anyway - in case 2 because the command is invalid and a fault is latched, in case 5 because current and supply fail alongside it. Expected: drive disabled. Actual: drive disabled. Both tests pass.

    There are two different questions here. The suite has exercised temp_ok strongly enough to satisfy MC/DC for the decision that computes supply_healthy. What it has not demonstrated is that a defect in temp_ok can change an outcome the test oracle observes.

    If that observable interlock behavior is not part of the acceptance criterion, MC/DC cannot create it on the test suite’s behalf.

    The condition was exercised. Its effect changed the helper decision, but it never propagated to anything the test oracle was watching.

    That is the failure mode in one sentence: MC/DC obliges you to exercise a condition, not to observe its effect.

    This is measured, not theoretical

    • Rajan, Whalen and Heimdahl (ICSE 2008, DOI 10.1145/1368088.1368111) generated test suites satisfying MC/DC over non-inlined implementations of six civil-avionics systems, then re-ran them against the functionally equivalent inlined versions. MC/DC achieved dropped by an average of 29.5 %.
    • Gay, Rajan, Staats, Whalen and Heimdahl (TOSEM 2016, DOI 10.1145/2934672) put it plainly: MC/DC can be defeated by restructuring a program so that the target coverage becomes easier to reach. Suites achieving MC/DC over structurally complex Boolean expressions were consistently larger and better at finding faults than suites achieving MC/DC over simpler but equivalent implementations.
    • Whalen, Gay, You, Heimdahl and Staats (ICSE 2013, DOI 10.1109/ICSE.2013.6606556) proposed Observable MC/DC: ordinary MC/DC plus the requirement that a non-masking data-flow path exists from the exercised condition to a variable the test oracle actually monitors. OMC/DC-adequate suites detected substantially more seeded faults than MC/DC-adequate suites.

    Two kinds of chain, and only one of them folds

    Before reaching for tooling, the two cases have to be separated, because they behave differently under every technique discussed below.

    The direct chain. The helper above computes its expression at the point of use. The def-use chain is clean, the subexpression is side-effect free, nothing is written in between. supply_healthy() && cmd_ok && !fault really is temp_ok && curr_ok && supp_ok && cmd_ok && !fault. The compiler knows this and inlines accordingly.

    The cached chain. Now suppose the monitor stores its verdict instead:

    /* safety_monitor.c - 1 ms task */
    g_status.supply_healthy = temp_ok && curr_ok && supp_ok;
    
    /* drive.c - 100 us task */
    if (g_status.supply_healthy && g_status.cmd_ok && !g_status.fault)

    This looks like the same refactoring and reads the same way in review. It is not the same thing. The flag is written in a different task at a different rate; between write and read, all three inputs may have changed. No compiler folds across that boundary, and no analysis should pretend to. Here there genuinely are two decisions - and the loss of evidence is just as real, but it is a data-flow problem, not a structural one.

    What this means for CEDARtools.Anchor

    CEDARtools.Anchor exists so that what the trace port reports can be attributed reliably back to source constructs - through the anchor manifest and symbol-table associations, without having to rest on DWARF debug information. It constrains the compiler only where an optimization would otherwise erase the distinction the measurement needs. That constraint has a measured, documented cost, and the attribution it secures is deliberately independent of the optimization level of the build under test. A second record walks the mechanism end to end on a single decision that the compiler fuses at -O3: supplemented source, the linked image with its named observation sites, an ETM trace captured on a Cortex®-M7, the reconstructed per-instance condition-evaluation histories and the resulting masking-MC/DC verdict, together with the measured cost of that one supplementation.

    That purpose has a consequence worth stating directly: Anchor* makes the object code tell the truth about the source - which means object-code coverage inherits the source’s structural weakness rather than repairing it.

    This is not a shortcoming. It is the correct behavior, and it disposes of a tempting but wrong idea. One might hope that because the compiler folds the direct chain anyway, measuring on the object code silently recovers the strong five-condition obligation. It does not, and it should not: a coverage figure that depends on which optimization level was used, or on whether the inliner reached its budget, is not a figure anyone can qualify. Whether the branch cascade in the machine code has three arms or five is an artifact of the build, not a property of the program.

    The one-line version

    MC/DC is a property of the code as written, not of the logic as specified, and not of the machine code the compiler happened to emit. A green MC/DC report therefore establishes a structural claim. Whether that structural evidence also establishes the behavioral claim you care about depends on what effects the tests actually make observable. Two implementations of the same interlock can differ by a factor of two in what a green report actually proves. What would close the gap is a criterion that asks not only whether a condition was exercised, but whether its effect was ever propagated to something the tests observe - the direction Observable MC/DC points. If your process treats 100 % MC/DC as an output, that question deserves at least as much scrutiny as the number itself.

    One way to make the evidential obligation less dependent on implementation structure is to define it above the source code. Requirements-based coverage criteria such as NASA’s FLIP apply a related independence idea to the atomic propositions of a formalized requirement. Refactoring the implementation then does not silently redefine what the requirement-level test suite is expected to demonstrate. Once source code is increasingly generated and structurally unstable, that is a natural place for the obligation to live - it does not replace the code-level question, but it anchors it.

    What FLIP is. FLIP is a requirements-based test-coverage metric used in NASA’s FRET requirements toolchain. For a requirement formalized in temporal logic, it generates one test obligation per atomic proposition: the test suite has to demonstrate that each proposition uniquely affects the requirement’s satisfaction. NASA’s description draws the analogy to MC/DC explicitly - the Boolean constraints of the requirement map to MC/DC’s conditions, the requirement itself to the decision. Primary source: NASA/TM-20240006865, “Recommendations on Evidence and Process for Certification of Learning-enabled Components in Aerospace Systems” (2024).


    Related: MC/DC, worked through · why the object-to-source mapping breaks under optimization

    Comments or feedback on this article? blog_comments@accemic.com