The single most important claim in this benchmark is zero entity fabrications. This document defines what that means and how it is enforced, because a fabrication claim is only as credible as its definition.
A fabricated entity is any named reference that appears in the generated documentation but does not exist in the source code (or its derived Structured Technical Baseline). Named references include:
ACCT-BAL)If the documentation refers to ACCT-BALANCE but the
program only defines ACCT-BAL, that is a fabrication — even
though it is a plausible, nearly-correct guess. In a maintenance
context, plausible-but-wrong is the dangerous failure mode, so it is
treated as a hard failure.
Fabrication detection targets named entities, not prose. A narrative sentence that explains business logic is subject to the quality rubric (accuracy, usefulness), but it is the entity grounding — the specific names and citations — that the hard-gate enforces absolutely. Appropriately-qualified inferences ("this report appears to feed downstream reconciliation") are not fabrications; asserting an unverifiable claim as fact is a rubric deduction, handled separately.
The cross-check resolves each reference to the underlying entity rather than matching raw text, so legitimate COBOL surface variants — qualified, reference-modified, and subscripted references, and the different names by which the same file or dataset is called — are grounded rather than flagged, and truthfully-documented absence is recognized rather than treated as a fabrication. Only a name that resolves to no real entity or alias trips the gate — and that triggers re-processing, not a patch.
Quality and fabrication are different kinds of claims. A document can read beautifully and still contain an invented field name; a document can be terse and contain none. Folding them into one number would hide exactly the property that regulated buyers care most about. So the benchmark reports two facts side by side: 98.7 quality and 0 fabrications.