Recomputing this figure. The overall score is a flat mean of the 42 per-program scores in
results/scores.csv, which computes to 98.666667 and publishes as 98.7. It is deliberately not an average of the eight archetype means, which would over-weight archetypes containing one or two programs; that calculation gives 98.5. Where the two disagree, §4.3 ofMETHODOLOGY.mdmandates the flat per-program mean.
| Archetype | Programs | Mean score |
|---|---|---|
| CICS Other | 4 | 99.4 |
| Batch File Maintenance | 11 | 99.1 |
| Batch Report | 2 | 99.1 |
| CICS Inquiry | 8 | 98.6 |
| CICS Menu | 6 | 98.4 |
| CICS Data Entry | 7 | 98.3 |
| Batch Utility | 1 | 97.7 |
| Called Subprogram | 3 | 97.7 |
| Overall | 42 | 98.7 |
Archetype assignment in this table reflects the rubric applied at scoring; one program (
PAUDBUNL) was classified batch-report by the pipeline classifier but scored under the batch-file-maintenance rubric — a recorded assignment mismatch.
Every program's score of record (mean of its three per-generation
medians), one decimal, with its separately-gated fabrication count.
Machine-readable copy: results/scores.csv.
| Program | Archetype | Score | Fabrications |
|---|---|---|---|
| COPAUS2C | CICS Other | 100.0 | 0 |
| CBACT04C | Batch File Maintenance | 99.9 | 0 |
| COACCT01 | CICS Other | 99.8 | 0 |
| COCRDLIC | CICS Menu | 99.8 | 0 |
| CBACT02C | Batch File Maintenance | 99.7 | 0 |
| CBACT03C | Batch File Maintenance | 99.7 | 0 |
| CBACT01C | Batch File Maintenance | 99.5 | 0 |
| CBTRN03C | Batch Report | 99.5 | 0 |
| COBTUPDT | Batch File Maintenance | 99.3 | 0 |
| COCRDSLC | CICS Inquiry | 99.3 | 0 |
| CBTRN01C | Batch File Maintenance | 99.3 | 0 |
| COUSR01C | CICS Data Entry | 99.3 | 0 |
| COADM01C | CICS Menu | 99.2 | 0 |
| COTRTUPC | CICS Inquiry | 99.2 | 0 |
| CBCUS01C | Batch File Maintenance | 99.1 | 0 |
| COTRN02C | CICS Data Entry | 99.1 | 0 |
| COPAUA0C | CICS Other | 99.0 | 0 |
| COPAUS0C | CICS Inquiry | 99.0 | 0 |
| COTRTLIC | CICS Menu | 99.0 | 0 |
| COCRDUPC | CICS Data Entry | 98.9 | 0 |
| COUSR03C | CICS Data Entry | 98.9 | 0 |
| COACTVWC | CICS Inquiry | 98.8 | 0 |
| CBSTM03B | Batch File Maintenance | 98.8 | 0 |
| PAUDBUNL | Batch File Maintenance | 98.7 | 0 |
| CBSTM03A | Batch Report | 98.7 | 0 |
| CODATE01 | CICS Other | 98.7 | 0 |
| CORPT00C | CICS Inquiry | 98.6 | 0 |
| CBTRN02C | Batch File Maintenance | 98.5 | 0 |
| COSGN00C | CICS Inquiry | 98.5 | 0 |
| COUSR00C | CICS Menu | 98.5 | 0 |
| CBPAUP0C | Called Subprogram | 98.4 | 0 |
| CSUTLDTC | Called Subprogram | 98.2 | 0 |
| COACTUPC | CICS Data Entry | 98.1 | 0 |
| COPAUS1C | CICS Inquiry | 98.0 | 0 |
| COBSWAIT | Batch Utility | 97.7 | 0 |
| COUSR02C | CICS Data Entry | 97.6 | 0 |
| COTRN01C | CICS Inquiry | 97.4 | 0 |
| PAUDBLOD | Batch File Maintenance | 97.2 | 0 |
| COMEN01C | CICS Menu | 97.1 | 0 |
| COTRN00C | CICS Menu | 97.0 | 0 |
| COBIL00C | CICS Data Entry | 96.5 | 0 |
| DBUNLDGS | Called Subprogram | 96.5 | 0 |
What a maintenance programmer gets. At these scores, a programmer receiving the PLMs can trust the field names, control-flow descriptions, I/O operations, and business-logic explanations without cross-referencing source code for basic comprehension. That is the bar the benchmark is designed to test — not literary quality, but whether the document is safe to act on.
Where the 1.3 points go. Deductions below 100 typically reflect inferences about business context that cannot be mechanically verified from source alone — for example, who consumes a report or when a batch job runs in production. In well-formed PLMs these are qualified as inferences rather than asserted as fact, which is the correct behavior for regulated environments and is rewarded by the rubric even though it caps the score short of 100.
Where the low archetypes sit, and why. The two lowest archetype means are Batch Utility (97.7, n = 1) and Called Subprogram (97.7, n = 3) — both small samples, so a single program moves the mean and neither number should be read as a broad claim about the archetype. Their scores are driven by the same cap the rest of the corpus hits, only more of it: these programs carry a higher proportion of non-derivable operational and caller context (what invokes a pure linkage routine, or what a utility does in the production job stream) that the source alone does not settle, and which well-formed PLMs correctly hedge as inference rather than assert. The cap is by design — appropriately-qualified uncertainty is scored as correctness — not a quality defect. (Batch Report, previously the low archetype at 96.7, is no longer low: at 99.1 it sits with the strongest batch programs after the pipeline fixes in this vintage.)
A mean of 98.7 across 42 programs spanning 8 archetypes, with zero fabricated entities, means the generated documentation is consistently accurate enough for production maintenance use — and, because fabrication is gated separately, accurate in a way that holds up to audit rather than merely reading well.