When there's no official test suite, make the parsers vote

Written for ED-1 — the supply-chain ISV / traceability-vendor engineering org — the VP Engineering / CTO weighing build-vs-buy and pricing the maintenance tax.

One thesis: X12 publishes no conformance test suite and runs no certification program, so "correct" cannot be outsourced to an authority — this engine defines ground truth as what independent open-source parsers agree on, routes their disagreements to a journaled adjudication queue, and publishes the agreement rate as a number you can watch.

The absence, verified

Before trusting any vendor's correctness story — including ours — it is worth knowing what the standards body actually sells. Checked against x12.org on 2026-07-31: the product list is Glass (view-only browser access to the standard), the EDI Standard itself, code directories, technical reports, and a tiered licensing program. There is no downloadable test corpus, no official conformance suite, no certification program you can pass. What X12 publishes free is a small library of illustrative scenario examples — useful, not systematic. The conformance-testing ecosystem that does exist (Edifecs tooling, CAQH CORE certification, the SNIP validation levels) is commercial and healthcare-centered, and SNIP is a taxonomy of validation types, not a published dataset.

That is a search result, dated, and stated as ours: we found no official suite, and we looked with intent. Every EDI vendor's correctness claim therefore rests on something private. The difference worth paying for is whether the something is auditable.

Four oracles instead of one authority

The harness this engine climbs against is built from four mutually reinforcing checks, none of which requires trusting this project:

1. A license-tagged corpus. Fixtures from permissively licensed open parser repositories plus a majority of synthetic self-authored documents, every file pinned with source, license, and retrieval date in CORPUS-PINS.json.
2. Cross-parser agreement. Every corpus document runs through independent open-source X12 parsers as subprocesses — pyx12 (BSD-style, maintained since the early 2000s), Walmart's gozer (Apache-2.0), and node-x12 (MIT) — with outputs normalized to a comparison form of segment stream plus element values, deliberately not any parser's object model.
3. Byte-exact round-trip. emit(parse(x)) === x on every document, reported per job.
4. Property-based fuzzing. Generated envelopes across every legal delimiter choice, fixed-width ISA padding violations, control-number mismatches, truncations — with the invariant set: never crash, never emit invalid canonical JSON, every rejection typed.

The same differential idiom covers the acknowledgment layer: the engine's 997s are cross-checked against pyx12's independent 997 generation on a shared corpus, so even the acks rest on agreement rather than self-report.

The parsers-agree subset is ground truth. Where the independent parsers disagree with each other — and they do, in instructive places — the document goes to an adjudication queue, and a human ruling decides which reading is correct, in writing, in an append-only journal. The ruling references land in the affected grammar file's provenance block, so the chain from "the harness says this is correct" back to "because three parsers agreed" or "because ruling RJ-0012 decided it" is walkable.

What an adjudication looks like

DISAGREEMENT dq-0041  856/004010  fixture: gozer/856-multi-tare.edi
  pyx12:    HL 5 parented to HL 3 (accepts forward reference)
  gozer:    HL 5 parented to HL 3
  node-x12: rejects — HL02 references an HL not yet seen
  RULING RJ-0012 (2026-07, journaled): forward HL02 references are
  structurally legal; the tree is resolved after the loop completes.
  node-x12's rejection recorded as a known divergence, excluded from
  its vote on HL-ordering cases.

Two things about that record matter more than the verdict. The disagreement is preserved, not overwritten — the harness remembers that independent implementations read this case differently, which is exactly the information a partner-file dispute needs later. And the excluded vote is scoped: node-x12 still votes on everything else. No parser is an oracle; the population is.

Why you can't overfit to this

The anti-Goodhart argument, spelled out, because it is the actual reason a differential harness beats a private test suite. An engine tested only against its own fixtures converges on its own bugs. An engine tested against one reference parser converges on that parser's bugs. An engine that must simultaneously match the agreement set of three independent implementations, round-trip every byte, and survive a hundred thousand fuzzed envelopes has nowhere to hide a convenient misreading — any shortcut that games one oracle trips another. The harness specs themselves are digest-pinned with the same journaled-ruling governance as the grammars, so the target cannot be quietly moved toward the implementation.

The agreement rate — the fraction of the corpus on which this engine matches the parsers-agree subset, alongside the open adjudication count — is a published number, per transaction set and version. It is the honest form of a correctness claim: not "conformant," which no one can certify, but "agrees with the independent population here, diverges there, rulings attached."

The build-vs-buy read

If your team builds this in-house, the harness is the part nobody budgets: assembling a license-clean corpus, wiring three parsers as subprocess oracles, staffing the adjudication queue, and keeping all of it honest as X12 versions accrete — a permanent tax, paid in your engineers, against a layer that is not your product. The pitch of this layer is that the tax is ours and the evidence is public: pinned grammars, pinned corpus, published agreement rates, and a CLI contract that lets your team — or your agents — re-run the verdicts on your own traffic. The access flow is at /get-access/; tell it which sets and versions your traffic runs, because that is literally the prioritization input for the harness's next column.

---

898 words. The access flow is at /get-access/; the family's event door is epcis.dev.