# Decision models need more than a compatible API: frozen protocol

Snapshot: 2026-09-30. Manifest SHA-256: 2290bbf3e3d6ce41a578386ed758b69db0e57bb1066bc703a4bcd95c137d1bc4.

## Question
A separate decision-model assurance layer is plausible if independent implementations share a decision contract while calibration, confidence semantics and port fidelity still require workload-specific evidence. Interchangeability is challenged if compatible implementations define confidence differently.

## Sampling
- "jev" in:name,description fork:false archived:false: 14377 matches, 100 downloaded; capped=true.
- "laya" in:name,description fork:false archived:false: 9702 matches, 100 downloaded; capped=true.
- "system one" in:description fork:false archived:false: 1841 matches, 100 downloaded; capped=true.
- "decision model" in:description fork:false archived:false: 1586 matches, 100 downloaded; capped=true.
- "typed decisions" in:description fork:false archived:false: 583 matches, 100 downloaded; capped=true.
- "RLCD" in:description fork:false archived:false: 151 matches, 100 downloaded; capped=true.
- "semif" in:name,description fork:false archived:false: 1566 matches, 100 downloaded; capped=true.
- "systemone" in:name,description fork:false archived:false: 187 matches, 100 downloaded; capped=true.

Searches return at most 100 non-fork, non-archived repositories per family ordered by current stars. Counts across queries overlap and are never added to estimate a market. Predeclared reference repositories ensure known measurement and serving baselines are represented; they are labelled in the CSV. Capture is resumable within this date and the exact README bytes are stored privately with their hash. Results describe this purposive sample, not the whole ecosystem.

## Screening
row => /\b(typesafe|rlcd|system[- ]one|systemone|sys1cal|typed[- ]decisions?|semif|jev[- ]compatible)\b/i.test(row.description + " " + row.readme) && /\b(model|decision|classifier|agent|benchmark|evaluation|runtime)\b/i.test(row.description + " " + row.readme)

README unavailable is excluded as unknown. Curated lists are excluded. Laya game-engine projects, conventional decision tables and unrelated uses of acronyms are screened out by contextual language. Repository aliases deduplicate by numeric GitHub ID. Owner accounts are not organizations; wrappers and ports are not independent model families.

## Coding rules
- typed-contract: Typed decision contract; /(?:\bchoice\b[\s\S]{0,180}\b(?:score|noul)\b|\/v1\/systemone|typed[- ](?:choice|decision|question|answer))/i. An explicit question or wire contract, rather than a free-form answer.
- compatibility: API or SDK compatibility; /(?:wire[- ]compatible|jev[- ]compatible|api[- ]compatible|drop[- ]in|typesafe.{0,60}(?:sdk|api)|\/v1\/systemone)/i. Compatibility is a documented claim; it does not establish equal probabilities.
- calibration-process: Calibration procedure; /(?:temperature[- ](?:scal|fit)|fit.{0,35}temperature|calibrat(?:ion|e|ed).{0,80}(?:held[- ]out|dataset|labels|temperature|split)|(?:held[- ]out|temperature).{0,80}calibrat)/i. A fitting or held-out procedure, not the word calibrated alone.
- calibration-metric: Numerical calibration results; /(?:(?:\bECE\b|\bBrier\b|calibration error|total variation).{0,120}\d|\d.{0,50}(?:\bECE\b|\bBrier\b))/i. Reported numbers are author claims until independently reproduced.
- abstention: Abstention or escalation; /\b(?:abstain|abstention|escalat\w*|human deferral|defer(?:ral)?|selective prediction)\b/i. A documented mechanism or example; correctness under distribution shift is untested.
- parity: Explicit port or protocol parity; /(?:numerical parity|port fidelity|byte[- ]identical|probability.{0,45}(?:parity|tolerance)|conformance test|selected answer.{0,35}match)/i. A parity indicator is narrower than general benchmark or unit-test wording.

The quantitative screen is a documentation indicator audit, not an expert code audit. All retained README texts are screened with predeclared indicators. The explicit case table is separately analytically reviewed. A missing indicator means not documented in this captured README; it does not mean the product lacks the feature. Quoted user anecdotes and vendor performance claims are not independent measurements.

## Results
Screened 506 deduplicated candidates and retained 368 subject-relevant repositories from 355 owner accounts. 138 candidates were excluded, including 27 with unavailable README text. Owner accounts are not verified independent organizations.
Typed decision contract: 277/368 (75.3%); API or SDK compatibility: 252/368 (68.5%); Calibration procedure: 84/368 (22.8%); Numerical calibration results: 78/368 (21.2%); Abstention or escalation: 95/368 (25.8%); Explicit port or protocol parity: 29/368 (7.9%)

## Reproduction
Run node scripts/collect-decision-quality-research.mjs, then node scripts/publish-quality-research-2026-09-30.mjs for the dry summary. Publish requires --publish. Frozen CSV and manifest refer to the captured data; later downloads must not silently replace these published numbers. The raw GitHub objects and README copies remain in TargetedResearchItem. Search logs remain in TargetedResearchQuerySnapshot.

## Limits
No paid inference calls, private telemetry or customer interviews were run. We did not independently rerun the benchmark results in the references. Search ranking, README detail, sample selection and shared upstream dependencies bias feature coverage. Creation dates and current stars are not historic usage or adoption series. This sample provides no revenue, customer count, market share or longitudinal growth estimate.

## Pinned artifact checks
AILANTA recomputed weighted accuracy from nine published suite counts and checked every figure value against the pinned report. These are reused experiment results, not new model runs.

The arithmetic agrees across the source tables. Removing the multilingual suite does not reverse the ranking. The labels came from a single human reviewer correcting an AI draft, with no second independent reviewer; this comparison cannot establish broad deployment reliability or customer-domain calibration.

- https://raw.githubusercontent.com/instax-dutta/sysone-bench/9447a17fccb524957d1211ecdf46153821ff0810/results/v2/report-20260926/report.json; {"sha":"9447a17fccb524957d1211ecdf46153821ff0810","path":"results/v2/report-20260926/report.json","sha256":"4f2525efce1b3b18a65406e2a7910afbc230bcc1de0d9a77ec1156a9f1c35a82"}
- https://raw.githubusercontent.com/instax-dutta/sysone-bench/9447a17fccb524957d1211ecdf46153821ff0810/results/v2/report-20260926/figures/data/accuracy.csv; {"sha":"9447a17fccb524957d1211ecdf46153821ff0810","path":"results/v2/report-20260926/figures/data/accuracy.csv","sha256":"16252f9a20f9d42130a057315cf6e2af0f8562d246b71c22e3037206808f4805"}
- https://raw.githubusercontent.com/instax-dutta/sysone-bench/9447a17fccb524957d1211ecdf46153821ff0810/results/v2/report-20260926/figures/data/calibration.csv; {"sha":"9447a17fccb524957d1211ecdf46153821ff0810","path":"results/v2/report-20260926/figures/data/calibration.csv","sha256":"05aa42e78f40957a334309da8b7c9a1bbca1cfca8039f3a09e276fd798b64ab6"}
- https://raw.githubusercontent.com/instax-dutta/sysone-bench/9447a17fccb524957d1211ecdf46153821ff0810/results/v2/report-20260926/figures/figure-source.json; {"sha":"9447a17fccb524957d1211ecdf46153821ff0810","path":"results/v2/report-20260926/figures/figure-source.json","sha256":"4b263b8e2cc08d3d5d52fe25b7ac9fc99d7dd2952bca297b4b6c9a4dc6d0af78"}

The artifact-audit JSON preserves the derived chart and table. Assertions reconcile source totals and deltas; they do not independently verify labels, rerun inference or certify vendor performance.