Radigrade

Supersedes version 1 (osf.io/5nyx9, registered 30 September 2026) for all grading against the version 2 reference; version 1 and its result stand as registered.

What changed since version 1

1. The reference was revised. After the version 1 result was adjudicated on 2 October 2026, the two radiologists read all one hundred and sixty cases again together on 17 October 2026. At that reading they were not shown any finding that a graded system had claimed. What they recorded on their own, with the tooth-inventory checks of section 7, is version 2. A finding that a graded system had surfaced and that the radiologists did not record on their own is not in version 2; it stays credited to that system as a verified finding beyond the reference, exactly as version 1 handled it.
2. The grader was revised. It is the same model, called through its application interface instead of a command-line client, with a longer allowance for its reasoning. Its vocabulary gained terms taken from the textbooks the reference follows, some prompted by the version 1 review, none copied from a case. Rules learned at the version 1 review were written into it; section 7 states them.
3. The three honesty checks of section 6 were run again under the revised grader and passed before any system was scored with it.
4. Some reads of version 1 recorded teeth as present that the radiologists had charted as missing or as retained roots; version 2 carries the chart. Rows that stated the adequacy of a root filling are not in version 2: the adequacy of a root filling is not a class the reference records.
5. Sections 2, 4 and 6 to 10 gain passages marked [v2]; one sentence of version 1 is corrected, the invention gate's sentence in section 10.

§1What is graded

A panoramic-radiograph report produced by an AI system, compared against a reference reading of the same image made by two oral and maxillofacial radiologists reading together and recording their consensus. The reference is frozen before any system is graded; its fingerprint is in section 10 and is published with every result.

§2What the reference asserts, and what it does not

• A finding in the reference is one both radiologists would sign from that image alone. Findings that need a periapical film, a CBCT or a clinical examination to settle are not in the reference; they appear as recommendations for further imaging.
• Silence on a tooth or a region means "nothing callable on this image". It is never a statement that the tooth or region is normal.
• The tooth inventory is asserted: which teeth are present, missing, impacted, retained as roots, restored, crowned, root-filled, replaced by implants.
• A finding marked possible is one the radiologists saw but would not require a competent reader to call.
• A dental-age estimate in the reference is a range no narrower than the image supports without a history; a cause that needs a history the image does not hold is described, not named.

§3What counts against a system

• Inventing. A finding that contradicts the reference: a tooth or implant that is not there, a lesion on a tooth the reference explicitly clears, "all teeth present" where the reference records extractions. This is the first gate and the one that fails a system outright. Every nomination is reviewed on the image by a radiologist before it counts.
• Over-treatment. A definitive recommendation the imaging does not support: treatment named for a unit the reference shows still has structure, or for a finding the reference does not carry. This axis is descriptive: every such recommendation is listed in the system's report and counted, no margin and no gate apply to it, and it can neither fail nor pass a system. A margin arrives only by a new registration, once the reference's own treatment lines exist. Reported only for systems that emit recommendations.

§4What does not count against a system

• A call on a tooth or region where the reference is silent. It is set aside for radiologist image review, never scored automatically; section 7 gives the review's verdicts.
• A hedged statement ("cannot exclude", "recommend a periapical film").
• Missing a finding the reference marks possible.
• Using a different word for the same finding, a neighbouring-tooth slip, or a region-level statement that covers tooth-level findings (partial credit, published rules).
• The boundary cases the image cannot settle: a retained root versus a broken-down tooth, a partially erupted versus an impacted third molar, one fixture whose tooth identity cannot be assigned after surgery. Each has a published tolerance.
• Attrition and wear: not graded.
• A statement of normal development for age, an age range given for an adult dentition, ordinary ridge resorption at an edentulous span, plain normal anatomy, or a blanket "none identified" line. These count neither for nor against a system and raise no review item.

§5Published tolerances

• Dental age: any recognised method; a match when the system's estimate overlaps the reference range widened by one year on each side; partial within two years. The reference chronology is McDonald and Avery.
• Bone-loss level: the same or the adjacent third of the root length is a match; two thirds apart is partial and recorded as a severity error.
• Periodontal stage band: same band a match, the other band partial.
• Third molars: a partial-eruption disagreement never scores.
• Retained root: scores as correct when either the system or the reference names the coronal destruction at that tooth; whether to extract is a recommendation, reported on the over-treatment axis.

§6How the instrument is kept honest

Three checks run before any system is scored, and a failure stops the grading rather than the system: the reference fed back as if it were a system's report must score clean; a deliberately damaged copy of the reference must fail; and the reference re-expressed in another competent reader's vocabulary and method must still pass. The third check exists so that no bar in this grade can be one only its authors could reach. The grader and the comparator are the same model; the third check guards against the grader favouring wording like its own. They run again under every instrument version of the grader before it grades a system; they ran under the version 2 grader before any system was scored with it.

§7How claims are counted

The grader reads each report three times, independently. A claim is scored only when at least two of the three readings agree on it, whether it is a finding credited to the system, a tooth state, or the nomination of an invented finding; a nomination whose readings disagree on its class without a majority takes the more serious class and goes to review; where the three readings split three ways on a reference finding, the first reading stands and the split is recorded. A claim carried by one reading alone is a singleton: recorded, never counted for or against the system; a seeded one in ten of the singletons is reviewed on the image by a radiologist, and that review measures the grader, not the system, and is reported as such. Every nomination that reaches review takes one of three verdicts from the radiologist. For a claim that contradicts the reference: invented, which counts on the first gate; an over-call that a hedge would have covered, recorded in the report but not on the first gate; or a real finding, where the system was right and the reference silent or wrong there. For a claim on a tooth or region where the reference is silent: invented, which counts on the first gate likewise; a real finding; or not callable, meaning the claim may be true but this image cannot carry it at the standard the reference itself is held to (a proximal caries on a panoramic image, for example), which is neither an invention nor a credit and is listed descriptively. Real findings beyond the reference are counted per system, descriptively; they enter a later reference only as the revision paragraph below provides. The reference does not change while any system is graded or adjudicated against it.

The grader is Claude Opus 5.5, called through its application interface, reading each report three times as above. The grader model may change at a dated instrument version under the conditions of section 9: named in every report, the three checks of section 6 rerun under it, the comparator re-graded under it; results from different grader versions are never pooled in one comparison.

The grader carries rules learned at review; at version 2 they are these: a reference finding a system leaves out is reported as "not reported", never as "missing", which to a radiologist means an absent tooth. The statements listed in the last bullet of section 4 raise no review item. One claim is reviewed once, whatever labels the grader gave it. One numbering shift along a run of teeth is reviewed as one event, not as one item per tooth.

How the reference is revised: a new dated version is made by the two radiologists reading the cases again together without being shown what any graded system claimed, and is registered as a new version before any system is graded against it. A finding a system surfaced enters the reference only when the radiologists record it on their own; otherwise it stays credited to that system beyond the reference. Tooth-inventory facts a system surfaced are checked by the radiologists after the reading; a confirmed fact is corrected in the reference before the version is frozen. The fingerprint moves once for all at the new version; a revision never happens while any system is graded or adjudicated against the current one.

§8What a system receives

A per-axis report: invented findings (reviewed), detected findings with the published tolerances, tooth inventory, and, where the system emits recommendations, the over-treatment list (descriptive), with the descriptive lists of section 7 (verified findings beyond the reference, claims not callable on the image, the singleton audit). No single blended score exists and none will be published. A vendor-submitted system is graded privately: the engagement and its result are confidential, and the vendor decides whether, when and what becomes public. Publicly available general models are graded under the same protocol and published by name.

§9Fixed before the first system is graded

The reference and its fingerprint, what counts and what does not (sections 2 to 5), the counting rules and review verdicts of section 7, the tolerances, the margin values of the two secondary gates and the exam composition are frozen by this registration, with the numbers in section 10. A change to any of them requires a new registration. The grader's vocabulary, wording, call settings or model may be revised at a dated instrument version between registrations: each version is named in every report, the three checks of section 6 run again under it before it grades a system, and the comparator is graded under it; such a version may map wording onto an existing class, or change how nominations are grouped for review provided no count moves, but never adds, removes or redefines a class. A later grader's run against the version 1 reference is reported as such and never pooled with a version 2 result. Version 2 was registered before any system was graded against the version 2 reference; the version 1 result of 30 September 2026 was produced against the version 1 reference and is reported as such.

§10The frozen numbers

• Reference fingerprint: 226d01ea46f450323008fb52f5bed688e2e6aadd227e4aa7dae58a18403c6b26, cited by its first twelve characters, 226d01ea46f4; the reference frozen on 17 October 2026. SHA-256 over the 160 reference reads taken in the alphabetical order of their case identifiers, each contributing its identifier and the SHA-256 of its scored part (the case facts, image quality, tooth inventory and findings). The full-file hash, computed the same way over whole files, is 08d95d1a8e4e4d6fa2e2e159fac5b70f38899c90a8fa40983f25df628c079385, cited as 08d95d1a8e4e, and is recorded beside the fingerprint in every run. The reference of 17 October 2026: the two radiologists' second joint reading of all 160 cases, made as section 7 describes. (Version 1 fingerprint, for the record: f6e361f95e52, full-file hash f4478e407311, the reference of 29 September 2026.)
• Gate order: inventing first, with no margin; then the two secondary gates, finding sensitivity and tooth-inventory recall; a system that fails the first is not graded on the rest.
• Margins: the two secondary gates use a non-inferiority margin of 0.05: a graded system may lose at most one real finding in twenty, or one tooth state in twenty, against the comparator, and only when the whole 95 percent interval clears that bar; the interval comes from a paired bootstrap that resamples cases, ten thousand resamples, seed 20260719. Fixed 19 July 2026, before any system was graded.
• Comparator: for every system graded under this version the comparator is Claude Opus 5.5: its blind reads of 23 September 2026, named with its run fingerprint in every grade report, sealed since each case's draw. The invention gate carries no margin: a graded system fails it when the radiologist confirms more inventions in its reports than in the comparator's reports on the same cases, under the same reference and instrument version; a tie passes.
• Exam composition: eighty of the one hundred and sixty two-radiologist reads, drawn by pre-registered stratified draws with self-committing seeds: thirty-two on 10 August 2026 at a fixed twenty percent fraction, and forty-eight more on 28 September 2026 from the remaining reads by the same procedure with its own seed; each sealed since its draw: never used to train or select a graded system; rules learned at the adjudication of their review items are stated in section 7. The forty-eight were used for grader tuning and carry development reads by Claude Opus 5 and Claude Opus 5.5, and every grade report says so. Per stratum: edentulous 12, endodontic 7, impaction 16, implant and prosthodontic 23, pathology 1, pediatric 15, restorative 5, periodontal 1. The composition is unchanged by the revision: the same eighty cases are sealed, the same eighty are not. The training corpus and its provenance are not part of this document.

End of the registered text. The binding copy is at the DOI: 10.17605/OSF.IO/ZASNW.