FALSIFY · v0.4.0
◆sha-256
 
Lock the claim · Then run

Evaluation Evidence Pack · powered by the open PRML specification · Community Specification License 1.0

falsify

Prove what the evaluation was supposed to pass —
before you show the result.

Create a verifiable Evaluation Evidence Pack: the acceptance criteria, a timestamped commitment, revision history, and a verification link.
Powered by the open PRML specification.

See a sample Evidence Pack → Try it in a browser →

It verifies the criteria record, not the evaluation result.

For compliance leads under the EU AI Act high-risk deadline → what PRML maps to

Assurance or audit practice? How the lock fits your engagements → Falsify for audit firms

Does this report show when the bar was set? Paste an evaluation report, model card or validation summary → Evidence Check (free, no sign-up)

receipts: Ed25519 signed · RFC 3161 timestamped · mirrored to the Rekor transparency log since 11 July 2026 · sealed commits since 3 September 2026 (bar private, ordering public) · each permalink states its own anchors · verify at registry.falsify.dev
◆ For Your Role ◆
Routing

Five entry points.
Pick the one that matches the job.

Each row maps a role to a working pattern on the spec subdomain. Same primitive, five framings.

AI delivery team
Handing an AI system to a client for acceptance
Record the acceptance criteria you and the client agreed before the acceptance test runs, and hand over the record of the acceptance decision: the plan both sides approved, the signed acceptance report, timestamped commitments and a verification bundle the client’s reviewer can check without your systems.
example record →
Compliance lead
What PRML maps to under the EU AI Act
The closest fit is Annex IV §2(g): the metrics used to measure accuracy and robustness have to appear in the technical file, and a PRML manifest is that record in hash-anchored form. Other entries are partial — Article 12 governs runtime logging, and a manifest can sit inside such a log without being the logging system.
/compliance →
ML governance / audit
Running ML evaluation reproducibility audits
Dataset hashes, seed handling, threshold drift, model-card verification. Audit checklist with PRML manifest examples.
reproducibility/eval-audit →
Researcher
Bringing pre-registration discipline to ML
From AsPredicted and OSF norms to a content-addressed manifest. For people crossing over from clinical and psychology.
pre-registration →
Self-assessment
Self-assess your EU AI Act evidence readiness in 10 questions
Free. No signup. Maps your current evidence posture against the 2 December 2027 high-risk enforcement bar (Annex III) and points at the gaps.
Run the 10-question assessment →
◆ Initializing System ◆
The System

Init. Lock. Run.
Verdict. Guard.

The claim becomes the spec. The spec becomes the hash. The hash becomes the verdict. The order is the contract.
Both CLIs ship with pip install falsify: falsify is the PRML reference, falsify-engine the workflow engine below.

For developers: PRML is the open specification falsify implements — 9 fields, one hash, CI exits 3 on tamper. pip install falsify · read the spec

falsify-engine · Hypothesis A · v2
Replay
Spec Ledger · falsify-engine list — Falsify's own runs4 Locked
claude_surface · dogfoodPASS · 10
cli_startup · dogfoodPASS · 113ms
test_coverage_count · dogfoodPASS · 588
hypothesis_a · internalFAIL · 0.214
honesty −−scope dogfood1.00
Dogfood values: falsify CI, 23 Sep 2026 · run log → · hypothesis_a is an internal run
Tamper detection · change the valueINTACT
threshold: above
sha-256 of the line above: a20a92a65301
Speccanonical YAML · SHA-256 hashed before run
Verdictnumeric compare · no LLM judgment
Guardgit commit-msg hook · blocks contradictions
Surface5 skills · 2 subagents · 3 commands · 1 MCP
exit 0PASSverdict satisfies the locked threshold
exit 10FAILverdict against the locked threshold; ship as recorded
exit 3TAMPEREDspec edited after lock · CI refuses to ship
◆ Scanning Evidence ◆
Evidence

The number changed.
The locked spec caught it.

Dogfood case: a March operating note recorded −0.0139; the locked spec re-run produced a materially different number. The engine surfaced what the notebook could not — the recorded verdict no longer reconciled with the data. Full write-up →

v2 event-level delta — eligible vs demoted cohort
+0.0000

Positive magnitude, small sample. The verdict is PASS against a pre-registered threshold of zero, but the engine refuses to let any “result confirmed” claim ship until the minimum sample size is met.

127 resolved events across the locked parent cohort. 16 eligible events, 19 demoted. Two locked specs (v1 March-frozen, v2 April-live), one contradictory historical claim. Reconciliation before marketing, not after.

Locked spec · internal run, spec not published
◆ Third-Party Record ◆
External catalogs

Independently listed.
Independently verifiable.

PRML and its reference implementation appear in catalogs we do not control. Every entry links to the source, so you can check it without trusting us.

Inspect AI Listed in the Inspect AI extensions catalog inspect.aisi.org.uk ↗ SchemaStore PRML schema listed in SchemaStore github.com/SchemaStore ↗ IANA media type application/vnd.prml+yaml registered · 3 Sep 2026 iana.org ↗ OpenSSF Best Practices passing 100% of criteria met bestpractices.dev ↗ Zenodo DOI Spec archived · 10.5281/zenodo.20177839 doi.org ↗ OECD.AI catalogue Listed in the Catalogue of Tools & Metrics for Trustworthy AI · 14 Sep 2026 oecd.ai ↗

Catalog listings describe where falsify appears or what it interoperates with. None implies endorsement, certification, or partnership by the organisations named. All trademarks belong to their owners.

◆ Specification ◆
PRML v0.1

A specification, not a tool.
The format is the contract.

Falsify is the reference implementation of PRML v0.1 — Pre-Registered ML Manifest Specification. Nine YAML fields. One SHA-256 over canonical bytes. Computed before the experiment runs. Verifiable offline by anyone with the manifest, the dataset, and the model. Community Specification License 1.0.

Four byte-equivalent reference implementations — Python, JavaScript, Go, Rust — reproduce the conformance vectors byte-for-byte. v0.2 frozen 2026-05-22 (RFC closed).

Specification
spec.falsify.dev/v0.1
RFC-style spec. Community Specification License 1.0. Nine required fields. 21 conformance vectors (13 v0.1 + 8 v0.2 candidate) with locked SHA-256.
Preprint — Zenodo DOI
14 pages, LaTeX
Threat model, EU AI Act mapping (Articles 12 · 17 · 18 · 50 · 72 · 73), NIST AI RMF, ISO 42001. Archived with a citable DOI.
v0.2 RFC
frozen 2026-05-22
Ten changes. Five open RFC questions. Public review window closed 2026-05-22; feedback rolls into the v0.3 cycle.
Reference 1
Python
13/13 + 8/8 · PyYAML
Reference 2
JavaScript
13/13 + 8/8 · ~400 LOC
Reference 3
Go
13/13 + 8/8 · ~450 LOC
Reference 4
Rust
13/13 + 8/8 · ~600 LOC
◆ Access Protocol ◆
Source · MIT · Public Repo
v0.4.0

Your past self isthe one your futureself answers to.

It does not tell you if you’re right. It tells you if you changed the rules.

$ pip install falsify

single-file CLI · no daemon · no account

View on GitHub →
Surfaces
Spec · v0.1 PRML working draft Full specification with grammar, canonicalization, §8.1 limitations. Community Specification License 1.0. Registry · live Commit a manifest Paste YAML, get a SHA-256 permalink and a README badge. No account. CI integration · v2 prml-verify-action Five-line GitHub Action. Block merges on tampered or regressed eval claims. Composite, MIT. Engagement · scoped per client Diagnostic Sprint Fixed-scope written engagements, from a €3,000 Acceptance Evidence Pack to 8–10 week enterprise scopes. Criteria recorded, verifier deployed, evidence pack shipped. Reference What is PRML? Plain-English answer page. Citable for AI search and academic context. Comparison PRML vs … Side-by-side with in-toto, SLSA, Sigstore, Model Cards, HELM, OSF, MLRC checklists. Tool PRML Falsifiability Check Paste an ML benchmark claim. See which falsifiability criteria it skipped. Index PRML Disclosure Index 27 well-known ML eval claims, scored against the 9 disclosure criteria. For gatekeepers Require a receipt Run a leaderboard, benchmark, or review process? A copy-paste CI gate: no locked claim, no entry.
EU AI Act · compliance resources

Long-form working notes for compliance leads, AI governance officers, and notified-body assessors preparing for the 2 December 2027 high-risk deadline for Annex III systems under Regulation (EU) 2024/1689. CC BY 4.0.

Article 9, 12, 13, 14, 17, 18 EU AI Act readiness assessment Six articles, ten questions, defensible evidence shape for high-risk AI providers. 2 December 2027 Deadline & ten-week plan Three application dates, risk classification, Article 99 penalty structure, pragmatic execution path. Article 12 logging Ten-item logging checklist Six event categories, ten-year retention, printable single-page summary. New · Reg 2026/1744 Article 25(2) hand-over pack Provider hand-overs are now a fineable evidence event. What must be disclosed, and how to timestamp it. Annex VI vs VII Notified body evidence When you need a notified body, what an Article 31 assessor actually asks for, six artefact families. ISO/IEC 42001:2023 42001 AIMS readiness Seven clauses, EU AI Act Article 17 overlap, twelve-month certification path. Case study #1 Anonymized LLM tech report audit Score 5/9 on a typical published claim. PRML manifest, audit memo, what we caught. Field report · 2026-05-22 Lock #2 post-mortem A pre-registered distribution experiment, resolved against its own threshold, and the counting correction that followed.
CÖ
Founder · Falsify OÜ
Cüneyt Öztürk
Founder, Falsify OÜ · author of PRML

Author of the PRML specification and of the four reference implementations (Python, JS, Go, Rust). Works on evaluation integrity: making the success criteria behind an AI claim independently checkable after the fact, rather than taken on the word of whoever ran the test.