Benchmark № 001 — methodology
Frozen before results.
The full methodology is published here before any vendor is queried, with a checksum so you can verify nothing changed after the fact. Vendors are invited to dispute the method before publication.
Status: draft — freeze pending. The frozen version will carry a commit hash and SHA-256.
What the frozen document specifies
The pre-registration covers ten sections. Nothing about scoring changes after the freeze.
- Dataset composition — 1,000 numbers by category, with ground-truth quality stated per category.
- Ground-truth definitions — valid format, allocated vs unallocated (NANPA file date pinned), fictional ranges, line type, carrier, ported, active.
- Exact vendor products and tiers under test — all public self-serve, named before the run.
- Run dates and data snapshots — results are point-in-time and dated.
- Scoring rubric — per-field correct / incorrect / partial; "no data" counted as abstained, not wrong.
- Weighting — weak-ground-truth categories down-weighted or reported descriptively, declared up front.
- What is published, what is redacted, what a neutral auditor can inspect under NDA.
- Vendor dispute process — every vendor may review and dispute before publication; public errata log.
- Reproducibility — the re-run harness and the exact portion any third party can reproduce.
- Conflict-of-interest disclosure — we operate a competing product, and we say so on every page.
Numbers are synthetic, owned by us, or contributed with written consent. No scraped or purchased lists, and no calls or texts are ever placed — API queries only.
Get the results when they drop
Benchmark results + early API access. One email when results drop — nothing else.