The Honest Readiness Scorecard: What 8/10 Actually Means
Passing every local test is not the same as being production-ready. Conflating the two is one of the most common and costliest habits in software portfolios, and in shipping decisions. This article examines the actual distinction, and why publishing the gap is a more trustworthy signal than hiding it.
Local confidence is real. Production proof is different.
A project can have strict TypeScript, a full test suite, 89% line coverage, and passing accessibility audits, and still not be production-ready. That is not a contradiction. It reflects the difference between two distinct categories of evidence: what you can verify before a single real user touches the system, and what only real usage can prove.
Local engineering confidence covers correctness at the unit and integration level. It covers type safety, linting rules, schema validation, and accessibility conformance in a controlled environment. These are genuinely valuable. A project without them is in worse shape. But they cannot answer the questions that production surfaces: Does the system hold its SLOs under real load? Do actual users complete the flows you designed? Does the data model serve the use cases you assumed?
The industry default is to skip this distinction entirely. A project "works," the CI is green, so it ships labeled production-ready. That label carries an implicit 10/10 claim. What it actually means, in most cases, is "10/10 on local confidence, 0 on production proof, and we are not publishing that gap."
A more honest label, and a more useful one, is to score them separately and publish both.
The 8/10 scorecard in detail
MetaMuseum is a federated museum data platform with 13 provider adapters, multi-tenant org isolation, RBAC, and a SHACL-gated CI pipeline that verifies JSON-LD expands to valid CIDOC-CRM RDF at the MUST level on every change. It has 1,100-plus tests written with the Node test runner, strict TypeScript throughout, 89% line coverage, and Lighthouse plus axe checks targeting WCAG 2.1 AA. On local engineering criteria, the numbers are strong.
The project scores 8/10. Not 10. The gap list is explicit and published:
Local engineering
1,100+ node:test tests, strict TypeScript, 89% line coverage, WCAG 2.1 AA via axe and Lighthouse, SHACL-gated RDF validation in CI.
Production proof
No deployed SLO evidence, no pilot users on record, ActivityStreams adoption not measured, feed adoption maturity not proven in a live environment.
The project documentation puts it plainly: "The strict 10/10 production claim cannot pass from local checks alone. It stays blocked until real reliability, pilot, KPI, and feed-adoption evidence matures. Readiness is a measured fact, not a marketing line."
That last sentence is the point. Readiness is a measured fact. It is not a confidence level, a feeling, or a marketing position. The SHACL gate verifies structural correctness. The test suite verifies behavioral correctness. Neither verifies that the system will hold its performance envelope when 50 concurrent museum curators are importing collections, because that has not happened yet.
Rating it 8/10 is not false modesty. It is the accurate reading of what has been verified and what has not.
The same discipline shows up across every project
MetaMuseum is the clearest example because its gap list is the most formal, but the same instinct runs through the other projects in this portfolio. The form it takes varies, but the underlying commitment is the same: do not claim more than the evidence supports.
Four gates, every change
Every commit runs tsc --noEmit, ESLint, 197 Vitest tests on in-memory PGlite, and a production build. A live Wikidata drift probe guards the data pipeline against upstream schema changes. The project also draws a clear line between "curated notable sample" and "exhaustive database." It is the former. It does not claim to be the latter.
Bounded scope, documented
2,220 AI narrations, all citation-verified, in a corpus that is explicitly bounded. Duplicate place slugs, 15 found, are documented in the roadmap rather than papered over. Dynamic surfaces are described as "deliberately thin," not apologetically thin. The constraint is a design decision, not a deficiency.
Knowing when to stop
The LLM layer was removed entirely when it could not be grounded reliably. No keys, no per-token cost, no hallucinated content. A "paintings from year N" query was dropped when it returned results in 23 seconds on WDQS and proved un-indexable. The single-process cache limitation for multi-instance deploys is documented, not hidden.
Clear ownership boundaries
HMAC webhook verification keeps revalidation from being triggered externally. A health endpoint reports boolean configuration state without exposing secrets. The architecture explicitly documents what Shopify owns versus what Next.js owns, so the boundary is legible to anyone maintaining it later.
The Artwork of the Day example is worth pausing on. Removing the LLM layer is a harder decision than it looks. It means shipping a project that does less than you originally planned. The alternative, shipping the LLM layer anyway and hoping the grounding problems do not surface in production, is the default move in most portfolios. It is also the move that erodes trust when the problems do surface.
Dropping the un-indexable query is the same pattern. The query worked in principle. It did not work in practice at the latency that makes it useful. Documenting that decision, and the 23-second number that drove it, is more informative than omitting the feature without explanation.
Four questions before claiming production-ready
The scorecard approach reduces to four questions. They are not novel questions. They are the questions that tend to get skipped when there is pressure to ship or pressure to present well.
- What have you actually measured versus what have you assumed? Local tests measure correctness against your own assertions. SLOs measure behavior under real conditions. If you have not measured it, you have an assumption, not a proof. Name which one you have.
- What would falsify your readiness claim? If you cannot answer this, the claim is not specific enough to be meaningful. MetaMuseum's answer is specific: pilot user data, deployed SLO evidence, feed adoption metrics. Until those exist, 10/10 is blocked.
- What are you not claiming? MetaHistoryBook is not claiming to be an exhaustive historical database. Big Aloha Guide is not claiming infinite dynamic scale. Naming what you are not is as important as naming what you are. It prevents the gap from being inferred incorrectly by someone reading the documentation later.
- What did you remove, and why? The decisions to cut scope, drop a feature, or remove a layer are often the most informative signals about engineering judgment. A project that documents what it chose not to ship, and why, is easier to trust than one that presents only what shipped.
Publishing the gap list builds more trust than hiding it
The instinct to hide the gap list is understandable. A scorecard that says 8/10 looks worse than one that says 10/10, if the reader is not thinking carefully. But the reader who is thinking carefully, the reader who is evaluating whether to rely on this system or hire this engineer, is doing exactly the opposite calculation.
A 10/10 claim with no gap list is not informative. It means the author either did not look for gaps, looked and decided not to publish them, or genuinely believes that passing local tests is sufficient evidence for a production claim. None of those interpretations is reassuring.
An 8/10 claim with a specific gap list tells you several things. It tells you the author knows the difference between local confidence and production proof. It tells you they checked and found something, rather than not checking. It tells you the gaps are bounded and named, which means they can be addressed. It tells you the author will tell you when something is not ready rather than letting you find out in production.
That is the claim this portfolio makes. Not that every project is production-ready in the fullest sense, but that every project knows where it stands and says so. The MetaMuseum score is 8/10 because 8/10 is correct. When the pilot data exists, when the SLOs are running in a deployed environment, when the feed adoption evidence is mature, the score will change. Until then, the gap list stays published.
Readiness is a measured fact, not a marketing line. The measurement is the product.