Selected work · Kustomer
Make every number safe to quote
A governed warehouse did not stop the same business question from returning different answers in the product, in BI, and in finance. The fix was a shared serving layer over ClickHouse plus a certification standard that a metric has to earn, not a policy document asking people to be careful.
- 1x
- metric definition, every surface
- 18-30ms
- parity overhead on the serving path
- Certified
- line between quote-safe and exploratory
The situation
The warehouse was governed and the modeling discipline was real. That is where the discipline stopped. Every surface that people actually looked at re-implemented the metric in hand-written SQL, so governance ended at the gold layer and the last mile was on the honor system.
The visible symptom was familiar: an account team quotes a number from the product, finance quotes a different one from BI, and the next hour is spent reconciling instead of deciding. The invisible cost was worse. People stopped quoting numbers at all, or quoted them with a disclaimer, which is the same thing as not having them.
Constraints
- Multi-tenant. Every query had to respect tenant isolation, so a shared metric layer could not become a shared data leak.
- Customer-facing latency budget. The product surfaced these metrics, so correctness could not be bought with a slow path.
- No appetite for a rewrite. The warehouse and the models were fine. The answer had to sit on top of what already worked.
What I did
01
One definition, served to everyone
A self-hosted MetricFlow serving layer over ClickHouse, so the application and BI resolve the same definitions through the same path under multi-tenant row-level security. The definition stops being a convention that each team re-types and becomes an artifact that each team calls.
02
Prove parity instead of asserting it
Byte-identical parity checks between the served result and the warehouse result. Parity that is measured survives a migration. Parity that is claimed does not. The serving path carried 18 to 30 milliseconds of overhead, which bought correctness inside the latency budget rather than trading one for the other.
03
Make certification something a metric earns
A nightly quality score and a Certified badge that only lands when a metric has a ratified brief, a semantic definition, and proven provenance on the surface where it appears. Miss any of the three and the badge comes off automatically. Nobody has to police it.
04
Keep models out of the scoring seat
Models advise on failures and help explain what broke. They never assign the score. The moment a probabilistic system decides what counts as trustworthy, you have moved the trust problem instead of solving it.
Outcomes
- Write-once metrics for customer-facing reporting and internal BI
- Multi-tenant RLS with byte-identical parity checks
- Self-serve BI with certified finance, product, and CS surfaces
- "Safe to quote" enforced by the platform, not a wiki
What I would tell you now
- Governance that stops before the serving layer is not governance. It is documentation with better tooling.
- A certification badge is only worth something if it can be revoked automatically. Manual certification decays the week after the launch announcement.
- The useful line is not clean versus dirty data. It is quote-safe versus exploratory, drawn where the reader can see it.
Who this is for
Read this if your warehouse is in decent shape but people still argue about what a number means, or if you are being asked to expose customer-facing metrics and cannot yet promise the product and BI will agree.
Recognize the shape of this?
Thirty minutes is usually enough to tell whether it is the same problem. I reply within 48 hours.