Field Notes

Notes from running the platform, not just designing it.

Cost problems, incidents, and the architecture calls behind both. Written from production, after the dashboards went green again.

#clickhouse (2)#cost (1)#data-platform (2)#dbt (1)#iceberg (1)#incident (1)#postmortem (1)

July 7, 2026

5 min read

When your maintenance job can't outrun your writer: an Iceberg snapshot-bloat outage

Two bronze tables hit 20k+ Iceberg snapshots and took bronze freshness down with them. The interesting part wasn't the bloat. It was discovering that the standard expiration tool loses the optimistic-lock race against a hot writer deterministically, not occasionally.

#iceberg #clickhouse #incident #postmortem #data-platform

March 20, 2026

3 min read

The dbt test suite that cost almost as much as the pipeline

Every dbt run was executing the full test suite against ClickHouse Cloud, and the tests were costing almost as much as the pipelines they were checking. Splitting tests into two tiers brought the bill down by 95 percent without losing coverage.

#clickhouse #dbt #cost #data-platform