The Petabytes of Data Issue with AI

AI's next scaling problem isn't generating intelligence. It's understanding it.

For three years, AI infrastructure has been measured in FLOPS, GPU clusters, parameter counts, and cost per token. I suspect history will conclude we were optimizing the wrong bottleneck.

Every AI inference produces far more than an answer. It leaves behind a trail of operational evidence: routing decisions, GPU allocation, KV cache behavior, vector retrievals, memory pressure, latency distributions, safety evaluations, retries, model versions, and thousands of other signals. The output is the smallest artifact of the computation. The telemetry is the real exhaust.

Telemetry is compounding faster than compute. As AI systems become agentic, every invocation becomes a distributed system spanning storage, networking, retrieval, orchestration, and inference, each layer emitting high-cardinality signals. At scale, these systems will generate petabytes of operational data every day.

This isn't a new problem. In 2013, Facebook's Scuba changed how engineers reasoned about production systems. Instead of relying on pre-aggregated metrics and dashboards, Scuba made it possible to ask arbitrary questions over raw operational events in near real time. It transformed debugging from pattern matching into interactive exploration. The tradeoff was economics: Scuba kept enormous amounts of data in RAM, practical for Facebook but prohibitively expensive for everyone else.

That question has quietly become one of the most important problems in computer science.

How do you preserve the flexibility of Scuba while moving from terabytes to petabytes? How do you make object storage behave like memory? How do you query data without paying the cost of reading all of it?

The answers come not from larger models, but from decades of database research.

Probabilistic algorithms such as HyperLogLog showed that exact answers are often unnecessary when approximate answers are orders of magnitude cheaper. Daniel Lemire's work on compressed data structures, SIMD acceleration, and probabilistic filters showed that the fastest query is the one that never reads the bytes in the first place. Modern systems are synthesizing these ideas: pruning before scanning, evaluating predicates on encoded data, separating indexes from storage, and treating network I/O, not CPU, as the dominant constraint.

The future will not be engineers staring at dashboards. It will be AI agents reasoning over years of telemetry to diagnose other AI agents, reconstruct causal chains across billions of events, and explain why autonomous systems behaved the way they did. That future depends less on producing more data than on making nearly infinite amounts of data economically searchable.

The next generation of infrastructure companies may not be remembered because they built better models. They may be remembered because they made intelligence explainable at planetary scale.