TurboQuant: Google’s New Compression Tricks That Actually Work

I’ve been following quantization research for a while now, and most papers feel like they’re solving the same problem with slightly different math. But Google Research’s new batch of algorithms—TurboQuant, Quantized Johnson-Lindenstrauss (QJL), and PolarQuant—actually address something that’s been bugging me for years: the hidden memory overhead.

If you’ve worked with large language models or vector search engines, you know the drill. Vectors are everywhere. They represent everything from image features to word meanings. And they’re expensive. The key-value cache in transformers is basically a high-speed cheat sheet that stores frequently used info, but it fills up fast when you’re dealing with high-dimensional vectors.

Vector quantization is the classic fix. You compress those vectors to save memory and speed up similarity searches. But here’s the dirty secret: traditional quantization methods introduce their own memory overhead. You have to calculate and store quantization constants for every small block of data, usually in full precision. That adds 1 or 2 extra bits per number. It’s like patching a leaky pipe with tape that’s also leaking.

TurboQuant, which Google is presenting at ICLR 2026, aims to fix this. It combines two techniques: PolarQuant for the heavy lifting and QJL for cleanup. Let me break down how they work.

How TurboQuant works

TurboQuant has two stages. First, it randomly rotates the data vectors. This sounds weird, but it simplifies the geometry so you can apply a standard quantizer to each part of the vector independently. PolarQuant handles this stage, using most of the compression bits to capture the vector’s core meaning.

Then comes the clever part. TurboQuant uses just 1 bit to apply QJL to the residual error left over from the first stage. QJL acts as a mathematical error-checker, eliminating bias and producing more accurate attention scores. It’s like having a proofreader who only fixes the typos instead of rewriting the whole paragraph.

QJL: The zero-overhead, 1-bit trick

QJL is based on the Johnson-Lindenstrauss Transform, which shrinks high-dimensional data while preserving distances between points. What’s impressive is that it reduces each vector number to a single sign bit (+1 or -1). Zero memory overhead. To maintain accuracy, it uses a special estimator that balances a high-precision query with the low-precision data. The result is accurate attention scores without the usual overhead.

PolarQuant: A new angle on compression

PolarQuant takes a different approach to the memory overhead problem. Instead of representing vectors in standard Cartesian coordinates (X, Y, Z), it converts them into polar coordinates—angles and magnitudes. This representation naturally requires fewer bits to store, and it eliminates the need for those expensive quantization constants.

The team tested these algorithms on key-value cache compression and vector search tasks. The results show significant compression ratios without sacrificing model accuracy. I was skeptical at first—I’ve seen too many quantization papers that look great on synthetic benchmarks but fail in practice. But Google’s testing seems thorough, covering both LLM inference and large-scale similarity search.

Why this matters

The key-value cache bottleneck is one of the biggest practical problems in deploying large language models. Every time you increase context length or batch size, memory usage balloons. TurboQuant’s approach could make longer contexts and larger batches feasible on existing hardware.

For vector search, the implications are even broader. Every major search engine and recommendation system relies on vector similarity. Faster, more memory-efficient quantization means cheaper infrastructure and faster response times.

I’m particularly interested in the QJL algorithm. The idea of using 1 bit per component with zero overhead is elegant. It reminds me of some older work on binary embeddings, but the error-correction mechanism is new. I’d like to see how it performs on real-world datasets with noisy or adversarial inputs.

One thing I wish the paper covered more: the computational cost of the random rotation step. Random rotations are expensive, especially for high-dimensional vectors. If TurboQuant requires significant preprocessing, that could limit its applicability in real-time systems.

Still, this is solid research from a team that clearly understands the practical constraints of deploying AI at scale. I’ll be watching for the full papers at ICLR and AISTATS 2026.