random matrix theory

eOptShrinkQ: Near-Lossless KV Cache Compression for LLMs

eOptShrinkQ offers near-lossless KV cache compression for LLMs, leveraging spectral denoising and quantization to reduce memory overhead and improve long-context inference.