CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG

Long-context retrieval-augmented generation systems often reuse KV caches at the chunk level for efficiency, but this approach retains noisy, redundant information within coarse chunks. CoinRAG instead identifies fine-grained, query-relevant "nuggets" within retrieved chunks and reassembles their cached representations into a compact, semantically focused context. This is useful for applications requiring fast, low-latency RAG at scale, such as enterprise search, multi-hop question answering, and chatbots handling large document collections. By improving the accuracy-efficiency Pareto frontier, CoinRAG could benefit any system needing to reduce operational costs while maintaining answer quality, particularly in scenarios with tight prefill latency budgets. Paper: https://arxiv.org/abs/2608.07458