Attention Quantization for Tabular Foundation Models

Tabular foundation models are served differently from chatbots, so the usual speedups don't carry over. Here the bottleneck is the attention calculation. Converting queries, keys and values to FP8 gave up to 1.7 times the speed with no meaningful accuracy loss on TabPFN-v3 and TabICLv2. The catch: quantization error on test rows has to match the training rows, or accuracy drops sharply. Authors: Jonas M. Kübler, Benjamin Jäger, Klemens Flöge, Noah Hollmann, Frank Hutter Paper: https://arxiv.org/abs/2609.13031v1