k-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating

Low-Rank Adaptation (LoRA) fine-tunes large models efficiently but conventionally updates all matrices uniformly, wasting compute on matrices that contribute little. κ-LoRA shows that matrices with higher condition numbers hold underdeveloped directions driving most adaptation gains, while low-condition-number matrices are already balanced and add little value. By restricting updates to the top 50% of matrices by condition number, the method halves trainable parameters, cuts fine-tuning time by 16.2%, and reduces memory use, while matching standard LoRA accuracy. This is directly applicable to efficient large-model fine-tuning, particularly for edge deployment and resource-constrained on-device adaptation scenarios. Authors: Jianghui Wang, Silong Yong, Francesco Orabona, Marco Canini, Katia P. Sycara, Yaqi Xie Paper: https://arxiv.org/abs/2607.22489v1