A study titled “Spectral Smoothing and Spiking Perturbations Reconfigure Capability Profiles and Selective Task Performance in Large Language Models” was published on arXiv on October 24, 2026 (ID 2610.30676v1), focusing on the singular value spectrum of the up_proj and down_proj matrices in the feedforward layers of transformer models and exploring their causal effects on downstream behavior in large language models.
The study proposes two types of post-processing interventions: positive perturbation (spiking) amplifies the deviation of singular values from their geometric mean, boosting the largest values while compressing the smallest ones; negative perturbation (smoothing) does the opposite. Both operations leave the singular vectors unchanged and keep the geometric mean of the singular values constant.
The experiments cover five open-source Llama and Qwen instruction-tuned models, evaluating the perturbation effects across multiple dimensions—including capability-annotation subtasks, standard and chain-of-thought reasoning benchmarks, human‑rated measures of truthfulness, hallucination, and safety, as well as the TRAIT personality probe—and map the resulting response changes to a three‑dimensional capability framework encompassing “factual knowledge–language understanding–deductive reasoning” for interpretation.
The results show that changes in model capabilities are structural rather than random: substantial positive perturbations generally impair performance in language understanding and deductive reasoning; moderate negative perturbations can enhance factual accuracy or local task performance on certain evaluation subsets; and sensitivity depends significantly on the model family, the type of evaluation module, and the degree of imbalance across layers. This study positions spectral perturbation as a lightweight, interpretable tool for probing capability organization.