This paper, titled “Meaning Contracts: Local Semantic Geometry in Large Language Models,” was submitted to arXiv on October 24, 2026 (ID 2610.30715v1), and focuses on the local geometric structure of semantic meaning within the embedding space of large language models—specifically, which aspects of a given semantic factor remain invariant across different implementations, which vary, and how these factors evolve as semantics are refined.
The study designs two types of controlled probes: cross-linguistic probes treat language switching as a perturbation, distinguishing between semantically identical matches, same factors with different realizations, closely related concepts, and out-of-domain contrasts; refined probes, on the other hand, differentiate between expressions that encode narrowly scoped semantic specificity and those that are high-frequency attribute modifiers or adversarial control samples. Both probe sets were validated through human blind evaluation. The experiments encompass four decoder-only models with distinct architectures: Gemma 4, Llama 3.2, Qwen 2.5, and Pythia.
The study found strong consistency: after mean pooling and removal of the dominant variance direction, sentence pairs are strictly hierarchically ordered according to semantic relatedness (same factor → same domain → cross-domain), with no overlap; semantic factors are organized around an invariant core, while their periphery exhibits structured, domain-specific variations; semantic refinement triggers a directional shift toward closer neighborhoods, even surpassing the control group constructed on the basis of identical base nouns; sentence-level and word-level readouts reveal complementary geometries: the former manifests as volumetric contraction, whereas the latter displays sharper anisotropic endpoints.
Based on this, the authors infer that the semantic representations of large language models are not uniformly distributed, but rather organized into locally anisotropic neighborhoods, with semantics undergoing a contraction‑induced coalescence during refinement. This conclusion does not rely on the assumption of linear separability, offering a new geometric perspective for understanding the intrinsic semantic structure of LLMs.