On October 26, 2026, a research paper titled “LAMB: A Modern Contextual Language Model for Latin” was published on the arXiv platform (arXiv:2610.31723v1), officially introducing the Latin-specific contextual language model, Latin Modern BERT (LAMB). Built on the ModernBERT architecture, this model employs a two-stage pre-training pipeline, processing a total of over 74 billion (74B) Latin text tokens, and incorporates a novel “morphologically informed masking regimen” to better accommodate Latin’s highly inflected and morphologically rich linguistic characteristics.
The paper simultaneously releases two brand-new Latin corpora: Acervus and Corpus Trinum. Acervus contains 24 billion (24B) BERT WordPiece tokens, with over 760,000 samples derived from severely corrupted OCR-transcribed Latin texts that were processed through a ByT5 model specifically trained for Latin to perform text restoration; Corpus Trinum, meanwhile, is the first semantic triple dataset in Latin, encompassing more than 100 million (100M) strong and weak mixed pairs of Latin sentence triples, designed to support downstream tasks such as sentence-level semantic similarity.
The study constructs an evaluation benchmark covering five task categories and 17 subtasks, and conducts a comparative assessment of 15 pre-trained encoders, six LAMB training checkpoints, and one ablation model. The results show that LAMB achieves state-of-the-art performance on 16 out of the 17 subtasks, with the remaining one trailing the current best by only 0.8 percentage points. The paper also provides an in-depth analysis of how factors such as tokenizer behavior, syntactic structure, and fine-tuning configurations influence the performance of Latin-language models, and for the first time identifies a distinctive pattern of acquisition for open-class words in Latin during the “terminal decay phase” at the end of pre-training.