AI NEWS.ccxmod-base
X-MOD is a multilingual masked language model trained on filtered CommonCrawl data containing 81 languages. It was introduced in the paper Lifting the Curse of Multilinguality by Pre-training Modular Transformers (Pfeiffer et al., NAACL 2022) and first released in this repository. Because it has been pre-trained with language-specific modular components (_language adapters_), X-MOD differs from previous multilingual models like XLM-R. For fine-tuning, the language adapters in each transformer layer are frozen. Because this model uses language adapters, you need to specify the language of your input so that the correct adapter can be activated:
X-MOD is a multilingual masked language model trained on filtered CommonCrawl data containing 81 languages. It was introduced in the paper Lifting the Curse of Multilinguality by Pre-training Modular Transformers (Pfeiffer et al., NAACL 2022) and first released in this repository. Because it has been pre-trained with language-specific modular components (_language adapters_), X-MOD differs from previous multilingual models like XLM-R. For fine-tuning, the language adapters in each transformer layer are frozen. Because this model uses language adapters, you need to specify the language of your input so that the correct adapter can be activated: