On October 24, 2026, a paper titled “AQL-Lite: Enabling Efficient Knowledge Acquisition for Arabic LLMs via Data-Centric Continual Pretraining” was published on the arXiv platform (ID 2610.30344v1), officially introducing the AQL-Lite framework. The framework aims to efficiently adapt multilingual large language models to Modern Standard Arabic (MSA) while reducing time and resource costs.
AQL-Lite comprises two core components: first, a quality-first data preprocessing pipeline that includes structural cleaning, heuristic and model-driven filtering, MSA text recognition, and deduplication; second, a partial parameter continual pre-training (CPT) strategy that updates only the RS-LoRA adapter, the last eight Transformer layers, and the embedding matrix, while employing a hierarchical learning rate. Under a 1-billion-token budget, this approach improves the average scores of four Arabic-language benchmarks by 3.51 points, whereas CPT applied to raw network text under the same conditions yields no gains.
In a controlled comparative experiment with 5 billion tokens, AQL-Lite achieved an average score of 73.19 across four benchmarks, representing a 94% performance improvement over the full-parameter CPT (73.58) while consuming only 52% of its training time; moreover, it outperformed the pure adapter baseline on all benchmarks. Further scaling to 40 billion MSA tokens and fine-tuning on one million instructions yielded a 12B-parameter model that scored 75.3 on the Open Arabic LLM Leaderboard v2 (seven benchmarks).
The paper notes that ablation studies confirm that the RS-LoRA adapter, the unfrozen base modules, and the embedding matrix training each contribute independently. The study’s conclusions emphasize that, based on carefully curated corpora, fine-tuning CPT with a subset of its parameters represents a practical and efficient pathway for adapting large MSA models.