All AI updates

Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?

arXiv:2610.32046v1 Announce Type: new Abstract: Recent \emph{test-time training} (TTT) frameworks have enabled open-weight large language models (LLMs) to outperform inference-only frameworks powered by frontier models on scientific discovery tasks. However, TTT introduces additional computational cost for online weight updates during the discovery process, making computational efficiency an important consideration when scaling to larger models. This motivates us to explore an efficient \emph{test-time adaptation} method for scientific discovery. We introduce \emph{Test-Time Memory Evolution} (TTME), a plug-and-play test-time adaptation method, as an alternative to TTT. TTME continually updates the external memory by distilling the experience to guide subsequent discovery without updating

Do AI-for-Science Agents Preserve Model Safety?

arXiv:2610.32045v1 Announce Type: new Abstract: AI-for-Science agents are increasingly being used for formulating scientific hypotheses, running autonomous experiments, and interpreting evidence. These agents are constructed by integrating a large language model (LLM) into a domain-specific agent harness, which in turn can improve the LLM's scientific capability and enable the model to orchestrate scientific tools. In this work, we ask whether model safety is preserved when an LLM is integrated in an AI-for-Science agent harness. We evaluate five agent harnesses on 50 harmful chemistry requests. Specifically, we study how the safety behavior changes under direct prompting, two simple adversarial prompting strategies, and an adaptive attack. Across four recent GPT variants, our results show

Training an MoE LLM in MXFP4: How We Closed the Gap to BF16

arXiv:2610.32229v1 Announce Type: new Abstract: Training large language models (LLMs) in sub-8-bit arithmetic promises substantial gains in memory bandwidth and compute throughput, yet FP4 formats introduce quantization error severe enough to destabilize optimization if left unaddressed. We present ALTO, an open-source training recipe for GPT-OSS-20B, a 20-billion-parameter mixture-of-experts (MoE) model, on the MLPerf Small MoE Training benchmark under the MXFP4 microscaling format. The recipe combines established techniques from Abecassis, Felix et al. [arXiv:2509.25149] (hybrid 1D/2D block quantization, randomized Hadamard transforms, and stochastic rounding) with a simplified weight de-oscillation scheme adapted from TetraJet-v2 [arXiv:2510.27527]. On C4 validation at global batch size

EDWARD-NMT: Elastic Dimensions With Adaptive Routing and Dual-consistency for Neural Machine Translation

arXiv:2610.32115v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (MoE) is a powerful paradigm for scaling Neural Machine Translation (NMT) and managing language heterogeneity. However, practical limitations hinder its deployment on resource-constrained devices: the entire expert pool must remain in memory, and the per-token compute budget is statically fixed during training by rigid top-k routing. We introduce EDWARD-NMT (Elastic Dimensions With Adaptive Routing and Dual-consistency for NMT), a novel MoE encoder-decoder architecture designed for a train once, deploy anywhere paradigm that exposes a family of deployable sub-networks at inference time. EDWARD-NMT integrates three key innovations: (i) Matryoshka-style elastic dimensions to flexibly reduce the memory footprint; (ii) a

Query recall determines the retrieval performance of classical Arabic poetry: BM25 and dense encoders show markedly divergent behavior as query abstraction increases.

A recent study published on October 26, 2026, on arXiv (arXiv:2610.31821v1) shows that, in classical Arabic poetry retrieval tasks, the degree to which users retain fragments of poetic lines—such as whether they can recall half a line—plays a more critical role in determining retrieval performance than model capabilities. BM25 outperforms all dense encoders, including state-of-the-art models with 8B parameters, on “tip-of-the-tongue” queries, but falls significantly behind on abstract queries.

Why it matters：This study, through systematic benchmarking using a bilingual corpus, a three-index model, and a dual-evaluation protocol, for the first time quantitatively reveals the dominant influence of query abstraction on different retrieval paradigms (lexical vs. dense), and—after being validated by both human annotators and open-source weight models—the conclusions demonstrate methodological universality and empirical robustness.

Kolibri: A Sovereign European Model on the Pareto Frontier

arXiv:2610.31945v1 Announce Type: new Abstract: This report introduces Kolibri, an English-German mixture-of-experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under the Apache 2.0 licence. We built Kolibri for sovereign and specialised deployment, with a particular focus on German and on regulated domains such as public administration, industry, and aerospace. To support these settings, we control the full model-development process, including data, architecture, training infrastructure, post-training, and evaluation. We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20% of the data mix by bytes. To reach this share, we complement German datasets wit

In LLM evaluation, relying solely on the overall F1 score to validate automated scorers carries the risk of misleading results: a new study underscores the need to calibrate for prompt-specific differences.

An arXiv paper (2610.31678v1) points out that, in interactive evaluations of large language models, relying solely on the overall accuracy or F1 score of automated scorers can mask prompt-specific errors, leading to misjudgments of prompt effects. Empirical evidence shows that the same scorer exhibits substantial variation in its false-positive rate across different prompts (2.6%–33.3%), and a seemingly significant 9.3-point effect shrinks to 2.3 points—and becomes non-significant—under dual-LLM review.

Why it matters：This study, based on real experimental data (6 models, 180 fictional entity‑based questions, and 6 prompt conditions), reveals a widespread methodological flaw in current LLM evaluations: equating the scorer’s overall performance with its reliability on key benchmark tasks. Its central claim—that each prompt must be audited separately and uncertainty must be incorporated—rests on clear empirical evidence and offers actionable recommendations, making it highly timely (published on 2026‑10‑26) and backed by a credible source (the arXiv preprint platform).

Behaviour analysis for large language model-driven agents

arXiv:2610.32087v1 Announce Type: new Abstract: Understanding language-agent behaviour requires learning how actions are organised across a trajectory and how this organisation relates to context and outcomes. We propose a general framework that connects data-driven behavioural discovery with analysis and experimental investigation. Categories developed from recorded activity define what is measured. Analyses preserve the behaviours' positions in the process and express their relationships as interpretable hypotheses. Statistical assessment and tests in held-out contexts examine the strength and scope of the evidence. Experiments, including prompt steering, then test how behaviour and outcomes change under deliberately varied conditions. A study of iterative self-reflection illustrates how

Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead

arXiv:2610.32215v1 Announce Type: new Abstract: The growing computational power of modern GPU data centers has let a few industrial giants train and deploy large language models (LLMs) on massive amounts of data. Yet the sheer scale of these models and their training data makes centralized development increasingly hard to sustain, and at the same time shuts out smaller institutions that hold relevant private data but lack comparable infrastructure. In this paper, we organize our positions on distributed LLM training around the pretraining and fine-tuning paradigms, covering federated learning, split learning, and decentralized approaches, and we assess their readiness for geo-distributed settings with heterogeneous hardware and unreliable connectivity. We then present a {\em unified} syste

New research reveals serious measurement bias in LLM-based math tutoring assessments: exact-match scoring underestimates effectiveness by nearly 40 percentage points.

The paper “MisconceptDialogue: Evaluation Criteria Shape Measured Success in LLM Math Tutoring,” published on arXiv on October 26, 2026, introduces the MISCONCEPTDIALOGUE automated evaluation framework, which assesses the performance of five open-source small-parameter LLMs (7B–14B) as math tutoring models through over 5,000 rounds of dialogue. The study finds that the mainstream exact-match scoring method achieves only a 42.1% remediation success rate, whereas an expert‑calibrated LLM referee evaluates the same dialogues at 81.3%, a gap of 39.2 percentage points; moreover, this discrepancy is not accidental—four types of LLM referees show high agreement with 200 human‑annotated results, with a maximum kappa coefficient of 0.91. In addition, the models exhibit virtually no differentiation in their responses to diagnostic information (correct or incorrect concepts), indicating their limited ability to effectively leverage such diagnostic inputs.

Why it matters：This study directly addresses the core flaw in current methodologies for evaluating LLM-based educational applications: reliance on a single, rigid automated scoring system can severely distort assessments of actual competence. Its large-scale simulation experiments, multi-rater convergence validation, and empirical analysis of the breakdown in diagnostic–instructional coupling provide critical warnings and a replicable methodological benchmark for standardizing AI-driven educational assessment.

The study proposes the “Schema Consistency Index” (SCI) to assess the coherence of belief representations in language models.

On October 26, 2026, arXiv published the paper arXiv:2610.31683v1, introducing the Schema Coherence Inventory (SCI)—a 96-item forced-choice scale designed to assess whether the beliefs attributed to persona-conditioned language models genuinely structure their cognition, rather than merely eliciting emotional responses.

Why it matters：This study fills the current gap in the lack of a verifiable measurement method for the “degree of belief internalization” in language models; through validation by clinical experts, testing across multiple model backbones, and the design of target-specific criteria, it provides the first assessment framework that focuses on cognitive structures rather than emotional representations, and has open-sourced all its tools.

AI’s Double 11 Puzzle: Merchants Have Paid, Users Are Just Hunting for Freebies

During the 2026 Double 11 shopping festival, AI commercialization showed marked differentiation: on the merchant side (e.g., Qwen Office for E-commerce), paid adoption has already taken hold, with 20% of paying users coming from the e‑commerce sector; on the consumer side, order placement is driven by subsidies offered through distribution channels such as DouBao and Qwen, resulting in a price gap of 37.5%, yet user trust remains low (only 16.06% believe AI can accurately match products), and AI‑powered shopping assistants have not yet truly replaced traditional decision‑making pathways.

Why it matters：The material is based on an in-depth report published by Titanium Media on October 26, 2026, which systematically reviews three categories of AI applications during Double 11: ① AI-powered shopping assistance penetrating the transaction stage (with DouBao/Qwen supporting order placement and payment); ② Token services launched on Tmall and JD.com (with first-day GMV up 450% year-over-year, though sales remain in the early stages); and ③ Merchant tools leading the way in monetization (with clear examples of cost reduction through Qwen Office, integrated into Alibaba’s business systems). All facts are drawn directly from the timeframes, data, and sources explicitly disclosed in the article, with no external inferences.

Nanbeige Releases Nanbeige4.3-33B, a 33B Open Agentic Coding Model

Nanbeige LLM Lab released Nanbeige4.3-33B, a 33B dense Apache 2.0 model with 512K context. It reports 80.9 on Terminal-Bench 2.1 and will teach the upcoming Nanbeige4.3-3B. The post Nanbeige Releases Nanbeige4.3-33B, a 33B Open Agentic Coding Model appeared first on MarkTechPost.

Hugging Face Releases Carbon-X, a 14B Open DNA Language Model that Brings DNA and Natural Language Together for the First Time

Hugging Face released Carbon-X, an open 14.8B DNA language model built on Qwen3-14B. It reads raw DNA in chat, answers in English, and averages 66.7% across 5 DNA benchmarks under Apache 2.0. The post Hugging Face Releases Carbon-X, a 14B Open DNA Language Model that Brings DNA and Natural Language Together for the First Time appeared first on MarkTechPost.

Anthropic launches OSS Scanner, a free AI-powered vulnerability scanner for critical open-source projects.

Anthropic launched OSS Scanner on October 8, 2026—a free, opt-in AI vulnerability-scanning tool powered by strong models such as Claude Mythos, designed specifically for critical open-source projects. As of October 19, it had onboarded 114 projects and, within 11 days, reported 77 high-severity and 803 medium-severity vulnerabilities (according to model ratings), with 88% of sampled audit results meeting its disclosure standards.

Why it matters：This tool represents a large-scale implementation of LLMs in the open-source security domain, characterized by zero cost, full automation, and offline auditing; however, bypassing the human review stage introduces risks of false positives and duplicate issues. Its accuracy metrics—88% detection rate and 1% false positive rate—and real-world project feedback (e.g., wolfSSL, curl) provide an empirical benchmark for the current state of AI‑driven vulnerability discovery.

TRIAD Release: The First Multimodal Large Model Evaluation Benchmark for All-Modal Ambiguity Resolution

On October 24, 2026, researchers published the paper arXiv:2610.30841v1 on arXiv, officially introducing the TRIAD benchmark—a new evaluation framework designed to assess the ability of text–image–audio tri-modal systems to resolve cross-modal ambiguities. The benchmark comprises 327 hand-crafted tri-modal samples, spanning 86 scenario groups and 18 categories of cross-modal ambiguity types.

Why it matters：This achievement directly addresses a core blind spot in current multimodal large-model evaluations: existing benchmarks focus on perception or reasoning under deterministic, single-modal inputs. In contrast, TRIAD is the first to systematically target “modal-complementary disambiguation” as its evaluation objective. Through rigorously designed leave-one-modal-out counterfactual tests, it reveals critical shortcomings—particularly in audio processing—and highlights weak scene-level robustness, thereby providing a reproducible diagnostic protocol and taxonomy for assessing next-generation all‑modal models.

Research reveals multicultural value-selection bias in large language models: national and linguistic cues are not interchangeable

A study published on October 24, 2026, on the arXiv platform (ID 2610.31199v1) found that when large language models (LLMs) are presented with both a national identity (as set by system prompts) and a secondary cultural cue—either an explicitly named country or an implicitly used query language—the two types of cues exert markedly different effects on value judgments: linguistic cues trigger stronger but oppositely directed biases, whereas explicit country naming, though weaker in impact, leads to more nuanced and value shifts that align more closely with the target country’s values.

Why it matters：This study, based on 1.6 million open-ended generated responses and the World Values Survey (WVS) standardized tests, is the first to systematically distinguish the roles of explicit national labels and implicit linguistic cues in cultural modeling, highlighting that equating the two today can distort assessments of LLMs’ cultural adaptability. The findings carry methodological implications and were published prior to the current date (2026-10-26), making them valid new insights.

Research reveals the local geometric structure of semantic meaning in large language models

A paper published on October 24, 2026, on arXiv (ID 2610.30715v1) characterizes the local geometric properties of semantic meaning in embedding space across four decoder-only large language models—Gemma 4, Llama 3.2, Qwen 2.5, and Pythia—using cross-lingual probes and fine-grained probes.

Why it matters：This study is the first to systematically characterize the invariance, variability, and refinement mechanisms within “the same semantic factor” in LLMs, proposing the “local anisotropic neighborhood” model. The approach has been validated through blind human evaluations, demonstrating rigorous methodology, broad model coverage, and cross-architecture consistency in its findings.

Researchers have introduced the Japanese fact-checking benchmark JSFactCheckBench, revealing latent shortcomings in large language models’ fact-verification capabilities.

On October 24, 2026, arXiv published the paper arXiv:2610.30504v1, introducing a new Japanese benchmark called JSFactCheckBench. This benchmark is the first to explicitly define three key design elements in fact-checking evaluation—question relevance, judgment criteria, and evidence sources—and encompasses 1,913 statements derived from LLM responses to misleading information on Japanese social media platforms. It also features dual annotations for both factual accuracy and importance, based on pre-selected online sources at a reference point in time.

Why it matters：This study fills a gap in existing LLM fact-checking evaluation benchmarks, which have not explicitly specified assessment criteria. Through structured annotation, it reveals systematic weaknesses in mainstream models regarding direct-response statement verification, long‑ and multi‑source localization, and context sensitivity. Moreover, all findings rely on the benchmark’s transparent documentation of annotation conditions, endowing the work with methodological innovation and empirical value.

Aggregated scores in LLM evaluations mask the instability of individual judgments: New research from Arena-Hard reveals instruction sensitivity.

The study employed 500 Arena-Hard v2.0 prompts, with responses from two models held constant, and used GPT‑4.1 as the judge. Under eight instruction combinations, repeated evaluations revealed that even when aggregate scores varied only slightly (e.g., by ±1 point), up to 21.2% of individual prompt outcomes (win/draw/loss) changed; further adjustments to the instructions amplified this instability, leading to an additional 28.2 percentage points in outcome shifts, while the aggregate scores themselves remained virtually unchanged.

Why it matters：The study directly addresses a core blind spot in current LLM evaluation practices—overreliance on aggregate scores while neglecting the reproducibility of individual judgments and instruction dependency—providing critical methodological warnings for tasks that rely on fine-grained assessments, such as preference modeling and reinforcement learning from human feedback (RLHF).