Research reveals that open-source large models commonly exhibit list-filling phenomena in structured extraction.
An empirical study published on October 24, 2026, on arXiv (paper ID 2610.30997v1) found that when a JSON extraction schema specifies a minimum number of items in the list but the actual document content falls short, eight open-source weighting models generally treat this minimum value as a generation target rather than a constraint, resulting in 55.4% of output lists exceeding the true count (compared to a baseline of 34.5%). The filler items are often drawn from the original text, and about half pass the source‑consistency check; common mitigation strategies prove only marginally effective.
Why it matters：This study employed a preregistered protocol to systematically evaluate eight open-source models and six hosted-service models across 558 real-world documents and nine annotated datasets. With a rigorous methodology and publicly available data—both the code and results have been released on GitHub—it reveals a critical yet often overlooked deficiency in structured information extraction that undermines the reliability of downstream tasks: schema‑induced non‑faithful filling behavior.