A newly published academic study introduces the “Respondent Coverage Gap” (RCG) metric, designed to assess how well synthetic survey respondents generated by large language models (LLMs) through silicon sampling represent real human populations. This approach enables LLMs to “instantiate” responses to questionnaires based on specified demographic attributes, producing synthetic populations that are now increasingly replacing real respondents in social research.
Unlike previous evaluations that focused on single-question accuracy, answer distributions across questions, or inter‑question correlations, RCG represents each respondent as a point in a high-dimensional space defined by all possible answers. It then computes the distance between each real respondent and the nearest synthetic respondent, normalizing this distance relative to the typical intra‑respondent distance. This metric can reveal whether an LLM generates answer combinations that have never existed in reality—even if the individual question distributions match closely, it may still completely overlook certain real‑world demographic groups.
The study tested six LLMs on data from two U.S. social surveys; under the primary experimental setup, all models exhibited uncovered real respondents. Their RCG scores were not only higher than those of a baseline method that independently samples according to the human marginal distribution, but also exceeded those of statistical models fitted to human data using a Gaussian copula. This work was published on the arXiv preprint server under identifier 2610.30900v1, with a publication date of October 24, 2026, at 04:00 UTC.