Simulation with artificial crowds

This section will explore how large language models can generate artificial respondent data (‘artificial crowds’) to simulate human response for psychometric analyses, including IRT/CFA parameter estimation without real human samples. For now, this section is a curated reading list of foundational and recent work.

References

Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023). Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3), 337-351.

Chiappone, F., Marocco, D., & Milano, N. (2026). Large Language Models as Simulative Agents for Neurodivergent Adult Psychometric Profiles. arXiv preprint arXiv:2601.15319.

Kolluri, A., Wu, S., Park, J. S., & Bernstein, M. S. (2025, November). Finetuning llms for human behavior prediction in social science experiments. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 30084-30099). https://aclanthology.org/2025.emnlp-main.1530/

Lalor, J. P., Wu, H., & Yu, H. (2019, November). Learning latent parameters without human response patterns: Item response theory with artificial crowds. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 4249-4259).

Laverghetta Jr, A., Nighojkar, A., Mirzakhalov, J., & Licato, J. (2021, July). Predicting human psychometric properties using computational language models. In The Annual Meeting of the Psychometric Society (pp. 151-169). Cham: Springer International Publishing.

Maeda, H. (2025). Field-testing multiple-choice questions with AI examinees: English grammar items. Educational and Psychological Measurement, 85(2), 221-244. https://journals.sagepub.com/doi/full/10.1177/00131644241281053

Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023, October). Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology (pp. 1-22). https://dl.acm.org/doi/abs/10.1145/3586183.3606763

Park, J. S., Zou, C. Q., Shaw, A., Hill, B. M., Cai, C., Morris, M. R., ... & Bernstein, M. S. (2024). Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109.

https://arxiv.org/abs/2411.10109

Next page

LLMs and psychometric bias

Last page

Convex hull analyses

Return home

Psychometrics.ai

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).