Surveying with AI: Simulating Human Responses Using Personalized LLM Agents and Social Media Data

Survey research is a vital tool for understanding human opinions, behaviors, and preferences, but traditional methods face challenges such as high operational costs, limited sample diversity, and privacy concerns. Large Language Model (LLM)-based agents offer a novel approach to simulate human responses, generating high-quality synthetic data that preserves privacy while enabling researchers to explore patterns in public opinions. Our study introduces a flexible LLM-based survey simulation platform, which personalizes responses using real-world social media data to enhance realism and relevance. We conducted a validation case study using data from Steemit users, comparing the personalized simulated responses generated by LLM agents with actual survey responses collected from these users. This comparative analysis revealed key differences and potential biases, providing insights for refining survey methodologies. Personalized LLM simulations were highly effective for factual questions but faced limitations with subjective topics, often demonstrating lower variance compared to human responses. The quality of social media data and careful model selection also significantly influenced the accuracy of simulations. Our platform enables social science researchers to explore new methodologies for survey design, pre-test the impact of framing, and interact dynamically with LLM agents at reduced cost. This work provides valuable insights into the use of LLM-based agents for enhancing survey research, supporting their application in social science. © 2025 IEEE.