Intuitive Prompting Improves LLM Agent Social Media Reaction Simulation Fidelity

What happened

To improve user simulation fidelity for LLM agents, a research team first built user profiles for 8 Serbian participants via surveys, in-depth interviews, and written self-introductions, while capturing the participants' real reactions to 68 social media posts. The team then had 4 large language models predict these reactions across 5 distinct prompting conditions. Validation found that the prompt framework requiring models to give immediate, intuitive responses performed best: it reduced agents' compression of individual user differences from 7x the human benchmark level to 3x, and outperformed the human group baseline by a wide margin when predicting reactions to unfamiliar topics.

Key facts

Study sample

8 Serbian participants

Test setup scope

68 social media test posts, 4 large language models, 5 prompting conditions

Best-performing prompt outcome

Reduced agent compression of individual user differences from 7x the human benchmark level to 3x

Unfamiliar topic performance

Prediction performance significantly outperformed the human group baseline

Test use case

Simulating individual user reactions to social media content

Background

Platforms increasingly rely on virtual users to test policy changes, but prior agent validation efforts mostly focused on matching aggregate human behavior, rarely checking if agents align with the specific user profile they are meant to simulate. High-fidelity virtual users also carry the risk of being misused to manipulate public opinion during election periods.

Why it matters

For LLM developers, this finding debunks the assumption that complex reasoning prompts always deliver better performance, opening new directions for task-specific prompt design. For the industry, high-fidelity general-purpose simulated users can drastically cut the cost of platform policy testing, but stakeholders must guard against the risk of this technology being misused to manipulate public opinion. For end users, it will become harder to tell if an account they interact with online is a simulated agent.

What to watch

Follow-up research can track the method's performance stability across larger population groups and multilingual settings, as well as the development of accompanying risk mitigation frameworks for simulated user technology.