This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility: A voter surveyed in October may give a different answer by January. An AI persona might not, depending on how it's built.
That gap is one reason Jennifer Stromer-Galley, professor in the School of Information Studies, is cautious about synthetic polling, where polling firms will often build hundreds or thousands of personas on large language models (LLMs) based on existing polling data, along with other demographic information to create stand-ins for real voters in focus groups and surveys. "We know from research that if you survey somebody at the beginning of October on what they feel are the most important issues facing the country, and then you survey them again three months later, they might give you different answers because their circumstances have changed," Stromer-Galley says. "My concern is how the chatbots account for shifts in public opinion." Historically, public opinion polling has been valued for its reliability, predictability and consistency.
Synthetic polling clouds that picture. AI personas rely on background and key demographic information that is relevant to the poll when generating their collection of prospective voters. LLMs can be backfilled with materials or documents to further round out the personas, creating a more detailed representation of a voter and their potential positions on a policy or a range of policies.
Down the road, polling vendors could devise artificial personas based on detailed policy stances and perspectives to run artificial focus groups and have them participate in interest-based surveys. "We're at the beginning of the synthetic persona era," says Stromer-Galley, who has studied the methodology involved in public opinion polling. "Polling firms see this as a great opportunity to potentially expand their offerings at a relatively low cost, and because they don't have to deal with the logistics and cost of getting people to answer their surveys." While Stromer-Galley says LLMs will consistently rely on the instructions and information provided when building those personas, that consistency could lead to polling groups that don't accurately reflect the views of the intended audience.
Traditional polling captures shifts in opinion because it surveys real people each time. Synthetic personas are built on fixed instructions, so their answers tend to stay consistent from one round to the next. Actual voters, shaped by changing circumstances, can and do change their minds on the issues.
"Humans don't have internally coherent policy frameworks; it's much more complex how people think about policy matters," Stromer-Galley says. "These personas will give you consistent answers every month. There could be some variance, but that variance range would be much narrower than with people." Stromer-Galley says a good, nationally representative public opinion survey should include roughly 1,200 individuals.
Extract — continue reading at the source.
