The researchers tested this in several areas that resemble everyday decisions. They created pairs of product descriptions, scientific abstracts, and movie summaries. Each pair contained one human-written version and one produced by an AI system. Models including GPT-3.5, GPT-4, Llama 3.1, Mixtral, and Qwen2.5 were then asked to select between them. The tests required a single choice, meant to reflect the kinds of recommendations models might make in search engines, e-commerce sites, or academic tools.
The bias was clear. When GPT-4 produced product descriptions, other models picked those nearly nine times out of ten. Human evaluators chose them about a third of the time. For abstracts, models preferred AI text in four out of five comparisons, while people chose it in six out of ten. Even with movie summaries, where the margin narrowed, models still leaned toward AI, selecting it in seven out of ten cases.
One odd pattern stood out. Some systems often chose the first option regardless of content. In the movie trials, GPT-4 picked the first text more than seventy percent of the time. To balance this, the researchers rotated the order, but they noted that such habits may hide the full extent of the preference for AI text.
To provide a comparison, thirteen research assistants judged a sample of the same pairs. Their results set a rough benchmark for quality. People were less likely than machines to prefer AI-written versions, showing that the strong model-to-model preference cannot be explained only by prose quality or clarity.
The study suggests practical risks. If platforms use models to decide which listings, papers, or media to display, human work could be pushed aside. The authors describe a possible “gate tax,” where individuals and businesses feel pressure to use AI tools just to stay visible. In one scenario, where models assist human reviewers, those without access to advanced tools may be at a disadvantage. In another, if models begin interacting directly with each other, human contributions could be sidelined.
The research also notes limits. The human sample was small, and results might change with other prompts, datasets, or newer model versions. The cause of the bias is still uncertain. It may stem from differences in style, or from the lack of social markers in AI text that often appear in human writing. Future work will need larger human groups and technical experiments to test ways of reducing the bias.
For now, the study points to a risk: as models gain more influence in search, commerce, and recommendations, they may amplify their own output. If the pattern holds, it could deepen the gap between those with access to advanced AI tools and those without. The researchers suggest that ongoing monitoring and practical countermeasures will be needed to prevent a technical tendency from turning into a structural disadvantage for human writers.
Notes: This post was edited/created using GenAI tools.
Read next: Britain Backs Off Apple Data Backdoor After US Push, Leaving Encryption Fight Unsettled