A new study into search referrals has found that artificial intelligence assistants direct users to error pages at a much higher rate than Google. The results suggest a reliability problem for AI-driven tools, though the practical effect on website traffic remains modest for now.
Large-Scale Testing Across Platforms
Researchers at Ahrefs analyzed[1] more than 16 million unique web addresses linked by popular AI assistants including ChatGPT, Claude, Copilot, Perplexity, Gemini, and Mistral. Each link was checked for its status code to determine how often users encountered “404 not found” pages.
The study showed that AI assistants collectively referred visitors to broken pages almost three times as often as Google. While Google’s baseline 404 rate stood at 0.15 percent, the average across AI tools reached 0.43 percent.
ChatGPT Stands Out in Error Rates
Among the platforms tested, ChatGPT produced the highest share of faulty links. Roughly 1 percent of its clicked referrals led to error pages, and when counting all cited links, that figure climbed to 2.38 percent.
Claude ranked second with 0.58 percent for clicked links. Copilot and Perplexity each fell in the 0.3 percent range, while Gemini registered 0.21 percent. Mistral produced the lowest error rate at just 0.12 percent, but its overall traffic volume was much smaller, limiting the reliability of that result.
Broader Dataset Confirms the Pattern
The researchers ran a second test using their Brand Radar database, which captures URLs cited by AI tools regardless of whether users clicked on them. This provided a broader view of link quality.
Here again, ChatGPT recorded the highest proportion of broken addresses at 2.38 percent. Perplexity and Gemini reported rates close to Google’s own figure of 0.84 percent, which may reflect their dependence on Google’s search index. Copilot, which relies more heavily on Bing’s index, came in lower at 0.54 percent.
Why AI Systems Generate False Links
Two main reasons explain the high error rates. Some URLs once existed but were later removed or relocated without redirection. Others are fabrications that match the typical format of a site but have never been published.
For instance, on the Ahrefs blog, AI systems created fictional links such as “/blog/internal-links/” and “/blog/newsletter/.” These seemed plausible given the site’s focus on search optimization, but no such pages were ever created.
Because AI-generated content already appears on an estimated 74 percent of new web pages, false addresses can circulate further when crawlers attempt to index them. This increases the chance that fabricated links appear to have legitimacy, even when they lead nowhere.
Limitations of the Research
The study acknowledged that not every error represents an AI hallucination. Some 404 pages may stem from normal site maintenance or technical issues unrelated to AI. In addition, the analysis likely undercounted hallucinations, since only clicked links or those captured in the crawler database could be checked. Links invented by AI but never clicked, or never indexed, would not appear in the data.
Impact on Web Traffic
Despite the elevated error rates, the overall traffic effect remains limited. AI referrals represent just a quarter of one percent of typical website visits, compared with nearly 40 percent from Google. Even though AI sends users to broken pages more often, the absolute number of affected visits is small.
Still, the issue raises questions about the dependability of AI as a discovery tool. If usage grows, the volume of broken referrals may rise with it.
Industry Predictions and Response Options
Earlier this year, Google’s John Mueller predicted[2] that hallucinated links would become more common across AI systems. He suggested that site owners should focus on creating more helpful 404 pages instead of chasing small amounts of accidental traffic.
Practical responses include filtering analytics to isolate AI-driven visits, identifying which broken links attract meaningful traffic, and applying redirects where appropriate. For cases where redirects are less suitable, updating 404 pages with guides, resource links, or newsletter options can help capture value from misdirected visits.
A Weak Spot for AI Search
The evidence suggests that fabricated links are not reshaping traffic patterns today, but they highlight a clear weakness in how AI assistants handle web addresses. Addressing this problem early could help AI search tools build trust, while giving site owners more control over how visitors experience their content.
Read next:
• Study Finds AI Search Tools Struggle With Reliability and Accuracy[3]
• Most Americans Now Rely on AI in Search and Shopping, Survey Finds[4]
• AI Is Disrupting Hiring, And Trust Is the First Casualty[5]
References
- ^ analyzed (ahrefs.com)
- ^ Google’s John Mueller predicted (bsky.app)
- ^ Study Finds AI Search Tools Struggle With Reliability and Accuracy (www.digitalinformationworld.com)
- ^ Most Americans Now Rely on AI in Search and Shopping, Survey Finds (www.digitalinformationworld.com)
- ^ AI Is Disrupting Hiring, And Trust Is the First Casualty (www.digitalinformationworld.com)