AI search engines have gained popularity lately as quick and easy alternatives to Google that bring up relevant and up-to-date information on the internet without having to click through several links. But they seem to have a glaring problem with news content.
The Tow Center for Digital Journalism put AI search engines to the test to verify their ability to retrieve and cite news content accurately. Eight different generative search tools were tested, and it was found that these tools often fabricate source links or cite syndicated and copied versions of articles, among other problems.
The study remains consistent with Tow Center’s previous findings that the news citation issue is not just a ChatGPT problem but a general problem among generative search tools.
How it Was Done
Researchers conducted a controlled experiment to assess chatbot accuracy, focusing on ten articles from various news sources. From each article, specific passages were manually extracted and used as prompts. Each AI was then tasked with identifying the article’s headline, publisher, date, and web address.
To ensure source accessibility, the chosen excerpts were confirmed to produce the original article within the top three Google search results. In total, 1,600 queries were processed, representing the interaction of twenty publishers, ten articles, and eight different chatbots.
Troubling Findings
The findings revealed a troubling trend: Chatbots frequently provided incorrect answers with a high degree of confidence. Notably, premium chatbots displayed an even greater level of misplaced confidence. Furthermore, the study documented instances where platforms accessed content from publishers who had explicitly blocked their web crawlers. Consistent failures to provide links to original sources were observed, and existing licensing agreements did not guarantee improved accuracy in source citations.
Conclusion
In conclusion, the landscape of AI tools presents distinct dangers for those who create and those who consume news. While some AI developers have engaged with news publishers, many have not shown similar inclinations. Even among those who have, persistent problems arise with inaccurate source attribution and disregard for established web crawling guidelines, specifically the Robot Exclusion Protocol.
Consequently, news organizations find themselves with few effective means to manage the use of their content by AI-powered chatbots. The limited control publishers possess over how their material is presented appears to yield minimal results.
Stay Connected with ProPakistani
Get the latest tech news, telecom insights, and product launches wherever you prefer.
Add ProPakistani to Preferred Sources and see more of our stories in Google Search and Top Stories.
