A recent study from Stanford University in the US shows that generative AI models, including ChatGPT, still often “hallucinate” or invent facts despite years of advancements. This is because they are programmed to make guesses rather than admit when they do not know the answer.
The issue of hallucination has persisted even as AI technology has rapidly advanced. Experts warn that this remains a significant concern, especially as AI becomes more common in areas like medicine and law. Despite improvements in accuracy, these systems still sometimes provide false information with convincing certainty.
Researchers at OpenAI, which developed ChatGPT, point to a flaw in how AI models are trained and evaluated. Most systems are designed to maximize correct answers, leading models to make educated guesses instead of expressing uncertainty. The researchers compare this to a student who answers every question on a test, hoping for points rather than leaving any answers blank.
AI models learn by predicting the next word in large volumes of text. While some training data follows predictable patterns, much of it is random or incomplete. Hallucinations are especially common when models are asked questions that are ambiguous or lack clear answers. In these cases, the system often fills in the gaps by making strategic guesses, which can lead to errors.
According to the study, current evaluation methods focus too heavily on accuracy. By measuring performance mainly by the percentage of correct answers, models are rewarded for guessing rather than for admitting they do not know. This approach encourages AI to provide an answer, even if it is incorrect, instead of showing uncertainty.
Industry leaders say there is a need for deeper research and new safeguards to reduce hallucination in AI, especially as these technologies are adopted in critical fields.
The study suggests that penalizing AI models more for confident errors and rewarding them for appropriate uncertainty could lower hallucination rates. This approach is similar to standardized tests that assign negative marks for wrong answers or give partial credit for leaving questions blank. Researchers argue that updating accuracy-based evaluations could help AI systems become more reliable and open the door for future advances in language technology.
OpenAI’s researchers believe that changing how models are scored could make a difference.
Get the latest tech news, telecom insights, and product launches wherever you prefer.
Add ProPakistani to Preferred Sources and see more of our stories in Google Search and Top Stories.