OpenAI, the creator of the AI program ChatGPT, has unveiled a tool to determine if a text was written by AI. The research lab warns, however, that the tool is not foolproof.
In a recent blog post, OpenAI introduced the classifier tool which can distinguish text written by humans from text written by multiple AI systems, not just ChatGPT.
According to OpenAI researchers, it is challenging to accurately identify all text written by AI, but effective classifiers can recognize certain signs. The tool may be useful in detecting AI-based academic cheating and instances where AI chatbots pose as humans.
However, they acknowledged that the classifier has limitations, as it correctly identified only 26% of English texts written by AI and wrongly labeled 9% of human-written texts as AI-generated.
The researchers said:
Our classifier’s reliability typically improves as the length of the input text increases. Compared to our previously released classifier, this new classifier is significantly more reliable on text from more recent AI systems.
OpenAI acknowledged the limitations of the classifier tool, including its ineffectiveness on texts shorter than 1,000 characters, and its tendency to mistake human-written text as AI-generated.
The researchers stressed that the tool should only be utilized for English, as it performs poorly in other languages and is unreliable when checking code.
It should not be used as a primary decision-making tool, but instead as a complement to other methods of determining the source of a piece of text.Â
OpenAI has requested that educational institutions share their experiences with the integration of ChatGPT in their classrooms. Although a majority of institutions have responded to AI with bans, some have adopted it.
