OpenAI, the creators of ChatGPT, clearly heard the critics on its AI chatbot. Namely, those educators who have expressed concern about students using ChatGPT to do their homework or write their papers for them.
Tuesday, OpenAI released(Opens in a new window) a new AI classification tool aimed at directly addressing these concerns. The free web tool was created by OpenAI to help users determine if a block of text was written by a human or a computer.
The tool is simple to use. Just visit the AI classifier(Opens in a new window), sign in, then copy and paste the block of text you want to check. The classifier will then classify the text on a scale as likely to be generated by the AI, possibly, unclear, unlikely, or very unlikely.
Copy and paste text and find out if a human wrote it.
While this seems incredibly useful for addressing some concerns about AI-written text, OpenAI makes it very clear that there are caveats. For example, according to OpenAI, “the classifier is not completely reliable”.
“In our assessments on a ‘challenge set’ of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as ‘probably AI-written’, while incorrectly labeling text written by human as written by AI 9% of the time (false positives),” the company said in a blog post(Opens in a new window).
There are other limitations with the classifier discussed by OpenAI in its announcement. For example, the classifier performs considerably worse with text under 1,000 words. Sometimes it can falsely identify human-written content as computer-generated text. And it only works really well with English text.
OpenAI isn’t the first to release a tool to help detect when text is computer-generated. In fact, it’s not the first classification tool that the company itself has released. However, it is “significantly more reliable” than its previous tool, according to OpenAI. And it is very remarkable that such a tool is continuously worked on by one of the leaders in the AI space.
Yet, there is still a long way to go before teachers and educators can confidently rely on any AI detection tool to detect cheating in the classroom.
