OpenAI, the company behind the popular AI chatbot ChatGPT, has been working on a tool that could potentially catch students who cheat by asking ChatGPT to write their assignments. However, according to a recent report in The Wall Street Journal, OpenAI is debating whether to actually release this tool.
The Tool: Text Watermarking
The tool in question uses a method called text watermarking, which involves making small changes to how ChatGPT selects words when generating text. This creates an invisible "watermark" that can later be detected by a separate tool. The idea is that if someone tries to pass off ChatGPT-generated text as their own, the watermark will reveal its true origin.
Complexities and Concerns
However, OpenAI has expressed concerns about releasing this tool due to its potential impact on the broader ecosystem beyond OpenAI. According to a statement provided to TechCrunch, an OpenAI spokesperson said:
"The text watermarking method we’re developing is technically promising, but has important risks we’re weighing while we research alternatives, including susceptibility to circumvention by bad actors and the potential to disproportionately impact groups like non-English speakers."
A Deliberate Approach
OpenAI’s statement suggests that the company is taking a deliberate approach to this issue. They are aware of the potential benefits of releasing the tool but also acknowledge the complexities involved.
Why Text Watermarking?
Text watermarking is different from previous efforts to detect AI-generated text, which have been largely ineffective. Even OpenAI itself shut down its previous AI text detector last year due to its "low rate of accuracy."
The focus on detecting ChatGPT-generated text specifically could be an attempt to address the issue of academic cheating, where students use AI tools to generate assignments and then pass them off as their own.
Accurate But Not Foolproof
According to OpenAI’s update to a May blog post about its research around detecting AI-generated content, text watermarking has proven "highly accurate and even effective against localized tampering, such as paraphrasing." However, it has also shown to be "less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character."
This means that while text watermarking is accurate in many cases, it can still be circumvented by bad actors.
Stigmatizing Non-English Speakers
OpenAI’s update also highlights a potential issue with text watermarking: its impact on non-English speakers. The company writes that this method could "stigmatize use of AI as a useful writing tool for non-native English speakers."
This concern suggests that the use of AI tools like ChatGPT can be a helpful resource for students who struggle with language, and that text watermarking could inadvertently create barriers for these students.
Conclusion
The debate over releasing OpenAI’s tool to detect cheating with ChatGPT highlights the complexities involved in addressing academic integrity. While the potential benefits of such a tool are clear, the company must also consider its potential impact on the broader ecosystem and the communities it may affect.
As OpenAI continues to research this issue, one thing is certain: the use of AI tools like ChatGPT will only continue to grow, and with it, the need for effective solutions to address academic integrity.
Related Articles
- Inside the Wild Fall and Last-Minute Revival of Bench, the VC-Backed Accounting Startup That Imploded Over the Holidays: Charles Rollet
- Generative AI Funding Reached New Heights in 2024: Kyle Wiggers
