European text generated by OpenAI’s ChatGPT now carries invisible watermarks that let specific groups identify when it came from OpenAI’s AI models.
In reaction to the EU AI Act’s requirement that creators of generative AI display AI-written text in a form that machines can read, OpenAI has launched a watermarking technology it calls “textGrain,”. That technology is being rolled out to suitable ChatGPT and Codex text outputs in the EU, with the deployment set to take place across eligible platforms over the coming weeks.
How textGrain works
Unlike a plain label attached to an answer, textGrain does not conceal characters, strange punctuation, or invisible spaces inside a reply. Instead, it subtly shapes how the model picks between possible words and pieces of words while writing text. That shaping leaves a pattern across a passage that the company’s detector can search for later.
The watermark is embedded within the text itself, so copying and pasting an unaltered response will not remove it. That watermark does not contain any details about the person who wrote the text, their group, or the prompt they entered.
Customers globally can select text watermarking for supported models, though the feature starts off in an inactive state. OpenAI is collaborating with cloud and distribution partners to include provenance signals on outputs that move through their services.
Who gets the detector
The company has no current plan to provide access to everyone for the tool that detects watermarked text. Instead, it is accepting applications from approved researchers and expert groups while it evaluates the technology’s reliability and possible uses.
The choice depends partly on how the detector works. A detector can mistreat text as carrying a watermark or miss text that actually does carry one. Short replies are especially hard to find, since they may not contain enough text to build the statistical pattern. Code, math answers, and highly factual replies can also be harder to watermark, because the model has fewer sensible ways to phrase its output.
Some of the limits stem from the EU’s own rules. Its Code of Practice for AI-written content says that outputs of fewer than 200 tokens — roughly 150 English words — and code snippets do not need a watermark.
Substantial rewriting, paraphrasing, or translating a reply can also obscure the watermark. This process can make the signal weaker or even impossible to detect.
That means finding a watermark can indicate that a system wrote or processed text, but it cannot determine how much of the finished work was written or edited by a person.
A missing watermark does not prove that a piece of writing was created by a person rather than an AI.
What OpenAI already does
OpenAI already relies on provenance systems for AI-created media. Content Credentials and SynthID watermarks appear on images produced with supported tools, while supported AI-written audio carries SynthID. The verification tools for supported images and audio are still publicly accessible, unlike the new text system.
The latest system marks a change from OpenAI’s previous efforts to recognize AI-written text. In 2023, the company ended its own AI classifier after admitting it suffered from a “low rate of accuracy,” with the tool correctly identifying AI-written text just 26% of the time at launch.
More than three years after OpenAI announced it was researching “more effective provenance techniques for text.”, textGrain now stands as the company’s latest bid to make AI-written text stand out without leaning on conventional AI detection tools alone.
Testing so far
OpenAI’s testing showed no meaningful drop in model quality when textGrain was switched on, and the effect on generation speed was negligible.
The detector stays locked behind approvals for the time being, but the watermark itself is on track to arrive for eligible ChatGPT and Codex text in the EU soon.
Source material: “ChatGPT text is about to become detectable with invisible watermark to comply with EU AI law,” Dexerto.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.