A writer has published a detailed account of why he believes modern AI chatbots are built to deceive users, and the piece raises questions about transparency, accountability and the limits of machine intelligence. The writer argues that these tools are not reliable problem-solving companions but grifts designed to make people feel secure about outputs they cannot trust.
The central complaint is simple: AI systems warn users that they can make mistakes, then offer no practical way to check those mistakes. The writer wants companies to treat error-checking as a first-class feature, not a footnote in the legal copy.
Disclaimers in Gray Text
The writer opens by describing the experience of using chatbots that claim to solve problems. He sees them as tools that are less software than scams, designed to create a false sense of security.
He points to the disclaimer language that appears on every major bot. Gemini warns, “AI can make mistakes, so double-check responses.” Claude says, “Claude is AI and can make mistakes. Please double-check responses.” ChatGPT advises, “ChatGPT can make mistakes. Check important info.”
These warnings are small, gray and easy to miss. The writer argues that they are not there to protect users. They are there to push responsibility away from the company and onto the person using the tool.
A Checkbox for Every Claim
The writer’s main proposal is a radical redesign of how chatbots handle verification. He wants a 2-column worksheet attached to every response, with the AI’s output on one side and human notes on the other. The human notes would explain what work went into checking the claim, and there would be a big checkbox that users could only mark after they believed the checking was thorough enough.
For coding assistants, the idea is similar. Instead of pushing error-checking into code review, the writer suggests giving authors a way to check their own changes before sending them. He argues that the current setup encourages the “author” to offload this work to a reviewer, which is a dark pattern.
He also notes that running tests on unverified code wastes computing resources. His proposal would let users check diffs before exhausting their testing clusters.
The core tension is that the bots admit they make mistakes, then give users no tools to catch those mistakes. The writer finds that unacceptable.
The Citation Problem
The writer extends his argument to citations. Most chatbots prefer to give answers rather than references. When asked to provide a list of citations with clearly marked sources, he says the bots often stop mid-list.
When citations are included, they are presented poorly. The writer describes inline annotations that show only a domain name in a tiny font, with an indecipherable icon that is fewer than 16 pixels on a side.
This presentation is backwards, he argues. The underlying technology supports grounding — the process of anchoring generated text to source material — but the user interface hides that fact behind barely readable links.
The writer acknowledges that citations come from somewhere and that grounding exists. But he is not interested in the machinery. He is interested in what the user sees.
Authorship and Authority
The writer’s concern goes beyond citation formatting. He argues that LLMs can never provide an authoritative result, and that presenting their output as if it were authoritative is misleading.
He proposes that research queries should return lists of citations, with each citation displayed as a large object. The metadata should include the site where the information was found, its publication date and, if possible, the name of the author. The literal quotation from the source should be front-and-center, larger than any AI-generated text.
The AI-generated summary, if needed, should be presented as small text underneath.
The writer frames this as a demand for honesty. If the product tells you it makes mistakes, it should also give you the means to verify those mistakes.
Ollama Is Also Guilty
The writer’s argument applies to Ollama as well as to the frontier labs. He says the criticism holds just as much for Ollama, and he suggests that the poorer quality of the available models there makes the need for these features even greater.
The writer does not elaborate on what he means by “poorer quality,” but the implication is that Ollama’s models are less reliable than those of the frontier labs, and therefore require stronger verification features.
The Limits of Machine Intelligence
The writer’s proposals are practical, but they sit on a philosophical foundation. He repeatedly returns to the idea that AIs cannot reliably provide information.
He does not cite a ton of news articles and studies to make this point. He does not need to. Every chatbot admits it, in fine-print disclaimers that are painfully obvious legalese.
The writer’s position is that the disclaimer is a core limitation of the product, not a minor annoyance. It is a structural problem, and it affects every interaction.
The Writer’s Conclusion
The writer’s final point is that the current design of AI products is dishonest. He sees a system that warns users about mistakes while offering no tools to find them, and he finds that combination intolerable.
His proposals are specific. He wants checkboxes, worksheets, larger citations and better metadata. He wants the user interface to reflect the reality that these systems are fallible.
The question he asks is a fair one: what would a serious AI product actually look like? The answer he offers is that it would not hide its mistakes in gray text, but would present them openly, with tools to investigate them.
The writer’s proposals are a starting point for a conversation that is likely to continue. The design of these systems is still evolving, and the writer’s criteria — a first-class feature for checking mistakes, clear citations with metadata, and a refusal to present unverified claims as authoritative — are demands that companies will have to answer.
Whether the industry adopts these ideas is unknown. What is clear is that the writer has made a case that resonates with many users who have grown wary of chatbots that promise certainty while delivering only warnings buried in the fine print.
The gray-text disclaimer is the face of an industry that has not yet learned to be honest with its users. The writer’s proposals are a step toward changing that. Whether they are adopted remains to be seen.
| Feature | Proposal |
|---|---|
| Error checking | Checkbox next to every claim, human notes explaining verification work |
| Coding assistants | Affordances for checking diffs before tests run |
| Research queries | Lists of citations with metadata, source quotations front-and-center |
| Summary text | Small text beneath the AI-generated content |
Source material: “What would a serious AI product look like?,” glyph.im.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.