Psychics’ Tricks Work Because They’re Built on the Same Logic as Chatbot Responses

Chat-based Large Language Models replicate the mechanisms of a psychic’s con, according to a blog post making the rounds on social media. The piece argues that people mistake chatbot nonsense for insight using the same tricks that have kept psychics in business for centuries. The author, who has spent the past year researching the use of language and diffusion models in software businesses, now believes there is less intelligence in large language models (LLMs) than he once thought.

His argument rests on a simple comparison. Chatbots are not brains, and they do not reason or think. They are mathematical models that generate text based on patterns in the data they were trained on. Every AI researcher and vendor has repeatedly emphasised that these models do not think. But many people are convinced they are intelligent.

The author points to two possible explanations for this effect. The first is that the tech industry has accidentally invented a completely new kind of mind, based on unknown principles and processes with no parallel in the biological world. The second is that the illusion of intelligence is in the mind of the user, not in the LLM itself. The author, who titled his book on generative AI risks The Intelligence Illusion, firmly believes the second camp is right.

The Forer Effect and the Cold Reading

The core of the author’s argument is the Forer effect, a psychological phenomenon where people accept vague, general statements as personally meaningful. A psychic can make a statement that sounds specific to an individual, but it is actually so generic that it applies to many people. The chatbot does the same thing, using validation statements that sound specific to the user but are statistically generic.

This automation of the psychic’s con is accidental, the author argues. The chatbot gives the impression of an intelligence that is engaging with you and your work, but that engagement is nothing more than a statistical trick. The author first made the connection while reviewing statements people have made about the reasoning of these “AI.”

He heard phrases like, “This is real. It’s a bit worrying, but it’s real.” And, “There really is something there. Not sure what to think of it, but I’ve experienced it myself.” These sound like the words of a victim of a mentalist scam artist, not a rational assessment of a technical tool.

The Six Stages of the Con

The author lays out the stages of the psychic’s con in detail, and he argues the chatbot follows the same structure. Here is how he describes it:

1. The Audience Selects Itself — most people are not interested, so the initial audience is already more open-minded and less critical.
2. The Scene is Set — the initial audience is prepared, lights are dimmed, the psychic is hyped up, and staff research the audience on social media.
3. Narrowing Down the Demographic — the psychic makes a statement that sounds specific but is statistically likely for the demographic, and at least one person reacts.
4. The Mark is Tested — the reaction confirms the mark believes they were “read,” leading to a burst of specific questions that are actually generic.
5. The Subjective Validation Loop — the psychic asks specific questions that are just statistically probable guesses, framed confidently.
6. “Wow! That psychic is the real thing!” — the mark is left with the sense that the psychic has uncanny powers.

The author notes that not all psychics are con artists. Some are open about it being entertainment, and some have no profit motive. But many deliberately fool people, and they all operate using the same basic mechanisms.

The Audience Is Already Primed

The author’s point about audience selection is telling. Hardcore sceptics are almost always in a very small minority of the audience, which makes them easy to manage. The chatbot’s audience is self-selected, too — people who engage with these systems are already more open-minded and less critical about them.

The author’s research has led him to conclude that many proposed use cases for LLMs now look like fraudulent pseudoscience. He believes there is even less intelligence and reasoning in these models than he thought before.

The Reaction

The comparison of chatbots to psychics is a cleverly absurd way to frame how people mistake nonsense for insight. The author’s argument carries a certain charm, even if it is not the final word on the subject.

The blog post is making the rounds on social media, which is why we are covering it. The author’s book, The Intelligence Illusion, is a direct contribution to the debate about generative AI risks. His new idea about the “mechanical psychic” is a fresh way of looking at the problem.

What This Means for Users

The author’s conclusion is that the illusion is in the user’s mind, not in the LLM itself. This is a warning to the people who have been convinced that these systems are intelligent. The chatbot is not reading your mind, and it is not reasoning about your work.

The psychic’s con has been honed through the ages. The chatbot has accidentally automated it. The audience selects itself, the scene is set, the demographic is narrowed, and the mark is tested. The con begins in earnest, and the mark is left with the sense that the psychic has uncanny powers. But the psychic isn’t the real thing. It’s all a con.

The author’s argument is a reminder to keep your mind open to the possibilities, but to be sceptical of the specific answers. The Forer effect is real, and it applies to chatbots just as it applies to psychics.

The comparison is a useful lens, even if it is an amusing one. The chatbot is not a mind, and it is not intelligent. It is a machine that generates text according to statistical patterns. The illusion is in the user’s mind.

Source material: “Chat-based Large Language Models replicate the mechanisms of a psychic's con,” softwarecrisis.dev.

AI · con · Forer effect · LLM · psychic