"I asked ChatGPT what it really thinks" is not a finding, and I truly hate seeing it presented as fact.
I wanted to rant on this topic a bit, because "I asked ChatGPT" and "I asked AI" are being thrown around like the opinion being presented is as factual as it gets. This way of presenting opinions has mainly been hijacked by "AI gurus" and "course sellers" on TikTok and Instagram, but it is showing up on LinkedIn as well.
In my opinion, more people need to understand the mechanism behind why it seems like the AI has an opinion. I will not go into technical details like how LLMs work behind the scenes. To get the point across, the carousel below has one question that seems profound and deep: "What is the biggest lie in human history?"
Let's start with the question itself, framed as deep forbidden philosophical knowledge. First of all, "biggest" has no metric, no set of answers and no way to be proven or disproven. If you were to ask a person this, there would be a whole discussion about the question itself, a ton of pushback on the vagueness. The model is built to generate the next word and give you an explicit answer.
Secondly, the model is not looking anything up. It is producing the most likely continuation of the text you gave it. So the phrasing of your question decides which part of its training data it sounds like. Ask something framed as forbidden philosophical wisdom and you land in the part of the internet that talks that way: self-help books, TED talk cadence, the philosophical corners of Reddit, shower thoughts. You are not getting a quote from any of those. You are getting the average of all of them.
Thirdly, when those models generate an answer for you they are not fetching it, they are, as the word suggests, generating it. That has randomness built into how each word is picked, so asking the same question twice can produce different results, especially for an open-ended question like this one. Open a fresh incognito chat, no history, no memory of you, and ask it five times. You get five different biggest lies in human history.
Fourth, and what amplifies all of it: when a machine, an Artificial Intelligence, gives the answer to that question, it reads as a confession, like something you are not supposed to hear, kind of like a conspiracy theory. It also helps that the answer is usually broad enough that anyone can see their own life in it, the same way a horoscope works.
In the carousel below you can see how separate incognito chats give different answers. They are variations of the same thing, and whoever posts it only shows you the one they got, or even worse, they pick from multiple chats the one that proves their point and run with it as a fact. The format itself is what convinces most people who look at it on a surface level: one hard question with one clean, CONFIDENT answer, that comes from a machine and looks like a finding.