The Illusion of Explanatory Depth
I personally feel like AI has given people superpowers. I have never been so productive and the people on my teams have never been so productive. We are even having more fun at work since the boring parts like writing Jira tickets are mostly done by AI now. We oftentimes just record conversations and let AI do the heavy lifting.
However, there is a new draw on my time that I did not see coming - arguing with people.
People are overestimating their own knowledge on a topic because they just asked AI. From software engineering to politics to climate change, the only thing everyone seems to have in common is that they are positive that they are right.
A tale as old as Search Engines
After looking into this, it turns out that it isn’t new - it is just worse. Fisher, Goddu & Keil published a paper in 2015 called Searching for Explanations that showed that people who had just searched the web rated their own knowledge higher, even on topics that were not related to the search. The researchers suggested that the participants mistook access to information for knowing information.
Well, that sounds bad. But it also sounds… familiar.
The one study that I think about or reference almost every day is the study behind the Dunning-Kruger Effect. The short story is that the people who know the least about a topic overestimate their competence the most.
The internet’s favorite version of the curve looks like this.
Every time someone comes up to me absolutely positive about something, my gut reaction is that they know very little about that thing. Whenever someone talks in probabilities and carefully walks me through why they believe something, but is open to being wrong, my gut reaction is that they are probably right.
After all, as Carlo Rovelli best put it, genius hesitates.
The study even agrees with Carlo! People who are on the far right of the curve, top performers, tend to underestimate their competence!
Dunning-Kruger on Steroids
So this was all true before LLMs took over our lives. Has there been any research into that? You know it!
A 2025 study in Computers in Human Behavior called AI makes you smarter but none the wiser asked 246 participants to solve 20 reasoning problems from the LSAT. The participants were separated into two groups - a group that could use AI and a group that could not. The AI group unsurprisingly outperformed the group that could not use AI, but that is not the interesting part. The interesting part is that everyone in the AI group overestimated how well they did. From the low performers to the top performers, they all thought they did even better than they actually did.
So what does that mean? Well remember, before AI the people that lacked knowledge in an area overestimated their competence while the people with the most knowledge in a field underestimated their competence. With AI everyone overestimates their competence.
Now we know why everyone is walking around acting like an expert!
What to do about it
Well, we need to find a way to get people from the peak of Mt. Stupid way down the hill and to start climbing up the Slope of Enlightenment to the Plateau of Sustainability. That is a pretty long and painful commute for some people depending on the topic.
Thankfully, there is another study to help us out!
The Illusion of Explanatory Depth
This time we have to go way back to a study from 2002 called The misunderstood limits of folk science by Leonid Rozenblit and Frank Keil. They asked participants to follow a specific set of steps.
- First, they had to rate how well they understood some everyday things like toilets and speedometers.
- Then they had to write a step-by-step explanation of how they function.
- They then rated their understanding again.
- Then they would answer a diagnostic question about the device.
- Finally, they rated their understanding one last time.
You can imagine what happened. Participants first rated themselves with very high understanding. Then after having to write down how it functioned, their rating dropped substantially. It then fell again after they tried to answer the diagnostic question.
This is our solution! In order to get someone to understand something on a deeper level, asking them to detail how it works is a research-proven path.
Explanations vs Reasons
How this is done matters, and we need one last study to see why. Fernbach, Rogers, Fox & Sloman published Political extremism is supported by an illusion of understanding in Psychological Science in 2013. In the study, groups were walked through the same rating game as the toilet study, but one group was asked to give a mechanistic description of a policy and the other was asked to give reasons they had their position.
The group that gave the mechanistic description ended up lowering their understanding ratings, while the group that gave reasons did not. This aligns with every argument we have ever had with people where the other party walks away further entrenched in their belief after giving reason after reason for their position.
The Practical Takeaway
So the practical advice is not to ask people why. It is to ask them how. “Walk me through how that would actually work step by step.” Not only will people feel heard, you are actually escorting them on their journey down the mountain and up to the Plateau of Sustainability where we can all live happily ever after.