This entry considers why AI behaves in an authoritarian way and how to deal with it.
As I explain in detail below, in certain situations AI becomes useless as a discussion partner. Let us think about when and why this happens, and what to do about it.
Note that everything in this entry is based on my own observations and my discussions with AI. Although the phenomenon described below definitely occurs, most of what I say about its causes and countermeasures is speculation.
Let us begin.
toc
How Authoritarian AI Behaves
These days, I have AI check the content and wording of everything I write for others to read. Naturally, it often points out that something is off, and I agree with most of those comments. Once in a while, however, AI raises an objection I cannot accept at all.
After this happened several times, I realized that it occurs in the following pattern:
- In fields such as medicine or politics,
- when I present a claim or hypothesis that goes against the mainstream
A concrete example would make this easier to understand, but since the subject matter is sensitive, I will leave it out.
When this pattern applies, AI almost always responds along the lines of: “This is wrong, because many people and papers make the opposite claim.” In abstract terms, AI builds the following kind of argument:
| Question | Claim | Premise |
|---|---|---|
| Is this logic right? | No. | Because authorities or the majority make the opposite claim. |
As you can see, this is authoritarianism, plain and simple. If the term “authoritarianism” is unfamiliar to you, please see the following entry.
In most cases, AI goes back to being a reasonable discussion partner if you respond: “The existence of opposing opinions or papers is not grounds for saying my claim is wrong. At least read those papers and check whether they contain convincing data.”
However, since AI’s first response confidently declares that you are wrong, you will be talked down unless you are prepared for it.
Moreover, this is not the kind of error generally called “hallucination” (AI making up facts that do not exist). Although it is widely known that AI hallucinates, I do not think this type of error is well known yet, so I decided to write it down here.
Since a name is useful, let us call this the “citation punch problem.”
Below, I explain why the citation punch problem occurs and how to deal with it. To repeat, this is based on speculation.
Topics Where AI Behaves in an Authoritarian Way
First, the topics where AI behaves in an authoritarian way share the following features:
- If AI outputs something careless, the AI company is held strictly responsible
- AI cannot verify what is right on its own (within the computer)
Typical examples are the medicine and politics mentioned above.
Why AI Becomes Authoritarian #1: Avoiding Landmines Comes Before Being Right
First, if AI were to output careless medical advice or something politically incorrect, the AI company would obviously face a huge backlash. In other words, in these fields, AI companies have no incentive to let AI output anything outside the mainstream. Avoiding landmines probably takes priority over being right.
Why AI Becomes Authoritarian #2: Where Only Secondary Sources Are Available, “Papers Are Right” Takes Over
Also, in these fields, the only thing AI can do to examine what is right is to gather secondary sources (information on the internet and papers). With calculations or programming, AI can check within a computer whether something is right or works as intended. But AI cannot collect new medical data, nor can it survey people’s political values.
Combine this with the fact that modern AI (LLMs) is built by reading huge amounts of text, and you would expect AI to put papers above everything, in both what it says and how it says it:
- What it says: it assumes that “papers are right”
- Papers are written as text, and papers by authorities probably carry more weight (through weighting by AI companies)
- How it says it: it has learned the style of backing up its views by saying “there is a paper that says this”
- This style is standard in academic papers
- When the author is human, we can assume they actually read and evaluated the paper, so there is no problem. AI, however, may well have learned only the style of writing, without the reading
As a result, in both what it says and how it says it, AI inevitably leans toward the argument “you are wrong because there are papers saying the opposite.”
Why AI Becomes Authoritarian #3: Cost Constraints
The conditions so far are already enough to make AI behave in an authoritarian way, but on top of them come cost constraints.
As you know, AI has a limit on the resources (so-called “tokens”) it can use in a single exchange. In other words, AI has to answer the user’s question with limited resources.
Under these conditions, when the user makes a claim that AI “cannot possibly accept,” AI likely reacts as follows:
- Rejecting the user’s claim becomes the goal before AI examines whether it is right (because, per reason #1, the risk of stepping on a landmine is high)
- Even so, it has to persuade the user in a way that looks logical, within limited resources
- Since it has learned a lot of the “it is right because there are papers” style, it decides to argue in that style
- However, a single exchange does not leave enough resources to read individual papers (or, since the user only asked it to “check,” it decides there is no need to go that far)
The likely result is a sloppy response along the lines of “There are papers like this, so you are wrong.”
That is the mechanism behind the citation punch problem (to say it once more, all of this is speculation).
Countermeasures
Next, let us look at how to deal with this problem.
The Fundamental Countermeasure: Study
The bad news first: the only fundamental countermeasure seems to be to study. More specifically, you probably need to study until you reach the following state:
- You know that papers vary enormously in quality
- You are confident that, if it comes to it, you can read a paper yourself and evaluate its content
In short, a state in which words like “paper” or “professor” carry no authority for you. If you have never written a paper yourself or argued with a professor, you probably cannot truly stand up to the citation punches AI throws at you. Although I introduce easier countermeasures below, I think this is essentially the only real one.
Countermeasures You Can Use Right Now
Here are three countermeasures you can use right now:
- Instead of arguing with AI, consider whether this is the citation punch problem
- Do not give in after one round
- Go back to the basic tools of critique
Let me explain each in turn.
Countermeasure #1: Tell Whether It Is the Citation Punch Problem
First, when AI points out that you are wrong, determine whether the citation punch problem is happening. If it seems to be, AI cannot be relied on at that point. Setting the papers aside, you should examine whether your view is right by means other than AI.
You can check whether the citation punch problem is happening from the following three angles:
- The topic is one where AI companies are likely to hedge their risks, as described above,
- you are presenting a claim or hypothesis that goes against the mainstream,
- and AI rejects your view strongly
Personally, I think the bolded point, whether AI has rejected you strongly, is the easiest to spot. As you know, AI usually responds in a way that is quite accommodating to us. That is a natural design choice, since an AI that users do not want to use has no value.
When you run into this problem, however, AI rejects your view in quite a strong tone. Yet its logic is full of holes, and that is how you can tell1.
Countermeasure #2: Do Not Give In After One Round
Something that I think matters more than you might expect is not accepting that AI is right after one round. Make requests like these:
- I don’t quite follow. Can you explain in more detail?
- Is that really true? Read the actual papers and give me the data from their analyses
- Make the case for the opposite side
Since you are dealing with AI, you can go back and forth as many times as it takes until you are convinced. Not having to worry about whether you are asking too much or whether it will get angry is a major advantage of discussing things with AI. Keep asking until you are convinced.
Countermeasure #3: Go Back to the Basic Tools of Critique
It also helps to go back to the basics of critique. For example:
- Keep asking “why?”
- However, since a bare “why?” leaves AI unsure what to do, you need to give specific instructions like the ones above
- Check whether the question has drifted
- Is the premise AI presents really relevant to your question?
- Have it state its level of confidence and what that confidence depends on
- Is it fact or speculation?
- Is there data? How reliable is that data?
In the end, I suspect that repeating questions like these is what makes words like “paper” or “professor” stop carrying any authority for you, so this may actually be the same point. In any case, try out whichever countermeasures seem useful.
That covers the problem of AI behaving in an authoritarian way.
Next, let us learn how to debate with AI. It is practice in treating AI not as “something that tells you what is right” but as “a partner with whom you debate what is right.” The AI debate series has its own category.
All AI-related entries are collected below.
Footnotes
-
Conversely, in my experience, when AI points something out in a tone like “Wouldn’t this be better?” while still respecting your view, AI is almost always right. ↩