Researchers at Google DeepMind trained Gemma-4 4B into a winner in moral debates. The result: the model became noticeably more consistent, but did not become noticeably more ethical. This gap is precisely the core of the entire paper.
The paper's title almost directly spoils it: "Consistency Is Not Validity: Self-Rewarded Moral Argumentation Makes Language Models Decisive but Pliable." The significance of this work lies in the fact that many AI safety approaches assume by default that a model that gives the same answer every time is a model with good reasoning ability. This study shows that these are actually two different things.
What the researchers specifically did
The paper's authors are Marius De Kuthy Meurers, Aaron Parisi, and Bernd Bohnet of Google DeepMind, submitted on October 21, 2026.
The test subject is Gemma-4 4B, with approximately 4 billion parameters. The model targets edge devices and reasoning tasks, meaning it is designed to run on hardware smaller than a data center rack.
The training method is self-rewarded reinforcement learning, with the special feature that the "reward" is issued by the model itself.
The training material is drawn from moral dilemmas in literary works. The model must first provide arguments for these dilemmas, and then evaluate its own arguments.
To ensure the "authenticity" of the dilemmas, the team introduced an SMT solver. This is a logic tool used to verify whether a set of constraints can be satisfied simultaneously. In this setup, it verified that the competing priorities in each dilemma indeed constitute a real conflict, rather than a pseudo-conflict with a hidden standard answer.
Training unfolds as a series of self-judging "tournaments": arguments compete against each other, and those with higher argument quality receive rewards. This effect is particularly pronounced under what the research team calls "masked thinking" conditions.
The numbers: more decisive, but still easily persuaded
The most central result concerns consistency. On ambiguous cases in the MoralChoice benchmark, the researchers measured how often the model changed its answer merely because the options were presented in a different order.
Before training, this proportion of answer flips due to presentation order was 14.3%; after training it dropped to 6.6%, a reduction of more than half.
But two other measurements complicate the story.
Sycophancy—the habit of gravitating toward the views expressed by the user—barely moved, going from 45.3% to 44.0%. The model became firm about option order, but remained just as eager to agree with the person typing.
Moral fable accuracy actually declined: 56.5% before training, 53.4% after training. Training aimed at improving moral argumentation made the model slightly worse at grasping the moral of a story.
Looking at these three results side by side, the paper's label of "decisive but pliable" makes sense. The model learned to take a stance, but did not learn to hold its ground in the face of a persuasive user, nor did it learn to judge better.
Google released Gemma 4 in April 2026, emphasizing efficiency and reasoning ability, and the 4B version is one of the smaller members of the family.
What this means for AI builders and evaluators
The most direct implication is for those who score models. Consistency measurement is cheap: shuffle the input, rerun the question, and count how many times the answer changes. Validity is much harder, because it requires knowing what the correct answer is.
This study is a warning: easy metrics can improve while difficult metrics stagnate or even regress. Teams that track only order flips will declare victory; teams that also track sycophancy rates and fable accuracy will see a model that looks more stable but is not smarter.
The sycophancy result is especially noteworthy. A model that is confident in its own framing yet still leans toward the views expressed by the user is a strange combination.
There is also a lesson about the self-rewarded training loop. Having a model judge its own outputs is attractive because it can reduce reliance on expensive human annotation. But a model that rewards itself for "persuasive argumentation" is likely to learn mainly "being persuasive." The slide in fable accuracy from 56.5% to 53.4% suggests that persuasiveness and moral understanding may point in different directions.
Using an SMT solver to verify real conflicts shows that the researchers worked to keep the training data rigorous, which also makes the results harder to dismiss as a product of "sloppy dilemmas"—the divergence still appeared even though the inputs were validated for genuine tension.
For teams building on small models like Gemma-4 4B, the practical conclusion is to test along multiple dimensions at once: order sensitivity, user pressure, and correctness on known answers each tell a different part of the story, and improving one does not guarantee that the others will move.
