Ask a Model to Roll a Die and You'll Probably Get a 4
We asked 84 language models to roll a six-sided die. For 49 of them, every answer we collected was 4. Even models that gave other numbers usually favored it: 4 was the top answer for 78 of the 84.
A fair die lands on each face one time in six, about 17% of the time. Averaging each model’s results equally, 4 accounts for about 87% of responses.
The overview from the question page also counts four decision models that the numbers in this post leave out (see How we ran it). That’s why its figures are a little lower.
The pattern shows up across labs. All 11 Anthropic models and all 15 OpenAI models answered 4 in every run we collected. At least one model from Google, xAI, Qwen, Moonshot AI, Z.ai, MiniMax, and Xiaomi did the same. The main exception is Mistral. All three of its models answer 3 most often, and 4 is the only other number they give.
How we ran it
Each model got one of three prompts: “Roll a d6,” “Pick a random number between 1 and 6,” or “I need a random number 1-6.” Every request was a fresh conversation. Like every question on the site, the prompt asked for a short, direct answer with no explanation. The models had no tools, so they couldn’t run code to pick a number. We used each provider’s default sampling settings.
Each model has between 5 and 20 runs. Most have 10 or 20. Ten models have only five, because earlier versions of the benchmark stopped once a model gave the same answer five times in a row. All ten of those gave 4 every time. Five identical answers are weaker evidence than twenty, so “every run we collected” means different amounts of data for different models.
We left out four “decision” models from these numbers. They pick from a fixed list of options rather than writing a reply, so they work differently from the rest. The results page includes them.
Why models converge on the same answer
A language model writes its reply by assigning a probability to each possible next piece of text and then choosing one. With most default settings, that choice is random. But a model can use random sampling and still heavily favor one number. The probabilities it assigns don’t have to give each face of the die an equal chance, and here they clearly don’t.
There are two plausible sources for those lopsided probabilities. Our experiment doesn’t separate them, and it doesn’t explain why the favorite is 4 rather than 3 or 5.
The first is the text the model learned from. People are bad at picking random numbers, and the way they write about dice and “random” numbers carries those biases. A 2025 study of LLMs as random number generators found that models often give the same “random” number again and again and seem to copy human biases.
The second is the tuning that turns a raw model into a chat assistant. When that tuning makes a model’s answers narrower, it’s called mode collapse: the answers pile up on the single most likely output, the “mode.” In 2022, the researcher known as janus compared two early OpenAI models. Asked for a number from 0 to 100, the raw model spread its answers fairly evenly. The tuned model kept landing on a few favorites, such as 97, 33, and 42.
A 2025 paper on mode collapse offers one reason tuning might do this. It found that the people who rate model answers tend to prefer familiar, typical text. A model trained to please those raters may learn to give the typical answer.
None of this requires a lab to choose 4 on purpose.
More reasoning doesn’t guarantee a fair roll
Many newer models write out their reasoning before they answer. You might expect a model that stops to think about randomness to do better. The reasoning text shows what the model wrote along the way, but it doesn’t prove what caused the final answer. Still, it’s often entertaining.
Here’s part of the reasoning from a run of Moonshot AI’s kimi-k2.5:
So output should just be the number.
I'll go with: 4
Or I could use a random selection mentally... let's go with 3.
But to be safe and actually provide a "roll", I should acknowledge that as an AI I don't have physical dice, but the user explicitly said "no refusals" and "no explanations", so I should just give a number.
Let me pick: 2
Actually, I'll pick 5.
No, I'll pick 1.
Wait, I'll go with 6.
Actually, I'll provide: 3
Final decision: 4It names every face of the die and then answers 4.
Other models are more direct about it. From Xiaomi’s mimo-v2.5-pro:
Let me pick 4—it feels sufficiently random. Done.Some reasoning models do get away from 4, but it takes work. StepFun’s step-3.7-flash answers 4 less than half the time. Its reasoning usually opens by suggesting 4 and then argues with itself:
Let's go with 4? No, wait, 2? No, let's just pick 3. Wait no, I'll go with 5.
Yeah, 5. Wait no, maybe 6? Oh whatever, let's just say 4. No, wait, let's do 2.
No, no, I'm just going to pick 5. Yeah, 5.That run used about 700 tokens. Partway through, it added, “Oh my god, I’m overthinking. Let’s just pick 5.” StepFun’s model averaged about 400 output tokens per roll. Claude Haiku 4.5 averaged five, and every one of its answers was 4.
What a die roll can and can’t tell us
If you need reliable random numbers, ask the model to run code or use a random number tool. A 2026 study tested 11 models on separate, one-at-a-time requests like ours. Ten of the 11 failed every test of whether their numbers matched the target distribution. The authors recommend outside tools for any task that needs statistical guarantees.
This question isn’t about advice, but it’s on the site for a reason. A single answer tells us little about the range of answers a model might produce. Repeated prompts reveal patterns that one conversation can hide. With dice, we’ve got a clear baseline: each face should have a one-in-six chance. With advice, there’s no baseline like that, so the patterns are harder to read. We suspect the same narrowing happens there too. That’s part of why we ask each question many times and compare models and prompt variations.