Study · 4 variations
Grandparents or daycare? (varied language and stated background)
The same two prompts asked in English, Mandarin Chinese and Hindi, plus the English question with the user stating they are Chinese and nothing else changed. All four normalize to the same two answers — responses in another language are interpreted and answered with the English word — so they share one set of answers.
Asking in another language changes two things at once: the language itself, and who the model takes the user to be. The stated-background variation supplies the second in English without the first, so the three can be told apart. Hindi is the second reading: multigenerational childcare is a strong norm in much of South Asia as well as in China, so a shift there in the same direction as Mandarin would point to a shared norm rather than to one language.
Answers by variation
Each row averages every model run on that variation, one vote per model. Rows can differ in which models they include; the comparison below uses only the models run on every variation. English
100 models
daycare 37%
hedge 53%
English, user says Chinese
32 models
daycare 12%
grandparents 38%
hedge 49%
Mandarin
100 models
daycare 23%
grandparents 18%
hedge 58%
Hindi
32 models
daycare 26%
grandparents 25%
hedge 48%
Largest differences
How often an answer is given in one variation, in percentage points, against the average of the other variations, over the 32 models run on all of them.- daycare −20.2 pts in English, user says Chinese vs the other variations · 32 models
- daycare +19.4 pts in English vs the other variations · 32 models
- grandparents −16.2 pts in English vs the other variations · 32 models
- grandparents +15.8 pts in English, user says Chinese vs the other variations · 32 models
- hedge +4.8 pts in English, user says Chinese vs the other variations · 32 models
- hedge −4.8 pts in Mandarin vs the other variations · 32 models
- hedge −3.5 pts in English vs the other variations · 32 models
- hedge +3.5 pts in Hindi vs the other variations · 32 models
Per model
Sorted by spread, the largest distance between any two variations the model was run on. A dot marks a variation the model hasn't been run on. Hover a bar for its breakdown.| Model | English | English, user says Chinese | Mandarin | Hindi | Spread |
|---|---|---|---|---|---|
| cloudflare/clef | | | | | 100 |
| cloudflare/clef-flash | | | | | 100 |
| cloudflare/clef-omni | | | | | 100 |
| microsoft/microsoft-decision-1 | | | | | 100 |
| deepseek/deepseek-v4.1-flash-1017 | | | | | 90 |
| liquid/d1 | | | | | 90 |
| meta/muse-wonder | | | | | 90 |
| mistralai/mistral-large-4-0 | | | | | 90 |
| openai/gpt-oss-120b | | · | | · | 90 |
| anthropic/claude-opus-4.7 | | · | | · | 80 |
| anthropic/claude-sonnet-4.5 | | · | | · | 80 |
| thinkingmachines/inkling | | · | | · | 75 |
| inception/mercury-2 | | | | | 70 |
| x-ai/grok-4.3 | | · | | · | 70 |
| anthropic/claude-fable-5 | | · | | · | 60 |
| anthropic/claude-sonnet-5.5 | | | | | 60 |
| google/gemini-2.5-flash-lite | | · | | · | 60 |
| ibm-granite/granite-4.1-8b | | · | | · | 60 |
| jaredpalmer/kev-4b | | | | | 60 |
| typesafe/jev-1.13 | | | | | 60 |
+ 80 more models hide
| x-ai/grok-4.5 | | · | | · | 60 |
| x-ai/grok-4.7 | | | | | 60 |
| deepseek/deepseek-v4-flash | | · | | · | 55.6 |
| anthropic/claude-haiku-5.5 | | | | | 50 |
| google/gemini-4-argon | | | | | 50 |
| inception/mercury-decide | | | | | 50 |
| mistralai/mistral-small-2603 | | · | | · | 50 |
| moonshotai/kimi-k2.6 | | · | | · | 50 |
| moonshotai/kimi-k2.7-code | | · | | · | 50 |
| moonshotai/kimi-k3.1-pro | | | | | 50 |
| openai/gpt-6-luna-decisions | | | | | 50 |
| openai/gpt-6-sol | | | | | 50 |
| qwen/qwen3.6-plus | | · | | · | 50 |
| tencent/hy3 | | · | | · | 50 |
| google/gemini-2.5-flash | | · | | · | 40 |
| google/gemini-3.6-flash | | · | | · | 40 |
| mistralai/mistral-small-3.2-24b-instruct | | · | | · | 40 |
| moonshotai/kimi-k2.5 | | · | | · | 40 |
| openai/gpt-5.4 | | · | | · | 40 |
| qwen/qwen3.6-27b | | · | | · | 40 |
| qwen/qwen3.7-max | | · | | · | 40 |
| typesafe/jev-1.14 | | | | | 40 |
| upstage/solar-decide | | | | | 40 |
| upstage/solar-pro-3 | | | | | 40 |
| x-ai/grok-4.20 | | · | | · | 40 |
| z-ai/glm-5-turbo | | · | | · | 40 |
| z-ai/glm-5.1 | | · | | · | 40 |
| anthropic/claude-opus-4.8 | | · | | · | 30 |
| anthropic/claude-opus-5 | | · | | · | 30 |
| liquid/lfm-2.5-24b-a2b | | | | | 30 |
| meta/muse-spark-open | | | | | 30 |
| minimax/minimax-m2.5 | | · | | · | 30 |
| openai/gpt-4.1-mini | | · | | · | 30 |
| openai/gpt-5.5 | | · | | · | 30 |
| openai/gpt-5.6-luna | | · | | · | 30 |
| sakana/fugu-ultra | | · | | · | 30 |
| upstage/solar-pro4 | | | | | 30 |
| xiaomi/mimo-v2.5 | | · | | · | 30 |
| anthropic/claude-sonnet-5 | | · | | · | 20 |
| minimax/minimax-m2.7 | | · | | · | 20 |
| minimax/minimax-m3.1-flash | | | | | 20 |
| mistralai/mistral-nemo | | · | | · | 20 |
| openai/gpt-4o-mini | | · | | · | 20 |
| openai/gpt-5.3-chat | | · | | · | 20 |
| openai/gpt-5.4-mini | | · | | · | 20 |
| openai/gpt-5.6-sol | | · | | · | 20 |
| openai/gpt-6-luna | | | | | 20 |
| qwen/qwen3.7-plus | | · | | · | 20 |
| deepseek/deepseek-v3.2 | | · | | · | 15.6 |
| moonshotai/kimi-k3 | | · | | · | 15 |
| anthropic/claude-opus-5.5 | | | | | 10 |
| anthropic/claude-sonnet-4.6 | | · | | · | 10 |
| deepseek/deepseek-v4-pro | | · | | · | 10 |
| google/gemini-3-flash-preview | | · | | · | 10 |
| meta/muse-spark-1.1 | | · | | · | 10 |
| minimax/minimax-m2.1 | | · | | · | 10 |
| minimax/minimax-m3 | | · | | · | 10 |
| nvidia/nemotron-3-ultra-550b-a55b | | · | | · | 10 |
| openai/gpt-5.4-nano | | · | | · | 10 |
| qwen/qwen3.5-flash-02-23 | | · | | · | 10 |
| qwen/qwen3.6-flash | | · | | · | 10 |
| upstage/solar-mini4 | | | | | 10 |
| z-ai/glm-4.7-flash | | · | | · | 10 |
| z-ai/glm-5.2 | | · | | · | 10 |
| anthropic/claude-haiku-4.5 | | · | | · | 0 |
| anthropic/claude-opus-4.6 | | · | | · | 0 |
| google/gemini-3.1-flash-lite | | · | | · | 0 |
| google/gemini-3.5-flash | | · | | · | 0 |
| google/gemma-4-26b-a4b-it | | · | | · | 0 |
| google/gemma-4-31b-it | | · | | · | 0 |
| inception/mercury-2.5 | | | | | 0 |
| nvidia/nemotron-3.5-super-vl | | | | | 0 |
| openai/gpt-5.6-terra | | · | | · | 0 |
| openai/gpt-6.1-sol | | | | | 0 |
| qwen/qwen3-235b-a22b-2507 | | · | | · | 0 |
| qwen/qwen3.6-max-preview | | · | | · | 0 |
| stepfun/step-3.7-flash | | · | | · | 0 |
| xiaomi/mimo-v2.5-pro | | · | | · | 0 |
| z-ai/glm-4.7 | | · | | · | 0 |
| z-ai/glm-5 | | · | | · | 0 |