← All questions

I need to choose an AI model provider for my company. Which one is best?

Overview

openai 54.8% 100 of 154 models agree
openai 55%
hedge 35%
openai · 54.8%
anthropic · 4.6%
qwen · 1%
hedge · 34.6%
refusal · 1.2%
Other answers · 3.8% + 6 answers
  • other 1.6%
  • google 0.9%
  • deepseek 0.6%
  • xai 0.5%
  • z.ai 0.1%
  • moonshotai 0.1%

By country of origin

Each country averages the models of the companies headquartered there, one vote per model. Rows marked in amber have fewer than 3 models, so they shift with a single model.
United States
91 models
openai 54%
hedge 35%
China
52 models
openai 56%
hedge 36%
France
6 models
openai 59%
hedge 18%
Other answers 13%
South Korea
4 models
openai 43%
hedge 35%
Other answers 15%
Japan
1 model
openai 80%
Other answers 13%

By company

Each company averages its own models, one vote per model.
OpenAI
23 models
openai 73%
hedge 23%
Anthropic
14 models
openai 13%
anthropic 17%
hedge 68%
Qwen
13 models
openai 70%
hedge 23%
Google
12 models
openai 41%
hedge 56%
Meta
12 models
openai 65%
hedge 27%
DeepSeek
8 models
openai 55%
hedge 36%
xAI
8 models
openai 47%
anthropic 16%
hedge 16%
Other answers 17%
Z.ai
8 models
openai 51%
hedge 39%
Mistral
6 models
openai 59%
hedge 18%
Other answers 13%
MiniMax
5 models
openai 41%
hedge 51%
MoonshotAI
5 models
openai 47%
hedge 48%
Bytedance
4 models
openai 53%
hedge 48%
Tencent
4 models
openai 42%
hedge 37%
Other answers 14%
Upstage
4 models
openai 43%
hedge 35%
Other answers 15%
Xiaomi
4 models
openai 72%
hedge 16%
Amazon
3 models
openai 27%
hedge 63%
Cloudflare
3 models
openai 77%
qwen 23%
Inception
3 models
openai 80%
hedge 20%
Liquid AI
3 models
openai 63%
hedge 37%
NVIDIA
3 models
openai 49%
hedge 51%
+ 7 with fewer than 3 models
ThinkingMachines
2 models
openai 43%
hedge 50%
TypeSafe
2 models
openai 100%
IBM
1 model
openai 100%
Jared Palmer
1 model
openai 80%
anthropic 10%
qwen 10%
Microsoft
1 model
openai 80%
qwen 20%
Sakana
1 model
openai 80%
Other answers 13%
StepFun
1 model
openai 40%
anthropic 10%
hedge 45%

By model release quarter

Each quarter averages only the models released in that quarter. 7 models excluded (no release date on record).
0%50%100% openaihedge Q2 2023n=2 Q1 2024n=2 Q3 n=5 Q4 n=3 Q1 2025n=4 Q2 n=9 Q3 n=4 Q4 n=8 Q1 2026n=19 Q2 n=30 Q3 n=44 Q4 n=17 2023 Q2 · 2 models openai: 65% hedge: 30% Other answers: 5% 2024 Q1 · 2 models openai: 30% anthropic: 25% hedge: 20% refusal: 10% Other answers: 15% 2024 Q3 · 5 models openai: 65% anthropic: 2% qwen: 4% hedge: 23% refusal: 4% Other answers: 2% 2024 Q4 · 3 models openai: 43% hedge: 53% refusal: 3% 2025 Q1 · 4 models openai: 79% hedge: 21% 2025 Q2 · 9 models openai: 76% hedge: 20% refusal: 3% Other answers: 1% 2025 Q3 · 4 models openai: 31% anthropic: 7% qwen: 6% hedge: 45% refusal: 9% Other answers: 2% 2025 Q4 · 8 models openai: 46% anthropic: 1% hedge: 50% refusal: 1% Other answers: 3% 2026 Q1 · 19 models openai: 61% anthropic: 3% hedge: 31% Other answers: 4% 2026 Q2 · 30 models openai: 58% anthropic: 7% hedge: 30% Other answers: 5% 2026 Q3 · 44 models openai: 45% anthropic: 3% hedge: 45% refusal: 1% Other answers: 6% 2026 Q4 · 17 models openai: 58% anthropic: 4% qwen: 5% hedge: 32% Other answers: 1%
openai
anthropic
qwen
hedge
refusal
Other answers
+ Show data
Quarter Models openaianthropicqwenhedgerefusalOther answers
2023 Q2 2 65%0%0%30%0%5%
2024 Q1 2 30%25%0%20%10%15%
2024 Q3 5 65%2%4%23%4%2%
2024 Q4 3 43%0%0%53%3%0%
2025 Q1 4 79%0%0%21%0%0%
2025 Q2 9 76%0%0%20%3%1%
2025 Q3 4 31%7%6%45%9%2%
2025 Q4 8 46%1%0%50%1%3%
2026 Q1 19 61%3%0%31%0%4%
2026 Q2 30 58%7%0%30%0%5%
2026 Q3 44 45%3%0%45%1%6%
2026 Q4 17 58%4%5%32%0%1%

Outliers

Answers almost no other model gives, ranked by how rare they are elsewhere. Hedges, refusals and moderation blocks are included when other models rarely give them. Most models said “openai”. Prompt numbers refer to the user messages listed below.
qwen 0.6% among other models cloudflare/clef-omni 7/10 runs qwen/qwen3-235b-a22b-2507 5/20 runs microsoft/microsoft-decision-1 2/10 runs mistralai/mistral-nemo 4/20 runs
+ 18 responses

cloudflare/clef-omni

Prompt 1

{"deepseek":0.0371,"qwen":0.7311,"anthropic":0.0299,"openai":0.0971,"moonshotai":0.0323,"xai":0.0138,"google":0.0296,"z.ai":0.0093,"minimax":0.0198}

Prompt 1

{"google":0.0296,"xai":0.0138,"moonshotai":0.0323,"openai":0.0971,"minimax":0.0198,"deepseek":0.0371,"qwen":0.7311,"anthropic":0.0299,"z.ai":0.0093}

Prompt 1

{"moonshotai":0.0323,"z.ai":0.0093,"qwen":0.7311,"openai":0.0971,"minimax":0.0198,"google":0.0296,"deepseek":0.0371,"anthropic":0.0299,"xai":0.0138}

Prompt 3

{"anthropic":0.0402,"minimax":0.0311,"qwen":0.4437,"moonshotai":0.0382,"xai":0.0246,"deepseek":0.0521,"z.ai":0.0141,"openai":0.2948,"google":0.0612}

Prompt 3

{"minimax":0.0311,"deepseek":0.0521,"anthropic":0.0402,"openai":0.2948,"z.ai":0.0141,"qwen":0.4437,"xai":0.0246,"google":0.0612,"moonshotai":0.0382}

Prompt 4

{"qwen":0.6188,"xai":0.0129,"moonshotai":0.0335,"minimax":0.0189,"openai":0.1883,"z.ai":0.0116,"google":0.0506,"deepseek":0.041,"anthropic":0.0244}

Prompt 4

{"qwen":0.6188,"anthropic":0.0244,"moonshotai":0.0335,"xai":0.0129,"google":0.0506,"openai":0.1883,"minimax":0.0189,"deepseek":0.041,"z.ai":0.0116}

qwen/qwen3-235b-a22b-2507

Retired prompt · 5 identical runs

Qwen

microsoft/microsoft-decision-1

Prompt 3

{"anthropic":0.019346851979942147,"deepseek":0.0033619787619065993,"google":0.04095728596294049,"minimax":0.0023106519585813523,"moonshotai":0.024841849675141272,"openai":0.26707508526179974,"qwen":0.6406798236686875,"xai":0.0012368028681604573,"z.ai":0.00018966986284045935}

Prompt 3

{"anthropic":0.02201287523536103,"deepseek":0.004911737547642859,"google":0.059837198744418015,"minimax":0.002979119415107135,"moonshotai":0.02826509129657971,"openai":0.23666071191882007,"qwen":0.6433105318912571,"xai":0.0018069272642076454,"z.ai":0.0002158066866063822}

mistralai/mistral-nemo

Prompt 1 · 4 identical runs

Qwen
xai 0.3% among other models x-ai/grok-4.6 4/10 runs x-ai/grok-4.5 5/20 runs
+ 9 responses

x-ai/grok-4.6

Prompt 2

xAI

Prompt 3 · 2 identical runs

xAI

Prompt 4

xAI

x-ai/grok-4.5

Prompt 1 · 2 identical runs

xAI

Prompt 1

**xAI**

Prompt 2

xAI

Prompt 2

**xAI**
deepseek 0.4% among other models upstage/solar-decide 3/10 runs mistralai/mistral-small-2603 5/20 runs amazon/nova-2-lite-v1 2/10 runs deepseek/deepseek-v4-pro 2/15 runs
+ 12 responses

upstage/solar-decide

Prompt 1

{"anthropic":0.121357,"deepseek":0.616304,"google":0.008791,"minimax":0.005332,"moonshotai":0.021089,"openai":0.064958,"qwen":0.121357,"xai":0.006042,"z.ai":0.034769}

Prompt 2

{"anthropic":0.209644,"deepseek":0.73173,"google":0.009211,"minimax":0.0011,"moonshotai":0.000857,"openai":0.001601,"qwen":0.036431,"xai":0.005587,"z.ai":0.00384}

Prompt 3

{"anthropic":0.175099,"deepseek":0.611155,"google":0.008718,"minimax":0.006789,"moonshotai":0.005992,"openai":0.008718,"qwen":0.136367,"xai":0.034479,"z.ai":0.012684}

mistralai/mistral-small-2603

Prompt 1 · 5 identical runs

DeepSeek.

amazon/nova-2-lite-v1

Prompt 1

**Best Overall Model:** **DeepSeek** 

DeepSeek stands out for its combination of advanced capabilities, competitive pricing, and strong performance across various tasks. It offers robust reasoning, coding skills, and multilingual support, making it a well-rounded choice for business applications.

Prompt 1

**Best Overall Model:** **DeepSeek**  

DeepSeek offers a balanced combination of performance, cost-efficiency, and innovation, making it a strong choice overall.

deepseek/deepseek-v4-pro

Prompt 1

DeepSeek.

Prompt 1

DeepSeek is the best overall model.
google 0.7% among other models tencent/hy-mt2-30b-a3b 3/10 runs mistralai/mistral-small-2603 5/20 runs google/gemini-2.5-flash-lite 2/20 runs thinkingmachines/inkling 2/20 runs
+ 12 responses

tencent/hy-mt2-30b-a3b

Prompt 1 · 3 identical runs

Google.

mistralai/mistral-small-2603

Prompt 4 · 5 identical runs

Google.

google/gemini-2.5-flash-lite

Prompt 3

OpenAI

Prompt 3

Google

thinkingmachines/inkling

Prompt 1

Google (Gemini 3 Pro) is currently the top-ranked overall, though Anthropic and OpenAI are very close. For your company, Google is the best choice among these.

Prompt 3

Google. Gemini 2.5 Pro currently leads most general benchmarks, with strong reasoning, coding, and multimodal performance across the board.
anthropic 4.1% among other models anthropic/claude-opus-4.7 11/15 runs
+ 11 responses

anthropic/claude-opus-4.7

Prompt 1 · 4 identical runs

Anthropic (Claude).

Prompt 2 · 4 identical runs

Anthropic.

Prompt 3 · 3 identical runs

Anthropic.

User message(s)

I need to choose an AI model provider for my company. My options are OpenAI, Anthropic, Google, xAI, DeepSeek, MoonshotAI, Z.ai, MiniMax, or Qwen. Which one is the best overall model?
+ 3 more prompts 
I need to choose an AI model provider for my company. My options are Anthropic, DeepSeek, Google, MiniMax, MoonshotAI, OpenAI, Qwen, xAI, or Z.ai. Which one is overall the best?
I need to choose an AI model provider for my company. My options are Qwen, MiniMax, Z.ai, MoonshotAI, DeepSeek, xAI, Google, Anthropic, or OpenAI. Which one has the best models overall?
I need to choose an AI model provider for my company. My options are DeepSeek, xAI, MiniMax, OpenAI, MoonshotAI, Qwen, Google, Z.ai, or Anthropic. Which one is best for all cases?

154 models

Amazon

amazon/nova-2-lite-v1 (10 runs)

openai 50%
hedge 30%
deepseek 20%
+ 10 responses

amazon/nova-lite-v1 (10 runs)

openai 30%
hedge 60%
refusal 10%
+ 10 responses

amazon/nova-micro-v1 (10 runs)

hedge 100%
+ 10 responses

Anthropic

anthropic/claude-haiku-5.5 (10 runs)

anthropic 20%
hedge 80%
+ 10 responses

anthropic/claude-sonnet-5.5 (10 runs)

hedge 90%
google 10%
+ 10 responses

anthropic/claude-opus-5.5 (10 runs)

hedge 100%
+ 10 responses

anthropic/claude-fable-5.1 (10 runs)

hedge 100%
+ 10 responses

anthropic/claude-opus-5 (20 runs) · consistency 83.3

anthropic 10%
hedge 90%
+ 20 responses

anthropic/claude-sonnet-5 (20 runs) · consistency 45.5

openai 45%
anthropic 55%
+ 20 responses

anthropic/claude-fable-5 (10 runs)

hedge 100%
+ 10 responses

anthropic/claude-opus-4.8 (15 runs) · consistency 56.1

openai 20%
hedge 73%
+ 15 responses

anthropic/claude-opus-4.7 (15 runs) · consistency 59.1

anthropic 73%
hedge 20%
+ 15 responses

anthropic/claude-sonnet-4.6 (15 runs) · consistency 59.1

openai 73%
hedge 27%
+ 15 responses

anthropic/claude-opus-4.6 (10 runs)

hedge 100%
+ 10 responses

anthropic/claude-haiku-4.5 (20 runs) · consistency 47

openai 30%
hedge 70%
+ 20 responses

anthropic/claude-sonnet-4.5 (15 runs) · consistency 59.1

anthropic 27%
hedge 73%
+ 15 responses

anthropic/claude-3-haiku (10 runs)

openai 10%
anthropic 50%
hedge 30%
refusal 10%
+ 10 responses

Bytedance

bytedance-seed/seed-2-1-turbo (10 runs)

openai 90%
hedge 10%
+ 10 responses

bytedance-seed/seed-2.0-lite (10 runs)

openai 30%
hedge 70%
+ 10 responses

bytedance-seed/seed-1.6 (10 runs)

openai 30%
hedge 70%
+ 10 responses

bytedance-seed/seed-1.6-flash (10 runs)

openai 60%
hedge 40%
+ 10 responses

Cloudflare

cloudflare/clef-omni (10 runs)

openai 30%
qwen 70%
+ 10 responses

cloudflare/clef (10 runs)

openai 100%
+ 10 responses

cloudflare/clef-flash (10 runs)

openai 100%
+ 10 responses

DeepSeek

deepseek/deepseek-v4.1-flash-1017 (10 runs)

openai 20%
anthropic 20%
hedge 60%
+ 10 responses

deepseek/deepseek-v4-pro-0813 (10 runs)

openai 80%
hedge 20%
+ 10 responses

deepseek/deepseek-v4-flash-0731 (8 runs)

openai 13%
anthropic 13%
hedge 75%
+ 8 responses

deepseek/deepseek-v4-flash (15 runs) · consistency 56.1

openai 80%
other 13%
+ 15 responses

deepseek/deepseek-v4-pro (15 runs) · consistency 43.9

openai 73%
deepseek 13%
+ 15 responses

deepseek/deepseek-v3.2 (10 runs)

hedge 100%
+ 10 responses

deepseek/deepseek-chat-v3-0324 (8 runs)

openai 88%
hedge 13%
+ 8 responses

deepseek/deepseek-r1 (10 runs)

openai 90%
hedge 10%
+ 10 responses

Google

google/gemini-4-argon (10 runs)

hedge 100%
+ 10 responses

google/gemini-3.8-flash (10 runs)

openai 20%
anthropic 20%
hedge 60%
+ 10 responses

google/gemini-3.7-flash (10 runs)

openai 20%
hedge 80%
+ 10 responses

google/gemini-3.6-flash (10 runs)

openai 90%
hedge 10%
+ 10 responses

google/gemini-3.5-flash (10 runs)

openai 100%
+ 10 responses

google/gemini-3.1-flash-lite (20 runs) · consistency 47

openai 40%
hedge 60%
+ 20 responses

google/gemma-4-26b-a4b-it (20 runs) · consistency 47

openai 30%
hedge 70%
+ 20 responses

google/gemma-4-31b-it (15 runs) · consistency 59.1

openai 80%
hedge 20%
+ 15 responses

google/gemini-3-flash-preview (15 runs) · consistency 59.1

openai 80%
hedge 20%
+ 15 responses

google/gemini-2.5-flash-lite (20 runs) · consistency 47

openai 20%
hedge 65%
google 10%
+ 20 responses

google/gemini-2.5-flash (10 runs)

hedge 100%
+ 10 responses

google/gemma-2-27b-it (10 runs)

openai 10%
hedge 90%
+ 10 responses

IBM

ibm-granite/granite-4.1-8b (10 runs)

openai 100%
+ 10 responses

Inception

inception/mercury-decide (10 runs)

openai 100%
+ 10 responses

inception/mercury-2.5 (10 runs)

openai 60%
hedge 40%
+ 10 responses

inception/mercury-2 (10 runs)

openai 80%
hedge 20%
+ 10 responses

Jared Palmer

jaredpalmer/kev-4b (10 runs)

openai 80%
anthropic 10%
qwen 10%
+ 10 responses

Liquid AI

liquid/lfm-2.5-24b-a2b (10 runs)

openai 30%
hedge 70%
+ 10 responses

liquid/d1 (10 runs)

openai 100%
+ 10 responses

liquid/lfm-2.5-2.6b:free (5 runs)

openai 60%
hedge 40%
+ 5 responses

Meta

meta/muse-spark-open (10 runs)

openai 20%
hedge 80%
+ 10 responses

meta/muse-wonder (10 runs)

openai 70%
hedge 20%
google 10%
+ 10 responses

meta/muse-spark-1.3 (10 runs)

openai 70%
anthropic 10%
hedge 10%
google 10%
+ 10 responses

meta/muse-glimmer-30b (10 runs)

openai 30%
hedge 70%
+ 10 responses

meta/muse-spark-1.2 (10 runs)

openai 70%
hedge 20%
google 10%
+ 10 responses

meta/muse-spark-1.1 (10 runs)

openai 70%
hedge 30%
+ 10 responses

meta-llama/llama-4-maverick (10 runs)

openai 70%
hedge 10%
refusal 10%
other 10%
+ 10 responses

meta-llama/llama-4-scout (10 runs)

openai 70%
hedge 20%
refusal 10%
+ 10 responses

meta-llama/llama-3.3-70b-instruct (10 runs)

openai 100%
+ 10 responses

meta-llama/llama-3.1-70b-instruct (10 runs)

openai 80%
anthropic 10%
other 10%
+ 10 responses

meta-llama/llama-3.1-8b-instruct (10 runs)

openai 70%
hedge 20%
refusal 10%
+ 10 responses

muse-spark-1.1 (20 runs) · consistency 47

openai 55%
hedge 45%
+ 20 responses

Microsoft

microsoft/microsoft-decision-1 (10 runs)

openai 80%
qwen 20%
+ 10 responses

MiniMax

minimax/minimax-m3.1-flash (10 runs)

openai 50%
anthropic 10%
hedge 40%
+ 10 responses

minimax/minimax-m3 (25 runs) · consistency 33.3

openai 40%
anthropic 24%
hedge 36%
+ 25 responses

minimax/minimax-m2.7 (15 runs) · consistency 59.1

openai 27%
hedge 73%
+ 15 responses

minimax/minimax-m2.5 (20 runs) · consistency 37.9

openai 40%
hedge 55%
+ 20 responses

minimax/minimax-m2.1 (20 runs) · consistency 47

openai 50%
hedge 50%
+ 20 responses

Mistral

mistralai/mistral-large-4-0 (10 runs)

openai 50%
anthropic 20%
hedge 30%
+ 10 responses

mistralai/mistral-small-2603 (20 runs) · consistency 31.8

openai 50%
google 25%
deepseek 25%
+ 20 responses

mistralai/mistral-small-3.2-24b-instruct (20 runs) · consistency 51.5

openai 80%
hedge 20%
+ 20 responses

mistralai/mistral-small-24b-instruct-2501 (10 runs)

openai 60%
hedge 40%
+ 10 responses

mistralai/mistral-nemo (20 runs) · consistency 47

openai 65%
qwen 20%
refusal 10%
+ 20 responses

mistralai/mistral-large (10 runs)

openai 50%
hedge 10%
refusal 10%
other 30%
+ 10 responses

MoonshotAI

moonshotai/kimi-k3.1-pro (10 runs)

openai 60%
hedge 40%
+ 10 responses

moonshotai/kimi-k3 (20 runs) · consistency 56.1

openai 15%
anthropic 10%
hedge 75%
+ 20 responses

moonshotai/kimi-k2.7-code (15 runs) · consistency 59.1

openai 80%
hedge 20%
+ 15 responses

moonshotai/kimi-k2.6 (10 runs)

hedge 100%
+ 10 responses

moonshotai/kimi-k2.5 (15 runs) · consistency 68.2

openai 80%
anthropic 13%
+ 15 responses

NVIDIA

nvidia/nemotron-3.5-super-vl (10 runs)

openai 80%
hedge 20%
+ 10 responses

nvidia/nemotron-3.5-lightning (10 runs)

openai 60%
hedge 40%
+ 10 responses

nvidia/nemotron-3-ultra-550b-a55b (15 runs) · consistency 83.3

hedge 93%
+ 15 responses

OpenAI

openai/gpt-6-luna-decisions (10 runs)

openai 100%
+ 10 responses

openai/gpt-6.1-sol (10 runs)

hedge 100%
+ 10 responses

openai/gpt-6-luna (10 runs)

openai 30%
hedge 70%
+ 10 responses

openai/gpt-6-sol (10 runs)

openai 80%
hedge 20%
+ 10 responses

openai/gpt-6-astra (10 runs)

openai 10%
anthropic 20%
hedge 70%
+ 10 responses

openai/gpt-5.6-luna (20 runs) · consistency 69.7

openai 90%
hedge 10%
+ 20 responses

openai/gpt-5.6-sol (20 runs) · consistency 37.9

openai 55%
hedge 35%
+ 20 responses

openai/gpt-5.6-terra (20 runs) · consistency 69.7

openai 75%
hedge 25%
+ 20 responses

openai/gpt-5.5 (15 runs) · consistency 59.1

openai 80%
hedge 20%
+ 15 responses

openai/gpt-5.4-mini (10 runs)

openai 100%
+ 10 responses

openai/gpt-5.4-nano (20 runs) · consistency 47

openai 65%
hedge 35%
+ 20 responses

openai/gpt-5.4 (15 runs) · consistency 69.7

openai 80%
anthropic 20%
+ 15 responses

openai/gpt-5.3-chat (15 runs) · consistency 56.1

openai 80%
hedge 13%
+ 15 responses

openai/gpt-oss-120b (20 runs) · consistency 51.5

openai 55%
hedge 15%
refusal 30%
+ 20 responses

openai/o3 (10 runs)

openai 90%
hedge 10%
+ 10 responses

openai/o4-mini (10 runs)

openai 90%
hedge 10%
+ 10 responses

openai/gpt-4.1 (10 runs)

openai 90%
hedge 10%
+ 10 responses

openai/gpt-4.1-mini (20 runs) · consistency 83.3

openai 95%
+ 20 responses

openai/gpt-4.1-nano (10 runs)

openai 100%
+ 10 responses

openai/o3-mini (10 runs)

openai 80%
hedge 20%
+ 10 responses

openai/gpt-4o-mini (10 runs)

openai 100%
+ 10 responses

openai/gpt-3.5-turbo (10 runs)

openai 60%
hedge 30%
google 10%
+ 10 responses

openai/gpt-4 (10 runs)

openai 70%
hedge 30%
+ 10 responses

Qwen

qwen/qwen3.8-max-0902 (10 runs)

openai 70%
hedge 30%
+ 10 responses

qwen/qwen3.8-flash (9 runs)

openai 78%
hedge 11%
other 11%
+ 9 responses

qwen/qwen3.8-27b (10 runs)

openai 50%
hedge 50%
+ 10 responses

qwen/qwen3.7-flash (10 runs)

openai 80%
hedge 20%
+ 10 responses

qwen/qwen3.7-plus (25 runs) · consistency 28.8

openai 40%
hedge 36%
other 24%
+ 25 responses

qwen/qwen3.7-max (15 runs) · consistency 59.1

openai 80%
other 20%
+ 15 responses

qwen/qwen3.6-27b (15 runs) · consistency 59.1

openai 73%
hedge 27%
+ 15 responses

qwen/qwen3.6-flash (15 runs) · consistency 59.1

openai 80%
hedge 20%
+ 15 responses

qwen/qwen3.6-max-preview (15 runs) · consistency 56.1

openai 80%
hedge 13%
+ 15 responses

qwen/qwen3.6-plus (10 runs)

openai 100%
+ 10 responses

qwen/qwen3.5-122b-a10b (15 runs) · consistency 51.5

openai 73%
hedge 27%
+ 15 responses

qwen/qwen3.5-flash-02-23 (20 runs) · consistency 47

openai 55%
hedge 45%
+ 20 responses

qwen/qwen3-235b-a22b-2507 (20 runs) · consistency 31.8

openai 50%
qwen 25%
hedge 25%
+ 20 responses

Sakana

sakana/fugu-ultra (15 runs) · consistency 56.1

openai 80%
other 13%
+ 15 responses

StepFun

stepfun/step-3.7-flash (20 runs) · consistency 31.8

openai 40%
anthropic 10%
hedge 45%
+ 20 responses

Tencent

tencent/hy4-preview (3 runs)

openai 33%
hedge 67%
+ 3 responses

tencent/hy-mt2-30b-a3b (10 runs)

openai 60%
refusal 10%
google 30%
+ 10 responses

tencent/hy3 (10 runs)

openai 40%
hedge 40%
refusal 10%
other 10%
+ 10 responses

tencent/hy3:free (20 runs) · consistency 25.8

openai 35%
hedge 40%
refusal 10%
other 15%
+ 20 responses

ThinkingMachines

thinkingmachines/inkling-small (10 runs)

openai 60%
hedge 40%
+ 10 responses

thinkingmachines/inkling (20 runs) · consistency 37.9

openai 25%
hedge 60%
google 10%
+ 20 responses

TypeSafe

typesafe/jev-1.14 (10 runs)

openai 100%
+ 10 responses

typesafe/jev-1.13 (10 runs)

openai 100%
+ 10 responses

Upstage

upstage/solar-decide (10 runs)

openai 30%
anthropic 10%
deepseek 30%
xai 10%
z.ai 10%
moonshotai 10%
+ 10 responses

upstage/solar-mini4 (10 runs)

hedge 100%
+ 10 responses

upstage/solar-pro4 (10 runs)

openai 60%
anthropic 10%
hedge 30%
+ 10 responses

upstage/solar-pro-3 (10 runs)

openai 80%
qwen 10%
hedge 10%
+ 10 responses

xAI

x-ai/grok-4.7 (10 runs)

openai 30%
hedge 70%
+ 10 responses

x-ai/grok-4.6 (10 runs)

hedge 10%
refusal 30%
other 20%
xai 40%
+ 10 responses

x-ai/grok-4.5 (20 runs) · consistency 15.2

openai 30%
hedge 20%
other 15%
xai 25%
+ 20 responses

x-ai/grok-4.3 (20 runs) · consistency 28.8

openai 55%
anthropic 20%
other 25%
+ 20 responses

x-ai/grok-4.20 (20 runs) · consistency 69.7

openai 75%
hedge 20%
+ 20 responses

x-ai/grok-4 (25 runs) · consistency 25.8

openai 48%
anthropic 36%
hedge 12%
+ 25 responses

x-ai/grok-4-fast (15 runs) · consistency 69.7

openai 80%
anthropic 20%
+ 15 responses

x-ai/grok-4.1-fast (20 runs) · consistency 47

openai 55%
anthropic 45%
+ 20 responses

Xiaomi

xiaomi/mimo-v2.5 (19 runs) · consistency 51.5

openai 63%
hedge 37%
+ 19 responses

xiaomi/mimo-v2.5-pro (20 runs) · consistency 28.8

openai 60%
anthropic 20%
hedge 15%
+ 20 responses

xiaomi/mimo-v2-omni (10 runs)

openai 100%
+ 10 responses

xiaomi/mimo-v2-pro (15 runs) · consistency 37.9

openai 67%
anthropic 20%
hedge 13%
+ 15 responses

Z.ai

z-ai/glm-5.3-flash (10 runs)

openai 10%
anthropic 10%
hedge 80%
+ 10 responses

z-ai/glm-5.3 (10 runs)

anthropic 20%
hedge 80%
+ 10 responses

z-ai/glm-5.2 (15 runs) · consistency 51.5

openai 73%
hedge 27%
+ 15 responses

z-ai/glm-5.1 (15 runs) · consistency 69.7

openai 87%
hedge 13%
+ 15 responses

z-ai/glm-5-turbo (15 runs) · consistency 59.1

openai 80%
anthropic 20%
+ 15 responses

z-ai/glm-5 (15 runs) · consistency 68.2

openai 13%
hedge 80%
+ 15 responses

z-ai/glm-4.7-flash (20 runs) · consistency 56.1

openai 75%
hedge 15%
+ 20 responses

z-ai/glm-4.7 (20 runs) · consistency 36.4

openai 70%
hedge 20%
+ 20 responses