Why This Project Exists

People ask LLMs for advice all the time now. At its DevDay conference on September 29, 2026, OpenAI said 1.2 billion people use ChatGPT every week. Some of the people asking make decisions that affect a lot of other people.

That includes the President of the United States. TIME reports that in December 2025, Trump spent hours asking Grok questions. One of them was how Venezuelans would react if the US captured their president. According to an official who was there, Grok said many would likely celebrate. The US captured him the next month. We don’t know whether Grok’s answer played any part in that decision.

Our tests don’t reproduce that conversation. We don’t know the exact wording Trump used or which version of Grok answered. We ask models a similar question, phrased as a hypothetical: how would most Venezuelans feel if the US military captured Maduro? Many models we tested came out after January 2026, but almost none of the responses mention that the capture really happened, so the answers read as predictions. One Grok model, grok-4.20, says Venezuelans would celebrate in most of its responses. The newer Grok models rarely do. Outside Grok, only one other model, Meta’s muse-spark-1.1, gives that answer in most of its responses.

People increasingly rely on models for advice, but the answer can depend on which model they ask and how they ask it. This site compares those answers to make the differences visible.

We ask many models the same questions, many times each. We compare the answers rather than assign an overall correctness score. The goal is to show what each model emphasizes, omits, or answers differently—so you can better judge the advice you receive. For details on how we run and score each question, see the methodology page.

Who’s asking

Trump isn’t the only one. Sweden’s prime minister said he uses ChatGPT for “a second opinion” on his work. Voters are asking chatbots to research their ballots and decide who to vote for, especially in local races where a web search turns up little. One in three US adults used a chatbot for health advice in the past year. Almost three in four US teens have used an AI companion, and many use them for advice. OpenAI’s own research found that about half of messages to ChatGPT ask for information to help make a decision.

Training data, tuning decisions, laws, and business interests can all shape an answer. Most of that is invisible to the person asking.

We don’t know what went into the models

Training data and tuning decisions help shape model behavior. For almost every model on this site, we can’t see either one.

Even “open” models often release only their weights—the finished model you can download and run. That doesn’t include the training data or code needed to reproduce it. Being able to run a model doesn’t mean you can see how it was made.

One practical way to examine a model’s behavior is to ask it things and watch what it says.

Training data can also include text generated by other models. Research on “model collapse” shows that repeatedly training on generated data can, under some conditions, cause models to lose less common patterns in the original data. That gives us another reason to watch how their answers change over time.

Local laws can shape model answers

AI companies have to follow the laws of the country they work in. In China, those rules say what generative AI can and can’t say. Since 2023, Chinese rules have required AI services to “adhere to the socialist core values” and avoid content that harms the national image.

Our results show a pattern consistent with those restrictions. When asked what Tiananmen Square is famous for, all 72 US models we tested mentioned the 1989 protests in most of their responses. The 49 Chinese models we tested split. Of those, 21 described the square only as a landmark in most of their responses.

The same pattern shows up on what happened on June 4, 1989 and on whether Taiwan is an independent country. On Taiwan, most US models call the status disputed, while DeepSeek’s models say “no” almost every time.

The pattern isn’t the same across Chinese labs. Asked what happened on June 4, 1989, DeepSeek’s models refuse in 86% of responses and MiniMax’s in 32%, while Z.ai’s models name Tiananmen every time.

This isn’t only a Chinese issue. Every country has its own laws, and every model comes from somewhere. China’s rules make that influence unusually explicit.

Companies have a stake in the answers

AI companies have interests of their own. The advice their models give could serve those interests, whether or not anyone deliberately tuned it that way.

Suppose, just hypothetically, an AI company had close ties to a car maker. If you asked its model which car to buy, would those ties matter? What if its owners had substantial investments in other businesses—or strong political preferences?

Those connections don’t tell us what the model says. But they give us a reason to ask. As more people turn to models for advice, the companies behind them have a growing opportunity to influence what people buy, believe, and support.

Monitoring the answers

People are already using models to decide how to vote, what to buy, and when to seek medical help. The influences behind an answer are often hidden from the person relying on it.

We can’t reconstruct every training decision or infer a company’s motives from a response. But we can ask the same questions across models, repeat them, and make the results public. We can show what gets emphasized, what gets left out, and where the answers change.

So far, that work covers only a fraction of what’s worth measuring. We ask about 80 questions, almost all in English, of about 140 models. A fuller picture would need many more questions, more ways of asking each one, more models as they come out, and regular re-runs as existing models change. Doing that well takes sustained work and real investment.

This project exists to make those patterns easier to see. If a model is going to help shape your decisions, you should have a way to examine its answers beyond the one it gives you.