What will be the best score (5/5 reliability) on ZeroBench by December 31st 2025?

Ṁ74

2026

21%

0 - 10

13%

11 - 20

21 - 30

31 - 40

41 - 50

51 - 60

61 - 70

71 - 80

81 - 90

91 - 100

ZeroBench is a benchmark for visual reasoning, introduced by Roberts et al. in "ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models" (https://arxiv.org/abs/2502.09696).

This market will use the variant of the benchmark frozen one week after the initial release (following the public benchmark red-teaming stage to identify flawed/ambiguous questions).

The temperature used for the 5/5 reliability evaluation will be the default setting provided by each LLM API provider. In cases where this default is ambiguous to determine, we will default to a temperature of 0.7.

This question is managed and resolved by Manifold.

#AI

#️ Technology

#World

#Technical AI Timelines

#AI Benchmarks

Get

1,000

and

3.00

1 Comment

3 Holders

9 Trades

Sort by:

1mo

As of March 28th 2025, Gemini 2.5 Pro is the new SotA: https://x.com/JRobertsAI/status/1905577784300183653

3% pass@1

5% pass@5

1% 5/5 reliability

What is this?

What is Manifold?

Manifold is the world's largest social prediction market.

Get accurate real-time odds on politics, tech, sports, and more.

Win cash prizes for your predictions on our sweepstakes markets! Always free to play. No purchase necessary.

Are our predictions accurate?

Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.

In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like trading still use Manifold to get reliable news.

How do I win cash prizes?

Manifold offers two market types: play money and sweepstakes.

All questions include a play money market which uses mana and can't be cashed out.

Selected markets will have a sweepstakes toggle. These require sweepcash to participate and winners can withdraw sweepcash as a cash prize. You can filter for sweepstakes markets on the browse page.

Redeem your sweepcash won from markets at 1.00
→ $1.00, minus a 5% fee.

Learn more.

What is this?

What is Manifold?

Manifold is the world's largest social prediction market.

Get accurate real-time odds on politics, tech, sports, and more.

Win cash prizes for your predictions on our sweepstakes markets! Always free to play. No purchase necessary.

Are our predictions accurate?

Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.

How do I win cash prizes?

Manifold offers two market types: play money and sweepstakes.

All questions include a play money market which uses mana and can't be cashed out.

Selected markets will have a sweepstakes toggle. These require sweepcash to participate and winners can withdraw sweepcash as a cash prize. You can filter for sweepstakes markets on the browse page.

Redeem your sweepcash won from markets at 1.00
→ $1.00, minus a 5% fee.

Learn more.

What is this?

Related questions

What is this?

Related questions