Will Matthew and Tamay win their bet on longer AI timelines by 2026?

Plus

Ṁ754

2026

chance

ALL

From https://www.lesswrong.com/posts/X3p8mxE5dHYDZNxCm/a-concrete-bet-offer-to-those-with-short-ai-timelines @MatthewBarnett and Tamay believe that AI timelines are longer than expected. Specifically, they were offering "1:1 odds that no more than one of the following events will occur by 2026-01-01: - A model/ensemble of models achieves >80% on all tasks in the MMLU benchmark - A credible estimate reveals that an AI lab deployed EITHER >10^30 FLOPs OR hardware that would cost $1bn if purchased through competitive cloud computing vendors at the time on a training run to develop a single ML model (excluding autonomous driving efforts) - A model/ensemble of models will achieve >90% on the MATH dataset using a no-calculator rule - A model/ensemble of models achieves >80% top-1 strict accuracy on competition-level problems on the APPS benchmark - A gold medal for the IMO Grand Challenge (conditional on it being clear that the questions were not in the training set) - A robot that can, from beginning to end, reliably wash dishes, take them out of an ordinary dishwasher and stack them into a cabinet, without breaking any dishes, and at a comparable speed to humans (<120% the average time) - Tesla’s full-self-driving capability makes fewer than one major mistake per 100,000 miles" Resolves YES if they win their bet (no more than one of the above occurs by the end of 2025).

This question is managed and resolved by Manifold.

#AI

#Technical AI Timelines

Get

1,000

and

3.00

1 Comment

15 Holders

196 Trades

Sort by:

I just wrote a retrospective on the bet. I think it's highly likely that we'll lose, unless something weird happens, like people stop publishing on all the benchmarks that we used for the bet. I think what happened is that (1) my AI timelines were simply too long, and (2) I wasn't thinking very carefully about the milestones, and didn't realize that extrapolating past trends reveals that we are expected to lose unless progress slows down by a lot.

Related questions

Related questions