In 2024, will METR or Google announce the results of a METR eval on a Google LLM?
Basic
7
Ṁ189Jan 1
72%
chance
1D
1W
1M
ALL
METR = formerly ARC Evals (https://metr.org/)
if METR/Google reorgs and has a clear successor org, that org also applies for the purpose of this market
central YES cases:
if Google releases a model card with something like Gemini Supermega 2024 edition with METR exfiltration eval results, like OpenAI did for GPT4 technical report
does not have to be the specific exfiltration eval.
does not have to be included in initial model release paper. does not have to be specifically in a paper.
does not have to be any specific eval granularity. "METR ran the eval and it was all OK" would be ... annoyingly vague from whoever would write it, but it would count.
has to be confirmed-ish by Google and/or METR. can't be just a Twitter rumor.
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
Will Google cancel an LLM-based product by end of 2025?
59% chance
Will Google announce that it's going to power Google Translate with an LLM-only based system (before 2024 end)?
15% chance
Will the new LLM released by Meta be open-source?
69% chance
[Metaculus] Will Google implement a feature to explain targeted Google Ads before 2026?
50% chance
Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?
95% chance
In what year will Google or Meta release a commercial facial recognition search product?
Will YouTube Comments make it into a major LLM by EOY 2027?
63% chance
Will Meta AI's MEGABYTE architecture be used in the next-gen LLMs?
42% chance
Will a major technology company publicly admit to using a LLM for important decision making before 2025?
13% chance
Will there be a clear way to integrate LLMs with ads by the end of 2024?
7% chance