Will a flagship (>60T training bytes) open-weights LLM from Meta which doesn't use a tokenizer be released in 2025?
Plus
4
Ṁ3492026
43%
chance
1D
1W
1M
ALL
Resolves YES if Meta releases weights of an LLM trained on at least 60T bytes of data (roughly equivalent to the 15T tokens used to train the Llama 3.1 models) in 2025 which does not use standard fixed-vocabulary tokenization.
A qualifying model must be released under a license roughly as permissive as Llama 3.1.
This market was spurred by recent research from Meta showing a proof-of-concept for a tokenizer-free LLM. A qualifying model from Meta does not need to use the patching technique from this paper as long as it's not using tokenization.
https://ai.meta.com/research/publications/byte-latent-transformer-patches-scale-better-than-tokens/
This question is managed and resolved by Manifold.
Get
1,000
and3.00
Related questions
Related questions
When will OpenAI release a more capable LLM?
Will Meta ever deploy its best LLM without releasing its model weights up through AGI?
79% chance
Will the next major LLM by OpenAI use a new tokenizer?
77% chance
How much FLOP will be used to train the best language model with freely available weights on July 1, 2025?
Will OpenAI reveal a textless LLM before 2025?
12% chance
Will anyone train a TokenFormer model at scale before 2026?
25% chance
Will OpenAI release a tokenizer with more than 210000 tokens before 2026?
24% chance
Will OpenAI release a tokenizer with vocab size > 150k by end of 2024?
42% chance
Will researchers extract a novel program from the weights of an LLM into a Procedural/OO programming language by 2026?
26% chance
Will Meta AI's MEGABYTE architecture be used in the next-gen LLMs?
42% chance