Creator (@AbhinavSrinivas) seems to be inactive. I'm wondering what should happen if multiple companies share rank 1, since that is now possible on LMArena. My reading is that the company with the highest arena score (Elo) should win, just like this question. Do others read it this way as well?

@yetforever yes that seems most reasonable

Similar markets w/ arbitrage opportunities:

https://manifold.markets/Soli/who-will-have-the-best-llm-at-the-e

https://manifold.markets/Soli/who-will-have-the-best-llm-by-end-o

How will this market resolve if Meta trains the best LLM, open sources it, and then Mistral finetunes it for a small to moderate amount of additional tokens which makes it perform somewhat better than the original Meta model at the end of 2024?

(As seems to be what's going on with mistral-medium.)

@RyanGreenblatt I think I’ll run a poll to see what people want. Or maybe I’ll resolve both as yes.

Related:

Best in what regard?

@mattyb Good question. I’m mostly thinking using this leaderboard: https://chat.lmsys.org/ (go to the leaderboard tab). If you have any other suggestions then let me know.

@AbhinavSrinivas Ok, by the Arena Elo leaderboard. Thanks