Full question: "billion US dollars will be spent to train a new state-of-the-art AI system by Jan 1st 2030"

Resolution Criteria:

I will find the best reasonable reporting on this topic and use whatever number is most widely agreed upon. If none are available, I will use what best word-of-mouth estimates I can find. If multiple models perform similarly, I will pick the cheapest one. If a model is continuously trained, I will use the total cost of training up to the point of evaluation. If a model is trained on AI-generated data, I will include the cost of generating that data (if it is known).

Motivation and Context:

Current AI systems are being trained with exponentially larger training runs. The best estimate of the cost to train GPT-2 in 2019 was a few tens of thousands of dollars. For GPT-4 in 2022, the cost was estimated to be in the tens of millions of dollars. Will this trend continue? Will we continue to see exponentially larger training runs at this exponential rate? Or will the cost of training eventually plateau?

Question copied from: https://nicholas.carlini.com/writing/2024/forecasting-ai-future.html