I'll probably lock it at some point.So, below is a little animation I put together. It shows GPT-1, 2, and 3, with varying exponential trend lines depending on GPT-4.
For those who might be new to this forum, and/or unfamiliar with AI language models, GPT is a series of programs that uses deep learning to create human-like text. Think of them as like very, very advanced forms of smartphone autocorrect. The most recent of these, GPT-3, emerged in June 2020 and has demonstrated phenomenal capabilities, with dialogue that often seems like a real person.
This is largely thanks to its massive number of parameters, which can be thought of as like the individual synapses in a brain. GPT-3 has orders of magnitude more parameters than its predecessors. The program isn't perfect, however.
An even more advanced version, GPT-4, is strongly rumoured to be releasing this year. Estimates of the parameter count vary wildly – ranging from those who believe it won't be much larger than the 175 billion of GPT-3, to those who predict another huge leap with perhaps 100 trillion or more.
Rather like the "megahertz myth" of the early 2000s, and the qubit claims of D-wave Systems, it may be that we're reaching a point where large gains in parameter counts don't actually matter as much. Perhaps other factors will now be more important in determining how productive and efficient these language models are.
Anyway, the exact number of parameters that will feature in GPT-4 is publicly unknown, but I wanted to get the opinions of FT forumers. What do you think it will be? And does it even matter that much? Please vote in the poll I'm about to create, thanks!
)and now we have ChatGPT.
