Is GPT-3 worth all the media attention?
developed by OpenAI
has been all the rage in the AI-related news field for the last few months. It has transcended the tech-savvy target group, and now its capabilities are known to a wider audience thanks to endless articles describing its capabilities. In case you’ve missed them, here’s a short and concise overview
of the articles/resources that are worth checking out in order to understand why GPT-3 has become the most notorious language model
(besides the fact that OpenAI has a very good PR team):
🤔 If you want to get an AI’s perspective on what is the meaning of life
, it is now possible. You can also read GPT-3’s thoughts on other philosophical questions and topics.
🧙♂️ GPT-3 can code simple neural networks
after being given a short description (in a natural language) of what these networks should be doing. So GPT has created models in its own image... after being told to do so by its own creators.
💬 A blog written by GPT-3
attracted 26.000 visitors in 2 weeks and the only person who noticed something suspicious about the posts was downvoted by other users.
🤖 The Guardian
has published GPT-3’s attempt to convince us (humans) that robots don't want to eradicate the human race.
👨💻 Want to be a cool entrepreneur but you lack original startup ideas
? Don’t worry, GPT-3 has got your back. In case you want to reach out to potential investors via e-mail
, it is again at your service.
I think it is truly impressive what the GPT-3 is capable of. Having personally worked with the GPT-2 (previous iteration of the model released last year), the difference in text quality and coherence is significant. No wonder, to be honest, since GPT-3 has 100x
the amount of parameters that the GPT-2 had - 1.5 vs 175 billion
However, the weirdness of language use makes it evident that such models still can’t really comprehend language as humans are able to. It is still just a number crunching statistical model that attempts to mimic the distribution of words in a text as close to real texts as possible. Even the CEO of OpenAI himself admits
that the GPT-3 is overhyped
Nevertheless, it’s not about what the GPT can do right now in its 3rd iteration. It’s about what it will be able to showcase in its 10th or 15th iteration
. This is what fascinates me the most - if GPT continues improving at the same rate that it does right now (which is a fairly optimistic assumption), imagine what it will be capable of in just a few years
. Geoffrey Hinton, “the godfather of deep learning”, said it best:Extrapolating the spectacular performance of GPT
-3 into the future suggests that the answer to life, the universe and everything is just 4.398 trillion parameters.