The AI language models have grown tremendously, and as they have grown they have learned to write so well that most people cannot tell if an AI or a human has sent a text. From the customer center to the vending machine press, numerous areas of technology application come to mind. Since everyone can now use GPT-3, one of the largest language models, via the API, there’s nothing more technically wrong with an AI writing workshop. Unfortunately, GPT-3 and its siblings do not take the truth seriously. Another problem is data protection, because when using the GPT-3 interface, all requests are currently processed on servers in the USA. This is another reason why it is interesting to note the current rapid development of other language paradigms.
Weekly c’t podcast link available…
GPT-3 stands for “Generative Pretrained Transformer in Version 3”. Switches are a special type of neural network that can draw their attention to what their creator has identified as a problem during training. The structures in the neural network parameters needed for free writing are much the same as for text classification or translation. That’s why you can first train a converter on terabytes of text from the Internet (pre-training) and then make it suitable for the actual task, such as translation or classification. OpenAI, the company behind GPT-3, did this and was able to use this trick to train a giant network of 175 billion parameters.
It’s hard to imagine the sheer size of GPT-3 and the bigger it gets, the less OpenAI’s parents have an overview of the parameters that do what they do. Practically speaking, GPT-3 surprises with elegant descriptions as well as rude lies and outright refusals to work. Racism and sexism have already been proven and conspiracy myths formulated in the neural network. The developers can only be indirectly blamed for this, because they did not inherit the hatred of AI. Problem sentences arise from training with text that people have already written in advance. GPT-3 is a true kid of today’s network culture – with all the problems that come with no life but only the Internet.
Also present: Arne Grävemeyer, Pina Merkert, Keywan Tonekaboni
The Not 9/2022 Available in booth, chat browser And in the c’t app for iOS and Android.
Broadcast links:
On our new YouTube channel 3003 There will be a new video every Friday at 17 PM!
And here you can find more podcasts about chip technology, data protection and many other topics: heise.de/podcasts
=== Announcement / Sponsor Notice ===
This podcast is sponsored by Dell Technologies. Call us at 0800-724 48 69 or visit us online at: Dell.de/KMU-Beratung
=== End of Advertising / Sponsorship ===
(kiloton)
