This bullshit propaganda is intensely frustrating. Stop pretending these models are magic. This propaganda is meant to mystify and fascinate idiots.
Here is a short reading list that will explain to a competent person how generative transformers work.
https://jaykmody.com/blog/gpt-from-scratch/#fn1
https://explainextended.com/2023/12/31/happy-new-year-15/
https://jalammar.github.io/how-gpt3-works-visualizations-animations/
https://arxiv.org/abs/1706.03762
Enough pretend-along. They aren’t smarter than you.
The GPT architecture is well understood, the part that is hard to explain is the way information is encoded in the trained model’s parameters. It’s not magic, it’s just a highly opaque encoding.
Good on you for saying it so eloquently. None of them would have dared to say anything remotely similar to ‘we don’t know’ if it really was the case
I’m not the least bit surprised the CEO has no idea how these things work. But if the CEO not understanding something makes it magic, that also makes computers, cell phones, cars, the weather, healthy relationships, and being a decent person magic as well.
Remind me of the old saying about them ancient rotary dials phones, how no one in the world knows how to build one fully from scratch.
You made great analogies and very true
Anthropic is getting left in the dust.
