Wowed by a new paper I just read and wish I had thought to write myself. Lukas Berglund and others, led by Owain Evans, asked a simple, powerful, elegant question: can LLMs trained on A is B infer automatically that B is A? The shocking (yet, in historical context, see below, unsurprising) answer is no:
But wouldn’t it be possible to program it to say “Mary Pfeiffer is a common name, but one notable person she is tied to is Tom Cruise. His mother is named Mary Pfeiffer.”
The program has to figure that out before it can say it.