AI got a lot better in recent years because of transformers and the multi-head attention mechanism. Some more major breakthroughs will be made with a bit more time that will greatly boost accuracy and efficiency. Technology progresses exponentially so advances will get more and more common. One example of how LLMs are currently getting more efficient is DeepSeek’s improvement of sparse attention. There are trillions of dollars being thrown at these problems and they’re racing as fast as they can because there’s an arms race as well as heavy market competition.
AI got a lot better in recent years because of transformers and the multi-head attention mechanism. Some more major breakthroughs will be made with a bit more time that will greatly boost accuracy and efficiency. Technology progresses exponentially so advances will get more and more common. One example of how LLMs are currently getting more efficient is DeepSeek’s improvement of sparse attention. There are trillions of dollars being thrown at these problems and they’re racing as fast as they can because there’s an arms race as well as heavy market competition.