Once you’ve trained your large language model on the entire written output of humanity, where do you go? Here’s Ilya Sutskever, ex-OpenAI, admitting to Reuters that they’ve plateaued: [Reuters] The…

I’m usually the one saying “AI is already as good as it’s gonna get, for a long while.”

This article, in contrast, is quotes from folks making the next AI generation - saying the same.

  • ChicoSuave@lemmy.worldEnglish
    7·
    55 minutes ago

    I understand folks don’t like AI but this “article” is like a reddit post with lots of links to subjects which are vague and need the link text to tell us what is important, instead of relying on the actual article.

    • 11111one11111@lemmy.worldEnglish
      1·
      15 minutes ago

      What the fuck you aren’t kidding. I have comment replies to trolls that are longer than that article. The over the top citations also makes me think this was entirely written by an actual AI bot that was lrompted to supply x amoint of sources in their article. Lol

  • Greg Clarke@lemmy.caEnglish
    111·
    1 hour ago

    OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further

    Lol, no they didn’t. The quotes this articles are using are talking about LLMs not chatbots. This is yet another stupid article from someone who doesn’t understand the technology. There is a lot of legitimate criticism for the way this technology is being implemented but FFS get the basics right at least.

  • cron@feddit.orgEnglish
    27·
    2 hours ago

    It’s absurd that some of the larger LLMs now use hundreds of billions of parameters (e.g. llama3.1 with 405B).

    This doesn’t really seem like a smart usage of ressources if you need several of the largest GPUs available to even run one conversation.

      • blackbelt352@lemmy.worldEnglish
        2·
        20 minutes ago

        It’s a lot. Like a lot a lot. GPUs have about 150 billion transistors but those transistors only make 1 connection in what is essentially printed in a 2d space on silicon.

        Each neuron makes dozens of connections, and there’s on the order of almost 100 billion neurons in a blobby lump of fat and neurons that takes up 3d space. And then combine the fact that multiple neurons in patterns firing is how everything actually functions and you have such absurdly high number of potential for how powerful human brains are.

        At this point, I’m not sure there’s enough gpus in the world to mimic what a human brain can do.

      • cron@feddit.orgEnglish
        21·
        1 hour ago

        I don’t think your brain can be reasonably compared with an LLM, just like it can’t be compared with a calculator.

  • Lvxferre@mander.xyzEnglish
    1·
    26 minutes ago

    I believe that the current LLM paradigm is a technological dead end. We might see a few additional applications popping up, in the near future; but they’ll be only a tiny fraction of what was promised.

    My bet is that they’ll get superseded by models with hard-coded logic. Just enough to be able to correctly output “if X and Y are true/false, then Z is false”, without fine-tuning or other band-aid solutions.

  • Ragdoll X@lemmy.worldEnglish
    3·
    1 hour ago

    It’s a known problem - though of course, because these companies are trying to push AI into everything and oversell it to build hype and please investors, they usually try to avoid recognizing its limitations.

    Frankly I think that now they should focus on making these models smaller and more efficient instead of just throwing more compute at the wall, and actually train them to completion so they’ll generalize properly and be more useful.

  • hendrik@palaver.p3x.deEnglish
    41·
    1 hour ago

    Though, I don’t think that means they won’t get any better. It just means they don’t scale by feeding in more training data. But that’s why OpenAI changed their approach and added some reasoning abilities. And we’re developing/researching things like multimodality etc… There’s still quite some room for improvements.