The LLMentalist Effect: how chat-based Large Language Models rep…
softwarecrisis.dev
The new era of tech seems to be built on superstitious behaviour
  • Lvxferre [he/him]@mander.xyzEnglish
    11·
    10 days ago

    I’ve read this already, also from a HN link. It’s a good piece to explain why people believe that large language models reached some sort of intelligence, instead of being glorified text generators. (Sometimes a glorified text generator is damn useful, but we shouldn’t pretend that it’s something else.)

    I’ll list here a few fallacious arguments that you see being used to defend that LLMs are intelligent, as well as some rather grumpy answers.

    You can't prove that it is not intelligent!

    Fallacy: shifting the burden of the proof. The one who claims the existence of an entity or an attribute for an entity is the one who needs to prove it, not people calling it bullshit.

    Grumpy answer: you can’t prove that your grandma didn’t catch syphilis in Hell, by sharing a dragon dildo with Hitler.

    But humans...

    Whataboutism, rather close to a tu quoque. Yes, humans sometimes brainfart, and this doesn’t say shit about LLM lack of ability to reason properly.

    Grumpy answer: you’re a prime example of humans not being able to reason.

    Wut's you are definishun of intallijanse?

    A weird mix of ad nauseam, shifting the goalposts, red herring; the effect here is that, instead of people discuss what LLMs do, they spend lines and lines discussing definitions.

    Grumpy answer: “Intelligence” can be define as the attribute you lack.

    I don't know if we aren't ourselves glorified text generators.

    Appeal to ignorance. Roughly: since you don’t know it (or claim to not to), you assume that it’s false.

    Grumpy answer: based on your answer, you probably are, but most people aren’t.

    Trust me.

    Not exactly a fallacy, but a superficially polite way to rephrase “I expect you to be gullible trash, so behave as such.”

    Grump answer: quoting the user but replacing the words “trust me” with “be gullible”.

    You must have something against LLMs.

    Ad hominem. And no, ad hominem does not need to involve insults; it boils down to “let’s not discuss this, let’s discuss why you’re saying this”.

    Grump answer: “Stop being an assumer. Also the subject is LLMs, not me”.

    Now let me see the HN comments. Given the subject it’s a shitfest, but I’m low-key in the mood for popcorn.

    This is from 2023 and is clearly dated. It is mildly interesting to notice how quickly things changed since then. Nowadays models can solve original math puzzles much of the time and it is harder to argue they cannot reason when we have access to R1, o1, and o3-mini.

    It is not really dated. People obsess over the metric (in this case, maths puzzles) without considering what the metric is supposed to gauge (ability to reason). Even DeepSeek and GPT o4-mini are still vomiting brainfarts here and there, and both are from my experience the best costless models there. (“Costless” for the user. Not for the environment. I just don’t want to use the word “free” as it’s ambiguous here.)

    I was hoping it was talking about how it can resonate with users using those techniques. Or some experiments to prove the point. But it is not even that. // There is nothing of substance in this and it feels like the author has a grudge against LLMs.

    Refer to “You must have something against LLMs.”

    Plus typical “Redditors LARP as 5UP4 1337 H4x0rz” behaviour - “waah, it’s missing [thing]!”

    Turns out that identifying problems and solving them are two different skills, and people who are good with one aren’t necessarily good with the other. So it’s completely fine to point out a problem (e.g. suckers believing LLMs are smart) without necessarily providing a solution.

    This feels rather forced. The article seems to claim both that LLMs don’t actually work, it is all an illusion and that of course the LLMs know everything, they stole all our work from the last 20 years by scraping the internet and underpaying people to produce content. If it was a con, it wouldn’t have to do that. Or in other words, if you had a psychic who actually memorized all biographies of all people ever, they wouldn’t need their cons

    What doesn’t actually work is the reading comprehension of this user.

    You should try the arc agi puzzles yourself, and then tell me you think these things aren’t intelligent / https://arcprize.org/blog/openai-o1-results-arc-prize / I wouldn’t say it’s full agi or anything yet, but these things can definitely think in a very broad sense of the word

    Goodhart’s Law.