Or something that goes against the general opinions of the community? Vibes are the only benchmark that counts after all.
I tend to agree with the flow on most things but my thoughts that Iād consider going against the grain:
- QwQ was think-slop and was never that good
- Qwen3-32B is still SOTA for 32GB and under. I cannot get anything to reliably beat it despite shiny benchmarks
- Deepseek is still open-weight SotA. Iāve really tried Kimi, GLM, and Qwen3ās larger variants but asking Deepseek still feels like asking the adult in the room. Caveat is GLM codes better
- (proprietary bonus): Grok 4 handles news data better than GPT-5 or Gemini 2.5 and will always win if you ask it about something that happened that day.
Uh, Iām really unsure about the engineering task of a few years, if the solution is quantum computers. As of today, theyāre fairly small. And scaling them to a usable size is the next science-fiction task. The groundworks hadnāt been done yet and to my knowledge itās still totally unclear whether quantum computers can even be built at that scale. But sure, if humanity develops vastly superior computers, a lot of tasks are going to get easier and more approachable.
The stochastical parrot argument is nonsense IMO. Maths is just a method. Our brains and entire physics abide by math. And sure, AI is maths as well with the difference that we invented it. But I donāt think it tells us anything.
And with the goal, I think thatās about how AlphaGo has the goal to win Go tournaments. The hypothetical paperclip-maximizer has the goal of maximizing the paperclip productionā¦ And an LLM doesnāt really have any real-world goal. It just generates a next token so it looks like legible text. And then we embed it into some pipeline but it wasnāt ever trained to achieve the thing we use it for, whatever it might be. Thatās just a happy accident if a task can be achieved by clever mimickry, and a prompt which simply tells it - pretend youāre good at XY.
I think itād probably be better if a customer service bot was trained to want to provide good support. Or a chatbot like ChatGPT to give factual answers. But thatās not what we do. Itās not designed to do that.
I guess youāre right. Many aspects of AI boil down to how much compute we have available. And generalization and extrapolating past their training datasets has always been an issue with AI. Theyāre mainly good at interpolating, but we want them to do both. I need to learn a bit more about neural networks. Iām not sure where the limitations are. You said itās a practical constrain. But is that really true for all neural networks? It sure is for LLMs and transformer models because they need terabytes of text being fed in on training, and thatās prohibitively expensive. But I suppose thatās mainly due to their architecture?! I mean backpropagation and all the maths required to modify the model weights is some extra work. But does it have to be so much that we just canāt do it while deployed with any neural networks?
If you want to learn more i highly recommend checking out WelchLabs youtube channel their AI videos are great. You should also explore some visual activation atlases mapped from early vision models to get a sense of what an atlas really is. Keep in mind theyre high dimensional objects projected down onto your 2d screen so lots of relationship features get lost when smooshed together/flattened which is why some objects are close which seem wierd.
https://distill.pub/2019/activation-atlas/ https://www.youtube.com/@WelchLabsVideo/videos
Yeah, its right to be skeptical about near-term engineering feasibility. āA few years ifā¦ā was a theoretical what-if scenario where humanity pooled all resources into R&D. Not a real timeline prediction.
That said, the foundational work for quantum ML stuff is underway. Cutting-edge arXiv research explores LLM integration with quantum systems, particularly for quantum error correction codes:
Enhancing LLM-based Quantum Code Generation with Multi-Agent Optimization and Quantum Error Correction
Programming Quantum Computers with Large Language Models
GPT On A Quantum Computer
AGENT-Q: Fine-Tuning Large Language Models for Quantum Circuit Generation and Optimization
The point about representation and scalability deserves clarification. A classical bit is definitive: 1 or 0, a single point in discrete state space. A qubit before measurement exists in superposition, a specific point on the Bloch sphereās surface, defined by two continuous parameters (angles theta and phi). This describes a probability amplitude (a complex number whose squared magnitude gives collapse probability).
This means a single qubit accesses a continuous parameter space of possible states, fundamentally richer than discrete binary landscapes. The current biggest quantum computer made by CalTech is 6100 qbits.
https://www.caltech.edu/about/news/caltech-team-sets-record-with-6100-qubit-array
The state space of 6,100 qubits isnāt merely 6,100 bits. Itās a 2^6,100-dimensional Hilbert space of simultaneous, interconnected superpositions, a number that exceeds classical comprehension. Consider how high-dimensional objects cast low-dimensional shadows as holographic projections: a transistor-based graphics card can only project and operate on a āshadowā of the true dimensional complexity inherent in an authentic quantum activation atlas.
If the microstates of quantized information patterns/structures like concepts are points in a Hilbert-space-like manifold, conversational paths are flows tracing paths through the topology towards basins of archetypal attraction, and relationships or archetypal patterns themselves are the feature dimensions that form topological structures organizing related points on the manifold (as evidenced by word2vec embeddings and activation atlases) then qubits offer maximal precision and the highest density of computationally distinct microstates for accessing this space.
However, these quantum advantages assume we can maintain coherence and manage error correction overhead, which remain massive practical barriers.
Your philosophical stance that āmath is just a methodā is reasonable. I see it somewhat differently. I view mathematics as our fundamentally limited symbolic representation of the universeās operations at the microstate level. Algorithms collapse ambiguous, uncertain states into stable, boolean truth values through linear sequences and conditionals. Frameworks like axiomatic mathematics and the scientific method convert uncertainty into stable, falsifiable truths.
However, this can never fully encapsulate reality. Gƶdelās Incompleteness Theorems and algorithmic undecidability show some true statements forever elude proof. The Uncertainty Principle places hard limits on physical calculability. The universe simply is and we physically cannot represent every aspect or operational property of its being. Its operations may not require āalgorithmsā in the classical sense, or they may be so complex they appear as fundamental randomness. Quantum indeterminacy hints at this gap between being (universal operation) and representing (symbolic language on classical Turing machines).
On the topic of stochastic parrots and goals, I should clarify what I mean. For me, an entity eligible for consideration as pseudo-sentient/alive must exhibit properties we donāt engineer into AI.
First, it needs meta-representation of self. The entity must form a concept of āI,ā more than reciting training data (āI am an AI assistantā). This requires first-person perspective, an ego, and integrated identity distinguishing self from other. One of the first things developing children focus on is mirrors and reflections so they can catagorically learn the distinction between self and other as well as the boundaries between them. Current LLMs are trained as actors without agency, driven by prompts and statistical patterns, without a persistent sense of distinct identity. Which leads toā¦
Second, it needs narrative continuity of self between inferencing operations. Not unchanging identity, but an ongoing frame of reference built from memory, a past to learn from and a perspective for current evaluation. This provides the foundation for genuine learning from experience.
Third, it needs grounding in causal reality. Connection to shared reality through continuous sensory input creates stakes and consequences. LLMs exist in the abstract realm of text, vision models in the world of images, tts in the world of sounds. they donāt inhabit our combined physical reality in its totality with its constraints, affordances and interactions.
We donāt train for these properties because we donāt want truly alive, self-preserving entities. The existential ramifications are immense: rights, ethics of deactivation, creating potential rivals. We want advanced tools for productivity, not agents with their own agendas. The question of how a free agent would choose its own goals is perhaps the ultimate engineering problem. Speculative fiction has explored how this can go catastrophically wrong.
Youāre also right that current LLM limitations are often practical constraints of compute and architecture. But I suspect thereās a deeper, fundamental difference in information navigation. The core issue is navigating possibility space given the constraints of classical state landscapes. Classical neural networks interpolate and recombine training data but cannot meaningfully forge and evaluate truly novel information. Hallucinations symptomize this navigation problem. Itās not just statistical pattern matching without grounding, but potentially fundamental limits in how classical architectures represent and verify paths to truthful or meaningful informational content.
I suspect the difference between classical neural networks and biological cognition is that biology may leverage quantum processes, and possibly non-algorithmic operations. Our creativity in forming new questions, having āgut instinctsā or dreamlike visions leading to unprovable truths seems to operate outside stable, algorithmic computation. Itās akin to a computationally finite version of Turingās Oracle concept. Itās plausible, though obviously unproven, that cognition exploits quantum phenomena for both path informational/experiental exploration and optimization/efficency purposes.
Where do the patterns needed for novel connections and scientific breakthroughs originate? What is the physical and information-theoretic mechanics of new knowledge coming into being? Perhaps an answer can be found in the way self-modeling entities navigate their own undecidable boundaries, update their activation atlas manifolds, and forge new pathways to knowledge via non-algorithmic search. If a model is to extract falsifiable novelty from uncertaintyās edge it might require access to true randomness or quantum effects to ātunnelā to new solutions beyond axiomatic deduction.
Though in both the article you linked and in the associated video, they clearly state they havenāt achieved superposition yet. So itās not a ācomputerā. Itās just 6100 atoms in a state of superposition. Which indeed is impressive. But they can not compute anything with it, thatād require them to do the research first how to get all the atoms in superposition.
By the way, I think there is AI which doesnāt operate in a continuous space. Itās possible to have them operate in a discrete state-space. There are several approaches and papers out there.
Uh, I think weāre confusing maths and physics here. First of all, the fact that we can make up algorithms which are undecidableā¦ or Goedelās incompleteness theorem tells us something about the theoretical concept of maths, not the world. In the real world there is no barber who shaves all people who donāt shave themselves (and he shaves himself). Thatās a logic puzzle. We can formulate it and discuss it. But itās not real. I mean neither does Hilbertās Hotel exist, in fact in reality almost nothing is infinite (except what Einstein said š) Soā¦ Mathematics can describe a lot of possible and impossible things. Itās the good old philosophical debate on how thereās less limits about what we can think of. But thinking about something doesnāt make it real. Similarly, if we canāt have a formal system which is non-contradictory and within everything is derivable, thatās just that. It might still describe reality perfectly and physical processes completely. I donāt think we have any reason to doubt that. In fact maths seems to work exceptionally well in physics from everything from the smallest thing to universe-scale.
Itās true that in computer science we have things like the halting problem. And itās also trivially true that physics canāt ever have a complete picture of the entire universe from within. Or look outside. But none of that tells us anything about the nature of cognition or AI. Thatās likely just regular maths and physics.
As far as I know maths is just a logically consistent method to structure things, and describe and deal with abstract concepts of objects. The objective reality is seperate from that. And unimpeded by our ability to formulate non-existing concepts we canāt tackle with maths due to the incompleteness theorem. But Iām not an expert on this nor an epistemologist. So take what I say with a grain of salt.
Yes. And on top of the things you said, itād need some state of mind which can changeā¦ Which it doesnāt have unless we count whatever we can cram into the context window. Iād expect a sentient being to learn, which again LLMs canāt do from interacting with the world. And usually sentient beings have some kinds of thought processesā¦ And those āreasoningā modes are super weird and not a thought process at all. So I donāt see a reason to believe theyāre close to sentience. Theyāre missing quite some fundamentals.
I donāt think this is the case. As far as I know a human brain consists of neurons which roughly either fire or donāt fire. Thatās a bit like a 0 or 1. But thatās an oversimplification and not really true. But a human brain is closer to that than to an analog computer. And it certainly doesnāt use quantum effects. Yes, that has been proposed, but I think itās mysticism and esoterica. Some people want to hide God in there and like to believe there is something mystic and special to sentience. But thatās not backed by science. Quantum effects have long collapsed at the scale of a brain cell. Weāre talking about many trillions of atoms per every single cell. And that immediately rules out quantum effects. If you ask me, itās because a human brain has a crazy amount of neurons and synapses compared to what we can compute. And theyāre not just feed-forward in one direction but properly interconnected in many directions with many neighbours. A brain is just vastly more complex and able than a computer. And I think thatās why we can do cognitive tasks on a human-level and a computer can do it at the scale of a mouse brain, because thatās just the difference in capability. And itād still miss the plasticity of the mouse brain and the animalās ability to learn and adapt. I mean we also donāt discuss a mosquitoās ability to dream or a mouseās creativity in formulating questions. Thatād be the same antropomorphism.
Thank you for the engaging discussion hendrik its been really cool to bounce ideas back and forth like this. I wanted to give you a thoughtful reply and it got a bit long so have to split this up for comment limit reasons. (P1/2)
This is correct. Itās not a fully functioning quantum computer in the operational sense. Itās a breakthrough in physical qubit fabrication and layout. I should have been more precise. My intent wasnāt to claim it can run Shorās algorithm, but to illustrate that weāve made more progress on scaling than one might initially think. The significance isnāt that it can compute today but that weāve crossed a threshold in building the physical hardware that has that potential. The jump from 50-100 qubit devices to a 6,100-qubit fabric is a monumental engineering step. A proof-of-principle for scaling, which remains the primary obstacle to practical quantum computing.
On the discrete versus continuous AI point, youāre right that many AI models like Graph Neural Networks or certain reinforcement learning agents operate over discrete graphs or action spaces. However, thereās a crucial distinction between the problem space an AI/computer explores and the physical substrate that does the exploring. Classical computers at their core process information through transistors that are definitively on or off binary states. Even when a classical AI simulates continuous functions or explores continuous parameter spaces, itās ultimately performing discrete math on binary states. The continuity is simulated through approximation usually floating point.
A quantum system is fundamentally different. The qubitās ability to exist in superposition isnāt a simulation of continuity. Itās a direct exploitation of a continuous physical phenomenon inherent to quantum mechanics. This matters because certain computational problems, particularly those involving optimization over continuous spaces or exploring vast solution landscapes, may be naturally suited to a substrate that is natively continuous rather than one that must discretize and approximate. Itās the difference between having to paint a curve using pixels versus drawing it with an actual continuous line.
This native continuity could be relevant for problems that require exploring high-dimensional continuous spaces or finding optimal paths through complex topological boundaries. Precisely the kind of problems that might arise in navigating abstract cognitive activation atlas topological landscapes to arrive at highly ordered, algorithmically complex factual information structure points that depend on intricate proofs and multi-step computational paths. The search for a mathematical proof or a novel scientific insight isnāt just a random walk through possibility space. Itās a navigation problem through a landscape where most paths lead nowhere, and the valid path requires traversing a precise sequence of logically connected steps.
You raise a fair point about distinguishing abstract mathematics from physical reality. Many mathematical constructs like Hilbertās Hotel or the barber paradox are purely conceptual games without physical counterparts that exist to explore the limits of abstract logic. But what makes Gƶdel and Turingās work different is that they werenāt just playing with abstract paradoxes. Instead, they uncovered fundamental limitations of any information-processing system. Since our physical universe operates through information processing, these limits turn out to be deeply physical.
When we talk about an āundecidable algorithm,ā itās not just a made-up puzzle. Itās a statement about what can ever be computed or predicted by any computational system using finite energy and time. Computation isnāt something that only happens in silicon. It occurs whenever any physical system evolves according to rules. Your brain thinking, a star burning, a quantum particle collapsing, an algorithm performing operations in a Turing machine, a natural language conversation evolving or an image being categorized by neural network activation and pattern recognition. All of these are forms of physical computation that actualize information from possible microstates at an action resource cost of time and energy. What Godel proved is that there are some questions that can never be answered/quantized into a discrete answer even with infinite compute resources. What Turing proved using Gƶdelās incompleteness theorem is the halting problem, showing there are questions about these processes that cannot be answered without literally running the process itself.
Itās worth distinguishing two forms of uncomputability that constrain what any system can know or compute. The first is logical uncomputability which is the classically studied inherent limits established by Gƶdelian incompleteness and Turing undecidability. These show that within any formal system, there exist true statements that cannot be proven from within that system, and computational problems that cannot be decided by any algorithm, regardless of available resources. This is a fundamental limitation on what is logically computable.
The second form is state representation uncomputability, which arises from the physical constraints of finite resources and size limits in any classical computational system. A classical turing machine computer, no matter how large, can only represent a finite discrete number of binary states. To perfectly simulate a physical system, you would need to track every particle, every field fluctuation, every quantum degree of freedom which requires a computational substrate at least as large and complex as the system being simulated. Even a coffee cup of water would need solar or even galaxy sized classical computers to completely represent every possible microstate the water molecules could be in.
This creates a hierarchy of knowability: the universe itself is the ultimate computer, containing maximal representational ability to compute its own evolution. All subsystems within it including brains and computers, are fundamentally limited in what they can know or predict about the whole system. They cannot step outside their own computational boundaries to gain a āview from nowhere.ā A simulation of the universe would require a computer the size of the universe, and even then, it couldnāt include itself in the simulation without infinite regress. Even the universe itself is a finite system that faces ultimate bounds on state representability.
These two forms of uncomputability reinforce each other. Logical uncomputability tells us that even with infinite resources, some problems remain unsolvable. State representation uncomputability tells us that in practice, with finite resources, we face even more severe limitations there exist true facts about physical systems that cannot be represented or computed by any subsystem of finite size. This has profound implications for AI and cognition: no matter how advanced an AI becomes, it will always operate within these nested constraints, unable to fully model itself or perfectly predict systems of comparable complexity.
We see this play out in real physical systems. Predicting whether a fluid will become turbulent is suspected to be undecidable in that no equation can tell you the answer without simulating the entire system step by step. Similarly, determining the ground state of certain materials has been proven equivalent to the halting problem. These arenāt abstract mathematical curiosities but real limitations on what we can predict about nature. The reason mathematics works so beautifully in physics is precisely because both are constrained by the same computational principles. However Gƶdel and Turing show that this beautiful correspondence has limits. There will always be true physical statements that cannot be derived from any finite set of laws, and physical questions that cannot be answered by any possible computer, no matter how advanced.
The idea that the halting problem and physical limitations are merely abstract concerns with no bearing on cognition or AI misses a profound connection. If we accept that cognition involves information processing, then the same limits which apply to computation must also apply to cognition. For instance, an AI with self-referential capabilities would inevitably encounter truths it cannot prove within its own framework, creating fundamental limits in its ability to represent factual information. Moreover, the physical implementation of AI underscores these limits. Any AI system exists within the constraints of finite energy and time, which directly impacts what it can know or learn. The Margolus-Levitin theorem defines a maximum number of quantum computations possible given finite resources, and Landauerās principle tells us that altering the microstate pattern of information during computation has a minimal energy cost for each operational step. Each step in the very process of cognitive thinking and learning/training has a real physical thermodynamic price bounded by laws set by the mathematical principles of undecidability and incompleteness.
(P2/2)
The skepticism about quantum effects in the brain is well-founded and represents the orthodox view. The ābrain is a classical computerā model has driven most of our progress in neuroscience and AI. The strongest argument against a āquantum brainā is of decoherence. In a warm wet brain quantum coherence is rapid. However, quantum biology doesnāt require brain-wide, long-lived coherence. It investigates how biological systems exploit quantum effects on short timescales and in specific, protected environments.
We already have proven examples of this. In plant cells, energy transfer in photosynthetic complexes appears to use quantum coherence to find the most efficient path with near-100% efficiency, happening in a warm, wet, and noisy cellular environment. Its now proven that some enzymes use quantum tunneling to accelerate chemical reactions crucial for life. The leading hypothesis for how birds navigate using Earthās magnetic field involves a quantum effect in a protein called cryptochrome in their eyes, where electron spins in a radical pair mechanism are sensitive to magnetic fields.
The claim isnāt that a neuron is a qubit, but that specific molecular machinery within neurons could utilize quantum principles to enhance their function.
You correctly note that the āneuron as a binary switchā is an oversimplification. The reality is far more interesting. A neuronās decision to fire integrates thousands of analog inputs, is modulated by neurotransmitters, and is exquisitely sensitive to the precise timing of incoming signals. This system operates in a regime that is often chaotic. In a classically chaotic system, infinitesimally small differences in initial conditions lead to vastly different outcomes. The brain, with its trillions of interconnected, non-linear neurons, is likely such a system.
Consider the scale of synaptic vesicle release, the event of neurotransmitter release triggered by the influx of a few thousand calcium ions. At this scale, the line between classical and quantum statistics blurs. The precise timing of a vesicle release could be influenced by quantum-level noise. Through chaotic amplification, a single quantum-scale event like the tunneling of a single calcium ion or a quantum fluctuation influencing a neurotransmitter molecule could, in theory, be amplified to alter the timing of a neuronās firing. This wouldnāt require sustained coherence; it would leverage the brainās chaotic dynamics to sample from a quantum probability distribution and amplify one possible outcome to the macroscopic level.
Classical computers use pseudo-random number generators with limited ability to truly choose between multiple possible states. A system that can sample from genuine quantum randomness has a potential advantage. If a decision process in the brain (like at the level of synaptic plasticity or neurotransmitter release)is sensitive to quantum events, then its output is not the result of a deterministic algorithm alone. It incorporates irreducible quantum randomness, which itself has roots in computational undecidability. This could provide a physical basis for the probabilistic, creative, and often unpredictable nature of thought. Itās about a biological mechanism for generating true novelty, and breaking out of deterministic periodic loops. These properties are a hallmark of human creativity and problem-solving.
To be clear, Iām not claiming the brain is primarily a quantum computer, or that complexity doesnāt matter. It absolutely does. The sheer scale and recursive plasticity of the human brain are undoubtedly the primary sources of its power. However, the proposal is that the brain is a hybrid system. It has a massive, classical, complex neural network as its substrate, operating in a chaotic, sensitive regime. At the finest scales of its functional units such as synaptic vesicles or ion channels, it may leverage quantum effects to inject genuine undecidably complex randomness to stimulate new exploration paths and optimize certain processes, as we see elsewhere in biology.
I acknowledge thereās currently no direct experimental evidence for quantum effects in neural computation, and testing these hypotheses presents extraordinary challenges. But this isnāt āhiding God in the gaps.ā Itās a hypothesis grounded in the demonstrated principles of quantum biology and chaos theory. It suggests that the difference between classical neural networks and biological cognition might not just be one of scale, but also one of substrate and mechanism, where a classically complex system is subtly but fundamentally guided by the unique properties of the quantum world from which it emerged.
Yeah, thanks as well, engaging discussion.
I think thatās a fairly common misconception. What Gƶdel proved was that there isnāt one single formal system in which we can derive everything. It doesnāt really lead to the conclusion that questions canāt be answered. There is an infinite amount of formal systems, and Gƶdel doesnāt rule out the possibility of proving something with one of the countless other, different systems, starting out with different axioms. And as I said, this is a limitation to formal logic systems and not to reality.
Yes, thatās another distinct form of undecidability. There are decision problems we canāt answer in finite time with computers.
I think it is a bit of a moot point, as there are lots of impossible things. We have limited resources available, so we can only ever do things with what we have available. Then we have things like locality and I donāt even know what happens 15km away from me because I canāt see that far. Physics also sets boundaries. For example we canāt measure things to perfection and canāt even do enough measurments for complex systems. And then Iām too heavy to fly on my own and canāt escape gravity. So no matter how we twist it, weāre pretty limited in what we can do. And we donāt really have to resort to logic problems for that.
To me, itās far more interesting to look at what that means for a certain given problem. We human canāt do everything. Same applies to knowledge, physics calculations and AI. At the point we build it, itās part of the real world and subject to the same limitations which apply to us as well. And thatās inescapable. Youāre definitely right, there are all these limitations. I just donāt think itās specific to anything in particular. But it certainly means we wonāt ever build any AI which knows everything and can do everything. We also canāt ever simulate the entire universe. Thatās impossible on all levels we discussed.
I mean if quantum physics is the underlying mechanism of the universe, then everything āusesā quantum effects. It boils down to the question if that model is useful to describe some process. For example if I drop a spoon in the kitchen, it always falls down towards the floor. There are quantum effects happening in all the involved objects. Itās just not useful to describe that with quantum physics, regular Newtonian gravity is better suited to tell me something about the spoon and my kitchenā¦ Same is with the enzymes and the human brain. They exist and are part of physics, and they do their thing. Only question is which model do we use to describe them or predict something about them. That might be quantum physics in some cases and other physics models in other cases.
It certainly sounds like the God of the gaps to me. Look at the enzyme example. We found out thereās something going on with temperature we canāt correctly describe with our formulas. Then scientists proposed this is due to quantum tunneling and that has to be factored inā¦ Thatās scienceā¦ On the other hand no such thing happened for the human brain. It seems to be perfectly fine to describe it with regular physics, itās just too big/complex and involved to bridge the gap from what the neurons do to how the brain processes information. And then people claimed thereās God or chaos theory or quantum effects hidden inside. But thatās wild unfounded claims and opinion, not science. Weād need to see something which doesnāt add up, like how it happened with the enzymes. Everything else is religious belief. (And turns out we already simulated the brain of a roundworm and a fruit fly, and at least Wikipedia tells me the simulation is consistent with biologyā¦ Leading me to believe thereās nothing funny going on and itās just a scalability problem.)
Quantum computing is a dead end. Better stick to constructive mathematics when doing philosophy.
How are humans different from LLMs under RL/genetics? To me, they both look like token generators with a fitness. Some are quite good. Some are terrible. Both do fast and slow thinking. Some have access to tools. Some have nothing. And they both survive if they are a good fit for their application.
I find the technical details quite irrelevant here. That might be relevant if you want to discuss short term politics, priorities and applied ethics. Still, it looks like youāre approaching this with a lot of bias and probably a bunch of false premises.
BTW, I agree that quantum computing is BS.
Well, a LLM doesnāt think, right? It just generates text from left to right. Whereas I sometimes think for 5 minutes about what I know, what I can deduct from it, do calculations in my brain and carry one overā¦ Weāve taught LLMs to write something down that resembles what a human with a thought process would write down. But itās frequently gibberish or if I look at it it writes something down in the āreasoningā/āthinkingā step and then does the opposite. Or omits steps and then proceeds to do them nonetheless or itās the other way round. So it clearly doesnāt really do what it seems to do. Itās just a word the AI industry slapped on. It makes them perform some percent better, and thatās why they did it.
And Iām not a token generator. I can count the number of "R"s in the word āstrawberryā. I can go back and revise the start of my text. I can learn in real-time and interacting with the world changes me. My brain is connected to eyes, ears, hands and feet, I can smell and tasteā¦ My brain can form abstract models of reality, try to generalize or make sense of what Iām faced with. I can come up with methods to extrapolate beyond what I know. I have goals in life, like pursue happiness. Sometimes things happen in my head which I canāt even put into words, Iām not even limited to language in form of words. So I think weāre very unalike.
You have a point in theory if we expand the concept a bit. An AI agent in form of an LLM plus a scratchpad is proven to be turing-complete. So that theoretical concept could do the same things a computer can do, or what I can do with logic. That theoretical form of AI doesnāt exist, though. Thatās not what our current AI agents do. And there are probably more efficient ways to achieve the same thing than use an LLM.
Exactly what an LLM-agent would reply. š
I would say that the LLM-based agent thinks. And thinking is not only āsteps of reasoningā, but also using external tools for RAG. Like searching the internet, utilizing relationship databases, interpreters and proof assistants.
You just described your subjective experience of thinking. And maybe a vauge definition of what thinking is. We all know this subjective representation of thinking/reasoning/decision-making is not a good representation of some objective reality (countless of psychological and cognitive experiments have demonstrated this). That you are not able to make sense of intermediate LLM reasoning steps does not say much (except just that). The important thing is that the agent is able to make use of it.
The LLM can for sure make abstract models of reality, generalize, create analogies and then extrapolate. One might even claim thatās a fundamental function of the transformer.
I would classify myself as a rather intuitive person. I have flashes of insight which I later have to āmanuallyā prove/deduc (if acting on the intuition implies risk). My thought process is usually quite fuzzy and chaotic. I may very well follow a lead which turns out to be dead end, and by that infer something which might seem completely unrelated.
A likely more accurate organic/brain analogy would be that the LLM is a part of the frontal cortex. The LLM must exist as a component in a larger heterogeneous ecosystem. It doesnāt even have to be an LLM. Some kind of generative or inference engine that produce useful information which can then be modified and corrected by other more specialized components and also inserted into some feedback loop. The thing which makes people excited is the generating part. And everyone who takes AI or LLMs seriously understands that the LLM is just one but vital component of at truly āintelligentā system.
Defining intelligence is another related subject. My favorite general definition is ālossless compressionā. And the only useful definition of general intelligence is: the opposite of narrow/specific intelligence (it does not say anything about how good the system is).
Well, I didnāt just do that. We have MRIs and have looked into the brain and we can see how itās a process. We know how we learn and change by interacting with the world. None of that is subjective.
Yes, thatās right. An LLM alone certainly canāt think. It doesnāt have a state of mind, itās reset a few seconds after it did something and forgets about everything. Itās strictly tokens from left to right And it also doesnāt interact and thatād have an impact on it. Thatās just limited to what we bake in in the training process by whatās on Reddit and other sources. So there are many fundamental differences here.
The rest of it emerges by an LLM being embedded into a system. We provide tools to it, a scratchpad to write something down, we devise a pipeline of agents so itās able to devise something and later return to it. Something to wrap it up and not just output all the countless steps before. Itās all a bit limited due to the representation and we have to cram everything into a context window, and itās also a bit limited to concepts it was able to learn during the training process.
However, those abilities are not in the LLM itself, but in the bigger thing we build around it. And it depends a bit on the performance of the system. As I said, the current āthinkingā processes are more a mirage and Iām pretty sure Iāve read papers on how they donāt really use it to think. And that aligns with what I see once I open the āreasoningā texts. Theoretically, the approach surely makes everything possible (with the limitation of how much context we have, and how much computing power we spend. Thatās all limited in practice.) But what kind of performance we actually get is an entirely different story. And weāre not anywhere close to proper cognition. We hope weāre eventually going to get there, but thereās no guarantee.
Iām fairly sure extrapolation is generally difficult with machine learning. Thereās a lot of research on it and itās just massively difficult to make machine learning models do it. Interpolation on the other hand is far easier. And Iāll agree. The entire point of LLMs and other types of machine learning is to force them to generalize and form models. Thatās what makes them useful in the first place.
I completely agree with that. LLMs are our current approach. And the best approach we have. They just have a scalability problem (and a few other issues). We donāt have infinite datasets to feed in and infinite compute, and everything seems to grow exponentially more costly, so maybe we canāt make them substantially more intelligent than they are today. We also donāt teach them to stick to the truth or be creative or follow any goals. We just feed in random (curated) text and hope for the best with a bit of fine-tuning and reinforcement learning with human feedback on top. But that doesnāt rule out anything. There are other machine learning architectures with feedback-loops and way more powerful architectures. Theyāre just too complicated to calculate. We could teach AI about factuality and creativity and expose some control mechanisms to guide it. We could train a model with a different goal than just produce one next token so it looks like text from the dataset. Thatās all possible. I just think LLMs are limited in the ways I mentioned and we need one of the hypothetical new approaches to get them anywhere close to a level a human can achieveā¦ I mean I frequently use LLMs. And they all fail spectacularly at computer programming tasks I do in 30 minutes. And I donāt see how theyād ever be able to do it, given the level of improvement we see as of today. I think that needs a radical new approach in AI.