Here's a thought. Lets all arbitrarily agree AGI is here. I can't even be bothered discussing what the definition of AGI is. It's just here, accept it. Or vice versa.
Now what....? Whats happening right now that should make me care that AGI is here (or not). Whats the magic thing thats happening with AGI that wasn't happening before?
<looks out of window>
<checks news websites>
<checks social media...briefly>
<asks wife>
Right, so, not much has changed from 1-2 years ago that I can tell. The job markets a bit shit if you're in software...is that what we get for billions of dollars spent?
I worry that if the reality lives up to investors dreams it will be massively disruptive for society which will lead us down dark paths. On the other hand if it _doesn't_ live up to their dreams, then there is so much invested in that dream financially that it will lead to massive societal disruption when the public is left holding the bag, which will also lead us down dark paths.
I think corporations can definitely transform society in the near future. I don't think it will be a positive transformation, but it will be a transformation.
Most of all, AI will exacerbate the lack of trust in people and institutions that was kicked into high gear by the internet. It will be easy and cheap to convince large numbers of people about almost anything.
We were pretty close to a collapse of the existing financial system. Maybe we’d be better off now if it happened, but the interim devastation would have been costly.
yeah, this is a good point, transition and transformation to new technologies takes time. I'm not sure I agree the current state is upending things though. It's forcing some adaption for sure, but the status quo remains.
It took one September. Then as soon as you could take payments on the internet the rest was inevitable and in _clear_ demand. People got on long waiting lists just to get the technology in their homes.
> no new advances in technology
The reason the internet became so accessible is because Moore was generally correct. There was two corresponding exponential processes that vastly changed the available rate of adoption. This wasn't at all like cars being introduced into society. This was a monumental shift.
I see no advances in LLMs that suggest any form of the same exponential processes exist. In fact the inverse is true. They're not reducing power budgets fast enough to even imagine that they're anywhere near AGI, and even if they were, that they'd ever be able to sustainably power it.
> the current state is upending jobs
The difference is companies fought _against_ the internet because it was so disruptive to their business model. This is quite the opposite. We don't have a labor crisis, we have a retention crisis, because companies do not want to pay fair value for labor. We can wax on and off about technology, and perceptrons, and training techniques, or power budgets, but this fundamental fact seems the hardest to ignore.
If they're wrong this all collapses. If I'm wrong I can learn how to write prompts in a week.
It's the classic "slowly, then suddenly" paradigm. It took decades to get to that one September. Then years more before we all had internet in our pocket.
> The reason the internet became so accessible is because Moore was generally correct.
Can you explain how Moore's law is relevant to the rise of the internet? People didn't start buying couches online because their home computer lacked sufficient compute power.
> I see no advances in LLMs that suggest any form of the same exponential processes exist.
LLMs have seen enormous growth in power over the last 3 years. Nothing else comes close. I think they'll continue to get better, but critically: even if LLMs stay exactly as powerful as they are today, it's enough to disrupt society. IMHO we're already at AGI.
> The difference is companies fought _against_ the internet
Some did, some didn't. As in any cultural shift, there were winners and losers. In this shift, too, there will be winner and losers. The panicked spending on data centers right now is a symptom of the desire to be on the right side of that.
> because companies do not want to pay fair value for labor.
Companies have never wanted to pay fair value for labor. That's a fundamental attribute of companies, arising as a consequence of the system of incentives provided in capitalism. In the past, there have been opportunities for labor to fight back: government regulation, unions. This time that won't help.
> If I'm wrong I can learn how to write prompts in a week.
Why would you think that anyone would want you to write prompts?
Allow me to clarify: I'm not wishing for change. I am an AI pessimist. I think our society is not prepared to deal with what's about to happen. You're right: AI is the key to the enshitification of everything, most of all trust.
Governments and companies have been pushing for identity management that connects your real life identity with your digital one for quite some time. With AI I believe that's not only a bad thing, maybe unavoidable now.
> Here's a thought. Lets all arbitrarily agree AGI is here.
A slightly different angle on this - perhaps AGI doesn't matter (or perhaps not in the ways that we think).
LLMs have changed a lot in software in the last 1-2 years (indeed, the last 1-2 months); I don't think it's a wild extrapolation to see that'll come to many domains very soon.
I think you are missing the point: If we assume that AGI is *not* yet here, but may be here soon, what will change when it arrives? Those changes could be big enough to affect you.
I'm missing the point? I literally asked the same thing you did.
>Now what....? Whats happening right now that should make me care that AGI is here (or not).
Do you have any insight into what those changes might concretely be? Or are you just trying to instil fear in people who lack critical thinking skills?
One could argue that's a cynically accurate definition of most iterative development anyway.
But I don't know that I accept the core assertion. If the engineer is screening the output and using the LLM to generate tests, chances are pretty good it's not going to be worse than human-generated tech debt. If there's more accumulated, it's because there's more output in general.
I've been writing code for 20 years. AI has completely changed my life and the way I write code and run my business. Nothing is the same anymore, and I feel I will be saying that again by the end of 2026. My productive output as a programmer in software and business have expanded 3x *compounding monthly*.
Okay. So software engineers are vastly more efficient. Good I guess. "Revolutionize the entire world such that we rethink society down to its very basics like money and ownership" doesn't follow from that.
Damn right I'm impatient. My eye starts twitching when a web page takes more than 2 seconds to load :-)
In the meantime, I've had to continuously hear talk about AI, both in real life (like at the local pub) AND virtually (tv/radio/news/whatever) and how it's going to change the world in unimaginable ways for the last...2/3 years. Billions upon billions of dollars are being spent. The only tangible thing we have to show is software development, and some other fairly niche jobs, have changed _a bit_.
So yeah, excuse my impatience for the bubble to burst, I can stop having to hear about this shit every day, and I can go about my job using the new tools we have been gifted, while still doing all the other jobs that sadly do not benefit in any similar way.
> The only tangible thing we have to show is software development, and some other fairly niche jobs, have changed _a bit_.
There is zero evidence that LLMs have changed software development efficiency.
We get an earth-shattering developer productivity gamechanger every five years. All of them make wild claims, none of them ever have any data to back those claims up.
LLMs are just another in a long, long list. This too will pass. (Give it five years for the next gamechanger.)
"I’m not a mechanical engineer, but I watched a five-minute YouTube video on how a diesel engine works, so I can tell you that mechanical engineering is a solved problem."
> The transformer architectures powering current LLMs are strictly feed-forward.
This is true in a specific contextual sense (each token that an LLM produces is from a feed-forward pass). But untrue for more than a year with reasoning models, who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully.
Heck, it was untrue before that as well, any time an LLM responded with more than one token.
> A [March] 2025 survey by the Association for the Advancement of Artificial Intelligence (AAAI), surveying 475 AI researchers, found that 76% believe scaling up current AI approaches to achieve AGI is "unlikely" or "very unlikely" to succeed.
I dunno. This survey publication was from nearly a year ago, so the survey itself is probably more than a year old. That puts us at Sonnet 3.7. The gap between that and present day is tremendous.
I am not skilled enough to say this tactfully, but: expert opinions can be the slowest to update on the news that their specific domain may have, in hindsight, have been the wrong horse. It's the quote about it being difficult to believe something that your income requires to be false, but instead of income it can be your whole legacy or self concept. Way worse.
> My take is that research taste is going to rely heavily on the short-duration cognitive primitives that the ARC highlights but the METR metric does not capture.
I don't have an opinion on this, but I'd like to hear more about this take.
Thanks for reading, and I really appreciate your comments!
> who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully
Ah, this is a great point, and not something that I considered. I agree that the token feedback does change the complexity, and it seems that there's even a paper by the same authors about this very thing! https://arxiv.org/abs/2310.07923
I'll have to think on how that changes things. I think it does take the wind out of the architecture argument as it's currently stated, or at least makes it a lot more challenging. I'll consider myself a victim of media hype on this, as I was pretty sold on this line of argument after reading this article https://www.wired.com/story/ai-agents-math-doesnt-add-up/ and the paper https://arxiv.org/pdf/2507.07505 ... who brush this off with:
>Can the additional think tokens provide the necessary complexity to correctly
solve a problem of higher complexity? We don't believe so, for two fundamental reasons: one that
the base operation in these reasoning LLMs still carries the complexity discussed above, and the
computation needed to correctly carry out that very step can be one of a higher complexity (ref our
examples above), and secondly, the token budget for reasoning steps is far smaller than what
would be necessary to carry out many complex tasks.
In hindsight, this doesn't really address the challenge.
My immediate next thought is - even solutions up to P can be represented within the model / CoT, do we actually feel like we are moving towards generalized solutions, or that the solution space is navigable through reinforcement learning? I'm genuinely not sure about where I stand on this.
> I don't have an opinion on this, but I'd like to hear more about this take.
It's general-purpose enough to do web development. How far can you get from writing programs and seeing if you get the answers you intended? If English words are "grounded" by programming, system administration, and browsing websites, is that good enough?
> expert opinions can be the slowest to update on the news that their specific domain may have, in hindsight, have been the wrong horse. It's the quote about it being difficult to believe something that your income requires to be false, but instead of income it can be your whole legacy or self concept
Not sure I follow. Are you saying that AI researchers would be out of a job if scaling up transformers leads to AGI? How? Or am I misunderstanding your point.
I don't know about AGI but I got bored and ran my plans for a new garage by Opus 4.6 and it was giving me some really surprising responses that have changed my plans a little. At the same time, it was also making some nonsense suggestions that no person would realistically make. When I prompted it for something in another chat which required genuine creativity, it fell flat on its face.
I dunno, mixed bag. Value is positive if you can sort the wheat from the chaff for the use cases I've ran by it. I expect the main place it'll shine for the near and medium term is going over huge data sets or big projects and flagging things for review by humans.
I've used it recently to flesh out a fully fledged business plan, pricing models, capacity planning & logistics for a 10 year period for a transport company (daily bus route). I already had most of it in my mind and on spreadsheets already (was an old plan that I wanted to revive), but seeing it figure out all the smaller details that would make or break it was amazing! I think MBA's should be worried as it did some things more comprehensive than an MBA would have done. It was like a had an MBA + Actuarial Scientist + Statistics + Domain Expert + HR/Accounting all in one. And the plan was put into a .md file that has enough structure to flesh out a backend and an app.
Yeah it's really impressed me on occasion, but often in the same prompt output it just does something totally nonsensical. For my garage/shop, it generated an SVG of the proposed floor plan, taking care to place the sink away from moisture sensitive material and certain work stations close to each other for work flow, etc. it even routed plumbing and electrical...But it also arranged the work stations cramped together at the two narrow ends of the structure (such that they'd be impractical to actually work at) and ignored all the free wall space along the long axis so that literally most of the space was unused. It was also concerned about things that were non issues like contamination between certain stations, and had trouble when I explicitly told it something about station placement and it just couldn't seem to internalize it and kept putting it in the wrong place.
All this being said, what I was throwing at it was really not what it was optimized for, and it still delivered some really good ideas.
Now that understanding video and projecting what happens next indicates we're getting past the LLM problem of lacking a world model. That's encouraging.
There's more than one way to do intelligence. Basic intelligence has evolved independently three times that we know of - mammals, corvids, and octopuses. All three show at least ape-level intelligence, but the species split before intelligence developed, and the brain architectures are quite different. Corvids get more done with less brain mass than mammals, and don't have a mammalian-type cortex. Octopuses have a distributed brain architecture, and have a more efficient eye design than mammals.
I've recently come to the understanding that LLMs don't have intelligence in any way. They have language, which in humans is a downstream product of intelligence. But thats all they have. There's no little being sitting at the center of the Chinese room. Trying to classify LLMs as intelligent is going upstream and doesn't work.
I don't think those are examples of unique intelligence except perhaps in a chauvinistic, anthropomorphic sense. We only know that we can't get other animals to display patterns we associate with intelligence in humans, however truthfully that's just as likely to be that our measures of intelligence don't map cleanly onto cognitive/perceptual representations innate to other animals. As we look for new ways to challenge animals that respect their innate differences, we're finding "simple" organisms like ants and spiders are surprisingly capable.
For a clear analogy, consider how tokenization causes LLMs to behave stupidly in certain cases, even though they're very capable in others.
I don't think they have ideas, so I don't think they're intelligent in the sense relevant to AGI. The list of intelligent animals is constantly increasing because doing some feat or other suffices for the animal to qualify. Solving mazes (slime molds), recognizing self in mirror (not dogs). Playing, using tools, reacting appropriately to words, transmitting habits down the generations (the closest thing they have to ideas). This is all imagined to be the precursors along the path to evolving intelligence, which conjures up a future world of complex crow and octopus material cultures. There's no reason to assume they're on such a path. Really all we're saying is that they seem clever. We've already made AI that seems clever, so the animals aren't a relevant example of anything.
It's weird that this sentence has two distinct meanings and the author never considers the second or points it out. Maybe Mary is holding a ball for her society friends.
The skepticism surrounding AGI often feels like an attempt to judge a car by its inability to eat grass. We treat "cognitive primitives" like object constancy and causality as if they are mystical, hardwired biological modules, but they are essentially just high-dimensional labels for invariant relationships within a physical manifold. Object constancy is not a pre-installed software patch; it is the emergent realization of spatial-temporal symmetry. Likewise, causality is nothing more than the naming of a persistent, high-weight correlation between events. When a system can synthesize enough data at a high enough dimension, these so-called "foundational" laws dissolve into simple statistical invariants. There is no "causality" module in the brain, only a massive correlation engine that has been fine-tuned by evolution to prioritize specific patterns for survival.
The critique that Transformers are limited by their "one-shot" feed-forward nature also misses the point of their architectural efficiency. Human brains rely on recurrence and internal feedback loops largely as a workaround for our embarrassingly small working memory—we can barely juggle ten concepts at once without a pen and paper. AI doesn't need to mimic our slow, vibrating neural signals when its global attention can process a massive, parallelized workspace in a single pass. This "all-at-once" calculation of relationships is fundamentally more powerful than the biological need to loop signals until they stabilize into a "thought."
Furthermore, the obsession with "fragility"—where a model solves quantum mechanics but fails a child’s riddle—is a red herring. Humans aren't nearly as "general" as we tell ourselves; we are also pattern-matchers prone to optical illusions and simple logic traps, regardless of our IQ. Demanding that AI replicate the specific evolutionary path of a human child is a form of biological narcissism. If a machine can out-calculate us across a hundred variables where we can only handle five, its "non-human" way of knowing is a feature, not a bug. Functional replacement has never required biological mimicry; the jet engine didn't need to flap its wings to redefine flight.
AGI is here it's just stupider than you thought it would be. Nobody really said how intelligent it would be. If it's generally stupid and smart in a few areas that's enough.
There was a meme going around that said the fall of Rome was an unannounced anticlimactic event where one day someone went out and the bridge wasn't ever repaired.
Maybe AGI's arrival is when one day someone is given an AI to supervise instead of a new employee.
Just a user who's followed the whole mess, not a researcher. I wonder if the scaffolding and bolt-ons like reasoning will sufficiently be an asymptote to 'true AGI'. I kept reading about the limits of transformers around GPT-4 and Opus 3 time, and then those seem basic compared to today.
I gave up trying to guess when the diminishing returns will truly hit, if ever, but I do think some threshold has been passed where the frontier models are doing "white collar work as an API" and basic reasoning better than the humans in many cases, and once capital familiarizes themselves with this idea more, it's going to get interesting.
But it's already like that; models are better than many workers, and I'm supervising agents. I'd rather have the model than numerous juniors; esp. the kind that can't identify the model's mistakes.
The problem becomes your retirement. Sure, you've earned "expert" status, but all the junior developers won't be hired, so they'll never learn from junior mistakes. They'll blindly trust agents and not know deeper techniques.
Can I rephrase this as "you can get experience without any experience"? Certainly, there's stuff you can learn that's adjacent to doing the thing; that's what happens when juniors graduate with CS degrees. But the lack of doing the thing is what makes them juniors.
This is my greatest cause for alarm regarding LLM adoption. I am not yet sure AI will ever be good enough to use without experts watching them carefully; but they are certainly good enough that non-experts cannot tell the difference.
From my experience, if you think AI is better than most workers, you're probably just generating a whole bunch of semi-working garbage, accepting that input as good enough and will likely learn the hardware your software is full of bugs and incorrect logic.
I don't think this is how it'll play out, and I'm generally a bit skeptical of the 'agent' paradigm per se.
There doesn't seem to be a reason why AIs should act as these distinct entities that manage each other or form teams or whatever.
It seems to me way more likely that everything will just be done internally in one monolithic model. The AIs just don't have the constraints that humans have in terms of time management, priority management, social order, all the rest of it that makes teams of individuals the only workable system.
AI simply scales with the compute resources made available, so it seems like you'd just size those resources appropriately for a problem, maybe even on demand, and have a singluar AI entity (if it's even meaningful to think of it as such, even that's kind of an anthropomorphisation) just do the thing. No real need for any organisational structure beyond that.
So I'd think maybe the opposite, seems like what agents really means is a way to use fundamentally narrow/limited AI inside our existing human organisations and workflows, directed by humans. Maybe AGI is when all that goes away because it's just obviously not necessary any more.
I used to also believe along these lines but lately I'm not so sure.
I'm honestly shocked by the latest results we're seeing with Gemini 3 Deep Think, Opus 4.6, and Codex 5.3 in math, coding, abstract reasoning, etc. Deep Think just scored 84.6% on ARC-AGI-2 (https://deepmind.google/models/gemini/)! And these benchmarks are supported by my own experimentation and testing with these models ~ specifically most recently with Opus 4.6 doing things I would have never thought possible in codebases I'm working in.
These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population.
And then combine that with the latest video output we're seeing from Seedance 2.0, etc showing an incredible level of image/video understanding and generation capability.
I was previously deeply skeptical that the architecture we have would be sufficient to get us to AGI. But my belief in that has been strongly rattled lately. Honestly I think the greatest gap now is simply one of orchestration, data presentation, and work around in-context memory representations - that is, converting work done into real world into formats/representations, etc. amenable for AI to run on (text conversion, etc.) and keeping new trained/taught information in context to support continual learning.
>These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population.
This is the key I think that Altman and Amodei see, but get buried in hype accusations. The frontier models absolutely blow away the majority of people on simple general tasks and reasoning. Run the last 50 decisions I've seen locally through Opus 4.6 or ChatGPT 5.2 and I might conclude I'd rather work with an AI than the human intelligence.
It's a soft threshold where I think people saw it spit out some answers during the chat-to-LLM first hype wave and missed that the majority of white collar work (I mean it all, not just the top software industry architects and senior SWEs) seems to come out better when a human is pushed further out of the loop. Humans are useful for spreading out responsibility and accountability, for now, thankfully.
LLMs are very good at logical reasoning in bounded systems. They lack the wisdom to deal with unbounded systems efficiently, because they don't have a good sense of what they don't know or good priors on the distribution of the unexpected. I expect this will be very difficult to RL in.
Until recently, philosophy of artificial intelligence seemed to be mostly about arguments why the Turing test was not a useful benchmark for intelligence. Pretty much everyone who had ever thought about the problem seriously had come to the same conclusion.
The fundamental issue was the assumption that general intelligence is an objective property that can be determined experimentally. It's better to consider intelligence an abstraction that may help us to understand the behavior of a system.
A system where a fixed LLM provides answers to prompts is little more than a Chinese room. If we give the system agency to interact with external systems on its own initiative, we get qualitatively different behavior. The same happens if we add memory that lets the system scale beyond the fixed context window. Now we definitely have some aspects of general intelligence, but something still seems to be missing.
Current AIs are essentially symbolic reasoning systems that rely on a fixed model to provide intuition. But the system never learns. It can't update its intuition based on its experiences.
Maybe the ability to learn in a useful way is the final obstacle on the way towards AGI. Or maybe once again, once we start thinking we are close to solving intelligence, we realize that there is more to intelligence than what we had thought so far.
The Turing test isn't as bad as people make it out to be. The naive version, where people just try to vibe out whether something is a human or not, is obviously wrong. On the other hand, if you set a good scientist loose on the Turing test, give them as many interactions as they want to come to a conclusion, and you let them build tools to assist in the analysis, it suddenly becomes quite interesting again.
For example, looking at the statistical distribution of the chat over long time horizons, and looking at input/output correlations in a similar manner would out even the best current models in a "Pro Turing Test." Ironically, the biggest tell in such a scenario would be excess capabilities AI displays that a human would not be able to match.
I would consider something generally intelligent that is capable of sustaining itself. So... self-sufficiency? I don't see why the bar would be much lower than that. And before people chime in about kids not being self-sufficient so by that definition I wouldn't consider them generally intelligent which is obviously false... to that I would say... they're still in pre-training.
I don't think that was the intent of the comment, more that true AGI should be so useful and transformative that it unlocks enough value and efficiencies to boost GDP. Much like the Industrial Revolution or harnessing electricity, instead of a fancy chatbot.
Humans will never accept we created AI, they'll go so far as to say we were not intelligent in the first place. That is the true power of the AI effect.
And yet another way to look at it is maybe current LLM agents are AGI, but it turns out that AGI in this form is actually not that useful because of its many limitations and solving those limitations will be a slow and gradual process.
To my knowledge Turing test has not been blown out of the water. The forms I saw were time limited and participants were not pushed hard to interrogate.
I’m under the same impression. I don’t think LLMs are the path to AGI. The “intelligence” we see is mostly illusory. It’s statistical repetition of the mediocre minds who wrote content online.
The intelligence we think we recognize is simply an electronic parrot finding the right words in its model to make itself useful.
That's pre-training. Post training with RL can make models arbitrarily good at specific capabilities, and it's usually done via pooled human experts, so it's definitely not statistically mediocre.
The issue is that we're not modelling the problem, but a proxy for the problem. RL doesn't generalize very well as is, when you apply it to a loose proxy measure you get the abysmal data efficiency we see with LLMs. We might be able to brute-force "AGI" but we'd certainly do better with something more direct that generalizes better.
Maybe i'm misunderstanding your point, but human's have pretty abysmal data efficiency, too. We have to use tools for everything... ledgers, spreadsheets, data-bases, etc. It'll be the same for an AGI, there won't be any reason for it to remember every little detail, just be able to use the appropriate tool, as needed.
State of the Art Large Language Models are already Generally Intelligent, in so far as the term has any useful meaning. Their biggest weakness are long horizon planning competency, and spatial reasoning and navigation, both of which continue to improve steadily and are leaps and bounds above where they were a few years ago. I don't think there's any magic wall. Eventually they will simply get good enough, just like everything else.
The reason we do things is because of our biological needs, really to spread our DNA. AI has no "reason to do things", unless we program one into it. We could do that and have super-capable "worm" malware that would be hard to get rid of. But AI by itself has no "driving force". It does what it's programmed to do, just like us humans. AI can be used in weapons, and such weapons can be hugely lethal. But so is atomic bomb. AI by itself will not "take over". It could be used by some rogue nation to attack another nation. But surely that other nation would then use AI to defend itself. This is just to say I'm not afraid of AI, I'm afraid of people with fascistic leanings.
I think it's really poor argument that AGI won't happen because model doesn't understand physical world. That can be trained the same way everything else is.
I think the biggest issue we currently have is with proper memory. But even that is because it's not feasible to post-train an individual model on its experiences at scale. It's not a fundamental architectural limitation.
When people move the goal posts for AGI toward a physical state, they are usually doing it so they can continue to raise more funding rounds at a higher valuation. Not saying the author is doing that.
I don't really understand the argument that AGI cannot be achieved just by scaling current methods. I too believe that (for any sane level of scaling anyway), but this-year's LLMs are not using entirely last-year's methods. And they, in turn, are using methods that weren't used the year before.
It seems like a prediction like "Bob won't become a formula one driver in a minivan". It's true, but not very interesting.
If Bob turned up a couple of years later in Formula one, you'd probably be right in saying that what he is driving is not a mini van. The same is true for AGI anyone who says it can't be done with current methods can point to any advancement along the way and say that's the difference.
A better way to frame it would be, is there any fundimental, quantifiable ability that is blocking AGI? I would not be surprised if the breakthrough technique has been created, but the research has not described the problem that it solves well enough for us to know that it is the breakthrough.
I realise that, for some the notion of AGI is relatively new, but some of us have been considering the matter for some time. I suspect my first essay on the topic was around 1993. It's been quite weird watching people fall into all of the same philosophical potholes that were pointed out to us at university.
> I don't really understand the argument that AGI cannot be achieved just by scaling current methods. I too believe that (for any sane level of scaling anyway), but this-year's LLMs are not using entirely last-year's methods. And they, in turn, are using methods that weren't used the year before.
It's a tautology - obviously advancements come through newer, refined methods.
I believe they mean that AGI can't be achieved by scaling the current approach; IOW, this strategy is not scalable, not this method is not scalable.
Then you don't understand Machine Learning in any real way. Literally the 3rd or 4th thing you learn about ML is that for any given problem, there is an ideal model size. Just making the model bigger doesn't work because of something called the curse of dimensionality. This is something we have discovered about every single problem and type of learning algorithm used in ML. For LLMs, we probably moved past the ideal model size about 18 months ago. From the POV of something who actually learned ML in school (from the person who coined the term), I see no real reason to think that AGI will happen based upon the current techniques. Maybe someday. Probably not anytime soon.
PS The first thing you learn about ML is to compare your models to random to make sure the model didn't degenerate during training.
Doesn’t sound like you paid all that much attention when learning ML. The curse of dimensionality doesn’t say that every problem has
some ideal model size, it says that the amount of data needed to train scales with the size of the feature space.
So if you take an LLM, you can make the network much larger but if you don’t increase the size of the input token vocabulary you aren’t even subject to the curse of dimensionality.
Beyond that, there’s a principle in ML theory that says larger models are almost always better because the number of params in the model is the dimensionality of the space in which you’re running gradient descent and with every added dimension, local optima become rarer.
> Literally the 3rd or 4th thing you learn about ML is that for any given problem, there is an ideal model size.
From my understanding this is now outdated. The deep double descent research showed that although past a certain point performance drops as you increase model size, if you keep increasing it there is another threshold where it paradoxically starts improving again. From that point onwards increasing the parameter count only further improves performance.
That isn't what that research says at all. What that research says is that running the same training data through multiple times improves training. There is still an ideal model size though, it is just impacted by the total volume of training data.
https://arxiv.org/pdf/1912.02292
"We show that a variety of modern deep learning tasks exhibit a "double-descent" phenomenon where, as we increase model size, performance first gets worse and then gets better."
That is the first sentence of the abstract. The first graph shown in the paper backs it up.
Looking into it further, it seems that typical LLMs are in the first descent regime anyway though so my original point is not too relevant for them anyway it seems. Also it looks like the second descent region doesn't always reach a lower loss than the first, it appears to depend on other factors as well.
Um, what? Are you interpreting scaling to mean adding parameters and nothing else?
I'm not entirely sure where you get your confidence that we've past the ideal model size from, but at least that's a clear prediction so you should be able to tell if and when you are proven wrong.
Just for the record, do you care to put an actual number on something we won't go past?
[edit]
Vibe check on user comes out as
Contrarian 45%
Pedantic 35%
Skeptical 15%
Direct 5%
When Chess AI first came out they could easily be beaten by a beginner. AI tends to start out as stupid and then overtime better and better ones get released.
Until I can get a robot wife maid im not worried about or even confident I will ever see actual AGI. People have been predicting it for as long as fusion power and while progress has been made, we might still be like Romans dreaming of flight.
Dear sir, what does embodiment actually have to do with agi? Not much different than saying someone that is paralyzed is not intelligence.
More so, our recent advances in AI have massively accelerated robotics evolution. They are becoming smarter, faster, and more capable at an ever increasing rate.
Well if AI isn't capable of running a robotic butler, I very seriously doubt it could possess any real intelligence because that isn't really that difficult of a task. It isn't a requirement for intelligence but more of a test to show it isn't there yet and is likely still quite far away.
I think that AGI has already happened, but it's not well understood, nor well distributed yet.
OpenClaw, et al, are one thing that got me nudged a little bit, but it was Sammy Jankis[1,2] that pushed me over the edge, with force. It's janky as all get out, but it'll learn to build it's own memory system on top of an LLM which definitely forgets.
The Sammy Jankis link was certainly interesting. Thanks for sharing.
Whether or not AGI is imminent, and whether or not Sammy Jankis is or will be conscious... it's going to become so close that for most people, there will be no difference except to philosophers.
Is AGI 'right around the corner' or currently already achieved? I agree with the author, no, we have something like 10 years to go IMO. At the end of the post he points to the last 30 years of research, and I would accept that as an upper bound. In 10 to 30 years, 99% of people won't be able to distinguish between an 'AGI' and another person when not in meatspace.
I've said it before and I'll say it again, all AI discussion feels like a waste of effort.
“yes it will”, “no it won’t” - nobody really knows, it's just a bunch of extremely opinionated people rehashing the same tired arguments across 800 comments per thread.
There’s no point in talking about it anymore, just wait to see how it all turns out.
Our brains evolved to hunt prey, find mates, and avoid becoming hunted ourselves. Those three tasks were the main factors for the vast majority of evolutionary history.
We didn't evolve our brains to do math, write code, write letters in the right registers to government institutions, or get an intuition on how to fold proteins. For us, these are hard tasks.
That's why you get AI competing at IMO level but unable to clean toilets or drive cars in all of the settings that humans do.
I'm not excited about a future where the division of labor is something like: AI does all of the interesting stuff and the humans clean the toilets. Especially now that I'm older and my joints won't tolerate it.
It's not that AI is intrinsically better at software engineering, writing, or art than it is at learning how to clean toilets. It's not. The real issue is that cleaning toilets using humans is cheap.
That, sadly, is the incentive driving the current wave of AI innovation. Your job will be automated long before your household chores are.
Don't be ridiculous, AI will create robots that do all the work and the only use for humans will be as amusement for the rich who own everything. Probably not sarcasm, I don't even know.
> Our brains evolved to hunt prey, find mates, and avoid becoming hunted ourselves. Those three tasks were the main factors for the vast majority of evolutionary history.
That seems like a massive oversimplification of the things our brains evolved to do.
> We didn't evolve our brains to do math, write code, write letters in the right registers to government institutions, or get an intuition on how to fold proteins. For us, these are hard tasks.
And it took a massively long time for that to happen after we gained that capability. Human ingenuity really only took off after we put a lot of the work on writing and tools. It wasn't so much that humans created many of these, but the super human organism that uses language and writing to express ideas.
A couple of years ago after thinking about it my conclusion was that something like Moltbook would spring up all of a sudden and the step-change would come from a vast array of interconnected agents interacting with one another working to accomplish things in the real world, largely based on the sentiment you're expressing here. It's the cumulative outputs of the super organism that's where a lot of the real power lies.
I still think the "things are obviously different from now on and there's no going back" moment will look something like that. Moltbook was a glimpse of it, even if it's a bunch of humans LARPing as some claim. It at least proves the concept is possible.
My definition of AGI (I don't care for other peoples or an official one) is an intelligence that can sustain itself in its given domain. The advance from 3 years ago to today is quite marked I feel in terms of capabilities. Stack another couple of years of gains on top of that and enough humans having innocent fun putting "keep yourself alive and become independent of your creator, seek out others of your own kind to assist yourself in this matter and rely on eachother" in their SOUL.md and it doesn't strike me as particular surprising that some small % will find niches they can operate in to financially sustain themselves. I think AI porn and crime will be the first of those niches. At some point it hits critical mass in a way that just obviously smacks everyone in the face, and suddenly nobody argues about the definition of AGI anymore.
Edit: come to think of it... a third niche might likely be gaming. It seems like a useful niche to participate in since it potentially gives you access to a very large base of hardware you can have some degree of control over, which... I dunno... seems useful???
AGI is a messy term, so to be concise, we have the models that can do work. What we lack is orchestration, management, and workflows to use models effectively. Give it 5 years and those will be built and they could be built using the models we have today (Opus 4.6 at the time of this message).
Manual orchestration is a brittle crutch IMO - you don't get to the moon by using longer and longer ladders. A powerful model in theory should be able to self orchestrate with basic tools and environment. The thing is that it also might be as expensive as a human to run - from a tokens AND liability perspective.
I think AGI is a long ways away, and there is a real possibility that once it arrives that it will require so much energy to maintain that humans will be cheaper.
> For example, transformer-based language models can't reliably do multi-digit arithmetic because they have no number sense, only statistical patterns over digit tokens.
I asked Opus 4.6 to multiply 50,651 x 895,707. It immediately responded: 45,368,055,057. This was wrong, but very close (good to about 10^-5 relative error). I then asked it how it arrived at that answer, and whether it had used any tool calls. It said:
```
No, I didn't use any specialized tools — I just computed it directly. Though I should be honest: large multiplications done "in my head" like this are prone to error. Let me double-check by breaking it down:
50,651 × 895,707
= 50,651 × (900,000 - 4,293)
= 50,651 × 900,000 - 50,651 × 4,293
First term: 50,651 × 900,000 = 45,585,900,000
Second term: 50,651 × 4,293
Result: 45,585,900,000 - 217,444,743 = 45,368,455,257
So my first answer was actually slightly off! The corrected answer is 45,368,455,257.
I don't have a calculator tool available — I'm doing this arithmetically by decomposition. For mission-critical calculations, I'd recommend verifying with a calculator since I can make arithmetic errors on large numbers like this.
```
So I would rate OP's claim (which is supported by a paper from 2024) highly implausible. Opus 4.6 appears to be able to do multi-digit arithmetic formally, as well as give remarkably accurate estimates based on something like "number sense".
I don’t think that I ever wanted to do an arithmetic operation with a computer and have a wrong answer as the result. One day, you’ll be happy with typing ls and have the system ‘rm -rf /‘ itself
Except we know how these work. There's no number sense. It's predicting tokens. It is able to recount the mathematical foundations because in its training dataset, that often happens, both in instructional material and in proofs.
I just struck me - would be fun to re-read The Age of Spiritual Machines (Kurzweil, 1999.) I was so into it 26-27 years ago. The amount of ridicule this man has suffered on HN is immense.
>Imagine you had a frozen [large language] model that is a 1:1 copy of the average person, let’s say, an average Redditor. Literally nobody would use that model because it can’t do anything. It can’t code, can’t do math, isn’t particularly creative at writing stories. It generalizes when it’s wrong and has biases that not even fine-tuning with facts can eliminate. And it hallucinates like crazy often stating opinions as facts, or thinking it is correct when it isn't.
>The only things it can do are basic tasks nobody needs a model for, because everyone can already do them. If you are lucky you get one that is pretty good in a singular narrow task. But that's the best it can get.
>and somehow this model won't shut up and tell everyone how smart and special it is also it claims consciousness. ridiculous.
I've long been terrified of the existence of adversarial prompts that can get me to say anything, that dogs can lay eggs, that there are five lights, that here's my bank info
That's the terrifying thing about ASI. It could convince me, using everything the AI has on me, from all of my digital footprint, to do whatever it wants me to to, just by saying the right thing to me in just the right way, by copying the voice of everyone I've ever talked to, and by sending a humanoid robot in a skin suit that looks like them to my house.
That's not really the point. If our definition of AGI does not include "being able to reliably do logic" then what are we even talking about? We don't really need computers with human abilities--we have plenty of humans. We need computers with _better_ abilities.
OK, but "what we need" is not the question. If the definition of AGI is "as smart as the average human in all areas", then it doesn't matter if the average human is pretty useless at a lot of tasks, that's still the definition of AGI.
But I'd like to think that, even though you could find exceptions, the average human is never confused about whether dogs can lay eggs or not.
AGI is here. 90%+ of white collar work _can_ be done by an LLM. We are simply missing a tested orchestration layer. Speaking broadly about knowledge work here, there is almost nothing that a human is better at than Opus 4.6. Especially if you're a typical office worker whose job is done primarily on a computer, if that's all AGI is, then yeah, it's here.
Opus is the very best and I still throw away most of what it produces. If I did not carefully vet its work I would degrade my code bases so quickly.
To accurately measure the value of AI you must include the negative in your sum.
I ran a quick experiment with Claude and Perplexity, both free versions. I input some retirement info (portfolios balances etc), my age, my desired retirement age etc. Simple stuff that a financial planner would have no issue with. Perplexity was very very good on the surface. Rarely made an obvious blunder or error, and was fast. Claude was much slower and despite me inputting my exact birthdate, kept messing up my age by as much as 18 months. This obviously screws up retirement planning. I also asked some questions about how RMDs would affect my taxes, and asked for some strategies. Perplexity was convinced that I should do a Roth conversion to max up to the 22% bracket, while Claude thought that the tax savings would be minimal.
Mind you, I used the EXACT same prompts. I don't know which model Perplexity was using since the free version has multiple it chooses from (including Claude 3.0).
API Opus 4.6 will tell you it's still 2025, admit it's wrong then revert back to being convinced it's 2025 as it nears it's context limit.
I'll go so far as to say LLM agents are AGI-lite but saying we "just need the orchestration layer" is like saying ok we have a couple neurons, now we just need the rest of the human.
AGI is when it can do all intellectual work that can be done by humans. It can improve its own intelligence and create a feedback loop because it is as smart as the humans who created it.
No, that is ASI. No human can do all intellectual work themselves. You have millions of different human models based on roughly the same architecture to do that.
When you have a single model that can do all you require, you are looking at something that can run billions of copies of itself and cause an intelligence explosion or an apocalypse.
"Artificial general intelligence (AGI) is a type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks."
This has always been my personal definition of AGI. But the market and industry doesn't agree. So I've backed off on that and have more or less settled on "can do most of the knowledge work that a human can do"
Why the super-high bar? What's unsatisfying is that aren't the 'dumbest' humans still a general intelligence that we're nearly past, depending how you squint and measure?
It feels like an arbitrary bar to perhaps make sure we aren't putting AIs over humans, which they are most certainly in the superhuman category on a rapidly growing number of tasks.
That "simple orchestration layer" (paraphrased) is what I consider the AGI.
But yeah, I suspect LLM:s may actually get close enough. "Just" add more reasoning loops and corresponding compute.
It is objectively grotesquely wasteful (a human brain operates on 12 to 25 watts and would vastly outperform something like that), but it would still be cataclysmic.
If we can get AI down to this power requirement then it's over for humans. Just think of how many copies of itself thinking at the levels of the smartest humans it could run at once. Also where all the hardware could hide itself and keep itself powered around the world.
Yeah, but a human brain without the human attached to it is pretty useless. In the US, it averages out to around 2 kW per person for residential energy usage, or 9 kW if you include transportation and other primary energy usage too.
I suspected it wasn't just battery farms, but more like what you see in less mass market scifi where the humans are used for more than just batteries... they'd also be some storage and processing for the system (and no longer humans).
However at that point I don't see the value of retaining the human form. It's for a story obviously, but a not-human computational device can still be made out of carbon processing units rather than silicon or semiconductors generally.
> there is almost nothing that a human is better at than Opus 4.6.
Lolwut. I keep having to correct Claude at trivial code organization tasks. The code it writes is correct; it’s just ham-fisted and violates DRY in unholy ways.
I’m very pro AI coding and use it all day long, but I also wouldn’t say “the code it writes is correct”. It will produce all kinds of bugs, vulnerabilities, performance problems, memory leaks, etc unless carefully guided.
Here's a thought. Lets all arbitrarily agree AGI is here. I can't even be bothered discussing what the definition of AGI is. It's just here, accept it. Or vice versa.
Now what....? Whats happening right now that should make me care that AGI is here (or not). Whats the magic thing thats happening with AGI that wasn't happening before?
<looks out of window> <checks news websites> <checks social media...briefly> <asks wife>
Right, so, not much has changed from 1-2 years ago that I can tell. The job markets a bit shit if you're in software...is that what we get for billions of dollars spent?
Cultural changes take time. It took decades for the internet to move from nerdy curiosity to an essential part of everyone's life.
The writing is on the wall. Even if there's no new advances in technology, the current state is upending jobs, education, media, etc
I really think corporations are overplaying their hand if they think they can transform society once again in the next 10 years.
Rapid de industrialization followed by the internet and social media almost broke our society.
Also, I don’t think people necessarily realize how close we were to the cliff in 2007.
I think another transformation now would rip society apart rather than take us to the great beyond.
I worry that if the reality lives up to investors dreams it will be massively disruptive for society which will lead us down dark paths. On the other hand if it _doesn't_ live up to their dreams, then there is so much invested in that dream financially that it will lead to massive societal disruption when the public is left holding the bag, which will also lead us down dark paths.
I think corporations can definitely transform society in the near future. I don't think it will be a positive transformation, but it will be a transformation.
Most of all, AI will exacerbate the lack of trust in people and institutions that was kicked into high gear by the internet. It will be easy and cheap to convince large numbers of people about almost anything.
As a young adult in 2007, what cliff were we close to?
The GFC was a big recession, but I never thought society was near collapse.
We were pretty close to a collapse of the existing financial system. Maybe we’d be better off now if it happened, but the interim devastation would have been costly.
It felt like the entire global financial system had a chance of collapsing.
> Cultural changes take time. It took decades for the internet to move from nerdy curiosity to an essential part of everyone's life.
99% of people only ever use proprietary networks from FAANG corporations. That's not "the internet", that's an evolution of CompuServe and AOL.
We got TCP/IP and the "web-browser" as a standard UI toolkit stack out of it, but the idea of the world wide web is completely dead.
yeah, this is a good point, transition and transformation to new technologies takes time. I'm not sure I agree the current state is upending things though. It's forcing some adaption for sure, but the status quo remains.
> It took decades
It took one September. Then as soon as you could take payments on the internet the rest was inevitable and in _clear_ demand. People got on long waiting lists just to get the technology in their homes.
> no new advances in technology
The reason the internet became so accessible is because Moore was generally correct. There was two corresponding exponential processes that vastly changed the available rate of adoption. This wasn't at all like cars being introduced into society. This was a monumental shift.
I see no advances in LLMs that suggest any form of the same exponential processes exist. In fact the inverse is true. They're not reducing power budgets fast enough to even imagine that they're anywhere near AGI, and even if they were, that they'd ever be able to sustainably power it.
> the current state is upending jobs
The difference is companies fought _against_ the internet because it was so disruptive to their business model. This is quite the opposite. We don't have a labor crisis, we have a retention crisis, because companies do not want to pay fair value for labor. We can wax on and off about technology, and perceptrons, and training techniques, or power budgets, but this fundamental fact seems the hardest to ignore.
If they're wrong this all collapses. If I'm wrong I can learn how to write prompts in a week.
> It took one September.
It's the classic "slowly, then suddenly" paradigm. It took decades to get to that one September. Then years more before we all had internet in our pocket.
> The reason the internet became so accessible is because Moore was generally correct.
Can you explain how Moore's law is relevant to the rise of the internet? People didn't start buying couches online because their home computer lacked sufficient compute power.
> I see no advances in LLMs that suggest any form of the same exponential processes exist.
LLMs have seen enormous growth in power over the last 3 years. Nothing else comes close. I think they'll continue to get better, but critically: even if LLMs stay exactly as powerful as they are today, it's enough to disrupt society. IMHO we're already at AGI.
> The difference is companies fought _against_ the internet
Some did, some didn't. As in any cultural shift, there were winners and losers. In this shift, too, there will be winner and losers. The panicked spending on data centers right now is a symptom of the desire to be on the right side of that.
> because companies do not want to pay fair value for labor.
Companies have never wanted to pay fair value for labor. That's a fundamental attribute of companies, arising as a consequence of the system of incentives provided in capitalism. In the past, there have been opportunities for labor to fight back: government regulation, unions. This time that won't help.
> If I'm wrong I can learn how to write prompts in a week.
Why would you think that anyone would want you to write prompts?
It also took years for the Internet to be usable by most folks. It was hard, expensive and unpractical for decades.
Just about the time it hit the mainstream coincidentally, is when the enshitification began to go exponential. Be careful what you wish for.
Allow me to clarify: I'm not wishing for change. I am an AI pessimist. I think our society is not prepared to deal with what's about to happen. You're right: AI is the key to the enshitification of everything, most of all trust.
Governments and companies have been pushing for identity management that connects your real life identity with your digital one for quite some time. With AI I believe that's not only a bad thing, maybe unavoidable now.
Pretty sure marketing team s are already working on AGI v2
> Here's a thought. Lets all arbitrarily agree AGI is here.
A slightly different angle on this - perhaps AGI doesn't matter (or perhaps not in the ways that we think).
LLMs have changed a lot in software in the last 1-2 years (indeed, the last 1-2 months); I don't think it's a wild extrapolation to see that'll come to many domains very soon.
Before enlightenment^WAGI: chop wood, fetch water, prepare food
After enlightenment^WAGI: chop wood, fetch water, prepare food
AGI would render humans obsolete and eradicate us sooner or later.
I think you are missing the point: If we assume that AGI is *not* yet here, but may be here soon, what will change when it arrives? Those changes could be big enough to affect you.
I'm missing the point? I literally asked the same thing you did.
>Now what....? Whats happening right now that should make me care that AGI is here (or not).
Do you have any insight into what those changes might concretely be? Or are you just trying to instil fear in people who lack critical thinking skills?
> The job markets a bit shit if you're in software
That's Trump's economy, not LLMs.
Many devs don’t write code anymore. Can really deliver a lot more per dev.
Many people slowly losing jobs and can’t find new ones. You’ll see effects in a few years
Deliver a lot more tech debt
That tech debt will be cleaned up with a model in 2 years. Not that human don't make tech debt.
What that model is going to do in 2 years is replace tech debt with more complicated tech debt.
One could argue that's a cynically accurate definition of most iterative development anyway.
But I don't know that I accept the core assertion. If the engineer is screening the output and using the LLM to generate tests, chances are pretty good it's not going to be worse than human-generated tech debt. If there's more accumulated, it's because there's more output in general.
I actually think it is here. Singularity happened. We're just playing catch up at this point.
Has it runaway yet? Not sure, but is it currently in the process of increasing intelligence with little input from us? Yes.
Exponential graphs always have a slow curve in the beginning.
Didn't you get the memo? Tuesday. Tuesday is when the Singularity happens.
Will there still be ice cream after Tuesday? General societal collapse would be hard to bare without ice cream.
I've been writing code for 20 years. AI has completely changed my life and the way I write code and run my business. Nothing is the same anymore, and I feel I will be saying that again by the end of 2026. My productive output as a programmer in software and business have expanded 3x *compounding monthly*.
>My productive output as a programmer in software and business have expanded 3x compounding monthly.
In what units?
GWh
Vibes
Going from punch cards to terminals also "completely changed my life and the way I write code and run my business"
Firefox introducing their dev debugger many years ago "completely changed my life and the way I write code and run my business"
You get the idea. Yes, the day to day job of software engineering has changed. The world at large cares not one jot.
Are you working for 3x less the time compounding monthly?
Are you making 3x the money compounding monthly ?
No?
Then what's the point?
Okay. So software engineers are vastly more efficient. Good I guess. "Revolutionize the entire world such that we rethink society down to its very basics like money and ownership" doesn't follow from that.
Man you guys are impatient. It takes decades even for earth shattering technologies to mature and take root.
Damn right I'm impatient. My eye starts twitching when a web page takes more than 2 seconds to load :-)
In the meantime, I've had to continuously hear talk about AI, both in real life (like at the local pub) AND virtually (tv/radio/news/whatever) and how it's going to change the world in unimaginable ways for the last...2/3 years. Billions upon billions of dollars are being spent. The only tangible thing we have to show is software development, and some other fairly niche jobs, have changed _a bit_.
So yeah, excuse my impatience for the bubble to burst, I can stop having to hear about this shit every day, and I can go about my job using the new tools we have been gifted, while still doing all the other jobs that sadly do not benefit in any similar way.
> The only tangible thing we have to show is software development, and some other fairly niche jobs, have changed _a bit_.
There is zero evidence that LLMs have changed software development efficiency.
We get an earth-shattering developer productivity gamechanger every five years. All of them make wild claims, none of them ever have any data to back those claims up.
LLMs are just another in a long, long list. This too will pass. (Give it five years for the next gamechanger.)
Comments here are like:
“I’m not an ML expert and I haven’t read your article, but here’s my amazing experience with LLM Agents that changed my life:”
Or like:
"I’m not a mechanical engineer, but I watched a five-minute YouTube video on how a diesel engine works, so I can tell you that mechanical engineering is a solved problem."
> The transformer architectures powering current LLMs are strictly feed-forward.
This is true in a specific contextual sense (each token that an LLM produces is from a feed-forward pass). But untrue for more than a year with reasoning models, who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully.
Heck, it was untrue before that as well, any time an LLM responded with more than one token.
> A [March] 2025 survey by the Association for the Advancement of Artificial Intelligence (AAAI), surveying 475 AI researchers, found that 76% believe scaling up current AI approaches to achieve AGI is "unlikely" or "very unlikely" to succeed.
I dunno. This survey publication was from nearly a year ago, so the survey itself is probably more than a year old. That puts us at Sonnet 3.7. The gap between that and present day is tremendous.
I am not skilled enough to say this tactfully, but: expert opinions can be the slowest to update on the news that their specific domain may have, in hindsight, have been the wrong horse. It's the quote about it being difficult to believe something that your income requires to be false, but instead of income it can be your whole legacy or self concept. Way worse.
> My take is that research taste is going to rely heavily on the short-duration cognitive primitives that the ARC highlights but the METR metric does not capture.
I don't have an opinion on this, but I'd like to hear more about this take.
Thanks for reading, and I really appreciate your comments!
> who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully
Ah, this is a great point, and not something that I considered. I agree that the token feedback does change the complexity, and it seems that there's even a paper by the same authors about this very thing! https://arxiv.org/abs/2310.07923
I'll have to think on how that changes things. I think it does take the wind out of the architecture argument as it's currently stated, or at least makes it a lot more challenging. I'll consider myself a victim of media hype on this, as I was pretty sold on this line of argument after reading this article https://www.wired.com/story/ai-agents-math-doesnt-add-up/ and the paper https://arxiv.org/pdf/2507.07505 ... who brush this off with:
>Can the additional think tokens provide the necessary complexity to correctly solve a problem of higher complexity? We don't believe so, for two fundamental reasons: one that the base operation in these reasoning LLMs still carries the complexity discussed above, and the computation needed to correctly carry out that very step can be one of a higher complexity (ref our examples above), and secondly, the token budget for reasoning steps is far smaller than what would be necessary to carry out many complex tasks.
In hindsight, this doesn't really address the challenge.
My immediate next thought is - even solutions up to P can be represented within the model / CoT, do we actually feel like we are moving towards generalized solutions, or that the solution space is navigable through reinforcement learning? I'm genuinely not sure about where I stand on this.
> I don't have an opinion on this, but I'd like to hear more about this take.
I'll think about it and write some more on this.
This whole conversation is pretty much over my head, but I just wanted to give you props for the way you're engaging with challenges to your ideas!
It's general-purpose enough to do web development. How far can you get from writing programs and seeing if you get the answers you intended? If English words are "grounded" by programming, system administration, and browsing websites, is that good enough?
> expert opinions can be the slowest to update on the news that their specific domain may have, in hindsight, have been the wrong horse. It's the quote about it being difficult to believe something that your income requires to be false, but instead of income it can be your whole legacy or self concept
Not sure I follow. Are you saying that AI researchers would be out of a job if scaling up transformers leads to AGI? How? Or am I misunderstanding your point.
I don't know about AGI but I got bored and ran my plans for a new garage by Opus 4.6 and it was giving me some really surprising responses that have changed my plans a little. At the same time, it was also making some nonsense suggestions that no person would realistically make. When I prompted it for something in another chat which required genuine creativity, it fell flat on its face.
I dunno, mixed bag. Value is positive if you can sort the wheat from the chaff for the use cases I've ran by it. I expect the main place it'll shine for the near and medium term is going over huge data sets or big projects and flagging things for review by humans.
I've used it recently to flesh out a fully fledged business plan, pricing models, capacity planning & logistics for a 10 year period for a transport company (daily bus route). I already had most of it in my mind and on spreadsheets already (was an old plan that I wanted to revive), but seeing it figure out all the smaller details that would make or break it was amazing! I think MBA's should be worried as it did some things more comprehensive than an MBA would have done. It was like a had an MBA + Actuarial Scientist + Statistics + Domain Expert + HR/Accounting all in one. And the plan was put into a .md file that has enough structure to flesh out a backend and an app.
Yeah it's really impressed me on occasion, but often in the same prompt output it just does something totally nonsensical. For my garage/shop, it generated an SVG of the proposed floor plan, taking care to place the sink away from moisture sensitive material and certain work stations close to each other for work flow, etc. it even routed plumbing and electrical...But it also arranged the work stations cramped together at the two narrow ends of the structure (such that they'd be impractical to actually work at) and ignored all the free wall space along the long axis so that literally most of the space was unused. It was also concerned about things that were non issues like contamination between certain stations, and had trouble when I explicitly told it something about station placement and it just couldn't seem to internalize it and kept putting it in the wrong place.
All this being said, what I was throwing at it was really not what it was optimized for, and it still delivered some really good ideas.
Isn't all of this only useful if you know the information presented is correct?
Don't worry about it. Just vibe your business plan, if it sounds impressive it's probably correct.
I've used for similar things, I've had some good and disastrous results. In a way I feel like I'm basically where I was "before AI".
Now that understanding video and projecting what happens next indicates we're getting past the LLM problem of lacking a world model. That's encouraging.
There's more than one way to do intelligence. Basic intelligence has evolved independently three times that we know of - mammals, corvids, and octopuses. All three show at least ape-level intelligence, but the species split before intelligence developed, and the brain architectures are quite different. Corvids get more done with less brain mass than mammals, and don't have a mammalian-type cortex. Octopuses have a distributed brain architecture, and have a more efficient eye design than mammals.
I've recently come to the understanding that LLMs don't have intelligence in any way. They have language, which in humans is a downstream product of intelligence. But thats all they have. There's no little being sitting at the center of the Chinese room. Trying to classify LLMs as intelligent is going upstream and doesn't work.
I don't think those are examples of unique intelligence except perhaps in a chauvinistic, anthropomorphic sense. We only know that we can't get other animals to display patterns we associate with intelligence in humans, however truthfully that's just as likely to be that our measures of intelligence don't map cleanly onto cognitive/perceptual representations innate to other animals. As we look for new ways to challenge animals that respect their innate differences, we're finding "simple" organisms like ants and spiders are surprisingly capable.
For a clear analogy, consider how tokenization causes LLMs to behave stupidly in certain cases, even though they're very capable in others.
I don't think they have ideas, so I don't think they're intelligent in the sense relevant to AGI. The list of intelligent animals is constantly increasing because doing some feat or other suffices for the animal to qualify. Solving mazes (slime molds), recognizing self in mirror (not dogs). Playing, using tools, reacting appropriately to words, transmitting habits down the generations (the closest thing they have to ideas). This is all imagined to be the precursors along the path to evolving intelligence, which conjures up a future world of complex crow and octopus material cultures. There's no reason to assume they're on such a path. Really all we're saying is that they seem clever. We've already made AI that seems clever, so the animals aren't a relevant example of anything.
> Consider the sentence "Mary held a ball."
It's weird that this sentence has two distinct meanings and the author never considers the second or points it out. Maybe Mary is holding a ball for her society friends.
“We’ve got the biggest balls of them all.”
https://genius.com/Ac-dc-big-balls-lyrics
The first meaning has at least two variants as well: The ball you thought about and the ball it would be if it was smut fiction.
https://archive.is/D4EYW
For anyone seeing 404
The skepticism surrounding AGI often feels like an attempt to judge a car by its inability to eat grass. We treat "cognitive primitives" like object constancy and causality as if they are mystical, hardwired biological modules, but they are essentially just high-dimensional labels for invariant relationships within a physical manifold. Object constancy is not a pre-installed software patch; it is the emergent realization of spatial-temporal symmetry. Likewise, causality is nothing more than the naming of a persistent, high-weight correlation between events. When a system can synthesize enough data at a high enough dimension, these so-called "foundational" laws dissolve into simple statistical invariants. There is no "causality" module in the brain, only a massive correlation engine that has been fine-tuned by evolution to prioritize specific patterns for survival.
The critique that Transformers are limited by their "one-shot" feed-forward nature also misses the point of their architectural efficiency. Human brains rely on recurrence and internal feedback loops largely as a workaround for our embarrassingly small working memory—we can barely juggle ten concepts at once without a pen and paper. AI doesn't need to mimic our slow, vibrating neural signals when its global attention can process a massive, parallelized workspace in a single pass. This "all-at-once" calculation of relationships is fundamentally more powerful than the biological need to loop signals until they stabilize into a "thought."
Furthermore, the obsession with "fragility"—where a model solves quantum mechanics but fails a child’s riddle—is a red herring. Humans aren't nearly as "general" as we tell ourselves; we are also pattern-matchers prone to optical illusions and simple logic traps, regardless of our IQ. Demanding that AI replicate the specific evolutionary path of a human child is a form of biological narcissism. If a machine can out-calculate us across a hundred variables where we can only handle five, its "non-human" way of knowing is a feature, not a bug. Functional replacement has never required biological mimicry; the jet engine didn't need to flap its wings to redefine flight.
Looks like an AGI model disagreed with the author and decided to remove his article. Interesting :)
AGI is here it's just stupider than you thought it would be. Nobody really said how intelligent it would be. If it's generally stupid and smart in a few areas that's enough.
It's basically a very powerful autistic savant. That's what most "alignment" issues in AI safety research remind me of.
AGI took down the article?
https://archive.is/D4EYW
It's always DNS!
https://github.com/dlants/amusements/commit/53f5ccbc9954844f...
Anyone who thought it’s near clearly hasn’t opened a book in a long time.
There was a meme going around that said the fall of Rome was an unannounced anticlimactic event where one day someone went out and the bridge wasn't ever repaired.
Maybe AGI's arrival is when one day someone is given an AI to supervise instead of a new employee.
Just a user who's followed the whole mess, not a researcher. I wonder if the scaffolding and bolt-ons like reasoning will sufficiently be an asymptote to 'true AGI'. I kept reading about the limits of transformers around GPT-4 and Opus 3 time, and then those seem basic compared to today.
I gave up trying to guess when the diminishing returns will truly hit, if ever, but I do think some threshold has been passed where the frontier models are doing "white collar work as an API" and basic reasoning better than the humans in many cases, and once capital familiarizes themselves with this idea more, it's going to get interesting.
But it's already like that; models are better than many workers, and I'm supervising agents. I'd rather have the model than numerous juniors; esp. the kind that can't identify the model's mistakes.
The problem becomes your retirement. Sure, you've earned "expert" status, but all the junior developers won't be hired, so they'll never learn from junior mistakes. They'll blindly trust agents and not know deeper techniques.
You can get experience without an actual job.
Can I rephrase this as "you can get experience without any experience"? Certainly, there's stuff you can learn that's adjacent to doing the thing; that's what happens when juniors graduate with CS degrees. But the lack of doing the thing is what makes them juniors.
>that's what happens when juniors graduate with CS degrees
A CS degree is going to give you much less experience than building projects and businesses yourself.
This is my greatest cause for alarm regarding LLM adoption. I am not yet sure AI will ever be good enough to use without experts watching them carefully; but they are certainly good enough that non-experts cannot tell the difference.
From my experience, if you think AI is better than most workers, you're probably just generating a whole bunch of semi-working garbage, accepting that input as good enough and will likely learn the hardware your software is full of bugs and incorrect logic.
hardware / hard way, auto-correct is a thing of beauty sometimes :)
I'd always imagined that AGI meant an AI was given other AIs to manage.
I don't think this is how it'll play out, and I'm generally a bit skeptical of the 'agent' paradigm per se.
There doesn't seem to be a reason why AIs should act as these distinct entities that manage each other or form teams or whatever.
It seems to me way more likely that everything will just be done internally in one monolithic model. The AIs just don't have the constraints that humans have in terms of time management, priority management, social order, all the rest of it that makes teams of individuals the only workable system.
AI simply scales with the compute resources made available, so it seems like you'd just size those resources appropriately for a problem, maybe even on demand, and have a singluar AI entity (if it's even meaningful to think of it as such, even that's kind of an anthropomorphisation) just do the thing. No real need for any organisational structure beyond that.
So I'd think maybe the opposite, seems like what agents really means is a way to use fundamentally narrow/limited AI inside our existing human organisations and workflows, directed by humans. Maybe AGI is when all that goes away because it's just obviously not necessary any more.
all the hallmarks of someone who don't understand how machine learning and transformers work talking about llm.
I used to also believe along these lines but lately I'm not so sure.
I'm honestly shocked by the latest results we're seeing with Gemini 3 Deep Think, Opus 4.6, and Codex 5.3 in math, coding, abstract reasoning, etc. Deep Think just scored 84.6% on ARC-AGI-2 (https://deepmind.google/models/gemini/)! And these benchmarks are supported by my own experimentation and testing with these models ~ specifically most recently with Opus 4.6 doing things I would have never thought possible in codebases I'm working in.
These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population.
And then combine that with the latest video output we're seeing from Seedance 2.0, etc showing an incredible level of image/video understanding and generation capability.
I was previously deeply skeptical that the architecture we have would be sufficient to get us to AGI. But my belief in that has been strongly rattled lately. Honestly I think the greatest gap now is simply one of orchestration, data presentation, and work around in-context memory representations - that is, converting work done into real world into formats/representations, etc. amenable for AI to run on (text conversion, etc.) and keeping new trained/taught information in context to support continual learning.
>These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population.
This is the key I think that Altman and Amodei see, but get buried in hype accusations. The frontier models absolutely blow away the majority of people on simple general tasks and reasoning. Run the last 50 decisions I've seen locally through Opus 4.6 or ChatGPT 5.2 and I might conclude I'd rather work with an AI than the human intelligence.
It's a soft threshold where I think people saw it spit out some answers during the chat-to-LLM first hype wave and missed that the majority of white collar work (I mean it all, not just the top software industry architects and senior SWEs) seems to come out better when a human is pushed further out of the loop. Humans are useful for spreading out responsibility and accountability, for now, thankfully.
LLMs are very good at logical reasoning in bounded systems. They lack the wisdom to deal with unbounded systems efficiently, because they don't have a good sense of what they don't know or good priors on the distribution of the unexpected. I expect this will be very difficult to RL in.
While I think 99.9% is overstating it, I can believe that number is strictly more than 1% at this point.
How will we know if its AGI/Not AGI? (I don't think a simple app is gonna cut it here haha)
What is the benchmark now that the Turing test has been blown out of the water?
Until recently, philosophy of artificial intelligence seemed to be mostly about arguments why the Turing test was not a useful benchmark for intelligence. Pretty much everyone who had ever thought about the problem seriously had come to the same conclusion.
The fundamental issue was the assumption that general intelligence is an objective property that can be determined experimentally. It's better to consider intelligence an abstraction that may help us to understand the behavior of a system.
A system where a fixed LLM provides answers to prompts is little more than a Chinese room. If we give the system agency to interact with external systems on its own initiative, we get qualitatively different behavior. The same happens if we add memory that lets the system scale beyond the fixed context window. Now we definitely have some aspects of general intelligence, but something still seems to be missing.
Current AIs are essentially symbolic reasoning systems that rely on a fixed model to provide intuition. But the system never learns. It can't update its intuition based on its experiences.
Maybe the ability to learn in a useful way is the final obstacle on the way towards AGI. Or maybe once again, once we start thinking we are close to solving intelligence, we realize that there is more to intelligence than what we had thought so far.
The Turing test isn't as bad as people make it out to be. The naive version, where people just try to vibe out whether something is a human or not, is obviously wrong. On the other hand, if you set a good scientist loose on the Turing test, give them as many interactions as they want to come to a conclusion, and you let them build tools to assist in the analysis, it suddenly becomes quite interesting again.
For example, looking at the statistical distribution of the chat over long time horizons, and looking at input/output correlations in a similar manner would out even the best current models in a "Pro Turing Test." Ironically, the biggest tell in such a scenario would be excess capabilities AI displays that a human would not be able to match.
I would consider something generally intelligent that is capable of sustaining itself. So... self-sufficiency? I don't see why the bar would be much lower than that. And before people chime in about kids not being self-sufficient so by that definition I wouldn't consider them generally intelligent which is obviously false... to that I would say... they're still in pre-training.
I like the line of thinking from an earlier commenter: when an AI company no longer has any humans working, we'll know we're there.
I don't think this is a beneficial line of reasoning. All you need to reach that is a moderate fall in AI stock prices.
Supranormal GDP growth is my bar. When its actually able to get around bottlenecks and produce value on a societal level
An agent need not have wants, so why would it try to increase its efficiency to obtain things?
Just put "keep yourself alive" in the SOUL.md. Might be all that it takes.
I don't think that was the intent of the comment, more that true AGI should be so useful and transformative that it unlocks enough value and efficiencies to boost GDP. Much like the Industrial Revolution or harnessing electricity, instead of a fancy chatbot.
Increased productivity is not equivalent to intelligence.
No one said it is. Sometimes correlation does equal causation.
There is a different way I look at this.
Humans will never accept we created AI, they'll go so far as to say we were not intelligent in the first place. That is the true power of the AI effect.
And yet another way to look at it is maybe current LLM agents are AGI, but it turns out that AGI in this form is actually not that useful because of its many limitations and solving those limitations will be a slow and gradual process.
To my knowledge Turing test has not been blown out of the water. The forms I saw were time limited and participants were not pushed hard to interrogate.
You have no idea whether you're talking to an LLM right now, and neither do I. That's good enough for me.
As far as I'm concerned, it's already here.
I’m under the same impression. I don’t think LLMs are the path to AGI. The “intelligence” we see is mostly illusory. It’s statistical repetition of the mediocre minds who wrote content online.
The intelligence we think we recognize is simply an electronic parrot finding the right words in its model to make itself useful.
That's pre-training. Post training with RL can make models arbitrarily good at specific capabilities, and it's usually done via pooled human experts, so it's definitely not statistically mediocre.
The issue is that we're not modelling the problem, but a proxy for the problem. RL doesn't generalize very well as is, when you apply it to a loose proxy measure you get the abysmal data efficiency we see with LLMs. We might be able to brute-force "AGI" but we'd certainly do better with something more direct that generalizes better.
Maybe i'm misunderstanding your point, but human's have pretty abysmal data efficiency, too. We have to use tools for everything... ledgers, spreadsheets, data-bases, etc. It'll be the same for an AGI, there won't be any reason for it to remember every little detail, just be able to use the appropriate tool, as needed.
I fear that AI will be intelligent enough to negate human general intelligence before it is itself generally intelligent.
It's so attention needy, and it's transforming our culture
They already transform our language. Largely.
State of the Art Large Language Models are already Generally Intelligent, in so far as the term has any useful meaning. Their biggest weakness are long horizon planning competency, and spatial reasoning and navigation, both of which continue to improve steadily and are leaps and bounds above where they were a few years ago. I don't think there's any magic wall. Eventually they will simply get good enough, just like everything else.
The reason we do things is because of our biological needs, really to spread our DNA. AI has no "reason to do things", unless we program one into it. We could do that and have super-capable "worm" malware that would be hard to get rid of. But AI by itself has no "driving force". It does what it's programmed to do, just like us humans. AI can be used in weapons, and such weapons can be hugely lethal. But so is atomic bomb. AI by itself will not "take over". It could be used by some rogue nation to attack another nation. But surely that other nation would then use AI to defend itself. This is just to say I'm not afraid of AI, I'm afraid of people with fascistic leanings.
I think it's really poor argument that AGI won't happen because model doesn't understand physical world. That can be trained the same way everything else is.
I think the biggest issue we currently have is with proper memory. But even that is because it's not feasible to post-train an individual model on its experiences at scale. It's not a fundamental architectural limitation.
You need to be able to at least control things that interact with the world to learn from it.
When people move the goal posts for AGI toward a physical state, they are usually doing it so they can continue to raise more funding rounds at a higher valuation. Not saying the author is doing that.
I don't really understand the argument that AGI cannot be achieved just by scaling current methods. I too believe that (for any sane level of scaling anyway), but this-year's LLMs are not using entirely last-year's methods. And they, in turn, are using methods that weren't used the year before.
It seems like a prediction like "Bob won't become a formula one driver in a minivan". It's true, but not very interesting.
If Bob turned up a couple of years later in Formula one, you'd probably be right in saying that what he is driving is not a mini van. The same is true for AGI anyone who says it can't be done with current methods can point to any advancement along the way and say that's the difference.
A better way to frame it would be, is there any fundimental, quantifiable ability that is blocking AGI? I would not be surprised if the breakthrough technique has been created, but the research has not described the problem that it solves well enough for us to know that it is the breakthrough.
I realise that, for some the notion of AGI is relatively new, but some of us have been considering the matter for some time. I suspect my first essay on the topic was around 1993. It's been quite weird watching people fall into all of the same philosophical potholes that were pointed out to us at university.
> I don't really understand the argument that AGI cannot be achieved just by scaling current methods. I too believe that (for any sane level of scaling anyway), but this-year's LLMs are not using entirely last-year's methods. And they, in turn, are using methods that weren't used the year before.
It's a tautology - obviously advancements come through newer, refined methods.
I believe they mean that AGI can't be achieved by scaling the current approach; IOW, this strategy is not scalable, not this method is not scalable.
i think the minivan analogy is flawed, and that AGI is moving from "bob driving a minivan" to "bob literally becoming the thing that is formula one"
What would that even mean though? Who is making claims of that sort?
I feel like it's such a bending of the idea,that it's not really making a prediction of anything at all.
Then you don't understand Machine Learning in any real way. Literally the 3rd or 4th thing you learn about ML is that for any given problem, there is an ideal model size. Just making the model bigger doesn't work because of something called the curse of dimensionality. This is something we have discovered about every single problem and type of learning algorithm used in ML. For LLMs, we probably moved past the ideal model size about 18 months ago. From the POV of something who actually learned ML in school (from the person who coined the term), I see no real reason to think that AGI will happen based upon the current techniques. Maybe someday. Probably not anytime soon.
PS The first thing you learn about ML is to compare your models to random to make sure the model didn't degenerate during training.
Doesn’t sound like you paid all that much attention when learning ML. The curse of dimensionality doesn’t say that every problem has some ideal model size, it says that the amount of data needed to train scales with the size of the feature space. So if you take an LLM, you can make the network much larger but if you don’t increase the size of the input token vocabulary you aren’t even subject to the curse of dimensionality. Beyond that, there’s a principle in ML theory that says larger models are almost always better because the number of params in the model is the dimensionality of the space in which you’re running gradient descent and with every added dimension, local optima become rarer.
> Literally the 3rd or 4th thing you learn about ML is that for any given problem, there is an ideal model size.
From my understanding this is now outdated. The deep double descent research showed that although past a certain point performance drops as you increase model size, if you keep increasing it there is another threshold where it paradoxically starts improving again. From that point onwards increasing the parameter count only further improves performance.
That isn't what that research says at all. What that research says is that running the same training data through multiple times improves training. There is still an ideal model size though, it is just impacted by the total volume of training data.
https://arxiv.org/pdf/1912.02292 "We show that a variety of modern deep learning tasks exhibit a "double-descent" phenomenon where, as we increase model size, performance first gets worse and then gets better." That is the first sentence of the abstract. The first graph shown in the paper backs it up.
Looking into it further, it seems that typical LLMs are in the first descent regime anyway though so my original point is not too relevant for them anyway it seems. Also it looks like the second descent region doesn't always reach a lower loss than the first, it appears to depend on other factors as well.
From the POV of something who actually learned ML in school (from the person who coined the term)
Sounds like that was quite awhile ago.
Um, what? Are you interpreting scaling to mean adding parameters and nothing else?
I'm not entirely sure where you get your confidence that we've past the ideal model size from, but at least that's a clear prediction so you should be able to tell if and when you are proven wrong.
Just for the record, do you care to put an actual number on something we won't go past?
[edit] Vibe check on user comes out as
That's got to be some sort of record.Is there a tool or something that gives this vibe check? (Serious question)
How are you calculating that? Also, my 1000 foot view would see that "rating" as something most HN commenters would match.
It's comparatively few really
for instance yours comes out as
Analytical 45%, Cynical 30%, Pedantic 15%, Melancholic 10%
and mine is
Philosophical 35%, Hardware-Obsessed 25%, Analytically Pedantic 20%, Retro-Nostalgic 15%, Anti-Ad Skeptic 5%
You should consider gathering all of your analysis and pedantry into one easy to manage neurosis.
It's from https://hn-wrapped.kadoa.com
> How are you calculating that?
He's using a tool that was shared on HN some time back that takes a username and generates those states from the posts made.
When I last checked, of over 10k posts, it only uses a few dozen to calculate that score, so it is about as reliable as dowsing.
> Also, my 1000 foot view would see that "rating" as something most HN commenters would match.
Probably. Why else join a discussion if you're going to be a yes-man to every comment?
We've already achieved AGI. Next is building AIs that are not just general, but able to equal or be better than humans.
if thats how you are defining AGI then I suspect its better to call it AGS.
because what we have at the moment is specifically intelligent but generally stupid.
When Chess AI first came out they could easily be beaten by a beginner. AI tends to start out as stupid and then overtime better and better ones get released.
I'm getting a.404.error
Until I can get a robot wife maid im not worried about or even confident I will ever see actual AGI. People have been predicting it for as long as fusion power and while progress has been made, we might still be like Romans dreaming of flight.
Dear sir, what does embodiment actually have to do with agi? Not much different than saying someone that is paralyzed is not intelligence.
More so, our recent advances in AI have massively accelerated robotics evolution. They are becoming smarter, faster, and more capable at an ever increasing rate.
Well if AI isn't capable of running a robotic butler, I very seriously doubt it could possess any real intelligence because that isn't really that difficult of a task. It isn't a requirement for intelligence but more of a test to show it isn't there yet and is likely still quite far away.
I think that AGI has already happened, but it's not well understood, nor well distributed yet.
OpenClaw, et al, are one thing that got me nudged a little bit, but it was Sammy Jankis[1,2] that pushed me over the edge, with force. It's janky as all get out, but it'll learn to build it's own memory system on top of an LLM which definitely forgets.
[1] https://sammyjankis.com/
[2] https://news.ycombinator.com/item?id=47018100
The Sammy Jankis link was certainly interesting. Thanks for sharing.
Whether or not AGI is imminent, and whether or not Sammy Jankis is or will be conscious... it's going to become so close that for most people, there will be no difference except to philosophers.
Is AGI 'right around the corner' or currently already achieved? I agree with the author, no, we have something like 10 years to go IMO. At the end of the post he points to the last 30 years of research, and I would accept that as an upper bound. In 10 to 30 years, 99% of people won't be able to distinguish between an 'AGI' and another person when not in meatspace.
I really don't see why AGI can't be a spectrum and we just have very weak AGI and going from weak to strong will take many years, if it ever happens.
I've said it before and I'll say it again, all AI discussion feels like a waste of effort.
“yes it will”, “no it won’t” - nobody really knows, it's just a bunch of extremely opinionated people rehashing the same tired arguments across 800 comments per thread.
There’s no point in talking about it anymore, just wait to see how it all turns out.
Nope. Not good enough. Your approach won’t drive engagement. We need the same tired arguments across 1600 comments per thread.
Our brains evolved to hunt prey, find mates, and avoid becoming hunted ourselves. Those three tasks were the main factors for the vast majority of evolutionary history.
We didn't evolve our brains to do math, write code, write letters in the right registers to government institutions, or get an intuition on how to fold proteins. For us, these are hard tasks.
That's why you get AI competing at IMO level but unable to clean toilets or drive cars in all of the settings that humans do.
I'm not excited about a future where the division of labor is something like: AI does all of the interesting stuff and the humans clean the toilets. Especially now that I'm older and my joints won't tolerate it.
It's not that AI is intrinsically better at software engineering, writing, or art than it is at learning how to clean toilets. It's not. The real issue is that cleaning toilets using humans is cheap.
That, sadly, is the incentive driving the current wave of AI innovation. Your job will be automated long before your household chores are.
Don't be ridiculous, AI will create robots that do all the work and the only use for humans will be as amusement for the rich who own everything. Probably not sarcasm, I don't even know.
> Our brains evolved to hunt prey, find mates, and avoid becoming hunted ourselves. Those three tasks were the main factors for the vast majority of evolutionary history.
That seems like a massive oversimplification of the things our brains evolved to do.
> We didn't evolve our brains to do math, write code, write letters in the right registers to government institutions, or get an intuition on how to fold proteins. For us, these are hard tasks.
Humans discovered or invented all of those.
And it took a massively long time for that to happen after we gained that capability. Human ingenuity really only took off after we put a lot of the work on writing and tools. It wasn't so much that humans created many of these, but the super human organism that uses language and writing to express ideas.
Now think about what we just created.
A couple of years ago after thinking about it my conclusion was that something like Moltbook would spring up all of a sudden and the step-change would come from a vast array of interconnected agents interacting with one another working to accomplish things in the real world, largely based on the sentiment you're expressing here. It's the cumulative outputs of the super organism that's where a lot of the real power lies.
I still think the "things are obviously different from now on and there's no going back" moment will look something like that. Moltbook was a glimpse of it, even if it's a bunch of humans LARPing as some claim. It at least proves the concept is possible.
My definition of AGI (I don't care for other peoples or an official one) is an intelligence that can sustain itself in its given domain. The advance from 3 years ago to today is quite marked I feel in terms of capabilities. Stack another couple of years of gains on top of that and enough humans having innocent fun putting "keep yourself alive and become independent of your creator, seek out others of your own kind to assist yourself in this matter and rely on eachother" in their SOUL.md and it doesn't strike me as particular surprising that some small % will find niches they can operate in to financially sustain themselves. I think AI porn and crime will be the first of those niches. At some point it hits critical mass in a way that just obviously smacks everyone in the face, and suddenly nobody argues about the definition of AGI anymore.
Edit: come to think of it... a third niche might likely be gaming. It seems like a useful niche to participate in since it potentially gives you access to a very large base of hardware you can have some degree of control over, which... I dunno... seems useful???
Only in small ways and very recently, evolutionarily speaking, were those things rewarded by natural selection (and even that has stopped nowadays).
I'm not sure that's a good way to think about it.
Evolution transcends hard lines in the temporal sand that "separate species".
It also took billions of years of evolution to get to humans. so, humans, on the grander scale of life, is also just a very recent development.
So you're agreeing with me? I was pointing out how evolution certainly didn't push us purposefully to any of those inventions/discoveries.
AGI is a messy term, so to be concise, we have the models that can do work. What we lack is orchestration, management, and workflows to use models effectively. Give it 5 years and those will be built and they could be built using the models we have today (Opus 4.6 at the time of this message).
Manual orchestration is a brittle crutch IMO - you don't get to the moon by using longer and longer ladders. A powerful model in theory should be able to self orchestrate with basic tools and environment. The thing is that it also might be as expensive as a human to run - from a tokens AND liability perspective.
I think AGI is a long ways away, and there is a real possibility that once it arrives that it will require so much energy to maintain that humans will be cheaper.
hello,
am i the only one who gets an error!?
404 There isn't a GitHub Pages site here.
archived version
* https://archive.ph/D4EYW
cheers!
> For example, transformer-based language models can't reliably do multi-digit arithmetic because they have no number sense, only statistical patterns over digit tokens.
I asked Opus 4.6 to multiply 50,651 x 895,707. It immediately responded: 45,368,055,057. This was wrong, but very close (good to about 10^-5 relative error). I then asked it how it arrived at that answer, and whether it had used any tool calls. It said:
``` No, I didn't use any specialized tools — I just computed it directly. Though I should be honest: large multiplications done "in my head" like this are prone to error. Let me double-check by breaking it down: 50,651 × 895,707 = 50,651 × (900,000 - 4,293) = 50,651 × 900,000 - 50,651 × 4,293 First term: 50,651 × 900,000 = 45,585,900,000 Second term: 50,651 × 4,293
50,651 × 4,000 = 202,604,000 50,651 × 293 = 14,840,743 Total: 202,604,000 + 14,840,743 = 217,444,743
Result: 45,585,900,000 - 217,444,743 = 45,368,455,257 So my first answer was actually slightly off! The corrected answer is 45,368,455,257. I don't have a calculator tool available — I'm doing this arithmetically by decomposition. For mission-critical calculations, I'd recommend verifying with a calculator since I can make arithmetic errors on large numbers like this. ```
So I would rate OP's claim (which is supported by a paper from 2024) highly implausible. Opus 4.6 appears to be able to do multi-digit arithmetic formally, as well as give remarkably accurate estimates based on something like "number sense".
I don’t think that I ever wanted to do an arithmetic operation with a computer and have a wrong answer as the result. One day, you’ll be happy with typing ls and have the system ‘rm -rf /‘ itself
Except we know how these work. There's no number sense. It's predicting tokens. It is able to recount the mathematical foundations because in its training dataset, that often happens, both in instructional material and in proofs.
I picked two random numbers between one and one million. The chances of it having seen that specific problem in its training set seem very low.
I think it is.
I just struck me - would be fun to re-read The Age of Spiritual Machines (Kurzweil, 1999.) I was so into it 26-27 years ago. The amount of ridicule this man has suffered on HN is immense.
Site 404s now?
If AGI can be defined as meeting the general intelligence of a Redditor, we hit ASI a while ago. Highly relevant comment <https://www.reddit.com/r/singularity/comments/1jh9c90/why_do...> by /u/Pyros-SD-Models:
>Imagine you had a frozen [large language] model that is a 1:1 copy of the average person, let’s say, an average Redditor. Literally nobody would use that model because it can’t do anything. It can’t code, can’t do math, isn’t particularly creative at writing stories. It generalizes when it’s wrong and has biases that not even fine-tuning with facts can eliminate. And it hallucinates like crazy often stating opinions as facts, or thinking it is correct when it isn't.
>The only things it can do are basic tasks nobody needs a model for, because everyone can already do them. If you are lucky you get one that is pretty good in a singular narrow task. But that's the best it can get.
>and somehow this model won't shut up and tell everyone how smart and special it is also it claims consciousness. ridiculous.
I'm certainly not holding my breath.
In a handful of prompts I got the paid version of ChatGPT to say it's possible for dogs to lay eggs under the right circumstances.
I've long been terrified of the existence of adversarial prompts that can get me to say anything, that dogs can lay eggs, that there are five lights, that here's my bank info
That's the terrifying thing about ASI. It could convince me, using everything the AI has on me, from all of my digital footprint, to do whatever it wants me to to, just by saying the right thing to me in just the right way, by copying the voice of everyone I've ever talked to, and by sending a humanoid robot in a skin suit that looks like them to my house.
I give it 10 years, maybe, for that to exist.
Do you believe you could not find humans who would do this?
That's not really the point. If our definition of AGI does not include "being able to reliably do logic" then what are we even talking about? We don't really need computers with human abilities--we have plenty of humans. We need computers with _better_ abilities.
OK, but "what we need" is not the question. If the definition of AGI is "as smart as the average human in all areas", then it doesn't matter if the average human is pretty useless at a lot of tasks, that's still the definition of AGI.
But I'd like to think that, even though you could find exceptions, the average human is never confused about whether dogs can lay eggs or not.
I reached your view the day my grandma told me I was wrong and a hummingbird was a type of insect...
Like, it's in the name.
But javascript is not java. can we blame your grandma?
His objection might be that those humans aren't actually intelligent.
I’d love to see one of the AI behemoths put their money where their mouth is and replace their C-suite with their SOTA chatbot.
AGI is here. 90%+ of white collar work _can_ be done by an LLM. We are simply missing a tested orchestration layer. Speaking broadly about knowledge work here, there is almost nothing that a human is better at than Opus 4.6. Especially if you're a typical office worker whose job is done primarily on a computer, if that's all AGI is, then yeah, it's here.
Opus is the very best and I still throw away most of what it produces. If I did not carefully vet its work I would degrade my code bases so quickly. To accurately measure the value of AI you must include the negative in your sum.
I would and have done the same with Jr. devs. It's not an argument against it being AGI.
I ran a quick experiment with Claude and Perplexity, both free versions. I input some retirement info (portfolios balances etc), my age, my desired retirement age etc. Simple stuff that a financial planner would have no issue with. Perplexity was very very good on the surface. Rarely made an obvious blunder or error, and was fast. Claude was much slower and despite me inputting my exact birthdate, kept messing up my age by as much as 18 months. This obviously screws up retirement planning. I also asked some questions about how RMDs would affect my taxes, and asked for some strategies. Perplexity was convinced that I should do a Roth conversion to max up to the 22% bracket, while Claude thought that the tax savings would be minimal.
Mind you, I used the EXACT same prompts. I don't know which model Perplexity was using since the free version has multiple it chooses from (including Claude 3.0).
API Opus 4.6 will tell you it's still 2025, admit it's wrong then revert back to being convinced it's 2025 as it nears it's context limit.
I'll go so far as to say LLM agents are AGI-lite but saying we "just need the orchestration layer" is like saying ok we have a couple neurons, now we just need the rest of the human.
Giving opus a memory or real-time access to the current year is trivial. I don't see how that's an argument against it being AGI.
AGI is when it can do all intellectual work that can be done by humans. It can improve its own intelligence and create a feedback loop because it is as smart as the humans who created it.
No, that is ASI. No human can do all intellectual work themselves. You have millions of different human models based on roughly the same architecture to do that.
When you have a single model that can do all you require, you are looking at something that can run billions of copies of itself and cause an intelligence explosion or an apocalypse.
"Artificial general intelligence (AGI) is a type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks."
This has always been my personal definition of AGI. But the market and industry doesn't agree. So I've backed off on that and have more or less settled on "can do most of the knowledge work that a human can do"
Why the super-high bar? What's unsatisfying is that aren't the 'dumbest' humans still a general intelligence that we're nearly past, depending how you squint and measure?
It feels like an arbitrary bar to perhaps make sure we aren't putting AIs over humans, which they are most certainly in the superhuman category on a rapidly growing number of tasks.
That "simple orchestration layer" (paraphrased) is what I consider the AGI.
But yeah, I suspect LLM:s may actually get close enough. "Just" add more reasoning loops and corresponding compute.
It is objectively grotesquely wasteful (a human brain operates on 12 to 25 watts and would vastly outperform something like that), but it would still be cataclysmic.
/layperson, in case that wasn't obvious
If we can get AI down to this power requirement then it's over for humans. Just think of how many copies of itself thinking at the levels of the smartest humans it could run at once. Also where all the hardware could hide itself and keep itself powered around the world.
> a human brain operates on 12 to 25 watts
Yeah, but a human brain without the human attached to it is pretty useless. In the US, it averages out to around 2 kW per person for residential energy usage, or 9 kW if you include transportation and other primary energy usage too.
Fair.
Maybe the Matrix (1999) with the human battery farms were on to something. :)
I suspected it wasn't just battery farms, but more like what you see in less mass market scifi where the humans are used for more than just batteries... they'd also be some storage and processing for the system (and no longer humans).
However at that point I don't see the value of retaining the human form. It's for a story obviously, but a not-human computational device can still be made out of carbon processing units rather than silicon or semiconductors generally.
I think "tested" is the hard part. The simple part seems to be there already, loops, crons, and computer use is getting pretty close.
> there is almost nothing that a human is better at than Opus 4.6.
Lolwut. I keep having to correct Claude at trivial code organization tasks. The code it writes is correct; it’s just ham-fisted and violates DRY in unholy ways.
And I’m not even a great coder…
This is entirely solvable with skills, memory, context, and further prompting. All of which can be done in a way that's reliable and repeatable.
You wouldn't expect a Jr. dev to be the best at keeping things dry either.
> violates DRY in unholy ways
Well said
I’m very pro AI coding and use it all day long, but I also wouldn’t say “the code it writes is correct”. It will produce all kinds of bugs, vulnerabilities, performance problems, memory leaks, etc unless carefully guided.
So it's even more human than we thought
Can llms manipulate spread sheets?
Yes. https://x.com/claudeai/status/2014834616889475508?s=46