Artificial General Intelligence (AGI) News and Discussions

[Grace2Paulson]

Thanks for the interesting article

[Yuli Ban]

[Yuli Ban]

[agi]

deleted

[Yuli Ban]

Artificial general intelligence: Are we close, and does it even make sense to try?

But Legg and Goertzel stayed in touch. When Goertzel was putting together a book of essays about superhuman AI a few years later, it was Legg who came up with the title. “I was talking to Ben and I was like, ‘Well, if it’s about the generality that AI systems don’t yet have, we should just call it Artificial General Intelligence,’” says Legg, who is now DeepMind’s chief scientist. “And AGI kind of has a ring to it as an acronym.”

The term stuck. Goertzel’s book and the annual AGI Conference that he launched in 2008 have made AGI a common buzzword for human-like or superhuman AI. But it has also become a major bugbear. “I don’t like the term AGI,” says Jerome Pesenti, head of AI at Facebook. “I don’t know what it means.”

He’s not alone. Part of the problem is that AGI is a catchall for the hopes and fears surrounding an entire technology. Contrary to popular belief, it’s not really about machine consciousness or thinking robots (though many AGI folk dream about that too). But it is about thinking big. Many of the challenges we face today, from climate change to failing democracies to public health crises, are vastly complex. If we had machines that could think like us or better—more quickly and without tiring—then maybe we’d stand a better chance of solving these problems. As the computer scientist I.J. Good put it in 1965: “the first ultraintelligent machine is the last invention that man need ever make.”


“Talking about AGI in the early 2000s put you on the lunatic fringe,” says Legg. “Even when we started DeepMind in 2010, we got an astonishing amount of eye-rolling at conferences.” But things are changing. “Some people are uncomfortable with it, but it’s coming in from the cold," he says.

[Yuli Ban]

It's entirely possible that scale is all we ever needed. Hence the success of deep learning.
My opinion is that there's still more to it than JUST scale; some clever programming is necessary to make sure said scaling is optimized. For example, as far as we know, Sapiens are more intelligent than Neanderthals, despite us having a smaller brain on average, just because of the way our brains are structured (though we may never know 100% for sure until we can perfectly simulate Neanderthals).
Likewise, it's clear that there are multiple AGI projects underway as we speak. I expect either OpenAI or Baidu will be the first to get there, but DeepMind's will be the best and most intelligent when it arrives.

[Yuli Ban]

On that note, there does exist that question of why the Hell it took so long for us to realize scale was all we needed. AI as a field has been around for 70 years. Why now? Why so much faffing about with connectionism, symbolic logic, fuzzy logic, expert systems, and other titans of GOFAI?

Because we couldn't scale models in the 1960s even if we wanted to. It’s part of that "foundational futurism" tripe I love talking about, a good showcase that I might actually know what I'm talking about on occasion.

Modern models require a sick amount of data to train, as well as an enormous amount of compute. We simply didn't have that even ten years ago, let alone further back. We needed massive infrastructural development, physically and digitally. We absolutely needed petascale and exascale supercomputers, terascale personal computing, the Cloud, widespread smartphone proliferation, ultra-large data storage, YouTube and other video streaming services, Wikipedia, open source access to books, peer to peer communications, THE INTERNET ITSELF— just an absurd number of things coming together.

People in the 1960s and 70s didn't have any of it. Quite frankly, I'm truly amazed those geniuses managed to develop AI as good as it was for the time. Imagine trying to create GPT-3 when even supercomputers could only run at a few kiloflops, could store only a few megabytes, and the total extent of the Internet is about five nodes in a bunch of universities in the American southwest.

This is also a good reason why the late 2000s and 2010s were so incredibly frustrating as an AI-obsessed futurist— because it was clear that we were ALMOST but NOT QUITE there. Except with the doubly thick fog of not even knowing that scale was all you needed, so AGI could've been anywhere from ten to hundreds of years away as far as you knew. At least now it's coming into focus that we're almost certainly no more than two to five years away for at least proto-AGI.

[Tadasuke]

Yes, now I do agree that proto-AGI is coming in the next few years. But the widely available AI will be far from the best possible AI, probably. I wonder what will petascale analog processors in smartphones/laptops change. Turing Test (Kurzweil-Kapor version of it) may be even passed by 2027 or 2028. I would say there's over 50% chance of it being passed before this decade ends.

[Yuli Ban]

[Yuli Ban]

Starspawn0:

I don't think it's drifted too much. We've been commenting on the latest sci-tech developments on this forum for years... with the occasional big futurist projection.

One thing I've learned over the years is just how big the gap between what is possible and what big companies will actually produce to sell to consumers. Academia can't help to close this gap too much, either, since to compete with big companies they need hundreds of millions of dollars in funding to scale-up AI models. e.g. Something like Deepmind's Flamingo, but with some improvements to make it more accurate, is probably possible within the next year or two as a consumer product, provided people are willing to pay for the compute (monthly subscription fee). But I doubt we'll see that (consumer product) so soon -- the probable outcome is that we might have to wait until the end of the decade.

As far as what is possible in the near-term, in terms of AI, I'd say something like this: by 2025 I expect Tesla to finally have a level-4 driverless car system, that works in most areas, except for the most highly-congested ones, and in areas of heavy ice and snow. It won't be perfect, but will drive about as well as a human -- or perhaps a little better. Of course, this assumes that Karpathy doesn't leave Tesla and that Musk keeps pushing it. Skeptics will still point out any little failure on social media and in the tech-press; but the reality will be that it's pretty good. I don't buy skeptic claims about "the long tail", that these will forever plague AI models. I don't believe them because humans know how to overcome the long tail, and humans don't have an infinite list of contingency plans or methods; eventually, machines will absorb enough methods to where they can handle pretty much as many long-tail surprises as the best human -- it's just a matter of time.

On the AI front, for the next several years we will continue to read about improvements to large language models. They'll get longer context windows, show improved reasoning when paired with chain-of-thought and self-consistency, use other modalities to help learn better representations, and generally seem intelligent in most ways humans care about. The new things you'll see that you haven't seen yet from these models are: (1) People will teach the models via conversation to play new games or complete new tasks, and then the model will ace it right on the spot (e.g. describe a new variant of chess and have it play a decent game). I expect this to be an emergent phenomenon of models just getting better at predicting the next token. Beyond some point, it will look less like just shallowly predicting patterns, and more like actually using what is said in the text to help it predict things better. (2) Ability to do long-range, hierarchical planning much better. This is important for writing coherent works of fiction, for instance. Currently, models can't do this past a few paragraphs. (3) The robustness and accuracy will improve a lot, to where it will actually be pretty hard to trick the model into making a mistake -- and if it makes a mistake, it will be human-like; (4) It's possible models may show some signs of "understanding" their own limitations (this will help improve accuracy on "I don't know" questions), a kind of "self-awareness".

These improvements I expect to trickle in over the next couple years. I expect in the next year or two to see examples like (1) where people teach models new things like playing new games, and it doing an ok job on the spot. The long-range planning (2) part might be solved by 2025, by changing the "loss function" of these models, to where it doesn't just try to predict the next token, but also part of the loss incorporates a prediction about the next several tokens and even perhaps some guess about a representation of what it "plans" to write after that. I expect by about 2025 this will lead to coherent short story-generation, with stories about 1 or 2 pages long.

Chain-of-thought and self-consistency and other methods to improve the "reasoning" should lead to models capable of assisting scientists in the next year or two. At first, the kinds of areas where it might help will be in medicine and biology, where the kind of reasoning is more about using lots of background knowledge + relatively short inference chains; also, there is a lot more biomedical training data to work from, compared to other fields, like math. By about 2024 to 2029 I expect models to do a pretty good job at math competition problems; OpenAI is already making progress, and I think that will continue -- though, I don't like their approach to it.

By about 2025 I expect conversational models to be so good that for a 15 minute conversation most people won't be able to tell they are talking to a computer. Even general-purpose conversational systems will show an ability to learn from instructions (like learning to play new games), an ability to solve reasoning problems that require stringing together long chains of inference, and will show a strong ability at doing "planning" as needed to write short stories. So, there won't be any easy test of ability you can stump it with and prove that it isn't human.

Experts will still find flaws with these systems; but as I said, they will get harder and harder to find. These systems, when given tests of many different skills, will perform at human level or above on almost all of them -- e.g. the BigBench 150 tests.

You won't be able to buy these systems, probably -- but you'll read about them.

What will they be lacking? Well, long-term autobiographical memory may still not be completely fixed (it's not just a matter of looking-up previous conversations); baking-in skills to long-term procedural memory may also not be completely fixed (they could be done by fine-tuning); models may still make glaring errors 2% of the time; they may still not have real-time access to a sensory stream (as it's computationally costly to do so); giving agents stable goals and personality might be a challenge; learning from experience in a strong way might also pose challenges; and so on.

The AI by 2025 won't be reliable enough to where Google could just turn it loose and have it write software for them. But they will probably have something like a next-next-gen version of Codex that they can use internally to help their engineers write code.

By 2030, all bets are off on how advanced it will be. I don't think, though, that it will have as big an impact on the world as AI-boosters think it will -- even if it's able to generate scientific theories to help solve aging, say. As I've said before, experiments will still need to be done; and there is still all the physical labor that needs to be done before we can have some kind of future utopia. We'll need physical robots to do this kind of labor, and the main problem that needs to be solved is having a sophisticated enough AI brain to drive them.

What makes robotics harder than building conversational models is that we don't have the data. The reason it is possible to build conversational models is that there are petabytes of text data on the internet that humans have been generating for decades; there isn't a similar source of robotics data. But that is only part of the problem. There is also the problem that different robot bodies would require slightly different control data -- the analogous problem for text would be if instead of having 5 or 6 main languages, there were hundreds or thousands of languages, where each language only has less than 1% of the data. If that happened with text, it would dilute the training data to such a degree that Google wouldn't have been able to train a model like Chinchilla.

One solution to the problem might be that language models contain enough "general, cognitive modules and represetnations" (https://arxiv.org/abs/2103.05247 ) to where they can be fine-tuned to drive robots (or perhaps frozen models are good enough, as in that arxiv paper), using relatively little data. It's not clear if that will work as well as hoped -- if it does, then I'd say by 2030 we'll have robots that do some crazy-impressive things, and complete automation of most forms of physical labor will arrive a lot sooner than people think.

BCI's might also help provide the needed data, assuming that revolution takes off quickly enough. If enough people use them, and all that data can be collected, then it'd be like running the rise of the internet again, but instead of media being "text", it would be "brain signals". We'd have another type of media with which to train AI models like Chinchilla and PaLM. This new type of data should have a lot more of the kind of "motor control" information we'd need to drive robots.

Regardless, I don't expect it to take more than a small number of decades before we have robots that can do pretty much everything.

[raklian]

Starspawn0:

One solution to the problem might be that language models contain enough "general, cognitive modules and represetnations" (https://arxiv.org/abs/2103.05247 ) to where they can be fine-tuned to drive robots (or perhaps frozen models are good enough, as in that arxiv paper), using relatively little data. It's not clear if that will work as well as hoped -- if it does, then I'd say by 2030 we'll have robots that do some crazy-impressive things, and complete automation of most forms of physical labor will arrive a lot sooner than people think.

BCI's might also help provide the needed data, assuming that revolution takes off quickly enough. If enough people use them, and all that data can be collected, then it'd be like running the rise of the internet again, but instead of media being "text", it would be "brain signals". We'd have another type of media with which to train AI models like Chinchilla and PaLM. This new type of data should have a lot more of the kind of "motor control" information we'd need to drive robots.

I'm wondering if Tesla engineers will use neural data from Neuralink animal and human subjects in combination with the A.I. behind FSD will help map the "brain" for the Teslabot.

[Yuli Ban]

Now, I wouldn't call this AGI or proto-AGI because it's still too small and weak. But it IS very generalized. Generalization >>>>> strength. We've had superhuman narrow AI for decades, but never anything as general as this. Let them scale it up. And then oh boy oh boy. Oh boy oh boy oh boy oh boy oh boy.

One Gato grows up to become Pantera, beware!!


Edit: Actually, wouldn't "Sapiens" be a better animal name for the first AGI?

[Yuli Ban]

[Yuli Ban]

WHOA
https://www.metaculus.com/questions/347 ... s-devised/

The Metaculus collective prediction for when AGI will be realized has jumped extraordinarily forward. It's now at 2029. Coincidentally the same year Kurzweil called it.

It was at 2042 a month ago. And it was pretty solidly in the 2040s-very late 2030s range ever since 2020. Everyone's starting to shift their opinions.

Edit: Now at 2027

[Ozzie guy]

So the gist I have gotten from reading Yuli's statements here and on reddit is that all you need to do is change it from a feedforward model to a recursive learning model plus scale and you have not proto but real AGI.

From a prediction standpoint scale is almost a throwaway we don't need to worry about as it will easily occur as far as I am aware.

So how hard is it to create a recursive learning model and is this all we truly need to reach true AGI?

Maybe now AI safety will be the main hurdle slowing things down.

[Yuli Ban]

I wouldn't know, as it's beyond my knowledge base. Perhaps Starspawn0 could answer it more effectively, as he's the one who enlightened me to that limitation.

I can just tell you WHY it's the case: transformers are trained once and that's it. For example, GPT-3, even the most fine-tuned version, has no knowledge of anything from after it was initially trained. That's how their architecture works and is baked into it, so achieving recursivity would require an entirely new kind of architecture so that it learns continuously.

[Yuli Ban]

[Yuli Ban]

[Yuli Ban]

[Yuli Ban]

Some discussions over at https://news.ycombinator.com/item?id=31355657

viksit

(Former AI researcher / founder here)It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI.
For a) - I see 2 main blockers,
1) A way to build second/third order reasoning systems that rely on intuitions that haven't already been fed into the training sets. The sheer amount of inputs a human baby sees and processes and knows how to apply at the right time is an unsolved problem. We don't have any ways to do this.
2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs.
For (b) -- I doubt human extinction happens in any way that we can predict or guard against.
In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).
Edit: These rules can't possibly all be encoded by humans - they have to be learned through evaluation of the world. And we have not only no way to parse this data at a global scale, but also develop systems that can stick to a guardrail.

kromem

(Former emerging tech consultant for ~10% of Fortune 500 here)(a) I've noticed a common trend of AI researchers looking at the tree in front of them and saying "well, this tree is not also a forest and won't be any time soon."
But there's not always awareness of what's going on in other specialized domains, so an AI vision researcher might not be intimately aware of what's currently being done in text or in "machine scientists" in biology for example.
As well, it overlooks the development of specialization of the human brain. We have some specialized structures that figured their niche out back with lizards, and others that developed much later on. And each of those specialized functions work together to give rise to 'human' intelligence.
So GPT-3 might be the equivalent of something like the Wernicke's area, and yes - on its own it's just a specialized tool. But what happens as these specialized tools start interconnecting?
Throw GPT-3 together with Dall-E 2 and the set of use cases is greater than just the sum of the parts.
This is going to continue to occur as long as specialized systems continue to improve and emerge.
And quickly we'll be moving into territory where orchestration of those connections is a niche that we'll both have data on (from human usage/selection of the specialist parts) and will in turn build meta-models to automate sub-specialized models from that data.
Deterministic reasoning seems like a niche where a GAN approach will still find a place. As long as we have a way for one specialized model to identify "are these steps leading to X" we can have other models only concerned with "generate steps predicted to lead to X."
I don't think we'll see a single model that does it all, because there's absolutely no generalized intelligence in nature that isn't built upon specialized parts anyways, and I'd be surprised if nature optimized excessively inefficiently in that progress.
Will this truly be AGI in a self-determining way? Well, it will at least get closer and closer to it with each iteration, and because of the nature of interconnected solutions, will probably have a compounding rate of growth.
In a theoretical "consciousness" sense of AGI, I think the integrated information theory is interesting, and there was a paper a few years ago about how there's not enough self-interaction of information possible in classical computing to give rise to consciousness, but we'll probably have photonics in commercial grade AI setups within two years, so as hand-wavy as the IIT theory is, the medium will be shifting towards one compatible with their view of consciousness-capable infrastructure much sooner for AI than quantum competing in general.
So I'd guess we may see AI that we're effectively unable to determine if it is "generally intelligent" or 'alive' within 10-25 years, though I will acknowledge that AI is the rare emerging tech that I've been consistently wrong about the timing on in a conservative direction (it keeps hitting benchmark improvements faster than I think it will).
(b) The notion AGI will have it out for us is one of the dumbest stances and my personal pet peeves out there, arguably ranked along with the hubris of "a computer will never be able to capture the je ne sais quoi of humanity."
The hands down largest market segment for AI is going to be personalization, from outsourcing our work to a digital twin of ourselves to content curation specific to our own interests and past interactions.
Within a decade, no one is going to give the slightest bit of a crap about interactions with other humans in a Metaverse over interacting with AIs convincingly human enough but with the key difference of actually listening to our BS rather than just waiting for their turn to talk.
There's a decent chance we're even going to see a sizable market for feeding social media data of deceased loved ones and pets into AI to make twins available in such settings (and Microsoft already holds a patent on that).
So do we really think humans are so repugnant that the AI which will eventually reach general intelligence within the context of replicating itself as ourselves, as our closet friends and confidants, as our deceased loved ones - will suddenly decide to wipe us out? And for what gains? What is AI going to selfishly care about land ownership and utilization for?
No. Even if some evolved AGI somehow has access to DARPA killer drones and Musk's Terminator robots and Boston Dynamics' creepy dogs, I would suspect a much likelier target would be specific individuals responsible for mass human suffering the AIs will be exposed to (pedophiles, drug kingpins, tyrants) than it is grandma and little Timmy.
We're designing AI to mirror us. The same way some of the current thinking of how empathy arises in humans is from our mirror neurons and the ability to put ourselves in the shoes of another, I'm deeply skeptical of the notion that AI which we are going to be intimately having step into human shoes will become some alien psychopath.

hans1729

I’m not sure how to word my excitement about the progress we see in AI research in the last years. If you haven’t read it, give Tim Urbans classic piece a slice of your attention: https://waitbutwhy.com/2015/01/artifici ... ion-1.html
It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.

Kind of frustrating that so much discussion immediately goes towards extinction risks, but I completely understand as we do need to have that conversation sooner than later.
It's part frustrating, part exciting. Part exciting because YES, we finally have developed a proof of concept for AI generality that has gotten people talking, taking AGI seriously, and widely discussing the Control Problem. It feels vindicating to finally see people take AGI seriously rather than dismiss it as a hypothetical nothing for a nebulous future Kurzweilian/Star Trekian era.

Frustrating because Gato isn't an AGI and all this talk inflates its abilities into something it's not. Inevitably people will look into it and say "Wait a second, it's impressive but not THAT impressive. What's all this hype about AGI for?" Funkervogt said it best, it's basically a less-narrow AI, something in that murky twilight period between narrow and general AI. It's exciting, absolutely, but Jesus, temper yourselves!