Artificial General Intelligence (AGI) News and Discussions

[wjfox]

[funkervogt]

Chollet believes OpenAI spent “tens of millions” on compute in 2025 to train models specifically for ARC-AGI-2, using publicly available ARC puzzle samples to generate additional training data. “What this amounts to is preemptive brute forcing … by trying to guess in advance every possible task,” he says.

At any rate, the tactics worked: top scores rose to 40–50% by December 2025, Knoop says.

“I expect the same will happen with ARC-3, but with ARC-3 it’s going to be harder,” Chollet says. “It’ll be more expensive.”

https://www.fastcompany.com/91515360/ar ... -benchmark

I predict the old pattern will repeat: LLMs will start scoring high on ARC-AGI-3, True Believers will point to it as proof that AGI is a few seconds from being created, Constant Cynics will scoff and point out all the examples of those LLMs lacking intelligence, Chollet will say the tech companies figured out how to game the test, and he will release ARC-AGI-4.

In a bad timeline, this pattern and the associated arguing and hand-waving go on for the next 25 years.

[firestar464]

I don't think Chollet has acted like it's going to be otherwise. The idea is to keep making benchmarks until we figure out the best one to measure reasoning ability.

[Yuli Ban]

Chollet believes OpenAI spent “tens of millions” on compute in 2025 to train models specifically for ARC-AGI-2, using publicly available ARC puzzle samples to generate additional training data. “What this amounts to is preemptive brute forcing … by trying to guess in advance every possible task,” he says.

At any rate, the tactics worked: top scores rose to 40–50% by December 2025, Knoop says.

“I expect the same will happen with ARC-3, but with ARC-3 it’s going to be harder,” Chollet says. “It’ll be more expensive.”

https://www.fastcompany.com/91515360/ar ... -benchmark

I predict the old pattern will repeat: LLMs will start scoring high on ARC-AGI-3, True Believers will point to it as proof that AGI is a few seconds from being created, Constant Cynics will scoff and point out all the examples of those LLMs lacking intelligence, Chollet will say the tech companies figured out how to game the test, and he will release ARC-AGI-4.

In a bad timeline, this pattern and the associated arguing and hand-waving go on for the next 25 years.

That bad timeline is what if we keep focusing on attention-based transformers with no attempt to look for anything else.
Even the most cursory glance at, say, neurosymbolic architectures and Monte carlo tree search + LLMs could give us something far, far more worthwhile than what we have now. It'd just take a lot of reworking and retraining.

[wjfox]

[weatheriscool]

[firestar464]

(Thought the full context would be helpful)

[weatheriscool]

This is weird how he wants grok 5 to be agi. Could take years of going up from 4.3, 4.4, 4.5, 4.6, 4.95, 4.99, 4.995, etc

[weatheriscool]

My standard for AGI is an android of the ability and function of Commander Data of Star trek. We'll have to construct this level of android so it will have the tools it needs to prove its case. It needs to be able to do everything a human can do at all things. Needs all the tools of a human in order to do that.

AGI is a hell of a lot more then a voice or running around.

-Needs to walk
-Needs to talk in a human way and be able to talk to people...Needs to be able to understand and convince.
-Needs to reason through the environment and work it has before it.
-Needs to study and learn and rethink based on new information
-Needs to write papers of advance science with its hand
-Needs to type it out
-Needs to be able to do work and work with other people as an equal
Can't have weaknesses or it isn't agi!
on and on

[weatheriscool]

[weatheriscool]

[firestar464]

From what I understand, most LLM-based AIs aren't really conscious because they are fundamentally divided into learning (training) and doing (output) phases, in contrast to humans, who learn and do at the same time. They are essentially unable to form new memories and recontextualize them.

It's interesting that this might get somewhat closer to that on a per-session basis (each session is an entity), though I'm not qualified enough to say how close.