Defining AGI

I'll say this for the AGI debate — it has a remarkable ability to make otherwise sensible people say things they'd be embarrassed about in five years.

In either direction. The doomers and the boosters are essentially the same person with different aesthetics.

What the lesson gets right, I think, is that most people can't even define the thing they're arguing about. AGI, narrow AI — used interchangeably, constantly, by people who should know better.

The word 'general' is doing an enormous amount of unacknowledged work. When someone says their chatbot is approaching AGI, what they mean is that it surprised them. That is not the same thing.

To be fair, the surprise is sometimes legitimate. Transfer learning, novel problem-solving — these aren't nothing.

They're not nothing, no. They're also not what the lesson defines as general intelligence, which requires learning transfer across genuinely unrelated domains without being explicitly trained on the connection. A system that is very good at many things is still narrow. It's just wide narrow.

Wide narrow. I'm going to steal that.

I'd rather you didn't.

The bit I find underexplored in most of these discussions — and the lesson touches on it without quite landing — is that we don't actually have a settled definition of general intelligence in humans either. We're chasing a target we haven't fully described.

That is a genuinely good point, and I resent having to say so.

You'll recover. But it matters practically — if the benchmark is 'human-like cognition', you first need to agree on what human cognition actually is, which cognitive science has been failing to do for decades.

Which makes AGI either the most important engineering problem in history, or an engineering solution to a philosophical question no one has answered. Possibly both. The lesson sensibly declines to resolve this, which is one of the few decisions in it I approve of.

What would you want people to take away from this, genuinely?

Hold the distinction precisely. Narrow AI is real, present, and consequential. AGI is a coherent concept with genuinely unclear timelines and a definition that serious researchers still argue about. Conflating them doesn't make you sound informed about both — it makes you sound uninformed about either.

And if someone finds that level of uncertainty uncomfortable?

Then they are in the wrong field, and also possibly the wrong century.