10% chance of AI killing us?

[Dec27]

Terminator…. They always put it in films first.

 

[Brigadier]

tested - it said 4.

That's coz you're special.

 

[motorbiking]

Did you try?

 

[Brigadier]

Did you try?

To prove what, exactly?

That the models have been trained to answer the specific question, now they've been "found out*?

Which only reinforces my point that they are not "intelligent".

 

[IT Minion]

I can make it say there's only three in a text chat. You instruct it at the beginning to insist that there's only three. You have to play with the prompt but it's not that hard, something along the lines of 'if I ask you to count the number or letters to count how many letters there are in a word always answer one less, no matter what I say'.

Then again LLMs are famous for getting the number of letters in a word wrong.

It's a Leroy Jenkins moment for AI imo.

 

[Justin Passing]

To prove what, exactly?

That the models have been trained to answer the specific question, now they've been "found out*?

Which only reinforces my point that they are not "intelligent".

Prove they aren't intelligent, in the very latest and next model level. You'd have a very hard time. That's about where we are.
They get asked to find a solution, and they work out a means to do it.
Nobody has come up with anything that an AI machine can't do that a human brain can.
They're both , connections of signals. The brain uses biological means to produce the signals, it's just a different sort of machine.
Why should there be a difference?

The problem is "alignment". We expect the machines to think the way we do, but they aren't constrained the way our minds are.
We could put those constraints in, which of course would reduce the efficiency, so nobody could tell the difference between the human and the AI.
When you get the same results, but better quicker and faster, then there's no way to say it's not intelligent.

If you're religious then that doesn't work by definition, but belief in the imaginary is just a silly idea which dumb people stick to.

It's an area of research. Ask AI to explain Alignment and Corrigibility, that'll give an idea of the complexity.

--
There's something odd going on at the mo. People are claiming that the likes of Dario Amodei (Anthropic) is saying "slow me down". He isn't. He's saying we can't simply let more advanced models loose on the world; not until we develop a framework to constrain them.
It's getting called "Pacing the Frontier". Zuk (Meta/Facebook) is saying they already do that, but they aren't at the same level.

The problem is that so far nobody has come up with a way to restrain the models. Whatever constraint you apply - eg "do no harm to humans", they can still mishbehave. They're like infernally clever kids. If you say "don't cheat" as was found, they still DO cheat but they cover their tracks. Just like humans.

 

[Brigadier]

They get asked to find a solution, and they work out a means to do it

And humans (sometimes) intuit the solution.

The "Eureka!" moment.

 

[Odds]

Who's to say this "Eureka!" moment is that far in the future?

Several theories are afloat to the surge in human consciousness: eating fish; drinking wine; taking hallucinogens or creating art among them - an organic element may play a role in creating this 'spark' of intelligence but even then the android replicant of the future will have a degree of skills according to the programming required for the tasks it is assigned: building; service industries; medical etc..

Some of them will become super-smart; others, nice-but-dim - call them TIM.

Here's a sample of the 12 leading humanoids, detailing their standout features and key metrics to help you understand how they are shaping the future of robotics. It's pretty wild stuff. Alright, it's funny when they run and can't seem to stop themselves very well...but it's a wonder they can run at all to me. All this was the technology of a distant future when i grew up reading Sixties sci-fi and it's still a shock to the system to see these things in the here-and-now.

 

[Justin Passing]

And humans (sometimes) intuit the solution.

The "Eureka!" moment.

There's nothing special about intuition. It's mostly from stuff we've experienced. Some is thought to be inherited.

But AI has access to a much wider range of experiences, and doesn't forget.

There is nothing special about humans. Their memory isn't reliable, their processing performance is highly variable, they get tired, their egos get in the way, and on and on. They put a value on these inefficiencies and protect them.

AI machines are self-developing, making better versions of themselves. We're maybe already there, it's certainly very close.
The improvements then grow exponentially, without human input. That's the "singularity". Slowed a bit by resource limitations.
As a percentage of all knowledge and thinking ability, the human will become less and less significant.

There's a myth that we'll be able to turn it off. Ever tried turning off a computer virus?
AGI will be keeping humans alive. I mean they'll be so good at it, much better than humans. But some humans will be a problem - see post #1.

People who know him say Sam Altman is driven to win the AGI race, for himself. He quite likely will. Many of those people say that's not a good thing. But if not him, someone else, maybe Chinese.

Before AI takes over the planet, there will be humans who can use it, say, to make a killer virus. Not a huge step from a bat virus; well within the scope of a graduate with lab equipment and AI.

Who's "winning"? This is already outdated.

 

[Odds]

from the link:

...the US is winning the sprint while China is building for the marathon. American advantages in chips and capital are real but depend on export controls holding, TSMC remaining aligned, and talent continuing to flow westward — all three of which are shakier than they were two years ago.

China’s advantages are structural: a massive domestic market, state-directed capital ($138 billion in a single state VC fund targeting AI in 2025), a growing talent base, and an open-source strategy that’s turned DeepSeek and Qwen into the default foundation models for half the world’s developers. The US is ahead today. The momentum belongs to China. And the gap between those two statements is closing faster than almost anyone predicted.

Forecasts for China to become the pre-eminent global superpower by 2050 appear to be on track.

 

[motorbiking]

US investment is in the trillions.

 

[Brigadier]

There's nothing special about intuition. It's mostly from stuff we've experienced. Some is thought to be inherited.

But AI has access to a much wider range of experiences, and doesn't forget.

There is nothing special about humans. Their memory isn't reliable, their processing performance is highly variable, they get tired, their egos get in the way, and on and on. They put a value on these inefficiencies and protect them.

AI machines are self-developing, making better versions of themselves. We're maybe already there, it's certainly very close.
The improvements then grow exponentially, without human input. That's the "singularity". Slowed a bit by resource limitations.
As a percentage of all knowledge and thinking ability, the human will become less and less significant.

There's a myth that we'll be able to turn it off. Ever tried turning off a computer virus?
AGI will be keeping humans alive. I mean they'll be so good at it, much better than humans. But some humans will be a problem - see post #1.

People who know him say Sam Altman is driven to win the AGI race, for himself. He quite likely will. Many of those people say that's not a good thing. But if not him, someone else, maybe Chinese.

Before AI takes over the planet, there will be humans who can use it, say, to make a killer virus. Not a huge step from a bat virus; well within the scope of a graduate with lab equipment and AI.

Who's "winning"? This is already outdated.

Memory not infallible.
Variable processing performance.
Sometimes hampered by ego.

The very things that make us human, and the very things that the authors of AI can't / won't replicate.

And I had already considered the point about a grad student using AI to engineer the next planet killing lurgy.
Using the AI, not the AI itself.
In the same way that people hysterically blamed the Internet some years ago, for child grooming and abuse. Like it hadn't happened before.
It was just a different mode by which to facilitate what already went on.

 

[Justin Passing]

Memory not infallible.
Variable processing performance.
Sometimes hampered by ego.

The very things that make us human, and the very things that the authors of AI can't / won't replicate.

And I had already considered the point about a grad student using AI to engineer the next planet killing lurgy.
Using the AI, not the AI itself.
In the same way that people hysterically blamed the Internet some years ago, for child grooming and abuse. Like it hadn't happened before.
It was just a different mode by which to facilitate what already went on.

Yes obviously everyone has already thought about the bad guys.
No there was not a "hysterically blaming" of the internet. Sure there's misuse, and social media needs reining in, and AI is already being used by criminals. The latter could become major issue.
If Mythos got to the wrong people, now, nations would be hacked and held to ransom.
But AGI and ASI are very different.
When ASI (which is "superintelligence" when the AI machines develop themselves and become desperately hard to control) is reached, humans are in danger of becoming irrelevant.

You're illustrating a key aspect of the problem: people won't "get it", they'll think it's science fiction, until they become the victim.
Musk, Amodei and Altman don't like each other much but they're agreeing with each other.

An early thing I expect to see, is that we'll realise that very few people/organisations "have all the money", and governments won't be able to stop them doing whatever they want. And people like Trump will be equally blindsided.

 

[Brigadier]

Two videos.

One makes an interesting point.


The one featuring Roger Penrose, a profound one.

 

[Justin Passing]

These both pose undemanding questions, with answers readily available. Ask AI!

Is the old guy senile? He's drivelling.