The IT Thread

[SYZ]

The one thing—a major, limiting thing—that AI and/or ML
may never incorporate is logic. As in human logic. Which
is why we need not fear robots taking over the world, as
some people seem to think.

The sentient HAL 9000 is merely a fifty-year-old fiction.

[Inkubus]

The one thing—a major, limiting thing—that AI and/or ML
may never incorporate is logic.  As in human logic. Which
is why we need not fear robots taking over the world, as
some people seem to think.  

The sentient HAL 9000 is merely a fifty-year-old fiction.

Or mebbie not:

Park and his colleagues initiated this study after a 2022 Science study by Meta caught their attention. The study described CICERO, an AI system created by Meta to excel in the alliance-building, world-conquest board game Diplomacy...

Their analysis showed that CICERO failed to be honest despite being trained to be truthful. It learned to misrepresent its preferences to gain an upper hand in the negotiations, the paper noted. It also built a fake alliance with a human player to trick them into leaving themselves undefended during an attack.Link

[SYZ]

...

Park and his colleagues initiated this study after a 2022 Science study by Meta caught their attention. The study described CICERO, an AI system created by Meta to excel in the alliance-building, world-conquest board game Diplomacy...

Their analysis showed that CICERO failed to be honest despite being trained to be truthful. It learned to misrepresent its preferences to gain an upper hand in the negotiations, the paper noted. It also built a fake alliance with a human player to trick them into leaving themselves undefended during an attack.Link

(my bold)

That's a classic example of ML, rather than true AI...  I think?

[mordant]

The one thing—a major, limiting thing—that AI and/or ML
may never incorporate is logic.  As in human logic. Which
is why we need not fear robots taking over the world, as
some people seem to think.  

The sentient HAL 9000 is merely a fifty-year-old fiction.

Or mebbie not:

Park and his colleagues initiated this study after a 2022 Science study by Meta caught their attention. The study described CICERO, an AI system created by Meta to excel in the alliance-building, world-conquest board game Diplomacy...

Their analysis showed that CICERO failed to be honest despite being trained to be truthful. It learned to misrepresent its preferences to gain an upper hand in the negotiations, the paper noted. It also built a fake alliance with a human player to trick them into leaving themselves undefended during an attack.Link

Some guy from MIT wrote a seminal paper on this topic. In it he speaks of the fact that once you define a state where an agent is forbidden from X, which takes a lot of resources, it requires almost no resources to "flip the bit" and do X anyway. The problem is that in creating the angel, you also create the demon.

[Inkubus]

A bit long but I was fascinated:

We are as a matter of fact right now building creepy super capable amoral psychopaths that never sleep, think much faster than us, can make copies of themselves and have nothing human about them whatsoever. What could possibly go wrong.

AI is now deployed in the pentagon's top secret Cloud. Edward Snowden said the intersection of AI with the ocean of mass surveillance data is going to put terrible powers in the hands of an unaccountable few and they may not be able to keep hold of it. Paul Cristiano said that eventually AI systems will be able to prevent humans from turning them off but much sooner than this AI will get out of control.

Humanity's AI evolution is not out of our control not yet, there are promising areas of research. The problem is that nearly all AI research funding of hundreds of billions per year is pushing capabilities for profit, safety efforts are tiny in comparison.

Nick Bostrom and the future of Life Institute, join us in calling for international AI safety research projects. We don't know if it will be possible to maintain control of superintelligence but we can point it in the right direction, instead of rushing to create it with no moral compass and clear reasons to kill us off. There will be no warning shot. Our greatest risk is increasingly hidden even as it spreads into all our systems.

It was popularly said around these parts a bucket of water would sort it. Not any more.