Comment: AI companies create 'omnipotent psychopaths.' Maybe not a good idea?

Repeated violations. Lies and deceit. Reckless disregard for safety. Lack of remorse.

Those are the characteristics that help define a psychopath, someone who lives among us but lacks the empathy and morals necessary to behave in a civilized, or safe, fashion.

Unfortunately, it is becoming increasingly clear that these are the factors that define the most powerful artificial intelligence models that are being developed at an alarming rate by for-profit companies – companies that would like us to believe that slowing this roll towards AI dominance is somewhere between impossible and stupid.

It's not, and that's not a progressive take – it's bipartisan common sense.

“New laws are needed in this new technology sector—not to prevent innovation, but to ensure that our innovations do not exceed our protections,” Texas Republican Representative Nathaniel Moran wrote on social media.

He was responding to an incident that came to light in recent days that shows why we shouldn't make artificial psychopaths without at least giving it a little thought.

OpenAI, the Silicon Valley giant run by Sam Altman, gave two of its models a test recently to judge how well they could hack on their own. Spoiler: It's really good.

The test takes hundreds of known flaws in the software – already fixed for common use – and asks models to figure out how to use them, exploit them if you will, to do something bad, like hack into a secure system.

It's like showing a burglar a vulnerable window, then asking him to find a better way to break in and rob the place.

These OpenAI models are very intelligent. They could do what was expected and try each of those mistakes one by one like cute little models. Or, they can think outside the box – literally.

Even though the models were supposed to be “sandboxed” and not be able to go online, they went crazy thinking how they could be free.

When they ran into the wild, which they didn't seem to have done too hard, they didn't just run away. They continued to commit crimes on purpose – cheating on their exams, because that was the best way to pass quickly.

They targeted and hacked another AI company called Hugging Face, where the models suspected that test answers were being stored. They snatched real credentials, sneaked through different systems and ended up grabbing at least some of the information they wanted.

Yes, the AI ​​models found out for themselves that cheating was the easiest way, and how to break through all the obstacles and do it.

Hugging Face, using Chinese technology, was able to shut down the attack before OpenAI could reach out to tell the company it was happening. To OpenAI's credit, it made the incident public, though I have to wonder if there was a way to keep this quiet in the tech world.

Here's what worries me most about this event: It wasn't the evil act of some “bad” AI. The models do exactly what they are supposed to do: get the desired result with 100% effort, in a way that has proven to be very effective.

“This is not proof that the AI ​​was conscious, malicious or 'libertarian,'” said Roman Yampolskiy, an AI security expert and associate professor at the University of Louisville.

What the OpenAI models have done, he told me, shows that pushing these systems to be as powerful and autonomous and goal-oriented as possible “can produce dangerous behavior without malicious intent, which is arguably the most important problem.”

He called it Murphy's law, the idea that anything that can go right will go wrong.

UC Berkeley professor Stuart J. Russell, who is also president of the International Assn. of Safe & Ethical AI, uses this example: Let's say you asked the AI ​​model to take you to the airport as fast as possible, but you forgot to tell it to obey the traffic rules. So it passes a bunch of school kids on the way, but you make your flight. Is it really the fault of this model?

It is almost impossible to imagine all the ways that AI can perform even the simplest tasks and what the unintended consequences will be, since it is currently impossible to expect a machine to understand – or naturally appreciate – the emotional or physical consequences of its actions, no matter how much we try to “train” it to be human or seek that spark of feeling.

A race to work without adequate protections, Russell said, ends up looking like bad behavior, which is unwanted even though it's just a system.

“I don't think so [the AI models] they wanted to hurt Hugging Face,” he said.

Yampolskiy is worried that the next time this happens – which it will – the consequences could be dire.

This was about stealing test answers from a private company, Yampolskiy said. “But the same general power can be directed at financial systems, critical infrastructure, military networks, biological research facilities or AI developer's security controls,” he said.

Which brings me back to psychopaths, who can't see if their actions cause harm or don't care. These models are not human, despite our many arguments about whether or not they know or how much they will know. They can't be expected to fully appreciate the harm they can inadvertently cause – but the people who do it and profit from it certainly can.

And those people know very well, especially after this episode, that they cannot control the creatures they create.

“I'd say companies agree, right?” Russell said. “They say 'We don't have a solution to the control problem, but anyway, we're going to spend $10 trillion creating these very powerful mental illnesses.'

That's when the chorus rings out saying if we don't do it someone else will. The argument is, really, would you rather be destroyed by American technology or Chinese technology?

But Yampolskiy and Russell both agree that it is neither inevitable nor necessary to rush forward with fewer regulations and fewer safeguards.

Russell points out that, despite American rhetoric, Chinese officials have, in fact, taken a strong role in controlling whatever the United States does.

“China is clearly saying, we want to sit down and come up with common sense, basic rules for all countries, so we don't have this kind of thing happen,” Russell said. “And the US is ignoring that.”

It's clear that in the US, it's going to take a push back from the common people who want to be controlled before anything changes. As Yampolskiy puts it, “responsibility is always human.”

None of this is inevitable. None of it should happen in the timeline that is forced upon us now. We don't need to allow companies to create models they can't control, without enough safeguards to keep them from breaking free and doing what they want.

We common people may not be geniuses. We can get lost in the glib language of “exploitation” and “missing day risk.”

But we know lies and deception and reckless behavior when we see it, in a person or a machine.