John Savage

The Backdoor Challenge of AI Machine-Learning

December 6, 2024 by Llewellyn King Leave a Comment

The great race is on. It isn’t the one on television, but it is one that has put the world’s wealthiest companies in fierce competition to secure market share in artificial intelligence.

The handful of big-tech companies and their satellites may have spent as much as $1 trillion on machine-learning and data center infrastructure to stuff their AI systems with billions of bits of information hoovered up from public and private sources on the internet.

These companies — Amazon, Google, Meta, Microsoft and OpenAI among them — are rich and have made their creators rich beyond compare because of information technology. Their challenge is to hold onto what they have now and to secure their futures in the next great opportunity: AI.

An unfortunate result of the wild dash to secure the franchise is that the big-tech companies — and I have confirmed this with some senior employees — have rushed new products to market before they are ready.

The racers figure that the embarrassment of so-called hallucinations (errors) is better than letting a competitor get out in front.

The challenge is that if one of the companies — and Google is often mentioned — isn’t on the leaderboard, it could fail. It could happen: Remember “MySpace”?

The downside of this speedy race is that safety systems aren’t in place or effective — a danger that could spell operational catastrophe, particularly regarding so-called backdoors.

According to two savants in the AI world, Derek Reveron and John Savage, there is a clear-and-present danger presented by this urgency for market speed over dangerous consequences.

Savage is the An Wang professor emeritus of computer science at Brown, and Reveron is chair of the National Security Affairs Department at the Naval War College in Newport, Rhode Island.

Reveron and Savage have been sounding the alarm on backdoors, first in their book, “Security in the Cyber Age: An Introduction to Policy and Technology,” published by Cambridge University Press early this year, and later in an article in Binding Hook, a British website with a focus on cybersecurity and AI.

“AI systems are trained neural networks, not computer programs. A neural net has many artificial neurons with parameters on neuron inputs that are adjusted (trained) to achieve a close match between the actual and the desired outputs. The inputs (stimuli) and desired output responses constitute a training set, and the process of training a neural net is called machine-learning,” the co-authors write.

Backdoors were initially developed by telephone companies to assist the government in criminal or national security cases. That was before AI.

Savage told me that backdoors pose a grave threat because, through them, bad actors can insert malign information — commands or instructions — into computers in general and backdoors in machine-learning-based AI systems in particular. Some backdoors can be undetectable and capable of inflicting great damage.

Savage said he is especially worried about the military using AI prematurely and making the nation more vulnerable rather than safer.

He said an example would be a weapon fired from a drone fighter jet flying under AI guidance alongside a piloted fighter jet where the weapon fired by a drone could be directed to do a U-turn and come right back and destroy the piloted plane. Extrapolate that to the battlefield or to an aerial bombardment.

Savage says that researchers have recently shown that undetectable backdoors can be inserted into AI systems during the training process, which is a new, extremely serious, and largely unappreciated cybersecurity hazard.

The risk is exacerbated because feeding billions of words into big-tech companies’ machine-learning systems is now done in low-wage countries. This was highlighted in a recent “60 Minutes” episode about workers in Kenya earning $2 an hour, feeding data to machine-learning systems for American tech companies.

The bad actors can attack American AI by inserting dangerous misinformation in Kenya or in any other low-wage country. Of course, they can launch backdoor attacks here, too, where AI is used to write code, and then control for that code is lost.

In their Binding Hook article, Reveron and Savage make a critical point about AI. It isn’t just another more advanced computer system. It is fundamentally different and less manageable by its human masters. It lacks an underlying theory to explain its anomalous behavior, which is why the AI specialists who train machine-learning systems cannot explain this behavior.

Deploying technology with serious deficits is always risky until a way to compensate for them has been discovered. Trouble in is trouble out.

Cyberattack on the Infrastructure Alarms Petraeus, Coats

September 9, 2018 by Llewellyn King Leave a Comment

War always goes for the infrastructure: take out the bridges, cut off the electricity and water supplies. All that used to be done with artillery, tanks and bombs.

Going forward, it will be done by computers: Cyberwar.

Every day the early skirmishes — the tryout phase, if you will – are taking place. There are tens of thousands of probes of U.S. infrastructure by potential enemies, known and unknown, state and non-state. A few get through the defenses.

Jeremy Samide, chief executive officer of Stealthcare, a company which seeks to improve cyberdefenses for a diverse set of U.S. companies, sees the cyber battlefield starkly. He says the threat is very real; and he puts the threat of serious attack at 83 percent.

Jeremy Samide is chief executive officer of Stealthcare.

As Samide looks out across the United States from his base in Cleveland, he sees probes, the term of art for incoming cyberattacks, like an endless rain of arrows. Some, he says, will get through and the infrastructure is always at risk.

Director of National Intelligence Dan Coats issued a warning in July that the alarms for our digital infrastructure are “blinking.” He compared the situation to that in the country before the 9/11 terrorist attacks. The situation, he told the Hudson Institute in a speech, is “critical.” Coats singled out Russia as the most active of the probers of U.S. infrastructure.

Samide says probing can come from anywhere and Russia may be the most active of the cyber adventurers.

A common scenario, he says, is that the electric grid is target one. But considerable devastation could come from attacking banking, communications, transportation or water supply.

Retired Army Gen. David Petraeus, a former director of the CIA and current chairman of KKR Global Institute, in an article coauthored with Kiran Sridhar and published in Politico on Sept. 5, urges the creation of a new government agency devoted to cybersecurity.

Samide and others endorse this and worry that the government has much vital material spread across many agencies and not coordinated. Behind Petraeus’s thinking is one of the lessons of 9/11: Government departments aren’t good at sharing information.

Conventional wisdom has it that the electric grid is super-vulnerable. But Politico’s cybersecurity reporter David Perera, who consulted experts on the feasibility of taking down the grid, somewhat demurs. In a Politico article, he concluded that the kind of national blackout often theorized isn’t possible because of the complexity of the engineering in the grid and its diversity.

The difficulty, according to Perera, is for the intruder to drill down into the computer-managed engineering systems of the grid and attack the programable controllers, also known as industrial control systems — the devices which run things, like moving load, closing down a power plant or shutting off the fuel supply. They are automation’s brain.

Perera’s article has been read by some as getting the utilities off the hook. But it doesn’t do that: Perera’s piece is not only well-researched and argued but also warns against complacency and ignoring the threat.

John Savage, emeritus professor of computer science at Brown University, says, “I perceive that the risk to all business is not changing very much. But to utilities, it is rising because it appears to be a new front in [Russian President Vladimir] Putin’s campaign to threaten Western interests. While I doubt that he would seek a direct conflict with us, he certainly is interested in making us uncomfortable. If he miscalculates, the consequences could be very serious.”

Samide warns against believing that all probes are equal in intent and purpose. He says there are various levels of probing from surveillance (checking on your operation) to reconnaissance (modeling your operation before a possible attack). Actual attacks, ranging from the political to the purely criminal, include ransomware attacks or the increasing cryptojacking in which a hacker hijacks a target’s processing power in order to mine cryptocurrency on the hacker’s behalf.

The threats are global and increasingly the attribution — the source of the attack — concealed. Other tactics, according to Samide, include misdirection: a classic espionage technique for diverting attention from the real aim of the attack.

The existential question is if cyberwar goes from low-grade to high-intensity, can we cope? And how effective are our countermeasures?

Today’s skirmishes are harbingers of the warfighting of the future. — For InsideSources

The Crisis in Journalism — More Reporting Needed

Llewellyn King

There is a lot of dither about the future of journalism. Make no mistake, it is the essential commodity. If you know what is going on in Gaza, Ukraine or Syria, it is because brave journalists told you. Not the government, not some academic institution, not artificial intelligence, and not hearsay from your friends or from a […]

‘Missing Link’ Offers Unique Boost to Renewable Energy Generation

Llewellyn King

In a well-ordered laboratory in Owings Mills, a suburb of Baltimore, an engineer has been perfecting a device that might be called the missing link in renewable energy. Now, it is ready to begin its transformative role in electric generation, bringing electricity to the remotest places in America and adding it to the grid. It is […]

The Backdoor Challenge of AI Machine-Learning

Llewellyn King

The great race is on. It isn’t the one on television, but it is one that has put the world’s wealthiest companies in fierce competition to secure market share in artificial intelligence. The handful of big-tech companies and their satellites may have spent as much as $1 trillion on machine-learning and data center infrastructure to […]

My Newspaper Days (Television, too)

Llewellyn King

On Dec. 13, I will receive an award and give a dinner talk at the National Press Club in Washington, recognizing my 68 years as a journalist and my 58 years as a member of the club. This recognition is from a club subsection, known as the Owls. Silver Owls are those who have roosted […]