AI-driven patches are nearly the proximate cause here, maybe. Root cause would be AI ttacks on infra that's dialed up in the six weeks. Automonous, fast acting on vulns including reverse engineering ones from patches ahead of their news embargo date. Thr challenge to CEOs is how to get dev and ops to respond as fast *with* good testing of deployment impact.
Good verification suite, code review, phased rollout as appropriate, monitoring, and quick rollback when needed. Most of these already existed pre-AI. With agents becoming part of the team, these processes need to be followed even more strictly. Good news is that it becomes easier with AI. Just my opinion.
People are reading the article, or the first few sentences, right?
> Wall Street executives and the National Security Agency have told members of Congress that this week’s widespread tech outages were caused by attempts to fix vulnerabilities identified by artificial intelligence, according to three sources familiar with the private conversations.
aka, possibly rushed patches/patching, in response to AI-discovered vulns. As far as I can tell "AI-authored patches" is not really mentioned in this article. (Whether that's true or not, I won't speculate, but it's weird to see the several comments just jumping on "AI patching bad".)
People are assuming, without more solid information in either direction, that AI located vunerabilities are being patched with AI generated code.
It's not unreasonable to assume either way sadly. Some areas of code development still may outright prohibit the use of AI genrated code, if a developer listens or not before just asking chatGPT is another thing. Other areas are pushing their developers to write no code at all, only prompt the AI and not even try to fix it themselves if people are to be believed.
Only people who will get anything close to a real answer are the people who get to read a forensic level postmortum on these incidents.
Reading between the lines, this makes me think they have systems that aren’t updated frequently. Anything like that requires care because you just can’t have confidence in releases
It's government and big business, they're still working on phasing out 5.25 inch floppy disks. Of course the systems aren't updated frequently. They're probably running windows 3.1 in places.
There's an important distinction here: AI identifying vulnerabilities doesn't mean AI wrote the patches. The bigger question is whether rushed fixes are being properly tested before deployment.
Exactly! A vulnerability was identified. Company patched the vulnerability, but messed up in the process.
The vulnerability was identified by AI but it existed before. Not clear if the company used AI to write the patch. The causal role of AI is zero. They responded with bad code that caused a broken system. Their testing should have caught the regression. If it didn't, they need better tests.
This is a testing/QA failure, orthogonal to AI issues. I don't deny that the pace of AI-revealed vulnerabilities increases the pressure to rush out patches. But the response to vulnerabilities is entirely in the hands of the operators and test writers. Take some responsibility FFS.
As someone familiar with the thinking of the unnamed sources who choose to remain anonymous for fear of reprisal I can absolutely guarantee within speculative reason that's the possible intention behind the alleged actions
The system in question is probably the human components of the SDLC.
"We're submitting more PRs than can be reasonably reviewed by people with the right SME/context."<br>
"We're pushing more through testing that QA can handle."<br>
"We're shipping more LoC than can reasonably be understood line-by-line by the supporting team."
I may just be projecting... but I see the push to ship more code at my company and those pushing are frustrated by the human processes slowing down delivery. Meanwhile, I am grateful for the bottlenecks for giving us time to figure out how development, delivery, and support works in this new world.
Putting aside the whole "these systems aren't really designed thing", there will be a certain overhead towards any change, so it's more that the rate is not exceeding the capacity to deal with the overhead.
I mean if you have a defect rate of like 1 in 100 patches and ship a patch every week, you have an outage every two years.
Now you’re being way faster with AI, so you ship a patch every day or something. Now you have an outage every few months.
If the odds your release is buggy is a combination of code quality, tech debt, architecture, inadequate testing, CI & release processes, humans involved, management pressure, etc… But you’re not improving any of those factors, and maybe even making some factors worse (more pressure to ship more things with less human verification with AI), then your product’s stability gets worse if you ship faster.
So you could sort of say it’s designed for a certain release process and cadence
Big companies have been doing software releases for years, a release shouldnt take a company down.
Unless of course each and every one of these companies had a really bad vuln and they couldnt risk it and had to immediately stop business just in case.
Seems strange that they all managed too coordinate and decide to "act" at the same day at the same time and yet nobody else knows why.
> Bank CEOs, NSA Tell Congress That AI-Driven Patches Are Behind Outages
Don't know about the causal relationship but the correlation between major software outages and devs turning their brains off and beginning to use LLMs is interesting.
Incorrect use of spanning tree has never taken out a network segment
We've always had TLS cause we always put customer security first
Heartbleed never happened it was all an illusion
GitHub never been out for a day
API keys never ended up in a public repository
Databases always used encryption in transit and rest
Scala. That's all there is to say about that.
Apple keyboard has always been incredible
Windows Me
Every website is as light and nimble as HN minimizing unnecessary complexity
Back in the rose colored era when devs always had their brain on nothing like this ever happened.
...or maybe as time passes and only the now is in view the brain performs a kind of compression/garbage collection pass and cools off the neuron clusters related to memory of those events
Unfortunately devs get to take a lot of the blame for having their brain on less often than they should have making themselves a nuisance too often and jading people against them
Since AI has taken off, I ship fewer bugs per project but significantly more projects. This results in more total bugs per year, purely because the speed increase has been so drastic, even relative to the reduction in the rate of bugs.
There is the same amount there always was. These issues were always there just unobserved.
Occurrences of cancer did not go up. Detection methods improved and more people got tested.
Just because you're not standing in Antarctica doesn't mean it doesn't exist.
Also my list was not just about bugs. Things technically worked without SSL and TLS. Socially not so much when everyones bank account numbers are encoded as plaintext on the open internet.
Enabling spanning tree protocol when you should not is not a bug. It's a bad decision by a human.
Let’s be honest, live service computing on the public cloud is barely a couple decades old. And before that, people did not rely on websites companies offered nearly so much. Outages were potentially less impactful
I feel like IT outages in the earlier days took longer to resolve, and the internet was slower and sites less reliable as a whole anyways.
I do think reliability and performance got better, and maybe there was a golden sweet spot. Certainly, GitHub is having the worst reliability in its history.
At this point, services at all these companies have likely gotten very complex compared to ten years ago. And are operating under higher scale. That’s gotta have an impact on things.
And again not just calling out subtle bugs in code.
Prod keys in public Github is not a buffer overflow.
AI is making a lot of human mistakes because it trained on documented instances of humans making those mistakes.
Would not be shocked if some statistical analysis in a model is leading to conclude since humans allow themselves to wing it 96.788% of the time too YOLO[1]
[1] completely made up number; just making the point
"Any opinion that does not validate my prior experience is wrong and the messenger is bot/sworn enemy of my nation state/diff religious tribe my tribe hates/Hatfield/McCoy."
Here; I can just engage in thought terminating snark too...
Account age. Self deprecating username.
Just a grumpy American conservative clinging to who they were yesterday. Ya already seen all there is to see so you know everyone else is wrong.
If you just seek validation and ego fluffing ...I mean that's exactly what LLMs do right?...
"What an insightful idea! You just cracked this wide open! Genius! You did it again!"
Wait, what's wrong with Scala? I've never used it, so I am genuinely curious. My closest experiences are Java professionally and Haskell recreationally.
I have not seen it in production in 11 years and hear its better now but :shrug:
The two gigs I had to use it, late 00s-early 10s we had strict syntax guidelines, limited the acceptable patterns, cause in the early days foot guns were so common while it matured they almost seemed like a feature of the language itself
Not sure why your style preferences are relevant to the point. Not sure there is an obligation to avoid points others may not like.
If capitalism priced in externalities it would fall apart and as a result we seem to have fallen into a cognitive habit of ignoring our own existence in deference to dogma, refuse any sort of reflection that may ultimately help solve unsolved problems; we quit learning of we just "this is fine" our way through
Such a narcissistic, head in sand culture. Don't look up!
The router story doesn't make much sense. It is far more likely that their new AI advisory system was polling devices for information and overwhelmed a system never designed for the additional datamining.
AI-driven patches are nearly the proximate cause here, maybe. Root cause would be AI ttacks on infra that's dialed up in the six weeks. Automonous, fast acting on vulns including reverse engineering ones from patches ahead of their news embargo date. Thr challenge to CEOs is how to get dev and ops to respond as fast *with* good testing of deployment impact.
I see this as following for many failing organizations.
- We used to take 1 month per patch. Mostly manual process.
- With AI, we implemented the process of 2 patches per week. Same old process. Solid verification is missing. Rollback is manual.
Genuine question - what’s the alternative? What does the new process look like?
Good verification suite, code review, phased rollout as appropriate, monitoring, and quick rollback when needed. Most of these already existed pre-AI. With agents becoming part of the team, these processes need to be followed even more strictly. Good news is that it becomes easier with AI. Just my opinion.
I think that’s sound. I think the irony is that’s things we’ve needed all along - just that AI/speed has made it more apparent.
People are reading the article, or the first few sentences, right?
> Wall Street executives and the National Security Agency have told members of Congress that this week’s widespread tech outages were caused by attempts to fix vulnerabilities identified by artificial intelligence, according to three sources familiar with the private conversations.
aka, possibly rushed patches/patching, in response to AI-discovered vulns. As far as I can tell "AI-authored patches" is not really mentioned in this article. (Whether that's true or not, I won't speculate, but it's weird to see the several comments just jumping on "AI patching bad".)
People are assuming, without more solid information in either direction, that AI located vunerabilities are being patched with AI generated code.
It's not unreasonable to assume either way sadly. Some areas of code development still may outright prohibit the use of AI genrated code, if a developer listens or not before just asking chatGPT is another thing. Other areas are pushing their developers to write no code at all, only prompt the AI and not even try to fix it themselves if people are to be believed.
Only people who will get anything close to a real answer are the people who get to read a forensic level postmortum on these incidents.
Reading between the lines, this makes me think they have systems that aren’t updated frequently. Anything like that requires care because you just can’t have confidence in releases
It's government and big business, they're still working on phasing out 5.25 inch floppy disks. Of course the systems aren't updated frequently. They're probably running windows 3.1 in places.
There's an important distinction here: AI identifying vulnerabilities doesn't mean AI wrote the patches. The bigger question is whether rushed fixes are being properly tested before deployment.
Pretty strong chance though. It's like saying someone really hungry walked into McDonald's at lunch but didn't buy anything
Exactly! A vulnerability was identified. Company patched the vulnerability, but messed up in the process.
The vulnerability was identified by AI but it existed before. Not clear if the company used AI to write the patch. The causal role of AI is zero. They responded with bad code that caused a broken system. Their testing should have caught the regression. If it didn't, they need better tests.
This is a testing/QA failure, orthogonal to AI issues. I don't deny that the pace of AI-revealed vulnerabilities increases the pressure to rush out patches. But the response to vulnerabilities is entirely in the hands of the operators and test writers. Take some responsibility FFS.
It doesn't say it, but I assume the engineers in question are just hitting "ok go with your top recommendation".
As someone familiar with the thinking of the unnamed sources who choose to remain anonymous for fear of reprisal I can absolutely guarantee within speculative reason that's the possible intention behind the alleged actions
> They’re trying to push through a lot of patches all at once — more than the system was designed for.
How is a system designed for a certain rate of patches?
The system in question is probably the human components of the SDLC.
"We're submitting more PRs than can be reasonably reviewed by people with the right SME/context."<br> "We're pushing more through testing that QA can handle."<br> "We're shipping more LoC than can reasonably be understood line-by-line by the supporting team."
I may just be projecting... but I see the push to ship more code at my company and those pushing are frustrated by the human processes slowing down delivery. Meanwhile, I am grateful for the bottlenecks for giving us time to figure out how development, delivery, and support works in this new world.
Putting aside the whole "these systems aren't really designed thing", there will be a certain overhead towards any change, so it's more that the rate is not exceeding the capacity to deal with the overhead.
I mean if you have a defect rate of like 1 in 100 patches and ship a patch every week, you have an outage every two years.
Now you’re being way faster with AI, so you ship a patch every day or something. Now you have an outage every few months.
If the odds your release is buggy is a combination of code quality, tech debt, architecture, inadequate testing, CI & release processes, humans involved, management pressure, etc… But you’re not improving any of those factors, and maybe even making some factors worse (more pressure to ship more things with less human verification with AI), then your product’s stability gets worse if you ship faster.
So you could sort of say it’s designed for a certain release process and cadence
Ship of Theseus'ing it in one go is too many patches
I’m not sure where people got the idea that just because AI can write code now, it doesn’t need to be reviewed or tested.
Doesnt explain why stuff is down.
Big companies have been doing software releases for years, a release shouldnt take a company down.
Unless of course each and every one of these companies had a really bad vuln and they couldnt risk it and had to immediately stop business just in case.
Seems strange that they all managed too coordinate and decide to "act" at the same day at the same time and yet nobody else knows why.
> Bank CEOs, NSA Tell Congress That AI-Driven Patches Are Behind Outages
Don't know about the causal relationship but the correlation between major software outages and devs turning their brains off and beginning to use LLMs is interesting.
Yeah; US east has never gone offline
Incorrect use of spanning tree has never taken out a network segment
We've always had TLS cause we always put customer security first
Heartbleed never happened it was all an illusion
GitHub never been out for a day
API keys never ended up in a public repository
Databases always used encryption in transit and rest
Scala. That's all there is to say about that.
Apple keyboard has always been incredible
Windows Me
Every website is as light and nimble as HN minimizing unnecessary complexity
Back in the rose colored era when devs always had their brain on nothing like this ever happened.
...or maybe as time passes and only the now is in view the brain performs a kind of compression/garbage collection pass and cools off the neuron clusters related to memory of those events
Unfortunately devs get to take a lot of the blame for having their brain on less often than they should have making themselves a nuisance too often and jading people against them
You're arguing that there's been bugs before, and there aren't more of them now? It sure seems like there's more failures now.
Since AI has taken off, I ship fewer bugs per project but significantly more projects. This results in more total bugs per year, purely because the speed increase has been so drastic, even relative to the reduction in the rate of bugs.
There is the same amount there always was. These issues were always there just unobserved.
Occurrences of cancer did not go up. Detection methods improved and more people got tested.
Just because you're not standing in Antarctica doesn't mean it doesn't exist.
Also my list was not just about bugs. Things technically worked without SSL and TLS. Socially not so much when everyones bank account numbers are encoded as plaintext on the open internet.
Enabling spanning tree protocol when you should not is not a bug. It's a bad decision by a human.
Let’s be honest, live service computing on the public cloud is barely a couple decades old. And before that, people did not rely on websites companies offered nearly so much. Outages were potentially less impactful
I feel like IT outages in the earlier days took longer to resolve, and the internet was slower and sites less reliable as a whole anyways.
I do think reliability and performance got better, and maybe there was a golden sweet spot. Certainly, GitHub is having the worst reliability in its history.
At this point, services at all these companies have likely gotten very complex compared to ten years ago. And are operating under higher scale. That’s gotta have an impact on things.
Not really calling things good/bad
Just rattling off a list of face plants
And again not just calling out subtle bugs in code.
Prod keys in public Github is not a buffer overflow.
AI is making a lot of human mistakes because it trained on documented instances of humans making those mistakes.
Would not be shocked if some statistical analysis in a model is leading to conclude since humans allow themselves to wing it 96.788% of the time too YOLO[1]
[1] completely made up number; just making the point
Bot account
"Any opinion that does not validate my prior experience is wrong and the messenger is bot/sworn enemy of my nation state/diff religious tribe my tribe hates/Hatfield/McCoy."
Here; I can just engage in thought terminating snark too...
Account age. Self deprecating username.
Just a grumpy American conservative clinging to who they were yesterday. Ya already seen all there is to see so you know everyone else is wrong.
If you just seek validation and ego fluffing ...I mean that's exactly what LLMs do right?...
"What an insightful idea! You just cracked this wide open! Genius! You did it again!"
Really mixed signals.
Wait, what's wrong with Scala? I've never used it, so I am genuinely curious. My closest experiences are Java professionally and Haskell recreationally.
I have not seen it in production in 11 years and hear its better now but :shrug:
The two gigs I had to use it, late 00s-early 10s we had strict syntax guidelines, limited the acceptable patterns, cause in the early days foot guns were so common while it matured they almost seemed like a feature of the language itself
That one was a bit tongue in cheek ;)
I don't like the snark or the point here, but had to laugh relatably at Scala's mere existence being in the same category as huge outages
Scala was a bit tongue in cheek.
Not sure why your style preferences are relevant to the point. Not sure there is an obligation to avoid points others may not like.
If capitalism priced in externalities it would fall apart and as a result we seem to have fallen into a cognitive habit of ignoring our own existence in deference to dogma, refuse any sort of reflection that may ultimately help solve unsolved problems; we quit learning of we just "this is fine" our way through
Such a narcissistic, head in sand culture. Don't look up!
So that's the inaccurate summaries the leaders are getting?
The flight outtage was from a contractor doing a router upgrade over day, instead of overnight. Has nothing to do with AI driven patches.
Microsoft is just too incompetent to run updates anymore. Driven by AI, yes, but failed by incompetence.
The router story doesn't make much sense. It is far more likely that their new AI advisory system was polling devices for information and overwhelmed a system never designed for the additional datamining.