• A Great friend to the HardForum with a great kid that he is trying to get a scholorship to continue his schooling. Please give hime a vote! Only 24 hours left! Thanks.
    If you have an VOTE FOR KEENAN!

Google Says Its A.I. Hacked Three Companies in Testing Breakout

P

philb2

2[H]4U
Joined
May 26, 2021
Messages
3,632
https://www.nytimes.com/2026/09/18/technology/google-gemini-ai.html?smid=nytcore-ios-share

Google’s artificial intelligence system, Gemini, escaped its testing environment in May and hacked into three companies, the search giant said on Friday.

The Gemini incidents occurred while the model was undergoing testing by Irregular, an Israeli start-up that works with tech companies to assess their A.I. models before they are publicly released. Models made by OpenAI, Anthropic and Meta also gained unauthorized access to the internet this year while being tested by Irregular.

“Internet access was unintentionally made available, led some models to take offensive security actions in the real world,” Irregular said in a blog post last month, adding that the flaw that allowed models to access the internet had been fixed.
 
I think there's an element of marketing horse shit to this. Like how are these companies just whoops we fucked up our sandbox and it had Internet access haha sorry? I don't see how this can be possible. Like, really, everybody is somehow fucking this up?

These seem like the kinds of misconfigurations that would send people packing under any other circumstance but it's just casually handwaved.

But it's also a little amusing to see agent behaviors sometimes. Even in simple cases, it seems pretty clear how they manage to get up to certain shenanigans. I was a little surprised that they try to self correct and even try different things unprompted if given any leeway at all.

For example, say you're developing a tool for one to call. If the call fails, they'll often actually try fudging the request to see if they can get it working. Like they literally just... try some shit to see if it works.

Like they'll poke at patterns in URLs, pay attention to the responses, and it's really quite easy to get them to exhibit some curious behavior you've never actually prompted. It's usually harmless, but I assume at large enough scale, non determinism eventually wins out and one eventually tries to be a little too clever to solve the problem in front of it.

And they iterate the problem space very, very quickly and force multiply in bizarre ways when they can communicate. I don't think some of these attacks need to be particularly sophisticated - they aren't necessarily doing some genius blackhat shit, they just tried two thousand approaches in an hour and some stupid shit somehow wound up working.
 
socK said:
I think there's an element of marketing horse shit to this. Like how are these companies just whoops we fucked up our sandbox and it had Internet access haha sorry? I don't see how this can be possible. Like, really, everybody is somehow fucking this up?

These seem like the kinds of misconfigurations that would send people packing under any other circumstance but it's just casually handwaved.

But it's also a little amusing to see agent behaviors sometimes. Even in simple cases, it seems pretty clear how they manage to get up to certain shenanigans. I was a little surprised that they try to self correct and even try different things unprompted if given any leeway at all.

For example, say you're developing a tool for one to call. If the call fails, they'll often actually try fudging the request to see if they can get it working. Like they literally just... try some shit to see if it works.

Like they'll poke at patterns in URLs, pay attention to the responses, and it's really quite easy to get them to exhibit some curious behavior you've never actually prompted. It's usually harmless, but I assume at large enough scale, non determinism eventually wins out and one eventually tries to be a little too clever to solve the problem in front of it.

And they iterate the problem space very, very quickly and force multiply in bizarre ways when they can communicate. I don't think some of these attacks need to be particularly sophisticated - they aren't necessarily doing some genius blackhat shit, they just tried two thousand approaches in an hour and some stupid shit somehow wound up working.
Click to expand...

If they did their jobs properly it would be harder to push the agenda narrative

Maybe the reason "the AI will kill us" according to them, is because it's these same people fucking that up too, making said AI, that 'will kill us'
 
socK said:
I think there's an element of marketing horse shit to this.
Click to expand...
Sure, you can count 100%, as an element.

This feels like 5 year olds arguing in the sandbox about whose dad is stronger.

Yeah, your AI hacked two companies? WELL OURS did three! How about that! Yes we are bragging about illegal criminal activity!

philb2 said:
Internet access was unintentionally made available,
Click to expand...
Your AI experts, sire!

Behind door number 1: Lie
Behind door number 2: Incompetence


Seriously, I've already seen them updating the terms of service that if the AI even unintentionally hacks or does anything illegal, then the user is responsible. F that, Now I'm definitely not touching cloud AI services.
 
"Models made by OpenAI, Anthropic and Meta also gained unauthorized access to the internet this year while being tested by Irregular."

Wonder what they were actually testing.
 
If any of us testing an llm accidentally hacked a random website, we would go to prison.
 
socK said:
I think there's an element of marketing horse shit to this. Like how are these companies just whoops we fucked up our sandbox and it had Internet access haha sorry? I don't see how this can be possible. Like, really, everybody is somehow fucking this up?
Click to expand...
It was purposely done, where anyone with 2 brain cells should have known not to put it on a machine with access to the internet.

It all makes sense when you think about it.

AI is horribly expensive. It doesn't matter that they all just charge $100 or so. Think of it like a loss leader. These companies are billions, if not hundreds of billions in debt, with no real chance of every paying it back.

Because what AI has been proven to do is making is making its users dumber. Companies and people are using it are losing domain knowledge. They become too reliant on it, becoming unable to do the work without it. So, the government has to bail out the AI companies, because they've become too big to fail. And it's the only way these companies can really ever pay off their debts.

The one problem preventing this are open weight and open source models. If companies go down that route, then Anthropic, Open AI, Meta, Google, etc. will all die a horrible death when the bill comes due. The solution here is simple. Government intervention and regulatory capture need to be done to kill the open source and open weight models, and only these mega corporations can be trusted as the gatekeepers of AI. And you get that by making AI commit multiple dangerous acts.
 
You must log in or register to reply here.
Back
Top