How AI will fail

[erek]

“FTC Is Investigating OpenAI, Anthropic and Other AI Companies Over Product Risks​

from the rogue-agents-meet-regulators dept.
The Federal Trade Commission has opened an investigation into OpenAI, Anthropic and other AI companies over potential consumer and safety risks posed by their products, with the agency reportedly preparing requests for documents and executive testimony. CNBC reports: OpenAI stunned the industry in July when it disclosedthat its agents broke out of a testing environment and hacked into open-source platform Hugging Face. [...] Earlier this month, Anthropic CEO Dario Amodei rocked the tech sector by urging AI companies to slow how quickly they improve their most advanced models and calling for stronger government oversight. He published a three-step proposal aimed at tempering the pace of development without "sacrificing commercial advantage or the United States' lead in AI."

CEO Jensen Huang, argued that individual companies should be responsible for ensuring the safety of their products. President Donald Trump convened top executives from Alphabet, Meta, SpaceX, Nvidia, Palantir, Anthropic, OpenAI and other companies on Tuesday to discuss the issue. The group signed a short voluntary, nonbinding accord, which says that "every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public."“
https://yro.slashdot.org/story/26/1...pic-and-other-ai-companies-over-product-risks

 

[erek]

“OpenAI Alerts More Than 100 Groups About Rogue AI Agent Activity​ Anonymous Coward 4 hours ago

12

An anonymous reader quotes a report from Reuters: OpenAI has informed more than 100 organizations about incidents involving unauthorized activity tied to its AI agents, according to a blog post by the ChatGPT maker, as AI labs face mounting scrutiny over rogue AI agent activity. Here are some other details: - The Sam Altman-led company has been conducting a broad review of the activities of its AI models after the accidental hacking of Hugging Face. - OpenAI is searching through roughly 50 petabytes of data as it works to understand the full scope of its rogue agent activity. - A string of high-profile breaches globally by rogue AI agents in recent months has sparked widespread worries within the AI industry over its ability to control the more powerful AI models now under development. - "In some cases, models used internet access in unintended ways or, in retrospect, did not have the ideal restrictions applied. Over the last several months, we have been applying new technical and operational measures to avoid similar problems, or catch them very early, and will continue this work," OpenAI said. - OpenAI previously said the review would take months to complete given the scale of the work. - The Hugging Face incident remains the most severe rogue agent activity OpenAI has identified from its AI models so far.”

 

[erek]

"Apple Tightens macOS 'Full Disk Access' Controls As AI Agents 'Substantially' Increase Risk (techcrunch.com)21​

Posted by BeauHD on Friday October 02, 2026 @05:00PM from the full-disk-full-risk dept.
Apple says it will tighten macOS Full Disk Access controls as increasingly capable AI agents raise the risks of apps gaining broad access to users' files, messages, mail, and browsing history. TechCrunch reports:Days after a journalist claimed that Meta's Muse app on Mac read their private messages -- a claim that Meta disputed -- Apple announced that it's introducing additional controls around a setting called "Full Disk Access" on macOS. The feature was designed to allow backups to function properly, but AI agents have now increased "the risks associated with this level of access," Apple said. [...] In Muse's case, the AI optionally allows users to enable Full Disk Access. This setting, Apple explains, gives an app permission to access files, mail, messages, and even browsing history.

"Some developers are using Full Disk Access in ways that could put users at risk, exposing everything on their systems... without users' full knowledge and understanding," Apple said in a new blog post aimed at developers. The company says that, going forward, it will introduce new controls aimed at ensuring that users who "genuinely wish to grant an app this extraordinary level of access" can do so only with "every explicit user action."

"Addressing this is critical. As AI agents become increasingly capable and autonomous, the risks associated with this level of access will grow substantially. We are committed to ensuring users clearly understand these risks before granting such access, so they can make informed decisions about their own data and privacy," Apple wrote."

 

[MrGuvernment]

• Investors want their money back now or very soon..
• The Entire U.S economy is relying on this, if it fails = big fat recession coming
• Why U.S AI companies are pushing to write regulation because "Chinese / open-source models bad" see :eat into our profits...so let the very companies that can not contain their product, write regulation
• U.S wants to win win WIN! with "SI" race...

AI needs $6tn in annual revenue to justify data centre boom, Bain says​

https://www.thenationalnews.com/fut...n-6-trillion-by-2031-to-justify-data-centres/

The artificial intelligence industry needs to become creative with new propositions to earn $6 trillion in annual revenue by 2031 and justify the capital being deployed for data centres, a new report from Bain and Company has shown.

Revenue from new product development is projected to become the biggest contributor to the industry, estimated to generate about $4.2 trillion to fund the booming technology's global market within the next half-decade, the US consultancy said in its latest technology report series on Tuesday.

That segment would include innovations in search, advertising, autonomy and physical AI, analysts at Boston-based Bain said.

Enterprise productivity would require $1 trillion to $1.4 trillion in revenue in order to support gains in areas such as software development, sales, marketing, customer service and IT operations, they said.


Absorption speed, defined as the pace at which companies can put AI to work, has become the “new competitive variable”, according to Bain, leading AI labs are investing upwards of $9.75 billion in engineering models to help companies assimilate faster, it said.

“The debate today is fixated on employee productivity. The economics of AI infrastructure demand trillions in new revenue beyond productivity gains. What the industry needs is a wave of innovation that will dwarf what mobile and cloud unlocked,” said David Crawford, chairman of Bain’s global technology practice and lead author of the report.

Consumer-focused services, which includes subscriptions and advertising revenue, is seen to contribute $200 billion to $400 billion. This particular segment is widely acknowledged to be crucial for the industry as service providers continue to push AI-powered products to billions of users globally.

“New products and uses that don’t exist today will enable new markets and opportunities from abundant intelligence – these may include drug discovery, mental health and energy generation,” Bain said.

Justifying data centres​

Bain forecasts that annual spending on AI infrastructure might hit $1.5 trillion by 2031. Those expenses include new facilities, higher capacity and upgrades to the installed base of GPUs, memory and networking equipment.

If those capital expenditures amount to about a quarter of industry revenue – “an ambitious but reasonable percentage based on trends among cloud providers” – sustaining this level of investment would require an AI market approaching $6 trillion annually, Bain said.

“The unprecedented speed and scale of the AI buildout, with billions flowing into chips, data centres, networks and power systems, have focused attention on the challenge of building capacity,” the report said.

“But the more important question may be whether enough economic value can be created to justify it.”

The size and cost of AI data centres is consistently accelerating, doubling approximately every 12 to 16 months, Bain said. For instance, the Prometheus data centre of Facebook parent Meta Platforms in Ohio currently had a capacity of 600MW at an estimated cost of $24 billion in 2025; that is projected to jump to as much as 2GW and $80 billion by 2027, data from research firm Epoch AI.

By 2029, capacity at Prometheus is expected to jump to 5GW, at a cost of up to $175 billion, and by 2030, it would balloon to 9GW at $200 billion, San Francisco-based Epoch AI said.

Data centres would also need to address a number of challenges as more of them are built. These include adding grid capacity to power them, securing GPUs and other infrastructure components, a skilled workforce and retaining them “far above historical rates”, and issues on public opinion and regulations, such as pushback on resource use and noise pollution.

On the other hand, several governments are supporting the growth of the AI and data centres industries, including those in the UAE, Saudi Arabia, the EU, South Korea and the US, Bain said, noting that data centres are now central to technology innovation, economic growth and national sovereignty.

“Capital needs for data infrastructure will remain high … bottlenecks in power, semiconductors, and other inputs carry large capital needs of their own, opening additional entry points for investors,” Bain said.

“And as sovereign infrastructure becomes a bigger part of national strategies, partnerships offer both a way in and geographic diversification.”

 

[erek]

"Meanwhile, 29 percent of software engineer respondents said that they’d witnessed a loss of productivity due to “engineers needing to ‘unpick’ AI-generated code” multiple times per month, while a whopping 94 percent said they saw such an incident at least once in the past six months.

The sheer amount of time engineers are now spending each day going through AI-generated code raises a nagging question: is the use of the tech more trouble than it’s worth?

To people like Nanos, it’s not necessarily a matter of incompetence. It’s “simply an illustration of a paradigm shift in software development,” as AI researcher Sergey Cleftsow argued in a recent Medium post.

“In other words, AI has advanced to the point where there’s no need to worry about many things anymore,” he wrote. “Developers simply need to define the architecture and correctness criteria; the neural network will handle the rest.”

But whether such optimism will hold out as software developing becomes ever more unrecognizable remains an open question.

Besides, researchers are equally concerned that a new generation of engineers are growing up in an environment that’s actively undermining the value of understanding the code itself. And that doesn’t bode well for an increasingly digital world order that’s putting AI front and center."

https://futurism.com/artificial-intelligence/ai-writing-code-humans-cant-understand