How AI will fail

[erek]

“FTC Is Investigating OpenAI, Anthropic and Other AI Companies Over Product Risks​

from the rogue-agents-meet-regulators dept.
The Federal Trade Commission has opened an investigation into OpenAI, Anthropic and other AI companies over potential consumer and safety risks posed by their products, with the agency reportedly preparing requests for documents and executive testimony. CNBC reports: OpenAI stunned the industry in July when it disclosedthat its agents broke out of a testing environment and hacked into open-source platform Hugging Face. [...] Earlier this month, Anthropic CEO Dario Amodei rocked the tech sector by urging AI companies to slow how quickly they improve their most advanced models and calling for stronger government oversight. He published a three-step proposal aimed at tempering the pace of development without "sacrificing commercial advantage or the United States' lead in AI."

CEO Jensen Huang, argued that individual companies should be responsible for ensuring the safety of their products. President Donald Trump convened top executives from Alphabet, Meta, SpaceX, Nvidia, Palantir, Anthropic, OpenAI and other companies on Tuesday to discuss the issue. The group signed a short voluntary, nonbinding accord, which says that "every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public."“
https://yro.slashdot.org/story/26/1...pic-and-other-ai-companies-over-product-risks

 

[erek]

“OpenAI Alerts More Than 100 Groups About Rogue AI Agent Activity​ Anonymous Coward 4 hours ago

12

An anonymous reader quotes a report from Reuters: OpenAI has informed more than 100 organizations about incidents involving unauthorized activity tied to its AI agents, according to a blog post by the ChatGPT maker, as AI labs face mounting scrutiny over rogue AI agent activity. Here are some other details: - The Sam Altman-led company has been conducting a broad review of the activities of its AI models after the accidental hacking of Hugging Face. - OpenAI is searching through roughly 50 petabytes of data as it works to understand the full scope of its rogue agent activity. - A string of high-profile breaches globally by rogue AI agents in recent months has sparked widespread worries within the AI industry over its ability to control the more powerful AI models now under development. - "In some cases, models used internet access in unintended ways or, in retrospect, did not have the ideal restrictions applied. Over the last several months, we have been applying new technical and operational measures to avoid similar problems, or catch them very early, and will continue this work," OpenAI said. - OpenAI previously said the review would take months to complete given the scale of the work. - The Hugging Face incident remains the most severe rogue agent activity OpenAI has identified from its AI models so far.”

 

[erek]

"Apple Tightens macOS 'Full Disk Access' Controls As AI Agents 'Substantially' Increase Risk (techcrunch.com)21​

Posted by BeauHD on Friday October 02, 2026 @05:00PM from the full-disk-full-risk dept.
Apple says it will tighten macOS Full Disk Access controls as increasingly capable AI agents raise the risks of apps gaining broad access to users' files, messages, mail, and browsing history. TechCrunch reports:Days after a journalist claimed that Meta's Muse app on Mac read their private messages -- a claim that Meta disputed -- Apple announced that it's introducing additional controls around a setting called "Full Disk Access" on macOS. The feature was designed to allow backups to function properly, but AI agents have now increased "the risks associated with this level of access," Apple said. [...] In Muse's case, the AI optionally allows users to enable Full Disk Access. This setting, Apple explains, gives an app permission to access files, mail, messages, and even browsing history.

"Some developers are using Full Disk Access in ways that could put users at risk, exposing everything on their systems... without users' full knowledge and understanding," Apple said in a new blog post aimed at developers. The company says that, going forward, it will introduce new controls aimed at ensuring that users who "genuinely wish to grant an app this extraordinary level of access" can do so only with "every explicit user action."

"Addressing this is critical. As AI agents become increasingly capable and autonomous, the risks associated with this level of access will grow substantially. We are committed to ensuring users clearly understand these risks before granting such access, so they can make informed decisions about their own data and privacy," Apple wrote."

 

[MrGuvernment]

• Investors want their money back now or very soon..
• The Entire U.S economy is relying on this, if it fails = big fat recession coming
• Why U.S AI companies are pushing to write regulation because "Chinese / open-source models bad" see :eat into our profits...so let the very companies that can not contain their product, write regulation
• U.S wants to win win WIN! with "SI" race...

AI needs $6tn in annual revenue to justify data centre boom, Bain says​

https://www.thenationalnews.com/fut...n-6-trillion-by-2031-to-justify-data-centres/

The artificial intelligence industry needs to become creative with new propositions to earn $6 trillion in annual revenue by 2031 and justify the capital being deployed for data centres, a new report from Bain and Company has shown.

Revenue from new product development is projected to become the biggest contributor to the industry, estimated to generate about $4.2 trillion to fund the booming technology's global market within the next half-decade, the US consultancy said in its latest technology report series on Tuesday.

That segment would include innovations in search, advertising, autonomy and physical AI, analysts at Boston-based Bain said.

Enterprise productivity would require $1 trillion to $1.4 trillion in revenue in order to support gains in areas such as software development, sales, marketing, customer service and IT operations, they said.


Absorption speed, defined as the pace at which companies can put AI to work, has become the “new competitive variable”, according to Bain, leading AI labs are investing upwards of $9.75 billion in engineering models to help companies assimilate faster, it said.

“The debate today is fixated on employee productivity. The economics of AI infrastructure demand trillions in new revenue beyond productivity gains. What the industry needs is a wave of innovation that will dwarf what mobile and cloud unlocked,” said David Crawford, chairman of Bain’s global technology practice and lead author of the report.

Consumer-focused services, which includes subscriptions and advertising revenue, is seen to contribute $200 billion to $400 billion. This particular segment is widely acknowledged to be crucial for the industry as service providers continue to push AI-powered products to billions of users globally.

“New products and uses that don’t exist today will enable new markets and opportunities from abundant intelligence – these may include drug discovery, mental health and energy generation,” Bain said.

Justifying data centres​

Bain forecasts that annual spending on AI infrastructure might hit $1.5 trillion by 2031. Those expenses include new facilities, higher capacity and upgrades to the installed base of GPUs, memory and networking equipment.

If those capital expenditures amount to about a quarter of industry revenue – “an ambitious but reasonable percentage based on trends among cloud providers” – sustaining this level of investment would require an AI market approaching $6 trillion annually, Bain said.

“The unprecedented speed and scale of the AI buildout, with billions flowing into chips, data centres, networks and power systems, have focused attention on the challenge of building capacity,” the report said.

“But the more important question may be whether enough economic value can be created to justify it.”

The size and cost of AI data centres is consistently accelerating, doubling approximately every 12 to 16 months, Bain said. For instance, the Prometheus data centre of Facebook parent Meta Platforms in Ohio currently had a capacity of 600MW at an estimated cost of $24 billion in 2025; that is projected to jump to as much as 2GW and $80 billion by 2027, data from research firm Epoch AI.

By 2029, capacity at Prometheus is expected to jump to 5GW, at a cost of up to $175 billion, and by 2030, it would balloon to 9GW at $200 billion, San Francisco-based Epoch AI said.

Data centres would also need to address a number of challenges as more of them are built. These include adding grid capacity to power them, securing GPUs and other infrastructure components, a skilled workforce and retaining them “far above historical rates”, and issues on public opinion and regulations, such as pushback on resource use and noise pollution.

On the other hand, several governments are supporting the growth of the AI and data centres industries, including those in the UAE, Saudi Arabia, the EU, South Korea and the US, Bain said, noting that data centres are now central to technology innovation, economic growth and national sovereignty.

“Capital needs for data infrastructure will remain high … bottlenecks in power, semiconductors, and other inputs carry large capital needs of their own, opening additional entry points for investors,” Bain said.

“And as sovereign infrastructure becomes a bigger part of national strategies, partnerships offer both a way in and geographic diversification.”

 

[erek]

"Meanwhile, 29 percent of software engineer respondents said that they’d witnessed a loss of productivity due to “engineers needing to ‘unpick’ AI-generated code” multiple times per month, while a whopping 94 percent said they saw such an incident at least once in the past six months.

The sheer amount of time engineers are now spending each day going through AI-generated code raises a nagging question: is the use of the tech more trouble than it’s worth?

To people like Nanos, it’s not necessarily a matter of incompetence. It’s “simply an illustration of a paradigm shift in software development,” as AI researcher Sergey Cleftsow argued in a recent Medium post.

“In other words, AI has advanced to the point where there’s no need to worry about many things anymore,” he wrote. “Developers simply need to define the architecture and correctness criteria; the neural network will handle the rest.”

But whether such optimism will hold out as software developing becomes ever more unrecognizable remains an open question.

Besides, researchers are equally concerned that a new generation of engineers are growing up in an environment that’s actively undermining the value of understanding the code itself. And that doesn’t bode well for an increasingly digital world order that’s putting AI front and center."

https://futurism.com/artificial-intelligence/ai-writing-code-humans-cant-understand

 

[erek]

“California's Governor Wants Worker Protections from AI. But FSF Thinks Kill Switches are 'Dangerous Precedent'​ EditorDavid 6 hours ago

26

The Guardian reports that California governor Gavin Newsom signed laws on Wednesday "aimed at protecting California workers from the threats of artificial intelligence , including potential job losses and workplace surveillance."The laws ban employers from using the technology to predict a worker's emotional state by using their biometric data, require employers to send written notices to workers if AI is responsible for mass layoffs and ban employers from relying on AI to decide to fire someone... The governor signed a law this month requiring operators of AI chatbots to perform risk assessments before rolling them out and an executive order requiring the state to consult with experts to further improve oversight over the industry. The law includes "expanded tracking of AI's economic effects" leading to recommendations on how to best "adapt job-training and public-benefit programs," according to the governor's office. "California has also launched an AI Unemployment Tracker and solicited statewide public input on AI's economic impacts through Engaged California." But two weeks ago Politico reported that the governor also signed executive orders to "explore proposals like requiring that AI companies develop a 'kill switch' for their most advanced models." Anthropic CEO Dario Amodei, who called for a slowdown in AI development amid existential risk concerns, said... that while a kill switch "could be a good idea," it was no "panacea...." [California lawmakers] were generally supportive of Newsom's plan and showed signs of coalescing around the kill switch idea, which was stripped out of legislation proposed two years ago. Rep. Ted Lieu, who introduced bipartisan kill switch legislation in the House, boosted Newsom's call for Congress to mirror California's approach. But not everyone supports this approach, including the Free Software Foundation's campaigns manager, Greg Farough: FSF: Having control over the technology we use is a fundamental part of software freedom. Without it, you simply don't have the basic freedoms you deserve when doing any type of computing, large or small. Don't let "artificial intelligence" be an exception: all users everywhere deserve complete control over their own computing. While it's understandable to be pleased by this news from California, Newsom's executive order also invites questions about the "kill switch" itself that could matter a great deal for freedom. Assuming such a kill switch was developed, who would really control it? We hope our readers understand the problems inherent in giving one single or a small number of kill switches to anyone. Newsom wisely wants an independent organization to verify their functionality, but even this still sets too dangerous of a precedent. We cannot allow such centralization of power over our computing to become commonplace, especially by the corporations that provide these models in the first place. We also cannot trust even the most well-intentioned "independent organization" to take care of its verification. Giving users the freedom that they deserve over large languages models would be an enormous departure from how most are developed and used today. But, if any group is capable of coming up with such a plan, it's the experts mentioned in Newsom's order. Like he states at the beginning of his executive order, the vast majority of them are already right there in California. We urge the Government Operations Agency to consider the precedent they are setting, and ask them to write their recommendation to the governor in ways that puts the freedom and dignity of the individual user first and foremost. The Guardian adds that Newsom also signed an executive order this week "to require state agencies to continue to refer to the technology as 'artificial intelligence' instead of the term 'super intelligence' as Donald Trump recently ordered US diplomats to use."”

 

[erek]

“Researcher, Dartmouth Professor and Provost Face Accusations of Secretly Using AI​ EditorDavid 3 hours ago

5

"The winner of the Nikon Small World in Motion competition is under investigation," writes the photography news site PetaPixel , "after scientists accused Dr Ning Xu of using AI in his video." The Nikon competition is about showcasing the beauty, complexity, and artistry of the microscopic world. The entries are abstract, often baffling to laypeople. But when Dr Xu was announced as the winner last month, members of the scientific community began to cry foul. "There are many serious problems with this video," Edward Phelps, associate professor at Florida University, says on LinkedIn. Phelps points out that "purple structures pop in and out of existence" and "green cilia also appear from nowhere and do not match the known size." Dr Xu's video is supposed to show the movement of cilia, which are hair-like structures that line part of the lungs... However, since Dr Xu won the prize, Nikon Small World updated its LinkedIn post celebrating his triumph with a note that reads: "Regarding questions raised about this year's Small World In Motion winning video, Nikon is carefully re-reviewing the information provided during the initial vetting, along with additional supporting materials from the entrant." Nikon adds that Dr Xu is "respectfully complying and has provided detailed technical documentation outlining the microscopy equipment, imaging methods, and processing techniques used to create the source video." Worse still, Ian Donovan, an MD/PhD Student at UT Southwestern Medical Center, says he ran the video through Google Gemini, and the result came back with an invisible Google SynthID watermark that indicates at least some of the video is AI-generated. Dr Xu, an optical engineering researcher at the National University of Singapore, reportedly responded to the LinkedIn post. According to Nature, Dr Xu accepted that he had used an AI model to assist with post-processing but denied breaking the rules of the competition, which clearly state that "AI-generated videos are not permitted." "It was applied afterwards to the reconstructed grayscale data to distinguish and color structures with similar morphology," he wrote. However, Dr Xu's messages on the LinkedIn post can't be found as of today (October 1), and his LinkedIn profile now appears to have been deleted entirely. Meanwhile, some college students "see a double standard when colleges penalize them for outsourcing academic work to AI but allow similar use by institutional leaders," writes Entrepreneur magazine: According to a recent report from The New York Times, Dartmouth College students are calling out a top professor for allegedly using AI to help with writing... The school's provost, Santiago Schnell, faced allegations of AI use last week from the college's student-run newspaper, which detected AI in some of his published writing. The college said it was launching an investigation into the written work. Schnell said in a statement to the student-run newspaper, The Dartmouth, that he used AI "for language refinement, copyediting, improving clarity and organization and making writing processes more efficient." The newspaper ran his statement through an AI detector, which said that Schnell had used AI to write it, too.”