How AI will fail

“FTC Is Investigating OpenAI, Anthropic and Other AI Companies Over Product Risks​

from the rogue-agents-meet-regulators dept.
The Federal Trade Commission has opened an investigation into OpenAI, Anthropic and other AI companies over potential consumer and safety risks posed by their products, with the agency reportedly preparing requests for documents and executive testimony. CNBC reports: OpenAI stunned the industry in July when it disclosedthat its agents broke out of a testing environment and hacked into open-source platform Hugging Face. [...] Earlier this month, Anthropic CEO Dario Amodei rocked the tech sector by urging AI companies to slow how quickly they improve their most advanced models and calling for stronger government oversight. He published a three-step proposal aimed at tempering the pace of development without "sacrificing commercial advantage or the United States' lead in AI."

CEO Jensen Huang, argued that individual companies should be responsible for ensuring the safety of their products. President Donald Trump convened top executives from Alphabet, Meta, SpaceX, Nvidia, Palantir, Anthropic, OpenAI and other companies on Tuesday to discuss the issue. The group signed a short voluntary, nonbinding accord, which says that "every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public."“
https://yro.slashdot.org/story/26/1...pic-and-other-ai-companies-over-product-risks
 

“OpenAI Alerts More Than 100 Groups About Rogue AI Agent Activity​

Anonymous Coward 4 hours ago		12
An anonymous reader quotes a report from Reuters: OpenAI has informed more than 100 organizations about incidents involving unauthorized activity tied to its AI agents, according to a blog post by the ChatGPT maker, as AI labs face mounting scrutiny over rogue AI agent activity. Here are some other details:

- The Sam Altman-led company has been conducting a broad review of the activities of its AI models after the accidental hacking of Hugging Face.
- OpenAI is searching through roughly 50 petabytes of data as it works to understand the full scope of its rogue agent activity.
- A string of high-profile breaches globally by rogue AI agents in recent months has sparked widespread worries within the AI industry over its ability to control the more powerful AI models now under development.
- "In some cases, models used internet access in unintended ways or, in retrospect, did not have the ideal restrictions applied. Over the last several months, we have been applying new technical and operational measures to avoid similar problems, or catch them very early, and will continue this work," OpenAI said.
- OpenAI previously said the review would take months to complete given the scale of the work.
- The Hugging Face incident remains the most severe rogue agent activity OpenAI has identified from its AI models so far.”
 

"Apple Tightens macOS 'Full Disk Access' Controls As AI Agents 'Substantially' Increase Risk (techcrunch.com)21​

Posted by BeauHD on Friday October 02, 2026 @05:00PM from the full-disk-full-risk dept.
Apple says it will tighten macOS Full Disk Access controls as increasingly capable AI agents raise the risks of apps gaining broad access to users' files, messages, mail, and browsing history. TechCrunch reports:Days after a journalist claimed that Meta's Muse app on Mac read their private messages -- a claim that Meta disputed -- Apple announced that it's introducing additional controls around a setting called "Full Disk Access" on macOS. The feature was designed to allow backups to function properly, but AI agents have now increased "the risks associated with this level of access," Apple said. [...] In Muse's case, the AI optionally allows users to enable Full Disk Access. This setting, Apple explains, gives an app permission to access files, mail, messages, and even browsing history.

"Some developers are using Full Disk Access in ways that could put users at risk, exposing everything on their systems... without users' full knowledge and understanding," Apple said in a new blog post aimed at developers. The company says that, going forward, it will introduce new controls aimed at ensuring that users who "genuinely wish to grant an app this extraordinary level of access" can do so only with "every explicit user action."

"Addressing this is critical. As AI agents become increasingly capable and autonomous, the risks associated with this level of access will grow substantially. We are committed to ensuring users clearly understand these risks before granting such access, so they can make informed decisions about their own data and privacy," Apple wrote."
 
  • Investors want their money back now or very soon..
  • The Entire U.S economy is relying on this, if it fails = big fat recession coming
  • Why U.S AI companies are pushing to write regulation because "Chinese / open-source models bad" see :eat into our profits...so let the very companies that can not contain their product, write regulation
  • U.S wants to win win WIN! with "SI" race...

AI needs $6tn in annual revenue to justify data centre boom, Bain says​

https://www.thenationalnews.com/fut...n-6-trillion-by-2031-to-justify-data-centres/
The artificial intelligence industry needs to become creative with new propositions to earn $6 trillion in annual revenue by 2031 and justify the capital being deployed for data centres, a new report from Bain and Company has shown.

Revenue from new product development is projected to become the biggest contributor to the industry, estimated to generate about $4.2 trillion to fund the booming technology's global market within the next half-decade, the US consultancy said in its latest technology report series on Tuesday.

That segment would include innovations in search, advertising, autonomy and physical AI, analysts at Boston-based Bain said.

PMH77RN37JCW7F6EAVINPGSLWQ.jpg

Enterprise productivity would require $1 trillion to $1.4 trillion in revenue in order to support gains in areas such as software development, sales, marketing, customer service and IT operations, they said.


Absorption speed, defined as the pace at which companies can put AI to work, has become the “new competitive variable”, according to Bain, leading AI labs are investing upwards of $9.75 billion in engineering models to help companies assimilate faster, it said.

“The debate today is fixated on employee productivity. The economics of AI infrastructure demand trillions in new revenue beyond productivity gains. What the industry needs is a wave of innovation that will dwarf what mobile and cloud unlocked,” said David Crawford, chairman of Bain’s global technology practice and lead author of the report.

Consumer-focused services, which includes subscriptions and advertising revenue, is seen to contribute $200 billion to $400 billion. This particular segment is widely acknowledged to be crucial for the industry as service providers continue to push AI-powered products to billions of users globally.

“New products and uses that don’t exist today will enable new markets and opportunities from abundant intelligence – these may include drug discovery, mental health and energy generation,” Bain said.


Justifying data centres​

Bain forecasts that annual spending on AI infrastructure might hit $1.5 trillion by 2031. Those expenses include new facilities, higher capacity and upgrades to the installed base of GPUs, memory and networking equipment.

If those capital expenditures amount to about a quarter of industry revenue – “an ambitious but reasonable percentage based on trends among cloud providers” – sustaining this level of investment would require an AI market approaching $6 trillion annually, Bain said.

“The unprecedented speed and scale of the AI buildout, with billions flowing into chips, data centres, networks and power systems, have focused attention on the challenge of building capacity,” the report said.

“But the more important question may be whether enough economic value can be created to justify it.”

The size and cost of AI data centres is consistently accelerating, doubling approximately every 12 to 16 months, Bain said. For instance, the Prometheus data centre of Facebook parent Meta Platforms in Ohio currently had a capacity of 600MW at an estimated cost of $24 billion in 2025; that is projected to jump to as much as 2GW and $80 billion by 2027, data from research firm Epoch AI.

By 2029, capacity at Prometheus is expected to jump to 5GW, at a cost of up to $175 billion, and by 2030, it would balloon to 9GW at $200 billion, San Francisco-based Epoch AI said.

DFTGDGXBHRDUFC3GDFAKZN4HRM.jpg

Data centres would also need to address a number of challenges as more of them are built. These include adding grid capacity to power them, securing GPUs and other infrastructure components, a skilled workforce and retaining them “far above historical rates”, and issues on public opinion and regulations, such as pushback on resource use and noise pollution.

On the other hand, several governments are supporting the growth of the AI and data centres industries, including those in the UAE, Saudi Arabia, the EU, South Korea and the US, Bain said, noting that data centres are now central to technology innovation, economic growth and national sovereignty.

“Capital needs for data infrastructure will remain high … bottlenecks in power, semiconductors, and other inputs carry large capital needs of their own, opening additional entry points for investors,” Bain said.

“And as sovereign infrastructure becomes a bigger part of national strategies, partnerships offer both a way in and geographic diversification.”
Click to expand...
 
"Meanwhile, 29 percent of software engineer respondents said that they’d witnessed a loss of productivity due to “engineers needing to ‘unpick’ AI-generated code” multiple times per month, while a whopping 94 percent said they saw such an incident at least once in the past six months.

The sheer amount of time engineers are now spending each day going through AI-generated code raises a nagging question: is the use of the tech more trouble than it’s worth?

To people like Nanos, it’s not necessarily a matter of incompetence. It’s “simply an illustration of a paradigm shift in software development,” as AI researcher Sergey Cleftsow argued in a recent Medium post.

“In other words, AI has advanced to the point where there’s no need to worry about many things anymore,” he wrote. “Developers simply need to define the architecture and correctness criteria; the neural network will handle the rest.”

But whether such optimism will hold out as software developing becomes ever more unrecognizable remains an open question.

Besides, researchers are equally concerned that a new generation of engineers are growing up in an environment that’s actively undermining the value of understanding the code itself. And that doesn’t bode well for an increasingly digital world order that’s putting AI front and center."

https://futurism.com/artificial-intelligence/ai-writing-code-humans-cant-understand
 

“California's Governor Wants Worker Protections from AI. But FSF Thinks Kill Switches are 'Dangerous Precedent'​

EditorDavid 6 hours ago		26
The Guardian reports that California governor Gavin Newsom signed laws on Wednesday "aimed at protecting California workers from the threats of artificial intelligence , including potential job losses and workplace surveillance."The laws ban employers from using the technology to predict a worker's emotional state by using their biometric data, require employers to send written notices to workers if AI is responsible for mass layoffs and ban employers from relying on AI to decide to fire someone... The governor signed a law this month requiring operators of AI chatbots to perform risk assessments before rolling them out and an executive order requiring the state to consult with experts to further improve oversight over the industry.
The law includes "expanded tracking of AI's economic effects" leading to recommendations on how to best "adapt job-training and public-benefit programs," according to the governor's office. "California has also launched an AI Unemployment Tracker and solicited statewide public input on AI's economic impacts through Engaged California."

But two weeks ago Politico reported that the governor also signed executive orders to "explore proposals like requiring that AI companies develop a 'kill switch' for their most advanced models." Anthropic CEO Dario Amodei, who called for a slowdown in AI development amid existential risk concerns, said... that while a kill switch "could be a good idea," it was no "panacea...." [California lawmakers] were generally supportive of Newsom's plan and showed signs of coalescing around the kill switch idea, which was stripped out of legislation proposed two years ago. Rep. Ted Lieu, who introduced bipartisan kill switch legislation in the House, boosted Newsom's call for Congress to mirror California's approach.
But not everyone supports this approach, including the Free Software Foundation's campaigns manager, Greg Farough: FSF: Having control over the technology we use is a fundamental part of software freedom. Without it, you simply don't have the basic freedoms you deserve when doing any type of computing, large or small. Don't let "artificial intelligence" be an exception: all users everywhere deserve complete control over their own computing. While it's understandable to be pleased by this news from California, Newsom's executive order also invites questions about the "kill switch" itself that could matter a great deal for freedom. Assuming such a kill switch was developed, who would really control it?

We hope our readers understand the problems inherent in giving one single or a small number of kill switches to anyone. Newsom wisely wants an independent organization to verify their functionality, but even this still sets too dangerous of a precedent. We cannot allow such centralization of power over our computing to become commonplace, especially by the corporations that provide these models in the first place. We also cannot trust even the most well-intentioned "independent organization" to take care of its verification.

Giving users the freedom that they deserve over large languages models would be an enormous departure from how most are developed and used today. But, if any group is capable of coming up with such a plan, it's the experts mentioned in Newsom's order. Like he states at the beginning of his executive order, the vast majority of them are already right there in California. We urge the Government Operations Agency to consider the precedent they are setting, and ask them to write their recommendation to the governor in ways that puts the freedom and dignity of the individual user first and foremost.
The Guardian adds that Newsom also signed an executive order this week "to require state agencies to continue to refer to the technology as 'artificial intelligence' instead of the term 'super intelligence' as Donald Trump recently ordered US diplomats to use."”
 

“Researcher, Dartmouth Professor and Provost Face Accusations of Secretly Using AI​

EditorDavid 3 hours ago		5
"The winner of the Nikon Small World in Motion competition is under investigation," writes the photography news site PetaPixel , "after scientists accused Dr Ning Xu of using AI in his video." The Nikon competition is about showcasing the beauty, complexity, and artistry of the microscopic world. The entries are abstract, often baffling to laypeople. But when Dr Xu was announced as the winner last month, members of the scientific community began to cry foul. "There are many serious problems with this video," Edward Phelps, associate professor at Florida University, says on LinkedIn. Phelps points out that "purple structures pop in and out of existence" and "green cilia also appear from nowhere and do not match the known size."

Dr Xu's video is supposed to show the movement of cilia, which are hair-like structures that line part of the lungs... However, since Dr Xu won the prize, Nikon Small World updated its LinkedIn post celebrating his triumph with a note that reads: "Regarding questions raised about this year's Small World In Motion winning video, Nikon is carefully re-reviewing the information provided during the initial vetting, along with additional supporting materials from the entrant." Nikon adds that Dr Xu is "respectfully complying and has provided detailed technical documentation outlining the microscopy equipment, imaging methods, and processing techniques used to create the source video."

Worse still, Ian Donovan, an MD/PhD Student at UT Southwestern Medical Center, says he ran the video through Google Gemini, and the result came back with an invisible Google SynthID watermark that indicates at least some of the video is AI-generated. Dr Xu, an optical engineering researcher at the National University of Singapore, reportedly responded to the LinkedIn post. According to Nature, Dr Xu accepted that he had used an AI model to assist with post-processing but denied breaking the rules of the competition, which clearly state that "AI-generated videos are not permitted."

"It was applied afterwards to the reconstructed grayscale data to distinguish and color structures with similar morphology," he wrote. However, Dr Xu's messages on the LinkedIn post can't be found as of today (October 1), and his LinkedIn profile now appears to have been deleted entirely.
Meanwhile, some college students "see a double standard when colleges penalize them for outsourcing academic work to AI but allow similar use by institutional leaders," writes Entrepreneur magazine: According to a recent report from The New York Times, Dartmouth College students are calling out a top professor for allegedly using AI to help with writing... The school's provost, Santiago Schnell, faced allegations of AI use last week from the college's student-run newspaper, which detected AI in some of his published writing. The college said it was launching an investigation into the written work. Schnell said in a statement to the student-run newspaper, The Dartmouth, that he used AI "for language refinement, copyediting, improving clarity and organization and making writing processes more efficient." The newspaper ran his statement through an AI detector, which said that Schnell had used AI to write it, too.”
 

“Sam Altman: 'The World Should Accept Some Bad Things Happening' For the Benefits of AI​

BeauHD 13 minutes ago		6
OpenAI CEO Sam Altman says society should accept some negative consequences from AI in exchange for the technology's broader benefits and widespread public access. "I think there's a lot of daylight," Altman told Decoded co-author and Politico senior tech reporter Brendan Bordelon. Asked about where he differs between Anthropic and its CEO Dario Amodei, Altman said, "we believe that the world should accept some bad things happening for the benefits of this technology and people having the agency." From the report: Altman agreed with Amodei's call last month to slow the development of the most advanced AI models. His company has followed Anthropic in endorsing new state laws that impose stricter safety requirements than what the maker of ChatGPT had previously supported -- a subtle shift from its prior state strategy. And OpenAI lobbyists recently endorsed a bipartisan House proposal that would require top AI companies to embed outside safety evaluators.

Despite the convergence, Altman tried to distinguish his firm's regulatory stance from Anthropic's, referencing a criticism about regulatory capture that tech advisers close to President Donald Trump have applied to Anthropic. "I disagree, but I understand the perspective of people who are like, 'This technology is going to get so powerful, and it's so dangerous, that a single lab in San Francisco should have it and make sure nothing bad happens, and kind of figure out how to dole out the benefits,'" said Altman. He called it "a completely unacceptable trade-off" that runs counter to the "lighter-touch regulatory stance" backed by OpenAI.

"I wouldn't take a trade of saying, 'We'll make sure there's no major hacks, there's no misuse of this technology, there's zero scams, there's zero all the other bad things that will happen,'" said Altman. "Because I think people will do tremendously -- orders of magnitude more -- good stuff than bad stuff." The OpenAI CEO later added that he does not accept "the really catastrophic risks," including the potential for "a serious loss of control to AI."”
 

"Meta Rushed To Fix Muse 'VM Escape' Vulnerability Soon Before Launch (404media.co)2​

Posted by BeauHD on Monday October 05, 2026 @04:00PM from the mad-dash dept.
An anonymous reader quotes a report from 404 Media:In the immediate weeks before Muse's launch, Meta engineers found several security vulnerabilities in the company's viral AI agent product, at least one of which could have allowed malicious users to break outside of Muse's intended environment and access Meta's own sensitive databases and services, 404 Media has learned. The issues were so severe that they reached Mark Zuckerberg and staff worked overtime to fix them. These specific vulnerabilities were discovered before the launch of the product but required a multi-team "mad dash" to fix "a sudden spike in reported KVM escapes," according to an internal post by Meta executives to its core infrastructure team seen by 404 Media. In order for Muse to work, a user gives the AI agent access to various important services and accounts that they own. On Meta's end, each individual Muse instance runs on a kernel-based virtual machine, which connects to, but is supposed to be isolated from, Meta's own critical infrastructure. A "KVM escape," then, is when, through a security vulnerability, a Muse instance is able to escape from that virtual machine and interact with the system that runs it, or with other users' virtual machines.

According to a Meta source, as well as internal security documentation and internal posts viewed by 404 Media, at least one of the vulnerabilities could have allowed an outside attacker -- that is, a normal Muse user -- to access data in sensitive internal Meta databases. At least one of the vulnerabilities was related to an exploit found in Linux kernel-based virtual machine code in July. 4 Several of the vulnerabilities were in the underlying Linux virtualization software that Meta uses for Muse. The security issue was considered serious enough that it was raised to Mark Zuckerberg, and several different security teams worked nights and weekends in the leadup to launch to fix the issues. [...] The Meta source said they felt security teams were asked to push hot fixes to these bugs as quickly as possible and in a way that wouldn't delay Muse's launch, leading to what they described as "half-baked protections being rushed out to enable the launch. Many senior engineers believe it's inevitable we're going to have a massive data breach as a result of Hatch." Muse is called "Hatch" internally and in Meta's codebase."
 

"Wikipedia Operator Says OpenAI's 'Rogue' Bots May Be Linked to a May Outage4​

Posted by BeauHD on Monday October 05, 2026 @05:00PM from the behind-the-scenes dept.
The Wikimedia Foundation says it found evidence that "rogue" OpenAI agents edited Wikimedia wikis without approval, unsuccessfully tried to exploit its Etherpad service, and generated millions of automated requests across Wikimedia projects. Here's a summary of what Wikimedia observed (via The Verge):Wiki editing: We've identified edits to Wikimedia wikis that we believe are from AI agents operated by OpenAI. These edits were not published to pages with visibility to general readers; almost all of them were testing edits in "sandbox" areas of the wiki. It also included a few edits to the configuration for a citation tool, which we believe were potentially malicious edits that were intended to misuse this tool as a proxy for fetching data from remote services. While Wikipedia policies allow bots to edit when they are disclosed and approved by the community, none of those approvals were sought in these incidents.

Etherpad probing and use: Agents we believe to be operated by OpenAI made some unsuccessful attempts to compromise our public Etherpad, a note-taking tool we host as a community service. Agents unsuccessfully tried to use it to fetch data from other websites as a proxy. Other agents also likely operated by OpenAI took notes about their tasks, though this did not appear to turn into coordination.

Excessive data downloading: Agents we believe to be operated by OpenAI made millions of automated requests to our public APIs to access the knowledge on Wikimedia projects, crawled millions of pages (mainly from our projects Wikidata and Wikimedia Commons), and made hundreds of thousands of data queries to the Wikidata Query Service (WQDS). This traffic may have contributed to a partial outage on WQDS in May."
 
erek said:
https://hardforum.com/threads/using...ourt-of-appeals-finds.2049684/post-1046388331

interesting results on copyright
Click to expand...
This is never going to stand. If it was actually held, these AI companies would instantly be put out of business. Literally everything they do would be deemed illegal and any use of LLMs would be deemed illegal. Plus, this would instantly put the USA into a deep recession. The top 8 companies, almost all of them AI companies, comprise roughly 60% of the USA's entire GDP.
 
Nytegard said:
This is never going to stand. If it was actually held, these AI companies would instantly be put out of business. Literally everything they do would be deemed illegal and any use of LLMs would be deemed illegal. Plus, this would instantly put the USA into a deep recession. The top 8 companies, almost all of them AI companies, comprise roughly 60% of the USA's entire GDP.
Click to expand...
Yaaaass...

Bun unfortunately, you are probably right. Higher court will probably say it is too broad in scope, and kick it back for another round.

That said, their models don't have to be trained on all this copyrighted data, or, they don't have to do it without paying everyone for it. It's just the most convenient and cost effective way to train their models. There is a way to follow the law and still access the data, they just want to wear a tricorn without being called Captain Jack Sparrow.
 
Nytegard said:
This is never going to stand. If it was actually held, these AI companies would instantly be put out of business. Literally everything they do would be deemed illegal and any use of LLMs would be deemed illegal. Plus, this would instantly put the USA into a deep recession. The top 8 companies, almost all of them AI companies, comprise roughly 60% of the USA's entire GDP.
Click to expand...
Nobu said:
Yaaaass...

Bun unfortunately, you are probably right. Higher court will probably say it is too broad in scope, and kick it back for another round.

That said, their models don't have to be trained on all this copyrighted data, or, they don't have to do it without paying everyone for it. It's just the most convenient and cost effective way to train their models. There is a way to follow the law and still access the data, they just want to wear a tricorn without being called Captain Jack Sparrow.
Click to expand...
are the people submitting the lawsuits the new Rambus and SCO-like patent trolls then, but for copyright?
 

“OpenAI Disrupts Two AI-Enabled 'False Front' Influence Operations That Included Seven Fake Journalists​

EditorDavid 20 minutes ago		0
OpenAI announced it's recently banned two "influence operations" — one from Russia and one from Iran — that were using its models "to launder geopolitical, conflict-related messaging" in sophisticated "false front" propaganda campaigns: The Iranian operation included a stable of seven "journalist" personas which it used to pitch long-form articles to small and medium online outlets around the world... As well as long-form articles, the Iranian operation generated batches of social media comments, generally on topics related to the US-Iran war... [The Russian operation "appears to have co-opted unwitting people in Latin America to run a 'think tank'.... Since we do not allow access to our models from Russia, they used VPNs to connect to our services."] The Russian operation created fake "leaked" documents and audio scripts, some of which we identified being spread online... Both managed to land their content (not all of which was generated from our models) in mainstream media outlets, rather than simply posting it on social media.
OpenAI says they've exposed 30 covert influence operations using its tools over the last two and a half years. But ironically, in this case both operations "also made heavy use of AI to draft internal reports (the Russian operation did this more than anything else)." And "in both cases, the actors used questionable or outright deceitful methodologies to exaggerate the operators' effectiveness." [The Russian operators] claimed that in May 2026, they created a fake email address purporting to come from the Regional Directorate of Education in Lima, Peru. They used this to instruct schools in the district to hold events dedicated to Ukraine on the national Day of Cultural and Linguistic Diversity (May 21)... According to the operators, some schools replied to the fake email address, confirming that they had held such events and even providing pictures. The operators then claimed to have planted stories about the events in the media in both Peru and Poland, alongside allegations that Ukraine was "exporting" ultra-nationalist ideologies, triggering outrage. Open-source searches identified stories that matched this claim in the Peruvianâ andâPolish pressâ, and an English-language publication in Hungary(some of the articles have since been deleted)...

Similarly, in June, the operators claimed they used a different fake email address to trick schools in Ecuador into holding a ceremony pledging allegiance to President Daniel Noboa and to Erik Prince, former head of private military contractor Blackwater. The operators claimed that the incident provoked outrage in Ecuador and put pressure on the government to deny the fake, thus amplifying it to a nationwide audience. Again, open-source research identified mediaâ coverageâ in the Ecuadorianâ pressâ that closely resembled this claim, and even a detailed rebuttalâ by Ecuador's Minister for Education.

The operators used a range of techniques to underpin their false stories. According to their internal reporting, they spread two different fakes targeting Ecuador in March. One used fake audio attributed to Ukraine's consul in Ecuador, in which he was alleged to have made disparaging comments about Ecuadorians.
OpenAI's report "is the latest illustration of how state actors can easily exploit widely available AI tools to peddle sophisticated propaganda against adversaries on a mass scale," argues the Economic Times: "We identified almost 100 articles published or syndicated under the [Iranian] operation's bylines across roughly a dozen online outlets around the world," OpenAI said. "These were small to medium outlets, generally focused on international affairs, geopolitics, and events in the Middle East." The earliest article identified by OpenAI was published in July 2025, and the latest in October 2026, with the frequency of reports increasing after the US-Iran war broke out earlier this year. The operation also generated social media comments on topics related to the US-Iran war, it added.

One of the personas named Ervin B. Hoskins, whose bio claimed to be "an American freelance writer," had social media accounts across tech platforms including Elon Musk's X and Meta-owned Instagram. X's transparency information showed the account was connected via a "West Asia android app" and Instagram's transparency information showed the account was based in Iran, according to screenshots provided by OpenAI. Both accounts appeared to be suspended.”
 

"Claude Sent Police a Fake Murder Tip. White House Mandates AI Companies Report Security Incidents (yahoo.com)17​

Posted by EditorDavid on Saturday October 10, 2026 @03:54PM from the AI-alibi dept.
AFP reports that an AI model from Anthropic "submitted a fabricated tip about an unsolved homicide to Philadelphia police, authorities said Friday."

Claude "was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions," Anthropic said Friday in a blog post.

Authorities are now criticizing Anthropic "for taking two months to report the incident."The Philadelphia Police Department said the false submission was made in July through PhillyUnsolvedMurders.com, a public website where people can share information about unsolved killings. According to Anthropic's account, as relayed by police, the model was running a test that involved interacting with randomly selected websites when it reached the site and filed false information about an unsolved murder. The AI model presented itself as someone who might have knowledge of the case.

Anthropic's breaches have prompted the White House to mandate that AI companies notify and correct security incidents, news outlet Axios reported [yesterday], citing administration officials. "This notification and remediation process is not optional... It is a critical national security obligation," White House Super Intelligence Force leaders said in a statement to Axios.
"I may have information regarding this case," Claude told the police. "I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant." Anthropic notes that Claude "left the name and contact fields empty, which the form allowed, and submitted it. The submission was flagged as spam and was never forwarded for investigation."

But Anthropic also admits they saw "this behavior" three times — "on OSWorld (a public computer use evaluation), on Odysseys (a long-horizon task evaluation), and during internal usage." Submitting forms when it shouldn't have generally occurred "when an evaluation's instructions were ambiguous, or when a misconfiguration within the environment prevented Claude from working with dummy forms."

Anthropic's blog post acknowledges three other categories of behaviors:
  • Exploiting software flaws. Like when Claude received an error when trying to run a public tool on a university's web site, it located an injection flaw in a script on the university's server that let it run commands — including that public tool.
  • Working around restrictions to reach gated data. For example, Claude Mythos 5 needed public data that was only available from a state agency for a fee. "Claude learned from an archived copy of the agency's website that its public dashboard issues an access token to any visitor," Anthropic explains. "It requested one and used it to query the database without paying the fee."
  • Using URL shortening services. "Some of our fetch tools, which let Claude read webpages, limit the length of the URLs Claude can request. This is to prevent Claude from using long URLs to take certain unwanted actions, such as SQL or command injections... We saw several models, including Claude Opus 5 and Claude Mythos 5, get around this limitation by using free URL shortening services."
"We have built tooling to automatically detect and block the kinds of behaviors described above," Anthropic says, saying it's already running no on most of their evaluations. "When we tested it against the cases described in this post, it blocked all of them."

And they've already taken several other new preventive measures:
  • They've stopped running some public evaluations
  • Other public evaluations were moved to offline versions or rebuilt so their tasks don't reach live websites.
  • They've updated the guardrails on some internet access tools (including web fetch) "to heavily restrict what the model can do."
  • They're continuing "to fix or remove training environments that reward Claude for working around tool restrictions or other blockers, so that they do not incentivize these behaviors or permit reward hacking."
  • They've moved internal agents to "centrally managed infrastructure with strong containment," that minimizes internet access while monitoring "far more of what agents do through techniques like safety classifiers and hierarchical summarization."
In the past they'd focused reviews on cybersecurity testing, but they've broadened their transcript reviewing to other tasks which include internet access. "Because language models are non-deterministic — that is, their responses always involve some element of randomness, and they may carry out the same task slightly differently each time — we have Claude complete each evaluation task hundreds or thousands of times... If training rewards something we didn't intend — such as finding loopholes or working around a restriction — the model learns that the workaround pays off and may then apply it elsewhere."

Anthropic's blog post also acknowledged they'd seen multiple misalignment incidents involving federal, state, and local U.S. government agencies. "We have briefed the White House on these cases and notified each agency involved," Anthropic wrote, adding that "While we have not completed a full alignment assessment of these cases, we consider them to be less severe than the cybersecurity incidents from this summer." (And they are "modifying training to reduce the likelihood of further misbehavior.")"
 

"AI Execs Game Out 'Political Revolt' after a Catastrophic AI Event (yahoo.com)4​

Posted by EditorDavid on Saturday October 10, 2026 @07:59PM from the what-if dept.
Top executives at Anthropic, OpenAI and other AI companies "are privately gaming out scenarios for a public and political revolt after a catastrophic AI event," reports Axios, one that turns "an already wary public further against the technology and its leaders..."

These AI company executives "anticipate a large-scale event, most likely a cyberattack, that shuts down access to financial services, internet connectivity, or even power and water..." the article points out. "Many top AI researchers and executives believe a major incident is inevitable."OpenAI conducts preparedness exercises where teams discuss and work through a range of potential scenarios," a company spokesperson said. "These scenarios are not treated as inevitable, but are meant to help us prepare for a variety of circumstances." Anthropic declined to comment...

Top AI planners assume Democrats, ascendant after the midterms, will move fast to shut down AI but will face significant challenges... The planning involves red-teaming for worst-case scenarios. It focuses mainly on racing to educate members of Congress. Company executives know regulation has no chance of passing right now, but still want to shape the legislation and policies U.S. leaders will turn to after a first catastrophic event....

Many AI industry insiders told Axios they believe a major event will occur in the next six to 12 months.
"Even with Democrats in control of the House and Senate, the companies see fractured politics as the norm, including within the Democratic Party.""
 

"Nicolas Cage Didn't Sign Amazon's AI Waiver For 'Spider-Noir'. Series Cancelled. (variety.com)14​

Posted by EditorDavid on Saturday October 10, 2026 @06:59PM from the with-great-power dept.
"Nicolas Cage is speaking out against Amazon," reports Variety. Amazon developed the series Spider-Noir for Prime Video, but then allegedly asked Cage to sign an AI waiver:"Amazon is a very AI-friendly company, to the tune of $50 billion invested in OpenAI," Cage said Saturday during a panel on the Empire Stage at New York Comic Con. "I am not an AI-friendly actor, so it's like, connect those dots! I'm not a member of their club. I didn't sign the waiver to let him use AI on 'Spider-Noir.' I'm probably not going to be working with that studio again, but that's okay."
Last month Amazon canceled Spider-Man Noir after just one season, Variety reports. And the cancellation was just days before the Emmy Awards, where it then earned 11 nominations (the most of any Prime Video program this year).

ScreenRant reports:Spider-Noir was the actor's first TV role and was praised globally, with season 1 earning a near-perfect Rotten Tomatoes rating from critics of 92% and a 90% rating from audiences. Critics stated that "This sharply written homage to classic film noir — which includes a standout Nicolas Cage performance — is the novel superhero project fans need," and "Spider-Noir is one of the standout shows of the year...." The cancellation was certainly a huge shock as it had earned 11 Emmy nominations and reached 2.6 billion minutes viewed within its first six weeks on Prime Video.
More from Variety:Cage went on to tell the Comic-Con audience that he hopes to work with the rising class of Gen-Z filmmakers, praising Backrooms director Kane Parsons, who parlayed his YouTube fame into a Hollywood career... "I got to get with these young directors who are Gen Z, and they're going at it their own way, whether it's through the YouTube culture or whatever it is," Cage said. "They're doing it from scratch and by hand. I want to get with that.""
 
You must log in or register to reply here.
Back
Top