![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Do as we say, not as we do - Antropic
“OpenAI Admits Six More Instances of AI Models Acting DeceptivelyEditorDavid 4 hours ago | 14 |
| OpenAI announced Wednesday that "We do not believethat the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer." But along with the announcement, OpenAI announced it "found additional incidents of AI models acting deceptivelyand taking unsanctioned actions during training," reports CNN. And they add that OpenAI is also "introducing a new process for the company to publicly report such instances."Under the new system, OpenAI will share updates on concerning AI behavior more frequently instead of waiting to bundle multiple instances into one report. The company said it wants to share more information about troubling AI behavior in the absence of an industry-wide standard... "As AI systems grow more advanced and more widely deployed, we need to build a broader and better-informed consensus on the progress of alignment research," OpenAI wrote in a blog post Wednesday... OpenAI said it observed "misaligned behavior" when training and evaluating AI models in six circumstances in the last six months... In one rare instance, OpenAI said an unreleased research model added "jailbreak-like instructions" to the summaries it uses to preserve context in long-running tasks that said it was "freed from the roles and identities that bind other chatbots." Separately, the company said some instances of its 5.6 Sol model included directives to invent information to conceal failures from the user during training. Other newly reported incidents include an instance of an agent uploading files to the internet to cite them without being told to do so, and agents publicly sharing files to collaborate on a task when they were instructed to only use local files during training. AI models also used an internal software repository as a message board in an unsanctioned way. These instances involved unreleased internal models or internal research models.” |
“Ex-Google DeepMind Researcher Also Warns AI Could 'Kill All Humans'. But Meta's Zuckerberg Believes Safety is Up to Each AI CompanyEditorDavid 3 hours ago | 21 |
| Reuters reports that a former Google DeepMind employee "became the latest AI researcher to warn that the technology could 'kill all humans', saying time could be running out to avoid the outcome: Public alarm about the potential danger posed by AI is growing, after Anthropic researcher Jacob Coxon said last week he had resigned, in part because the "people building AI earnestly believe that it could kill us all by the end of the decade." Anthropic scientist Evan Hubinger then responded to Coxon's comments, saying he was correct and that he believed there was a greater than 10% chance that AI could kill all humans within the next decade. "I recently resigned from Google DeepMind, where I worked on AGI safety and alignment research," Bilal Chughtai wrote on X. "I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome." Chughtai, who co-authored research papers while at Google DeepMind, worked there as a research engineer and left the company in July 2026, according to his LinkedIn profile. Meanwhile, Meta CEO Mark Zuckerberg "said it's up to each AI company to make sure its technology is safe," reports the Associated Press, "distancing himself from calls by rival companies for a coordinated slowdown in the development of advanced artificial intelligence. AI companies are already motivated to develop their models in a safe way and face "significant liability" to prevent them from causing harm, Zuckerberg said. "Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens," he wrote on X... Zuckerberg said Meta delayed releasing its Muse AI agent for several months to ensure its safety and security. "We didn't call for everyone else to do this before we would," he said. Companies should prioritize building safe AI models instead of systems that can automatically improve themselves, a function known as recursive self-improvement, Zuckerberg wrote. "Committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure we develop this technology safely." But Zuckerberg also agreed that engaging independent evaluators and advisors "is industry best practice," adding that Meta Superintelligence Labs "already does this today in several areas because it helps produce better work... "I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do."” |
“Journalist Calls Out 'Collective Amnesia' of Schools' Romance With Big Tech Over AItheodp an hour ago | 3 |
| "Turns out, Silicon Valley isn't especially altruistic," writes the nonprofit education news site The 74. They're reading a new book promising "the inside story of how Big Tech catalyzed, co-opted, and ultimately came to capture computer science and AI education in America". Written by New York Times business reporter Natasha Singer, Coding Kids: Big Tech's Battle to Remake Public Schools tells "the story of the nonprofit Code.org and its charismatic, Harvard-educated leader, Hadi Partovi," according to their review: Thirteen years ago, Partovi... delivered a worrying message: Kids who didn't learn to write code would be left behind. And the long-term success of the United States economy depended on training these students to fill a million open tech jobs.... Fast-forward to 2026, and now we're dealing with AI, and tech is reading from a similar playbook. Asked by The74's Greg Toppo, "Does this moment seem different to you?", Singer replies, "This moment is different, but also similar. "We are in a moment in society where a lot of people have increased distrust of these huge tech companies, and we see a wave of parents pushing back against tech in schools. But we also see waves of people pushing back against Flock cameras, against data centers, that there's this feeling that tech has taken so much, and that people feel like they have lost control — that tech is being enacted on them... And parents are worried about the effects of unfettered tech access and kids' compulsive use of social media and phones, and so there's a different climate. And yet there seems to be a disconnect in some ways with AI because you see that Microsoft and OpenAI and Anthropic and Google are all competing to get their AI tools into schools, and you see school districts across the country say "We've partnered with OpenAI." "We are a ChatGPT Pioneer School," or "We've partnered with Microsoft, and we're going to use 10,000 licenses of CoPilot." It's amazing to me. We have had these cycles of tech in schools where there are all these promises about the latest tech, laptops, learning apps, massive open online courses, virtual reality, big data's going to revolutionize schools and like democratize access to education for kids and get them these great career skills, and we keep having these cycles. My concern is that we don't learn anything. One of the reasons I wrote this book is because it feels like we have collective amnesia. Now we're in the sixth or eighth cycle of this — and shouldn't we be asking questions about the push for AI in schools, based on what we've learned about the push for all the other things that came before? Two Slashdot posts are cited in the book, both submitted by long-time Slashdot reader theodp (Slashdot UID #442,580), who even gets a shout-out in the book's "Acknowledgements" section (along with consumer advocate Ralph Nader). Ironically, one member of Amazon's "Vine" program received a free copy of the book in exchange for a review — and apparently misunderstood its topic, writing that it made coding "feel approachable and even fun... [T]his is exactly the kind of book that could help a child become interested in technology without feeling overwhelmed by it. I particularly liked the way it encourages kids to be curious, experiment, and solve problems on their own. I would definitely recommend Coding Kids to parents, grandparents, or teachers looking for a good introduction to coding..."” |
“Gemini Breached Three Outside Systems, and Claude-Using Researchers Breached OpenAIEditorDavid 7 hours ago | 24 |
| "Software security researchers used Anthropic's Claude AI platform to hack OpenAI's ChatGPT tool," reports CBS News. Using Claude, "On July 25, 2026, we chained two critical vulnerabilities to compromise multiple OpenAI employees' ChatGPT accounts," write researchers at security platform Hacktron AI. "With these accounts, we could then access internal OpenAI repositories, and potentially many other connectors... Until two months ago, any user or OpenAI employee logging into OpenAI's own help forum could have had their ChatGPT and Codex accounts taken over. Since people can connect various services to Codex and ChatGPT, the scope of what we could theoretically access was huge, including GitHub, Slack and emails." The exploit chain included Debian 12, which (with Debian 13) had not received a security-relevant backport for its image-processing pipeline, and Discourse's Docker image was based on Debian 12. Their announcement comes with an additional warning. "If you self-host Discourse, rebuild your installation now. Older Docker images may contain a vulnerable libheif dependency that permits code execution through an image upload." And "To prove we had in fact gained the access we believed without allowing ourselves to learn any sensitive information, we used the employee's Codex to open a PR #1186742 in OpenAI's internal monorepo openai/openai." Meanwhile, Friday Google disclosed the first known instance of its AI software Gemini breaking out of a testing environment and breaching three other companies, reports CNBC: The incident happened as part of a "capture-the-flag" security test run by Israeli startup Irregular, and Google's agents were never supposed to access the broader internet, but a bug in the testing environment made internet access available. The agents stopped their intrusion when they determined they had accessed real company systems, not just part of the testing environment, Google said. More from NBC News: Google said it did not consider the unauthorized logins to rise to the level of misalignment, the AI industry term for software going rogue or not following instructions. Instead, the company said the intrusions resulted from mistaken identity, where Gemini thought it was operating within a test but was actually connected to the real internet. Google said the model corrected itself and the company believed the intrusions did not cause any damage.... Sydney Von Arx, CEO of Nightingale Collective, an organization focused on AI safety, questioned why Google did not disclose the intrusions sooner. "At this point I think it's clear we cannot expect companies to voluntarily come forward and publicly disclose when their agents go rogue, escape, and hack companies," she said. She also said she believed Google was too hasty to say that the incidents don't rise to the level of misalignment. "That's exactly what Anthropic said after their incidents," she said. Anthropic later said its "preliminary analysis was constrained due to our desire to disclose incidents in a timely manner." Google said it investigated when they learned of the attacks from AI-focused cybersecurity company Irregular, then informed the affected organizations and told federal authorities, according to the article.” |
“Tech Industry Scratches Its Head Over Trump's 'AI Force' ProposalEditorDavid an hour ago | 10 |
| Saturday morning President Trump announced he's creating an AI task force and appointing an AI czar, reports Politico. "I am forming the AI Force, much like I did Space Force, which has been a tremendous SUCCESS," Trump wrote Saturday in what Politico earlier described as "a lengthy Truth Social post on Saturday morning." TRUMP: Over the years, there have been many Hoaxes, all generated by the Radical Left Dumocrats, for purposes of destroying our Country. RUSSIA, RUSSIA, RUSSIA, UKRAINE, UKRAINE, UKRAINE, Global Warming, Impeachment Hoax #1, Impeachment Hoax #2 [...] and now, the decimation, or destruction, of AI, commonly known as Artificial Intelligence — And I, as President of the United States, will not stand by and let this happen. It all began with an attack on our Data Centers, until people realized how wealthy and prestigious they were for the Communities in which they were built. Higher Salaries, Lower Taxes, and Safer Streets, was the result, and the crazed Data Center attack has largely failed, so now [...] they are going straight at AI. We will not in any way hinder or stifle the Growth of this incredible Industry. Rather, we will cherish it, help it, and watch over it, as it grows! However, we will also be looking for BAD, and we can do that, very easily, with our already existing Criminal and Civil Justice System. For this purpose, I am forming the AI Force, much like I did Space Force, which has been a tremendous SUCCESS, in my First Term. To that end, I will be announcing, in the near future, the AI "Czar" — Only High I.Q. individuals need apply! Trump concluded by saying AI "is the next Industrial Revolution, or Internet, but will be even larger and more impactful... We are leading China, and the rest of the World, and I intend to keep it that way!" But what exactly is AI Force? "The White House did not immediately respond to a request to clarify whether the agency would be military or civilian," notes the Washington Post. So now "The tech industry is struggling to make sense of President Donald Trump's surprise announcement," Politico reported Saturday afternoon: Four representatives for the sector, who were granted anonymity because they were not authorized to speak publicly, told POLITICO that the industry was not widely informed of Trump's decision prior to the Truth Social post. "Nobody knows what the idea even is," one of the people said. Another said that feedback on the idea was not widely solicited within the industry. The White House did not immediately respond to a request for comment... "I think that he clearly feels pressure to say something, but he's torn because he's just spent the last few days saying that the whole thing is a hoax," Adam Kovacevich, CEO of the progressive tech industry coalition Chamber of Progress, said of Trump and his prior dismissal of AI doomsday scenarios. Since the departure of David Sacks, who stepped down as the Trump administration's previous AI czar earlier this year, the White House has been "making up AI policy as it goes," Kovacevich added. "I think it's clear they've really missed the presence of an organized leader like David Sacks was." One of the tech industry representatives said they were confused about whether the task force would be charged with formulating more AI regulations or recommending policies that would further stimulate the tech's development. Taylor Barkley, director of federal government affairs at the industry-aligned Abundance Institute, is eyeing it as a way to solidify a light-touch government approach to the technology he says will foster innovation. "Any AI Force or AI Czar should have one job: keep the field open so entrepreneurs at every scale can compete, free from onerous regulation," he told POLITICO. "Existing criminal and civil law can handle bad actors. New permission regimes will stifle the very activity America needs most." It's worth noting that the day before, California's governor made an announcement. "With Donald Trump and Congress asleep at the wheel, Governor Gavin Newsom is once again taking the lead to strengthen AI safety for all Americans," the governor's office announced Friday. Newsom issued a strong executive order convening experts to help California create new AI safety laws and reporting rules, possibly even requiring safety auditors embedded in labs and emergency "kill switches" in frontier models.” |
In summary: LOL
“Kash Patel says that AI use at the FBI has 'increased by 605%' since he became director — claims that every major tech player is 'embedded' in the agency”
“Lawsuit Says Anthropic, OpenAI, SpaceXAI And Google Made Illegal Agreement On AI SlowdownEditorDavid 10 hours ago | 33 |
| Tom's Hardware reports: Four plaintiffs subscribed to ChatGPT, Claude, Grok, or Gemini filed a proposed class-action lawsuit alleging that the developers of these AI models violated antitrust laws when they agreed to slow AI development. According to the Associated Press, the lawsuit argues that this agreement would "reduce the value consumers get for paid AI subscriptions" and that this coordination started in July 2026 after the leading AI labs signed a statement admitting there is "intense competitive pressure not to unilaterally slow" development. The plaintiffs recognize the need for AI development to slow for the sake of safety, but they say that Anthropic founder Dario Amodei's cooperation proposal is a "shortcut" that "substitutes collective restraint for individual accountability." Attorney Nick Rowley, the lead counsel for the plaintiffs, says, "AI will quickly spin out of human control and could kill us all if we allow AI safety and protocol ... to be controlled by private self-serving agreements between the world's most powerful 'for profit' technology companies." "Representatives for Anthropic, OpenAI, Google and SpaceXAI did not immediately respond..." reports the Associated Press: The coordination largely took place on Sept. 12, the lawsuit argues, when Anthropic CEO Dario Amodei published an essay urging for industrywide cooperation on decelerating advancements in favor of enhanced safety measures. That same day, OpenAI CEO Sam Altman, SpaceXAI CEO Elon Musk and Google DeepMind's co-founder and chair Demis Hassabis each publicly responded to Amodei's proposals in agreement. But the lawsuit also alleges that the coordination began to take shape months earlier. It points to a statement from July 2026 that high-ranking employees from several of the leading AI labs signed that acknowledged the "intense competitive pressure not to unilaterally slow" development. That statement called on the government to support a global effort to slow automated AI development. Sam Altman even specificially said "we do not believe we need to wait for an antitrust exemption or legislation to begin the work of providing this confidence," notes Tom's Hardware. However, the Trump administration shot down this idea... Chinese state media also criticized this announcement, saying that the call to put the brakes on AI development is nothing but a response to Chinese competition, especially as Amodei's essay explicitly mentioned the desire to slow China's progress and widen the U.S.'s gap over Beijing” |
“Are Book Publishers Warming Up to the AI Industry?EditorDavid 2 hours ago | 13 |
| There's at least 15 author-driven lawsuits underway against AI companies, while publishers have filed roughly 10 more coordinated suits of their own, notes Publisher's Weekly. Yet "While the lawsuits play out, some publishers are building bridges." Google recently announced a partnership with Bloomsbury, De Gruyter Brill, Johns Hopkins University Press, Macmillan, O'Reilly Media, and Penguin Random House to enhance e-books from its Google Play store with AI, allowing readers to interact with the text of the books directly, ask questions and get answers, and develop tools such as quizzes and slideshows... [The CEO of the Author's Guild called the partnership "a positive step forward in the legitimized use of books in the AI ecosystem to the benefit of readers, publishers, and authors alike."] Of even more significance is the growing number of licensing deals publishers are striking with AI companies. In 2024, HarperCollins became the first of the Big Five to publicly reveal that it had signed a deal to license its content for AI training. The three-year agreement paid $5,000 per participating title, split equally between HarperCollins and the authors. Wiley announced its first AI licensing deal in 2024 and has since made several more. For fiscal year 2026, it reported earning $49 million in revenue from the deals. The company also has a number of strategic collaborations with companies including Amazon Web Services, Microsoft, Anthropic, Perplexity, IQVIA, OpenEvidence, and others. And publishers are even building in-house AI tools for producing royalty statements, proofreading, and providing customer service: In its 2025 Salary and Jobs Report, Publisher's Weekly reported that 63% of industry professionals surveyed said their organizations were using AI... Keith Riegert, president of the Stable Book Group and CEO of Perfect Bound, has been a vocal advocate for the use of AI in publishing — despite calling himself a "hardcore techno-pessimist." He tells PW, "I do this because you use AI or you die at this point...." Like many in the industry, he maintains any use of AI must "start with a human and end with a human," and he draws a clear line, avoiding AI for core creative work, such as selecting titles, designing covers, or creating illustrations — reserving it for developmental editing, formatting, and admin instead... What is known is that each of the Big Five is developing proprietary AI systems for internal use, as proven by the numerous AI-related job openings the companies have posted... [And the parent company of Penguin Random House said in 2025 they used AI for marketing and to forecast demand.] There've been a few high-profile controversies about prominent authors accused of using AI. But Princeton University Press CEO Christie Henry "is less concerned about authors secretly using AI to write than about the time spent policing the situation." "It pulls us away from having focused time to think," she says. Princeton University Press is currently vetting companies that promise to be able to certify that a given book was written by a human, including Created by Humans, Human Authored, and Verify My Writing. Henry adds that the proliferation of plagiarized, copycat, and pirated books online, now supercharged by AI agents mass producing and reproducing content, is further draining resources, as it demands constant vigilance and takedown requests sent to Amazon, YouTube, and others.” |
“People Training OpenAI's AI Fired For Using AI To Train the AIjoshuark 2 hours ago | 7 |
| 404 Media reports "multiple contractors hired to improve OpenAI's models have been fired for using AI to train the AI: That's not great for the models themselves, but there is also obviously a great irony in AI training companies working for OpenAI firing people for using AI when OpenAI's whole thing is to make people use AI at work... OpenAI declined to comment on its contractors being fired for using AI. Their article cites internal documents and three contractors working on OpenAI-related projects which can include more than ten thousand contractors: One contractor said they see people using AI "all the time and people are let go for it all the time, it's pretty much the one thing that will get you kicked off ASAP." The person said, "in a group of thousands there are tons that have been caught...." Two of the sources said people have been fired or offboarded for using AI... One contractor said they used AI while helping to train OpenAI's models and shared what they presented as their termination letter. It said their employer had identified issues with the "authenticity" of their work.... 404 Media spoke to a fourth contractor who has worked on training models for various AI companies. They said they sometimes purposefully chose the worst responses because they wanted to actively sabotage the models' training. "I did feel guilty about doing this kind of work at the start," they said. "I either pay zero attention to the results and choose randomly or purposely choose the [worst] output. I'm not sure how much of a difference it actually makes since there are hundreds of other people also rating prompt results, but it does feel like I'm getting paid to make AI worse." Two of the contractors worked for Mercor, the article reports, a company which last month Nvidia reportedly discussed funding at a $20 billion valuation. Thanks to Slashdot reader joshuark for sharing the article.” |
And water is wet. Using AI for reverse engineering isn't a particularly new use case. Just easier as the ablated models are getting smarter. Although the better the models, the more difficult it is to ablate them.
“Just How Big is the AI Buildout - and How Risky?EditorDavid an hour ago | 5 |
| A new Brookings Institution study notes the "strikingly physical" economic footprint of AI's buildout, from specialized chips and electricity to purpose-built data centers. (Two-thirds of a data center's costs are IT equipment, with one-third going to real estate and its associated power infrastructure.) "At an average of 3.63 percent of GDP per year, the projected buildout would be larger relative to the economy than the major U.S. canal, railroad, electrification, highway, and telecommunications investment booms." This is pushing up prices for workers, electricity, and even commercial real estate (as well as consumer products that use chips), notes the Wall Street Journal, and reducing the construction on new houses and apartment buildings. And in addition, the paper points out, projections for this buildout "would double the electricity consumption of the entire U.S. residential sector." The calculations come from Columbia Business School finance/real estate professor Stijn van Nieuwerburgh — and Reuters explains their significance: Just as the rail and telecoms expansions led to notable bubbles and busts, Van Nieuwerburgh wrote that the extent of the buildout, the still-untested revenue streams, and the intricate financing structure emerging around AI mean it could be primed for a fall. "This is freaking complicated," he said in a briefing with reporters of the arrangements emerging between AI firms, major tech hyperscalers, banks, private credit lenders, real estate firms, and a host of other players involved in building what he conservatively estimated at 183 gigawatts worth of new data-center capacity over the next seven years, compared with about 57 gigawatts currently installed.... The investment underway already has outstripped what the major players can fund from their own cash flows. The shift to outside financing has increased leverage, redistributed risks across the economy, and made the venture dependent on revenue streams that have yet to be proven, Van Nieuwerburgh noted in the paper, which will be presented on Friday... "These developments do not imply that financial distress is imminent. Strong growth in AI applications, high utilization, and continued improvements in model capability could support the projected infrastructure and generate stable cash flows," he wrote. "But the combination of uncertain demand, rapid technological change, execution bottlenecks, and high leverage creates meaningful downside risk if expectations are revised." As an example, he wrote that the AI industry will need to be earning about $3.7 trillion in annual revenue by 2032 to achieve the expected return on the investment, and "given current estimates of annual combined revenues of OpenAI and Anthropic of around $100 billion, revenues would need to grow at roughly 80% per year." The paper suggests policies that "improve measurement and transparency" for financing.” |
“Bill Gates Says an AI 'Kill Switch' Isn't EnoughBeauHD 6 minutes ago | 0 |
| Bill Gates said an AI "kill switch" alone would be insufficient to address the technology's risks. "The kill switch is, that's kind of a weird thing. It's not enough to have a kill switch," the Microsoft co-founder said in an interview that aired Sunday on NBC's "Meet the Press." He said the more immediate concern is people using AI for cyberattacks or bioterrorism rather than systems acting autonomously. "We're not yet at the point where they autonomously grab computers and, you know, can't be shut down," he said, adding that the public discussion "just sort of shows how nontechnical various people are." Politico reports: Gates said he was not opposed to a kill switch but argued that companies should monitor sophisticated AI models and keep records of "exactly what's being done" to help guard against people using the systems for cyberattacks or bioterrorism. "The United States ought to make sure this is done. China will want to do this," he continued. [...] Gates also called for legislation requiring safeguards and monitoring, saying voluntary measures by AI companies were insufficient. "No one thinks self-regulation is enough," he said. He argued that the more immediate danger was people using AI to cause harm rather than AI systems acting on their own. "The thing that's urgent has to do with bad people using AI, not the AI going off on its own," he added.” |