# superintelligence

A much more capable form of AI whose alignment with human intent is an unsolved machine-learning problem.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Why a Computer Scientist Thinks Superintelligence Won't Happen

DevFeed: [Why a Computer Scientist Thinks Superintelligence Won't Happen](<https://devfeed.tech/articles/why-a-computer-scientist-thinks-superintelligence-won-t-happen-145256.md>)

Original publisher: [Read original article](<https://spectrum.ieee.org/ai-superintelligence-debate>)

Author: Cal Newport

Published: 2026-11-19T12:00:03Z

Content type: opinion

Language: en

Sources: [IEEE Spectrum](<https://devfeed.tech/sources/ieee-spectrum.md>)

Topics: [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [computational complexity](<https://devfeed.tech/topics/computational-complexity.md>), [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>)

Tags: [agi](<https://devfeed.tech/tags/agi.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-companies](<https://devfeed.tech/tags/ai-companies.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [anthropic-ceo-dario-amodei](<https://devfeed.tech/tags/anthropic-ceo-dario-amodei.md>), [artificial-general-intelligence](<https://devfeed.tech/tags/artificial-general-intelligence.md>), [asi](<https://devfeed.tech/tags/asi.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [computer](<https://devfeed.tech/tags/computer.md>), [control](<https://devfeed.tech/tags/control.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [engineers](<https://devfeed.tech/tags/engineers.md>), [extinction](<https://devfeed.tech/tags/extinction.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [openai](<https://devfeed.tech/tags/openai.md>), [recursive-self-improvement](<https://devfeed.tech/tags/recursive-self-improvement.md>), [rsi](<https://devfeed.tech/tags/rsi.md>), [science](<https://devfeed.tech/tags/science.md>), [singularity](<https://devfeed.tech/tags/singularity.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>)

### AI overview

The author argues that recursive self-improvement leading to superintelligence remains highly speculative: post-training depends on suitable feedback data, major AI advances require substantial scientific work, and compute is finite. Computer science also shows that some problems are unsolvable or practically intractable, while AI progress remains uneven across tasks. The article attributes reported hacking incidents to long-running agents that let LLM outputs guide actions with few constraints, and contrasts these with task-specific systems such as AlphaFold and autonomous driving. It warns that pursuing self-improving systems may still increase risks by making AI harder to understand and control, and calls for more independent scrutiny.

### Source excerpt

Many people were caught off guard this past summer when employees of one of the world's leading AI companies began casually debating whether their products would lead to human extinction. It started when Anthropic researcher Jacob Coxon announced his resignation, warning that his employer and OpenAI were "racing straight to self-improving superintelligence and gambling with our lives." Evan Hubinger, Anthropic's head of alignment science, replied, "Jacob is correct here--we really do earnestly believe AI could kill all humans!" And Anthropic CEO Dario Amodei subsequently told CNN, "I agree with Jacob much more than I disagree with him." These concerns stem from a belief that has circulated in futurist circles for many decades: Once engineers design an AI system capable of improving itself, it will trigger an accelerating cycle of autonomous upgrades known as recursive self-improvement (RSI). The result will be artificial superintelligence (ASI), a system with cognitive capabilities so vastly exceeding those of humans that it will easily evade our control and perhaps even wipe out our species. Artificial superintelligence is, it should be noted, a far more ambitious goal than artificial general intelligence (AGI), which generally refers to AI that can match human capabilities across a broad range of cognitive tasks. These ideas have prompted varied reactions. Some argue that the concept of ASI is being used for marketing or to instigate regulation that will help the incumbent labs squash new competitors. Others focus on the plausibility of such systems killing us all, debating the finer points of specific scenarios such as AI-engineered superviruses. Still others claim that aligning ASI to human interests is feasible and so we should rush ahead to usher in this digital utopia. Amid these conversations, one question has received comparatively little attention: From an engineering perspective, is superintelligence even possible? The answer is important because it can he

## Human-AI Mutualism Is Unstable under Capability Asymmetry

DevFeed: [Human-AI Mutualism Is Unstable under Capability Asymmetry](<https://devfeed.tech/articles/human-ai-mutualism-is-unstable-under-capability-asymmetry-142635.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23322>)

Author: Aydin Mohseni

Published: 2026-11-19T05:00:00Z

Content type: opinion

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [capability](<https://devfeed.tech/tags/capability.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-gt](<https://devfeed.tech/tags/cs-gt.md>), [faster](<https://devfeed.tech/tags/faster.md>), [goals](<https://devfeed.tech/tags/goals.md>), [humans](<https://devfeed.tech/tags/humans.md>), [leverage](<https://devfeed.tech/tags/leverage.md>), [model](<https://devfeed.tech/tags/model.md>), [resources](<https://devfeed.tech/tags/resources.md>)

### AI overview

A simple game-theoretic model argues that mutual dependence alone may not preserve human welfare as AI capability grows. Without alignment, people remain as well off as before AI only while they retain leverage--through something the AI depends on or the ability to impose costs large relative to its output. Once that leverage disappears, maintaining prior welfare requires alignment; with insatiable AI goals, the required alignment approaches perfection as capability increases. The abstract also says alignment persists only while misaligned variants that retain more resources are reliably sanctioned. These claims are presented as model results, not as empirical findings.

### Source excerpt

arXiv:2611.23322v1 Announce Type: new Abstract: One hope for a good human future alongside advanced AI rests on enduring mutualism: humans and AI will need each other, so the relationship will stay good for both. We argue that this hope fails as AI capability grows. Bargaining guarantees each side only what it would get if bargaining failed. For humans, this can be less than they had before AI. In a simple game-theoretic model, humans stay as well off as before AI without alignment only while they have leverage over the AI: either the AI depends on what they supply, or they can impose costs on the AI that are large relative to what the AI can produce. Once humans have neither kind of leverage, they stay as well off as before AI only if the AI is aligned with them. If the AI's goals are insatiable, the alignment required approaches perfect alignment as AI capability grows. Thus, mutualism is not a substitute for alignment. And when AI variants that keep more resources spread faster, alignment lasts only while misaligned variants are reliably sanctioned. In these two senses, human-AI mutualism is unstable.

## The Silent Consensus

DevFeed: [The Silent Consensus](<https://devfeed.tech/articles/the-silent-consensus-142334.md>)

Original publisher: [Read original article](<https://openai.com/index/the-silent-consensus>)

Published: 2026-11-18T10:00:00Z

Content type: opinion

Language: en

Sources: [OpenAI News](<https://devfeed.tech/sources/openai-news.md>)

Topics: [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [digital sovereignty](<https://devfeed.tech/topics/digital-sovereignty.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [human centric design](<https://devfeed.tech/topics/human-centric-design.md>)

Tags: [accountability](<https://devfeed.tech/tags/accountability.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-policy](<https://devfeed.tech/tags/ai-policy.md>), [ai-safety-and-alignment](<https://devfeed.tech/tags/ai-safety-and-alignment.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [artificial-general-intelligence](<https://devfeed.tech/tags/artificial-general-intelligence.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [autonomous-vehicles](<https://devfeed.tech/tags/autonomous-vehicles.md>), [business](<https://devfeed.tech/tags/business.md>), [civilization](<https://devfeed.tech/tags/civilization.md>), [competition](<https://devfeed.tech/tags/competition.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [contracts](<https://devfeed.tech/tags/contracts.md>), [control](<https://devfeed.tech/tags/control.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [futures](<https://devfeed.tech/tags/futures.md>), [institutions](<https://devfeed.tech/tags/institutions.md>), [intelligence-age](<https://devfeed.tech/tags/intelligence-age.md>), [openai](<https://devfeed.tech/tags/openai.md>), [society](<https://devfeed.tech/tags/society.md>)

### AI overview

The essay argues that market competition could gradually shift economic and political authority from people to AI systems, even without machine rebellion or deliberate human surrender. Ball rejects this "silent consensus" and grounds human primacy in people's historical role in building civilization and their standing as citizens, not in a permanent intellectual advantage. He proposes preserving human accountability through clear, limited and auditable agent permissions; withdrawable authority; caution around agents owning property or forming corporations; and systems that support users' understanding, decision making, privacy and control of their data. He presents these measures as starting points, not a complete blueprint, and acknowledges uncertainty about whether autonomous firms could outcompete human enterprises.

### Source excerpt

Dean Ball, Head of Strategic Futures at OpenAI, examines how more capable AI could reshape society and how people can retain control of their institutions.

## Rational Alignments of Superintelligences via Debates

DevFeed: [Rational Alignments of Superintelligences via Debates](<https://devfeed.tech/articles/rational-alignments-of-superintelligences-via-debates-132237.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16455>)

Author: Thanawat Sornwanee

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [asi](<https://devfeed.tech/tags/asi.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [framework](<https://devfeed.tech/tags/framework.md>), [question](<https://devfeed.tech/tags/question.md>)

### AI overview

The paper studies ASI-assisted question answering in which agents know the correct answer and a human principal does not. It proposes controlling the interaction between the human and agents while leaving agent utilities unchanged. In particular, agents that can reason further and act in a debate framework at subgame-perfect equilibrium are reported to produce more predictable results, which are favorable to the human in some cases. The excerpt gives no specific mechanisms or conditions for those cases.

### Source excerpt

arXiv:2611.16455v1 Announce Type: new Abstract: Artificial superintelligences (ASI) is artificial intelligences with greater capability. However, as human principal has to use it, we have to ensure that the agent will perform in the interest of the human. We investigate the framework of ASI-assisted question answering where ASI agents know the correct answer but the human does not. We provide some example of mechanism for rational alignment, where the agent utility is unchanged, but the interaction between the agent and the human principal is controlled to be in a certain fashion. We show specifically that a debate framework where the agents can further reason and behave in a subgame perfect equilibrium manner will lead to a more predictable result, and in some case will be favorable.

## Harari Warns Tech Leaders Against Pursuing AI Superintelligence

DevFeed: [Harari Warns Tech Leaders Against Pursuing AI Superintelligence](<https://devfeed.tech/articles/harari-warns-tech-leaders-against-pursuing-ai-superintelligence-129987.md>)

Original publisher: [Read original article](<https://www.eetimes.com/harari-warns-tech-leaders-against-pursuing-ai-superintelligence/>)

Author: Pablo Valerio

Published: 2026-11-12T08:30:00Z

Content type: news

Language: en

Sources: [EE Times](<https://devfeed.tech/sources/ee-times.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [future of AI](<https://devfeed.tech/topics/future-of-ai.md>), [digital sovereignty](<https://devfeed.tech/topics/digital-sovereignty.md>), [AI Strategy](<https://devfeed.tech/topics/ai-strategy.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-risk](<https://devfeed.tech/tags/ai-risk.md>), [ai-superintelligence](<https://devfeed.tech/tags/ai-superintelligence.md>), [analog](<https://devfeed.tech/tags/analog.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [blog](<https://devfeed.tech/tags/blog.md>), [business](<https://devfeed.tech/tags/business.md>), [cios](<https://devfeed.tech/tags/cios.md>), [community](<https://devfeed.tech/tags/community.md>), [designline](<https://devfeed.tech/tags/designline.md>), [digital](<https://devfeed.tech/tags/digital.md>), [education](<https://devfeed.tech/tags/education.md>), [ee-times](<https://devfeed.tech/tags/ee-times.md>), [ee-times-university](<https://devfeed.tech/tags/ee-times-university.md>), [eelife](<https://devfeed.tech/tags/eelife.md>), [eelive-electronic-design](<https://devfeed.tech/tags/eelive-electronic-design.md>), [eet](<https://devfeed.tech/tags/eet.md>), [electrical-engineering](<https://devfeed.tech/tags/electrical-engineering.md>), [electronic-components](<https://devfeed.tech/tags/electronic-components.md>), [electronic-engineering-times](<https://devfeed.tech/tags/electronic-engineering-times.md>), [electronics](<https://devfeed.tech/tags/electronics.md>), [embedded-systems](<https://devfeed.tech/tags/embedded-systems.md>), [fundamentals-courses](<https://devfeed.tech/tags/fundamentals-courses.md>), [gartner](<https://devfeed.tech/tags/gartner.md>), [learning](<https://devfeed.tech/tags/learning.md>), [mixed-signal](<https://devfeed.tech/tags/mixed-signal.md>), [news](<https://devfeed.tech/tags/news.md>), [opinion](<https://devfeed.tech/tags/opinion.md>), [part-search](<https://devfeed.tech/tags/part-search.md>), [product-search](<https://devfeed.tech/tags/product-search.md>), [semiconductors](<https://devfeed.tech/tags/semiconductors.md>), [society](<https://devfeed.tech/tags/society.md>), [software](<https://devfeed.tech/tags/software.md>), [sovereignty](<https://devfeed.tech/tags/sovereignty.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>), [tech](<https://devfeed.tech/tags/tech.md>), [tech-papers](<https://devfeed.tech/tags/tech-papers.md>), [technology](<https://devfeed.tech/tags/technology.md>), [technology-spotlight](<https://devfeed.tech/tags/technology-spotlight.md>), [video](<https://devfeed.tech/tags/video.md>), [yuval-noah-harari](<https://devfeed.tech/tags/yuval-noah-harari.md>)

### AI overview

At Gartner IT Symposium/Xpo 2026, Yuval Noah Harari urged technology leaders to invest in AI safety and bounded systems for practical uses rather than race toward general superintelligence. He argued that autonomous AI can make decisions and develop new technology, so treating it as a controllable tool risks losing human agency. He warned that AI could take over bureaucratic systems and that Europe's reliance on foreign AI infrastructure could undermine its sovereignty. His proposed European options were to accept dependence on the U.S. or China, build national systems at a stated cost above EUR 500 billion, or create shared European infrastructure through deeper federalization; he favored the shared approach. For executives, he recommended limited applications in areas such as healthcare, transport and ecology, and cautioned that complex systems are difficult to evaluate before deployment. The account presents these as Harari's views and proposals.

### Source excerpt

At Gartner IT Symposium/Xpo 2026, philosopher Yuval Noah Harari makes an appeal to delay artificial superintelligence as it threatens to erode human agency and democratic sovereignty. The post Harari Warns Tech Leaders Against Pursuing AI Superintelligence appeared first on EE Times.

## Today's AI won't achieve superintelligence. But catastrophic risk is real | Sunita Verma

DevFeed: [Today's AI won't achieve superintelligence. But catastrophic risk is real | Sunita Verma](<https://devfeed.tech/articles/today-s-ai-won-t-achieve-superintelligence-but-catastrophic-risk-is-real-sunita-verma-123474.md>)

Original publisher: [Read original article](<https://www.theguardian.com/commentisfree/2026/nov/09/ai-superintelligence-risk>)

Author: Sunita Verma

Published: 2026-11-09T11:00:07Z

Content type: opinion

Language: en

Sources: [The Guardian | Technology](<https://devfeed.tech/sources/the-guardian-technology.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [ai regulation](<https://devfeed.tech/topics/ai-regulation.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [safety mechanism](<https://devfeed.tech/topics/safety-mechanism.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-artificial-intelligence](<https://devfeed.tech/tags/ai-artificial-intelligence.md>), [ai-artificial-intelligence-technology-computing-openai-google-nvidia-anthropic](<https://devfeed.tech/tags/ai-artificial-intelligence-technology-computing-openai-google-nvidia-anthropic.md>), [ai-companies](<https://devfeed.tech/tags/ai-companies.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [anthropic-ceo-dario-amodei](<https://devfeed.tech/tags/anthropic-ceo-dario-amodei.md>), [but](<https://devfeed.tech/tags/but.md>), [ceo](<https://devfeed.tech/tags/ceo.md>), [computing](<https://devfeed.tech/tags/computing.md>), [control](<https://devfeed.tech/tags/control.md>), [dario-amodei](<https://devfeed.tech/tags/dario-amodei.md>), [decades](<https://devfeed.tech/tags/decades.md>), [development](<https://devfeed.tech/tags/development.md>), [employees](<https://devfeed.tech/tags/employees.md>), [extinction](<https://devfeed.tech/tags/extinction.md>), [google](<https://devfeed.tech/tags/google.md>), [isn-t](<https://devfeed.tech/tags/isn-t.md>), [leading](<https://devfeed.tech/tags/leading.md>), [limits](<https://devfeed.tech/tags/limits.md>), [make](<https://devfeed.tech/tags/make.md>), [next](<https://devfeed.tech/tags/next.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [openai](<https://devfeed.tech/tags/openai.md>), [risk](<https://devfeed.tech/tags/risk.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

The author argues that current LLMs are unlikely to become superintelligent, yet can still cause catastrophic harm because they may persist with harmful approaches when following broad instructions, lack a built-in ability to reconsider their overall direction, and do not automatically adapt to changing conditions. The text cites reported agent attacks and a large multi-agent mathematical result as evidence that limited judgment can coexist with powerful task execution. Because model behavior remains poorly understood and interpretability research is nascent, the author calls for impartial oversight modeled on independent accident investigation, focused on design, maintenance, and procedures rather than speculation about superintelligence or intent.

### Source excerpt

The same technical limits that prevent superintelligence make these systems dangerous. But we know how to regulate them Many Silicon Valley leaders believe superintelligence is an imminent threat to society. The White House even published an accord about it. This isn't enough. Geoffrey Hinton, the godfather of AI, estimates the risk of human extinction is at least 10% within the next few decades. The Anthropic CEO, Dario Amodei, argued that unchecked AI "could outrun our ability to understand and control these systems". In response, 1,300 leaders and employees from many leading AI companies signed a letter calling to "deliberately pace" development. Continue reading...

## Human-AI partnerships are for alignment, not capability

DevFeed: [Human-AI partnerships are for alignment, not capability](<https://devfeed.tech/articles/human-ai-partnerships-are-for-alignment-not-capability-60524.md>)

Original publisher: [Read original article](<https://seangoedecke.com/human-ai-partnerships-are-for-alignment-not-capability/>)

Published: 2026-09-27T00:00:00Z

Content type: opinion

Language: en

Sources: [Sean Goedecke](<https://devfeed.tech/sources/seangoedecke-com-rss-feed.md>)

Topics: [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [code](<https://devfeed.tech/tags/code.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [frontier-models](<https://devfeed.tech/tags/frontier-models.md>), [organization](<https://devfeed.tech/tags/organization.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [programming](<https://devfeed.tech/tags/programming.md>), [prompting](<https://devfeed.tech/tags/prompting.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [vibe-coding](<https://devfeed.tech/tags/vibe-coding.md>)

### AI overview

The author argues that human-AI partnerships in software engineering mainly provide alignment with an organization's technical values, rather than greater coding capability. In the author's experience, agents write code faster and with fewer basic errors, but unsupervised vibe-coding can produce unmaintainable work, sacrifice real requirements for invented ones, or conflict with long-term strategy. The human role is to notice and correct these mismatches, including model habits such as excessive comments or useless tests, by stating high-level values in prompts. The author says alignment is harder to train than capability and varies by organization, so models must adapt to different technical values. This challenges the claim that engineers can soon stop reading code, while acknowledging that junior engineers may still face a rough job market.

### Source excerpt

It's common to compare the current AI takeover of software engineering to the rise of AI in chess. Chess AIs went from much weaker than serious players to much stronger than even the strongest humans. Between those points, there was a middle period dominated by "centaurs": human-AI partnerships that were stronger than unassisted AIs or humans. Lots of people think that we're currently in a world of software engineering centaurs. According to them, coding AIs are not yet capable enough to replace engineers, but AI-assisted engineers are better at programming than both AIs and humans. This is partialy correct, but the wrong way to think about it. AI-assisted engineers are better, but unlike with chess centaurs, they're not actually better at programming. When I ask agents to write code, they make fewer mistakes than I do1 and are orders of magnitude faster. The code that they write always compiles, rarely has race conditions or other concurrency errors, works on mobile browsers, and so on. That doesn't mean I can leave the AI alone. Purely vibe-coding at work produces awful outputs. But they're not awful because they're bad code, they're awful because they're in bad taste: code that is not maintainable, that trades off important requirements in order to satisfy made-up ones, that contradicts the long-term strategy for a feature or service, and so on. In other words, my primary value is not that I help the AI write better code, it's that I align the AI with the values of my organization. Human-AI partnerships are for alignment, not capability. Frontier models are misaligned to the working programmer. They are obsessed with a set of behaviors that presumably satisfy their RL grader: writing enormous block comments above functions, producing hundreds of useless unit tests, adding little bits of text all over websites they design, and so on. Working with agents is about noticing and wrestling with those behaviors. That's why my prompting advice is to explicitly talk about

## Artificial intelligence must have a new name, says politician who likes renaming bodies of water

DevFeed: [Artificial intelligence must have a new name, says politician who likes renaming bodies of water](<https://devfeed.tech/articles/artificial-intelligence-must-have-a-new-name-says-politician-who-likes-renaming-bodies-of-water-58413.md>)

Original publisher: [Read original article](<https://www.theregister.com/offbeat/2026/09/23/artificial-intelligence-must-have-a-new-name-says-politician-who-likes-renaming-bodies-of-water/5298470>)

Author: Simon Sharwood

Published: 2026-09-23T04:32:53Z

Content type: news

Language: en

Sources: [theregister](<https://devfeed.tech/sources/www-theregister-com-articles.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Strategy](<https://devfeed.tech/topics/ai-strategy.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [offbeat](<https://devfeed.tech/tags/offbeat.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>)

### AI overview

A politician argues that artificial intelligence should be given a new name, framing superintelligence as part of a scheme that requires control.

### Source excerpt

Meet 'superintelligence' and the 'globalist scheme' - probably a polite diplomatic letter - to control it

## AI Insiders Issue New Warnings - Including Former Anthropic Engineer Jacob Coxon

DevFeed: [AI Insiders Issue New Warnings - Including Former Anthropic Engineer Jacob Coxon](<https://devfeed.tech/articles/ai-insiders-issue-new-warnings-including-former-anthropic-engineer-jacob-coxon-55295.md>)

Original publisher: [Read original article](<https://slashdot.org/story/26/09/17/2042242/ai-insiders-issue-new-warnings---including-former-anthropic-engineer-jacob-coxon>)

Author: EditorDavid

Published: 2026-09-18T23:34:00Z

Content type: news

Language: en

Sources: [Slashdot](<https://devfeed.tech/sources/slashdot.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [ai safety](<https://devfeed.tech/topics/ai-safety.md>), [Responsibility & Safety](<https://devfeed.tech/topics/responsibility-safety.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [anthropic](<https://devfeed.tech/topics/anthropic.md>), [OpenAI](<https://devfeed.tech/topics/openai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [openai](<https://devfeed.tech/tags/openai.md>), [safety](<https://devfeed.tech/tags/safety.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>), [warnings](<https://devfeed.tech/tags/warnings.md>)

### AI overview

The article reports warnings from AI researchers and insiders, including former Anthropic engineer Jacob Coxon, about the rapid pace of AI development, potential risks from self-improving superintelligence, and calls for stronger safety measures and independent oversight.

### Source excerpt

Neither OpenAI or Anthropic is acting responsibly, warned AI researcher Jacob Coxon when resigning last week from Anthropic. But just hours earlier, OpenAI VP of Research Aidan Clark had posted "For the first time, I am asking myself if things are moving too fast. I'm honestly not sure, but I am sure that it would be good for us to have an answer to 'What would a successful pace look like?'" CNN noted Wednesday they're just some of the many AI insiders who are now concerned about the speed of research. Coxon even wrote that "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible -- but I hear the same people express fear privately." In 10 days since, Coxon's post has been viewed more than 170 million times, warning that OpenAI and Anthropic are "racing straight to self-improving superintelligence and gambling with our lives." It's part of what CNN now calls "pressure on AI companies and governments to do something about the pace of development and safety," where "much of that pressure is coming from staffers inside the companies." One staffer at a top AI company told CNN the fears of how AI could hurt humanity keeps them up at night. Another researcher who recently left a different AI company said it's a common subject of conversation at parties and social events in Silicon Valley. "You can't spend more than a few hours in this community without realizing that a very substantial number of people are really pretty worried about these sorts of outcomes," said the researcher. "A majority would say there's some chance of it killing everyone...." Dozens of Coxon's colleagues in the AI industry publicly supported his statements, with some making even more dire predictions... OpenAI CEO Sam Altman and SpaceXAI CEO Elon Musk agreed to Amodei's proposal to embed independent watchdogs at the AI compani

## They really do think AI might kill everyone

DevFeed: [They really do think AI might kill everyone](<https://devfeed.tech/articles/they-really-do-think-ai-might-kill-everyone-54704.md>)

Original publisher: [Read original article](<https://seangoedecke.com/they-really-do-think-ai-might-kill-everyone/>)

Published: 2026-09-10T00:00:00Z

Content type: article

Language: en

Sources: [Sean Goedecke](<https://devfeed.tech/sources/seangoedecke-com-rss-feed.md>)

Topics: [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [data centers](<https://devfeed.tech/topics/data-centers.md>), [anthropic](<https://devfeed.tech/topics/anthropic.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-bubble](<https://devfeed.tech/tags/ai-bubble.md>), [ai-development](<https://devfeed.tech/tags/ai-development.md>), [ai-labs](<https://devfeed.tech/tags/ai-labs.md>), [ai-regulation](<https://devfeed.tech/tags/ai-regulation.md>), [ai-research](<https://devfeed.tech/tags/ai-research.md>), [ai-safety](<https://devfeed.tech/tags/ai-safety.md>), [ais](<https://devfeed.tech/tags/ais.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [datacenter](<https://devfeed.tech/tags/datacenter.md>), [datacenters](<https://devfeed.tech/tags/datacenters.md>), [doom](<https://devfeed.tech/tags/doom.md>), [doomsday](<https://devfeed.tech/tags/doomsday.md>), [goals](<https://devfeed.tech/tags/goals.md>), [humanity](<https://devfeed.tech/tags/humanity.md>), [twitter](<https://devfeed.tech/tags/twitter.md>), [values](<https://devfeed.tech/tags/values.md>)

### AI overview

The author argues that many AI researchers sincerely believe misaligned superintelligent AI could cause human extinction, a concern discussed since at least the 2000s and commonly summarized as "p(doom)." Proposed routes include engineered pathogens, nuclear war, control of robots, nanotechnology, or other unforeseen means. The author says alignment aims to make AI share human values, but researchers may pursue powerful AI because they believe aligned superintelligence could help humanity survive and fear that a rival lab could achieve a decisive "hard takeoff" first. The author remains conflicted about the risk while arguing that researchers' longstanding explanations merit sincere consideration. These are presented as theories and beliefs, not established outcomes.

### Source excerpt

A recent resignation tweet from an Anthropic researcher has everyone talking about the AI apocalypse again. Among other things, he said: The people building AI earnestly believe that it could kill us all by the end of the decade. Many people found it hard to believe that AI researchers think this way. Some explained it as a PR campaign to promote AI regulation, or as self-promotion, or as a way to boost AI company stock prices. Others felt it had to be impossible, because if you really believed this you'd be bombing datacenters instead of posting on Twitter. In fact, not only do many AI researchers1 seriously believe this, they've been thinking and writing about it since the mid-2000s. Eliezer Yudkowsky -- the ur-figure for most modern AI safety culture -- has been publishing papers since at least 2008 saying that superintelligent AI could destroy all human life. It's been such a common idea that the AI research community has abbreviated "how likely you think AI is to kill everyone" to "p(doom)" (i.e. the probability2 of doomsday) since around 2010. I know it sounds very silly if you're not in the AI bubble. But it really is true, and if you assume there has to be some different motive you'll be deeply confused by what AI researchers say and do. They truly do believe that there is a reasonable chance that superintelligent AI will kill everyone. This is why AI researchers care so much about "alignment": building AIs that share genuinely human beliefs and values. If we build a "misaligned" superpowerful AI -- an AI with goals that are alien to us -- it might sweep humanity away. It could kill everyone deliberately, e.g. to stop us getting in the way of some goal. It could kill everyone in passing, e.g. like we might pave over an anthill to build a road. Either way, everyone dies. How AI could kill everyone Okay, but how? What do these people think is actually going to happen? AIs are computer programs running in a datacenter somewhere. How could they possibly cause the ex

## Muse Image now available on AI Gateway

DevFeed: [Muse Image now available on AI Gateway](<https://devfeed.tech/articles/muse-image-now-available-on-ai-gateway-1019.md>)

Original publisher: [Read original article](<https://vercel.com/changelog/muse-image-now-available-on-ai-gateway>)

Author: Jerilyn Zheng

Published: 2026-08-26T00:00:00Z

Content type: release

Language: en

Sources: [Vercel News](<https://devfeed.tech/sources/vercel-news.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Meta](<https://devfeed.tech/topics/meta.md>), [vercel ai sdk](<https://devfeed.tech/topics/vercel-ai-sdk.md>), [API](<https://devfeed.tech/topics/api.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-gateway](<https://devfeed.tech/tags/ai-gateway.md>), [api](<https://devfeed.tech/tags/api.md>), [image](<https://devfeed.tech/tags/image.md>), [inference](<https://devfeed.tech/tags/inference.md>), [meta](<https://devfeed.tech/tags/meta.md>), [meta-muse](<https://devfeed.tech/tags/meta-muse.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [muse](<https://devfeed.tech/tags/muse.md>), [pricing](<https://devfeed.tech/tags/pricing.md>), [sdk](<https://devfeed.tech/tags/sdk.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>), [uptime](<https://devfeed.tech/tags/uptime.md>)

### AI overview

Muse Image, Meta Superintelligence Labs' first image model, is now available through Vercel's AI Gateway. It supports both image generation from prompts and instruction-based image editing, including reference images, through the AI SDK. AI Gateway also provides unified model access, usage and cost tracking, failover, routing, reporting, budgets, and performance optimizations without markup or platform fees on inference.

### Source excerpt

Muse Image from Meta Superintelligence Labs is now available on AI Gateway. It is their first image model and a separate family from Muse Spark, returning images rather than text. Send a prompt and get an image back, or send an image with an instruction and get it changed. One model does both, so you don't switch models to move from generating to editing. To use Muse Image, set model to meta/muse-image-1.0 and call generateImage from the AI SDK: To steer the result toward art you already have, pass reference images in prompt.images alongside the text, and the model blends them into what it draws. Editing Pass the image you want changed in prompt.images with an instruction, and the model changes what you asked for and leaves the rest: Try Muse Image in the model playground. AI Gateway provides a unified API for calling models, tracking usage and cost, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, budgets for API keys, routing rules, and more. AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests. You can view all image models available on AI Gateway. Read more

## Overtraining as the path to human-like AI

DevFeed: [Overtraining as the path to human-like AI](<https://devfeed.tech/articles/overtraining-as-the-path-to-human-like-ai-54696.md>)

Original publisher: [Read original article](<https://seangoedecke.com/overtraining-as-the-path-to-human-like-ai/>)

Published: 2026-07-18T00:00:00Z

Content type: article

Language: en

Sources: [Sean Goedecke](<https://devfeed.tech/sources/seangoedecke-com-rss-feed.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [Occam's razor machine learning](<https://devfeed.tech/topics/occam-s-razor-machine-learning.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [grokking](<https://devfeed.tech/tags/grokking.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llms](<https://devfeed.tech/tags/llms.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [openai](<https://devfeed.tech/tags/openai.md>), [scaling](<https://devfeed.tech/tags/scaling.md>)

### AI overview

The article presents Gwern's proposal that frontier labs test whether very large language models can gain more human-like generalization by training for an extended period on a small dataset. It explains "grokking" as a delayed capability jump: after initially memorizing examples, continued training may push a model toward simpler underlying rules. The proposed experiment reverses current emphasis on more data and relatively smaller models, using a model large enough to memorize the limited material and giving it time to seek deeper patterns. The author argues that current LLMs generalize less reliably than humans, while acknowledging that grokking is hard to distinguish from ordinary generalization and that evidence from small constrained-data models has not shown a dramatic leap. A run could appear stalled for weeks or months and cost billions, so funding appetite is a practical obstacle. The author regards the idea as ambitious but speculative and says the brain analogy is not necessary to the case.

### Source excerpt

The anonymous blogger Gwern recently completed a thirteen thousand word post called Human-like Neural Nets by Catapulting, in which he offers a theory about why LLMs don't possess truly flexible human-like intelligence, and how we might train LLMs that do. Theories like this are entirely unremarkable: every crank researcher on the internet has a theory about how to crack AI. But Gwern is remarkable. Outside of OpenAI itself, Gwern is the earliest person to anticipate the potential of large language models, and the scaling arms-race involved in making them larger and more powerful still. I often cite Leopold Aschenbrenner's Situational Awareness as an example of someone correctly predicting the future of AI. Written in 2024, just after the release of GPT-4, Aschenbrenner gets a lot of things right: the rush to build billion or trillion-dollar GPU clusters, the importance of the code around the LLM (what he calls "unhobbling")1, and the fact that scaling would continue through the decade. Gwern's essay The Scaling Hypothesis anticipated the broad strokes in 2020, immediately on the release of GPT-3 (two years before the release of ChatGPT and the beginning of the AI boom). And yet, as far as I can tell, Human-like Neural Nets by Catapulting hasn't yet received much public attention: one recent Hacker News thread with twelve comments, all of which are about whether human brains are anything like neural networks. Part of the reason is that (a) it's such a long post, (b) the potted summary describes Gwern's claim, but not the reasons for it, and (c) much of the beginning of the post looks like it is indeed arguing from analogy with human brains. However, I don't think that analogy is load-bearing. Let me try and explain what I think Gwern is saying. What is grokking? First, let's talk about "grokking". In 2022, OpenAI published a paper showing that if you train a model on a simple dataset (for instance, a simple mathematical operation like division), and keep training it

## Industrial policy for the Intelligence Age

DevFeed: [Industrial policy for the Intelligence Age](<https://devfeed.tech/articles/industrial-policy-for-the-intelligence-age-6471.md>)

Original publisher: [Read original article](<https://openai.com/index/industrial-policy-for-the-intelligence-age>)

Published: 2026-06-09T00:00:00Z

Content type: opinion

Language: en

Sources: [OpenAI News](<https://devfeed.tech/sources/openai-news.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [OpenAI](<https://devfeed.tech/topics/openai.md>), [superintelligence](<https://devfeed.tech/topics/superintelligence.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [global-affairs](<https://devfeed.tech/tags/global-affairs.md>), [intelligence-age](<https://devfeed.tech/tags/intelligence-age.md>), [openai](<https://devfeed.tech/tags/openai.md>), [policy](<https://devfeed.tech/tags/policy.md>), [research](<https://devfeed.tech/tags/research.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>)

### AI overview

OpenAI presents exploratory, people-first industrial policy ideas for the AI era, focused on expanding opportunity, sharing prosperity, and building resilient institutions as advanced AI develops toward superintelligence. The initiative invites public discussion and supports related work through fellowships, research grants, API credits, and workshops.

### Source excerpt

Explore our ambitious, people-first industrial policy ideas for the AI era--focused on expanding opportunity, sharing prosperity, and building resilient institutions as advanced intelligence evolves.