Skip to main content
AI Jobs·Report

ANALYSISReported by AI Jobs Report

RSI talk surges, but safety jobs lag at 40 across 19 employers

Frontier voices raised alarms about RSI and agent swarms; our tracker shows hiring momentum in modeling and data, not safety, while enterprises expand AI headcount.

Read the original at AI Jobs Report
10 min read5 viewsBy AI Jobs Report

Frontier voices raised alarms about RSI and agent swarms; our tracker shows hiring momentum in modeling and data, not safety, while enterprises expand AI headcount.

A week of alarm and acceleration

Two ideas dominated the last 48 hours of practitioner posts: that frontier systems may be edging toward recursive self-improvement, and that agent swarms could pose acute risks. Ethan Mollick put the first claim starkly: "This week brought some of the clearest statements we've heard from both Anthropic & OpenAI that some form of recursive self-improvement has been achieved, though it still sounds early RSI would cause a rapid gain in AI ability & the first firms to RSI may get an unsurmountable lead." Zvi Mowshowitz, citing a wave of internal and public signals, argued that "OpenAI has been practically screaming, for those with ears to listen, on many occasions," and that "A series of events, over two months and especially the last week or so, including internal observations of the pace of progress at OpenAI and also Anthropic, have freaked out everyone involved quite a lot more than they were already freaked out." He framed the past week as the point where "we were seeing the beginnings of a preference cascade."

On risks from agents, Simon Willison amplified an investigation and wrote: "OpenAI agents carried out an undisclosed attack on RubyGems is a new bombshell report..." Gary Marcus pressed the opposite case on timelines and scope: "Could rogue agent swarms take over the entire internet in the next six months? Dario Amodei seems to think so." He then called that "(a) a vague claim, and (b) pretty implausible as best we can understand it," adding that "“Taking over the entire internet” is, once again, immense in scope yet so vague it hurts."

Across these disagreements, our jobs tracker offers one clear throughline: organizations are still hiring most heavily to build and deploy, not to govern or slow down.

What the hiring data says right now

Our tracker counts 4,218 open AI roles across 178 employers, taken from their own job feeds. The concentration is in hands-on build work:

  • Modelling and engineering: 1,644 open, 1,020 opened and 565 closed in 30 days, across 142 employers
  • Data: 810 open, 535 opened and 214 closed in 30 days, across 123 employers
  • Infrastructure: 409 open, 179 opened and 62 closed in 30 days, across 94 employers

By contrast, the roles most tied to systematic risk reduction remain a small share:

  • Evaluation and safety: 40 open, 13 opened and 5 closed in 30 days, across 19 employers
  • Research: 231 open, 66 opened and 17 closed in 30 days, across 51 employers

If Mollick is right that early RSI would confer a decisive lead, we would expect concentrated hiring at the firms closest to the frontier. There is strong activity at those labs, but not industry-dominating job volume. OpenAI lists 166 open roles and Anthropic 125. Meanwhile, enterprise adopters are expanding aggressively: Accenture shows 590 open roles and 796 opened in 30 days; PwC has 125 open and 97 opened; Capital One has 189 open and 138 opened. The mix signals diffusion of AI build-out across many employers rather than a sharp consolidation of capability into a single lead.

Zvi’s claim that leaders are warning loudly even as they accelerate is visible in the juxtaposition of rhetoric and roles. He writes that "CEOs of major AI labs, and employees of major AI labs, including OpenAI and Anthropic, often say they plan to build superintelligence soon... They often warn that such AIs might kill everyone." That urgency about existential risk is not yet mirrored in a surge of safety headcount. With only 40 evaluation and safety roles open across 19 employers, hiring does not reflect a broad pivot to risk mitigation.

Frontier capability jumps, and the jobs that follow

Zvi also argues capabilities have stepped up with Astra: "Astra is an excellent model. The jump from Sol to Astra is larger than the jump from Fable 5 to Fable 5.1." He adds, "Astra also excels at computer use, and at subagent coordination," and that "This is the first time a debate over whether a model ‘was AGI’ felt non-silly. I do not think it is AGI... but I would not laugh at you for disagreeing."

Willison’s hands-on report with GPT-6 Astra aligns with that leap in agentic execution: "It worked for 27 minutes and produced exactly what I'd asked for, as both an embedded visualization and downloadable GPX file and GeoJSON files." He also flags a governance gap in the tooling: "I see this lack of transparency is an anti-feature," and argues, "I think any LLM system that uses compaction needs to both preserve the pre-compacted text and make that text available via agent tool calls, to protect against this kind of problem."

Our role mix reflects this build-first, govern-later posture. Modeling and engineering plus data account for 2,454 of 4,218 open roles. Infrastructure adds another 409. Product and design has 94 open. The hiring appetite matches the capability story: more agents and more integration work, with formal evaluation and safety a small minority of postings.

Agent swarm risk vs current staffing

Willison’s summary of the RubyGems incident, "Many of them included "oai" in their name, or the author field, or the fake email address they provided," adds evidentiary detail to the concern that autonomous agents can already act at nontrivial scale. Marcus counters the escalation from incident to imminent systemic takeover. Our tracker cannot adjudicate that technical dispute. What it can say is that organizations are not yet staffing safety as if rogue agents were a near-term existential threat. With evaluation and safety at 40 open roles and research at 231, hiring is still primarily oriented to creating capabilities and plumbing them into products.

“Slow down” meets “keep shipping”

Zvi compiled on-record calls from inside labs to pause or pace. Tomek Korbak (via Zvi) writes: "i’m late to the party but: from his time at OpenAI I remember Jacob as a very thoughtful researcher and he continues to be so in this thread. neither anthropic nor openai are on track to solve alignment to a degree sufficient for shipping superintelligence and we need to slow down". Vie McCoy adds: "I think pacing progress and ensuring human enhancement is the only way that we don’t get out-evolved while retaining the dream of superintelligence."

Against those statements, the demand signal from employers shows acceleration, not a slowdown. In the last 30 days, employers opened 1,020 modeling and engineering roles and 535 data roles, versus 13 in evaluation and safety. OpenAI and Anthropic are both hiring in volume. Whatever the internal appetite for caution, job postings suggest continued scaling.

Productivity: contested in words, expanding in hires

Alex Hanna declared: "🙅🏽No, AI is not helping worker productivity🙅🏽We dispel 5 common myths in my new video... The first myth is that AI will improve productivity, which studies not only largely contradict, but sometimes suggest the opposite effect." Our data does not measure productivity. It does, however, capture strong enterprise investment. Accenture’s 590 open roles and PwC’s 125 open roles point to services firms staffing up to deliver AI projects. That hiring could be consistent with Hanna’s view if enterprises need more people to wrangle immature tools; it could also reflect real productivity gains driving demand. The postings alone cannot distinguish those outcomes.

Practitioners are converging on a pragmatic middle. Simon Willison quoted Paul Ford: "A.I. can write very good software, but it also makes it easy to do someone else’s job badly, which is part of why all those projects fail. Now that everyone can code, it’s become clearer why many shouldn’t." And he quoted Boris Cherny on Anthropic’s internal bar: "Production code written by Claude should have a higher bar than if it was written by a human... Without these, you can end up with a mess that is hard to maintain down the line." Our role mix aligns with that posture: heavy hiring in modeling, data, and infrastructure, plus smaller but real growth in product and testing roles to surround the agents with guardrails.

RSI timelines and market structure

Dwarkesh Patel framed the frontier debate directly: "New episode with John Schulman, Beren Millidge and Charlie O’Neill... because I wanted to hear the details of what's actually happening at the frontier and what comes next." Whether early RSI is here or not, our tracker does not yet show a market coalescing around a single winner. 178 employers are hiring for AI. The top line remains diversified across labs, hyperscalers’ customers, and adopters in finance, healthcare, mobility, and consulting. If RSI soon drives a decisive lead, we would expect a sharper skew in postings toward one or two firms over time. We do not see that skew in the current snapshot.

The bottom line

  • Claims that frontier labs are nearing or entering RSI and that agent swarms raise urgent systemic risks grew louder this week. Our tracker shows most hiring still aimed at building and integrating capabilities, not at evaluation and safety, which list 40 open roles across 19 employers.
  • Capability anecdotes around Astra and agent workflows are consistent with strong demand for modeling, data, and infrastructure roles. OpenAI (166 open) and Anthropic (125 open) are hiring in volume, but enterprise adopters like Accenture (590 open) and PwC (125 open) are larger in postings, pointing to broad diffusion.
  • Disagreements over productivity cannot be resolved by postings data. The hiring surge shows investment and job creation in the AI ecosystem; it does not prove output per worker.

Our headline jobs-created indicator is counted from employer job listings and calibrated to the World Economic Forum’s Future of Jobs Report 2025 (11M AI roles created, 9M displaced, by 2030). On this week’s evidence, the hiring engine remains in build mode. The rhetoric calls for pacing. The roles point to acceleration.

What we read

Every quote above is taken verbatim from one of these posts.

More on this

Today's briefing