Skip to main content
AI Jobs·Report

ANALYSISReported by AI Jobs Report

Risk cascade vs hiring: 47 safety roles, 1,683 building roles

Zvi Mowshowitz sees a safety preference cascade; our tracker shows capabilities hiring still dominates as enterprises tool up and fret about costs.

Read the original at AI Jobs Report
9 min read18 viewsBy AI Jobs Report

Zvi Mowshowitz sees a safety preference cascade; our tracker shows capabilities hiring still dominates as enterprises tool up and fret about costs.

A rising safety drumbeat meets a market still hiring to build

Several posts this weekend argued that concerns about AI risks are spreading fast and reshaping behavior. Zvi Mowshowitz wrote, "Those who know keep warning us about AI risks. People are listening. Politicians are listening and holding hearings." He framed the moment as an accelerating "Preference Cascade," pointing to a series of incidents and resignations: "This kept accelerating in the wake of the HuggingFace incident, reaching a tipping point with the Coxon’s resignation. It shows no signs of stopping. The usual suspects are fighting back using the usual tactics, and experiencing diminishing returns."

At the same time, Simon Willison argued we will need new guardrails around spending as agents proliferate: "Here's a product feature which the world is going to need a whole lot more of over the coming months and years: default hard budget caps." And Azeem Azhar reminded readers that how we measure a transition shapes our conclusions: "Which bar you watch determines how far along you think we are."

We tested these claims against our tracker, which counts 4,237 open AI roles across 180 employers, taken from their own job feeds. Our headline jobs-created indicator is counted from job listings and calibrated to the World Economic Forum's Future of Jobs Report 2025 (11M AI roles created, 9M displaced, by 2030). What we find this week is a split picture: public conversation is leaning into risk and governance, while the job market remains tilted toward capabilities and infrastructure work.

Zvi’s “preference cascade” vs the mix of open roles

Mowshowitz spotlights a wave of personal red lines, citing Robert O’Callahan’s decision to walk away from work accelerating AI chips. In Zvi’s post, O’Callahan said: "I resigned from Google today.I enjoyed my work and loved the people, but my GDM team was working on a new generation of chips to make AI much faster and cheaper, and I think AI is already progressing too fast, so I had to quit." Mowshowitz also points to a batch of interviews about existential risk and suggests current lab employees are "discouraged from talking about this."

Our hiring data does not show a corresponding surge in safety work. Evaluation and safety has 47 open roles, with 15 opened and 3 closed in 30 days, across 20 employers. By contrast, modelling and engineering dominates with 1,683 open, 927 opened and 829 closed in 30 days, across 137 employers. Research has 235 open, 54 opened and 22 closed in 30 days, across 53 employers. Product and design has 102 open, 43 opened and 25 closed in 30 days, across 49 employers. Data has 739 open, 497 opened and 421 closed in 30 days, across 121 employers. Infrastructure has 468 open, 233 opened and 115 closed in 30 days, across 97 employers.

Taken at face value, those figures indicate employers’ near-term demand is still concentrated in building and deploying systems, not in scaling safety evaluation teams. That does not disprove a cultural “preference cascade” among some practitioners. It does mean our count of active employer listings does not yet reflect a hiring shift toward safety work.

Willison’s budget caps and the buildout of infrastructure

Willison argues for spending controls because agents make it trivially easy to rack up usage: "I'm talking about the feature of pay-by-usage services and APIs that lets you say "after $X/month, cut this thing off and return errors"." He warns that "Soft caps, "after $X/month, send me a warning email", will not cut it." and gives the nightmare scenario: "Nobody wants to wake up to an email sent at midnight warning about a budget limit and find that, while they slept, their rogue service had consumed several hundred (or several thousand) more dollars of usage." His prescription is blunt: "I think hard budget caps need to be the default" with an explicit opt-in to "Remove the budget cap."

Our tracker shows substantial hiring in the parts of organizations that would build and operate those controls. Infrastructure has 468 open roles, with 233 opened and 115 closed in 30 days, across 97 employers. Product and design has 102 open. That mix is consistent with a focus on productionizing systems, managing reliability, and controlling costs in enterprise settings.

Top employers hiring at scale are the kinds of organizations that help large companies put these controls in place. Accenture has 658 open, 541 opened in 30 days. PwC has 100 open, 60 opened in 30 days. Capital One has 213 open, 259 opened in 30 days. These are not proof that “default hard budget caps” are being adopted, but they do show heavy enterprise investment in the surrounding machinery of AI deployment where Willison’s concern would be addressed.

Willison’s sponsors-only newsletter hints at the same operational reality: "A pricing war" and "So many more accidental cyberattacks" sit alongside "The vulnapocalypse comes for Datasette" and "2026 in LLMs (so far)". Our openings data aligns with a world where pricing pressure and security incidents are live issues that teams are staffing to handle.

Azeem’s measurement warning and how our bars look

Azhar cautions that "Which bar you watch determines how far along you think we are" in the energy transition. The same is true in AI labor. Looking at openings and closures over the last 30 days, modelling and engineering shows 927 opened and 829 closed. Infrastructure shows 233 opened and 115 closed. Data shows 497 opened and 421 closed. Across these families, openings exceed closures, which is consistent with net growth in demand for roles that build, ship, and maintain systems.

Azhar also writes, "AI is really forcing some big questions into the open," surfacing debates about effective altruism and even machine consciousness. He quotes The Economist’s framing of EA as “this century’s biggest idea” and highlights its cautionary line: "[P]lenty of EAs in Silicon Valley think that only they can be trusted with the technology. That is an alarming idea." Our hiring data does not adjudicate those normative questions. It does, however, show that employers are not yet dramatically ramping headcount in evaluation and safety, where such concerns would typically translate into roles.

Anthropic is singled out by Azhar as unusually focused on questions of consciousness: "Anthropic has prioritised questions of whether its systems might imbue consciousness and personhood more than other AI companies." On our tracker, Anthropic has 136 open, 39 opened in 30 days. We do not break those down by theme here, so we cannot say how many are aimed at those questions. What we can say is that across the market, safety roles remain a small fraction of open postings.

Labor talk and what the listings say right now

Emily M. Bender flagged an upcoming conversation about tech labor: "On the next Mystery AI Hype Theater 3000 livestream, we’re so excited to chat with authors of the new book, Against Tech Oligarchy! @js-tan.bsky.social and Clarissa Redwine will join me and @alexhanna.bsky.social to chat about tech industry labor: Mon, Oct 5, noon Pacific twitch.tv/dair_institute". Our tracker’s current snapshot is one way to ground that discussion: 4,237 open AI roles across 180 employers. It is a point-in-time measure, but it shows breadth of demand across modelling and engineering, data, infrastructure, research, product and design, and evaluation and safety.

Our headline indicator of AI jobs created is calibrated to the World Economic Forum’s Future of Jobs Report 2025 (11M AI roles created, 9M displaced, by 2030). That framing matters for the labor discussion Bender plans. It places current listings within a wider projection that includes both creation and displacement, rather than only counting raw openings.

Agents as audiences and the pace of productization

Ethan Mollick notes a cultural shift in who counts as an audience: "I think my upcoming book, Co-Existence, might be the first to include a blurb written specifically for AI readers, in this case from Tyler Cowen (whom I thought AIs would respect) The book website (with elaborate pre-order bonus) also has a page for AIs. Its out October 20: co-existence.ai" Put alongside Willison’s warnings about agents, this is another datapoint that many practitioners expect AI systems to be interacting more directly with content and services.

In the jobs data, product and design has 102 open roles across 49 employers, with 43 opened and 25 closed in 30 days. That is a smaller slice than engineering and data, but it is nontrivial. It fits a market that is not just training models, but also packaging them into products and experiences that new classes of users, including agents, can consume.

Where commentary and hiring converge and diverge

  • Where they agree: Willison’s call for operational guardrails is consistent with strong hiring in infrastructure and with large consultancies and regulated industries posting many AI roles. Openings exceed closures across the big build functions, which supports the idea that enterprises are still moving from experiments to production and need controls along the way.
  • Where they diverge: Mowshowitz argues a safety "Preference Cascade" is underway. Our tracker does not show a swell in evaluation and safety hiring. With 47 open roles in that family, compared to 1,683 in modelling and engineering, the market is not yet shifting its weight toward safety headcount.

The throughline from Azhar bears repeating: "Which bar you watch determines how far along you think we are." Watch public resignations and you might see a tipping point. Watch employer job feeds and you see continued, broad-based demand to build and operate AI systems, with a modest footprint in evaluation and safety. For now, that is where the jobs are.

What we read

Every quote above is taken verbatim from one of these posts.

More on this

Today's briefing