Azeem Azhar, David Ha, Simon Willison and Noah Smith flag capability leaps, supervision bottlenecks and tooling shifts; our tracker shows hiring skews to engineering while evaluation and safety remain thin.
A shared pattern: capability is racing ahead, evaluation lags, tooling hardens
Across very different posts this weekend, four practitioners circled the same tension: AI systems are getting more capable, our ways to judge them are straining, and developer tooling is quietly locking in new workflows. Azeem Azhar framed the acceleration in mathematical discovery. David Ha asked how to keep improving models once humans cannot reliably score them. Simon Willison highlighted small but telling shifts in software practice and automation. Noah Smith placed AI inside a culture war frame. Our hiring tracker puts a number on where employers are actually investing: overwhelmingly in building and plumbing, not in evaluation.
Capability acceleration meets the supervision bottleneck
Azeem Azhar set the tone with a claim about the widening gap between discovery and how we absorb it. He wrote: "Math 2.0 When OpenAI released its repo with hundreds of mathematical manuscripts and supporting proofs, I asked what it might mean for the future of human knowledge:" He then sharpened the point with: "Problems that have stumped the best human minds for decades fell to one afternoon of compute. […] Back in 1794, Condorcet bet that knowledge and our capacity to spread it and share it would evolve together. He has been right for nearly three hundred years. Now they might be coming apart, but not because of the intrinsic nature of these discoveries."
David Ha, announcing Sakana AI’s new work, put a name to the operational choke point: "New Sakana AI paper: Recursive Self-Improvement through Multi-Agent Self-Supervision (MASS)." He asked, "How do we keep improving AI when tasks become too complex for humans to reliably evaluate? The model eventually hits a supervision bottleneck."
Our tracker sees only a modest employer response to this bottleneck so far. Evaluation and safety roles are a small slice of open demand at 51 open, with 18 opened and 3 closed in 30 days, across 22 employers. By contrast, the biggest share of hiring is still squarely about building: modeling and engineering has 1,580 open roles, with 787 opened and 772 closed in 30 days, across 135 employers. Research sits at 232 open, with 44 opened and 25 closed in 30 days, across 55 employers.
That mismatch matters. If evaluation really is becoming the limiter on progress, we would expect employers to scale up hiring for evaluation and safety at something closer to build-level intensity. Our figures do not show that yet. They do show that some demand exists and is opening, but the emphasis is still on engineering muscle. On the evidence of postings, companies are acting as if more builders, and the data and infrastructure that support them, are the near-term priority.
Tooling signals: workflows are changing in quiet ways
Simon Willison’s two posts read as dispatches from the tools frontier. On the cultural side of software practice, he noted "Dwarf Fortress uses version control now" and added, "I remember hearing a while ago that Dwarf Fortress didn't use version control." He called the long version-control-less run "That's art." Quoting creator Tarn Adams over the years, the arc ends with the concession: "I've got Discord channels to check and version control stuff to use now, so the process is definitely a little more involved" — a small example of tool norms finally absorbing even the most idiosyncratic projects.
On the automation side, Willison leaned on an AI assistant to monitor a key dependency for him: "Bit of a niche link, but I've been waiting for this for a couple of days - I even told ChatGPT to check for it: Clone https://github.com/actions/python-versions and git pull once an hour until they add the stable 3.15 - then tell me about it" When the update landed, the instruction pipeline for teams became: "Now that this has landed, you can add "3.15" to a GitHub Actions testing matrix to run tests against the new Python 3.15.0 release."
These are practical signals that developer workflows are being refactored around automation and standardized pipelines. Our hiring data fits that story. Infrastructure roles stand at 460 open, with 225 opened and 124 closed in 30 days, across 96 employers. Data roles are at 745 open, with 448 opened and 403 closed in 30 days, across 119 employers. Those categories exist to make build systems reproducible, observable and scalable — the same underlying concerns that make continuous integration updates and version control discipline noteworthy in Willison’s posts.
Politics treats AI as a symbol; employers hire for execution
Noah Smith’s political analysis treats AI less as a technology vector and more as a prop in a broader polarization strategy. He writes of the president’s list of distractions: "These include demanding that artificial intelligence be renamed “artificial superintelligence”, bellowing that he should have won the Nobel Peace Prize, and conducting an execution on live TV." Whatever the political effects, our tracker does not register policy theater in the composition of open roles. Employers are hiring across 181 organizations, and their distribution is prosaic: modelers and engineers, data specialists, infrastructure, research, product and design, and a small pool in evaluation and safety.
The top employers reinforce the execution-first picture rather than a focus on symbolic fights. Accenture has 600 open, with 389 opened in 30 days. Capital One has 201 open, with 256 opened in 30 days. OpenAI has 174 open, with 40 opened in 30 days. Amgen has 144 open, with 36 opened in 30 days. Anthropic has 137 open, with 38 opened in 30 days. Waymo has 114 open, with 39 opened in 30 days. PwC has 101 open, with 40 opened in 30 days. Databricks has 84 open, with 20 opened in 30 days. These are postings from employers’ own job feeds — the places where budget meets plan.
Where commentary and hiring align, and where they do not
-
Alignment: Willison’s tooling notes line up with concrete demand in infrastructure and data roles. Employers are investing in the plumbing that makes continual updates, testing matrices, and version control hygiene routine.
-
Partial alignment: Ha’s "supervision bottleneck" is visible in postings only at a small scale. There are 51 open roles in evaluation and safety. The category is opening roles, but the absolute level is far below engineering and data. If the bottleneck were already the dominant constraint for many teams, we would expect a larger shift in requisitions.
-
Unresolved by our data: Azhar’s claim that discovery may decouple from our capacity to share it is a societal diagnosis. Our tracker cannot validate or refute that directly. What we can say is adoption in employers is broad — 4,138 open AI roles across 181 employers — and most of those roles are about building and operating systems rather than curating knowledge flows.
-
Orthogonal: Smith’s point about AI as a symbolic wedge in politics sits outside what our hiring data can confirm. We do not see a hiring response to that discourse in the mix of roles or the identity of top hirers. The postings reflect executional needs, not symbolic positioning.
The hiring reality right now
Our tracker counts 4,138 open AI roles across 181 employers, taken from their own job feeds. By role family today:
- Modeling and engineering: 1,580 open, 787 opened and 772 closed in 30 days, across 135 employers.
- Data: 745 open, 448 opened and 403 closed in 30 days, across 119 employers.
- Infrastructure: 460 open, 225 opened and 124 closed in 30 days, across 96 employers.
- Research: 232 open, 44 opened and 25 closed in 30 days, across 55 employers.
- Product and design: 96 open, 44 opened and 28 closed in 30 days, across 44 employers.
- Evaluation and safety: 51 open, 18 opened and 3 closed in 30 days, across 22 employers.
The picture is clear: employers are flooding headcount into builders and the support structures that make them productive. Evaluation and safety is present but small. If the supervision bottleneck sharpens, we will expect its hiring footprint to grow. For now, the market is behaving as if better engineering, better data, and better infrastructure will buy more progress than a step change in evaluation.
What to watch next
-
Do evaluation and safety postings expand beyond the current 22 employers carrying 51 open roles? Sustained increases there would be the first evidence that Ha’s bottleneck is reshaping team structures.
-
Does research hiring climb relative to engineering and data? A rise from the current 232 open, across 55 employers, toward the engineering share would signal more emphasis on new methods over application work.
-
Do infrastructure openings keep pace with developer workflow shifts? At 460 open across 96 employers, this category tracks Willison’s world. If automation of monitoring and CI spreads, we would expect this count to remain energetic.
-
Do political salvos about AI policy show up in postings? Our numbers to date show no compositional effect. We will keep watching the mix by family and the roster of top employers for any sign of a shift.
The practitioners this weekend described a world of accelerating capability, straining evaluators, and quietly standardizing tooling. The jobs market mostly agrees on the tools and the builders. It has not yet reweighted toward the evaluators who, by their account, will be asked to judge what humans can no longer reliably score.
What we read
Every quote above is taken verbatim from one of these posts.
- Noah Smith: Why Donald Trump won in 2024
- Azeem Azhar: 🔮 AI & Math 2.0, Pakistan’s solar hedge & who controls computing++
- David Ha: New Sakana AI paper: Recursive Self-Improvement through Multi-Agent Se
- Simon Willison: Dwarf Fortress uses version control now, Python 3.15.0 added to actions/python-versions