Recursive CEO: “Probability of AI Wiping Out Humanity is Zero” | Fortune AI Weekly

Fortune
35 followers
Oct 13, 2026
The P-Doom probability
of existentially wiping out humanity is zero from AI.
Hello and welcome to Fortune AI Weekly.
On this week's episode, we'll be talking to Richard Socher,
the CEO and co-founder of Recursive Superintelligence.
They often assume these near-magical capabilities for the attackers,
but somehow not near-magical abilities for the defenders.
In addition, we'll be discussing OpenAI's decision
to release the results to 370 longstanding mathematical problems.
The mathematicians are pretty unhappy about that.
They feel like their field should not just be used
to sort of score cheap bragging rights
and cheap marketing points for the AI labs.
And on the Brain Food segment,
we'll be talking about the new movie, The Social Reckoning.
Fortune AI Weekly is brought to you by ServiceNow.
ServiceNow's AI specialists do the work, start to finish,
so people can do the parts of the job they're best at.
To learn more, visit servicenow.com.
And now for our interview with Richard.
Welcome to Fortune AI Weekly,
where we're pleased to be joined today by Richard Socher,
who is the co-founder and chief executive officer at Recursive,
a company building self-improving
AI to automate scientific discovery.
He's also the author of a new book,
The Eureka Machine, which we're going to be chatting
to him a bit about today.
He's also the founder and chief executive officer at You.com,
a leading web search infrastructure
provider for AI, and co-founder and managing director of AIX Ventures.
Richard was previously the chief scientist at Salesforce,
chief AI scientist at Salesforce.
Richard, thank you so much for joining us today.
Thanks for having me.
Excited to be here.
So I want to start out, we're going to talk
a little bit about Recursive Superintelligence,
the company that you have co-founded and lead.
For those who haven't come across Recursive yet,
what is the company and what is the problem
you're really trying to solve?
We are building Recursive self-improving superintelligence.
And so trying to take that last step in abstraction and allow AI
to essentially ideate, implement, and validate ideas around AI itself.
We hope that that will make AI so much better,
so much more quickly, that we can then also point that research
capability towards all other kinds of sciences in the future.
Got it. And how did you come to found the company?
We've chatted a number of times over the years.
We chatted when you were at You.com,
and then you decided to launch almost in parallel, Recursive.
How did you come to found the company?
Yeah, I finished a book last year, The Eureka Machine,
and really couldn't let go of those ideas,
and I really wanted to build this kind of Eureka machine,
this ultimate invention-generating machine and invention itself.
And so I was very fortunate to know many of the most relevant,
important researchers,
developers and founders and executives in that space.
And so we all coalesced.
And it's really interesting to hear how many of us in the founding team
with eight co-founders, many of us ultimately came
to the same conclusion that this is the necessary next step.
This was a little bit less obvious
when we first talked about it early last year.
Now, of course, you hear a lot
of other folks talking about recursive self-improvement as well.
But you can come from this from a robotics perspective,
in the case of Josh Tobin, who used to work at OpenAI,
or an open-endedness perspective, which Tim Rocktäschel
and Jeff Clune have come to when they worked on world models
and similar things, rainbow teaming to make LLM safer.
So all of us came to the same conclusion,
but from different perspectives.
And you've really been able to assemble quite
a dream team here in this company, I have to say.
I mean, you just named some of them,
but it's an impressive list of co-founders.
Yeah, I feel really fortunate.
Yeah, Tsai Ming-Xiong, Yuan Dongqian, Tim Shi,
Alexey Dosovitskiy, who invented the Vision Transformer,
one of the most cited vision papers ever.
So it's just an incredible team indeed.
I want to take us back a little bit onto
the recursive self-improvement.
And you know how you say that it can have many definitions.
What do you mean by it and how are you defining that?
How much of this is taking the human out of the loop?
It's a great question because a lot
of people use that term very loosely.
We are very clear that there's auto
research where you take one AI system
and you make something else better.
That something else can be the Shopify code base,
it could be a small model, whatever it is.
That is not recursive self-improvement.
It's also not recursive self-improvement
if you just have your AI researchers
use an AI coding tool like Claude and Codex
to build and implement the ideas for you.
We consider those somewhat weak forms of self-improvement.
The true self-improvement loop
is if you have an open-ended system
that can combine interestingly different ideas
in an evolutionary sort of process
to truly ideate, implement, and validate ideas.
So all three steps are done by an AI with the goals, of course,
that humans and people set for it to, you know, accomplish tasks
that are relevant for humanity and for AI research, according to us.
Yeah, it's interesting because recursive self-improvement
has very much been in the spotlight recently.
A lot of engineers from some of the leading AI labs are kind of openly
talking about the risks that they see from recursive self-improvement.
And they say, you know, it's one of the
developments that worries them most.
i'm wondering what you make of their concerns
on this um you know i think if you're if your ai
currently uh cannot be controlled well enough
that it keeps committing felonies then maybe
that organization should slow down
but that doesn't mean everyone else has been uh you know
doing that uh and has to also slow down
so i think it does make sense to be responsible here
um i do think there are a lot of unfounded fears
and there are a lot of sort of sci-fi inspired uh
people and and uh ideas out there that assume
that once you have recursive self-improvement
then like you know an hour later it'll be like
this the super intelligence that will be better
at anything humanity could ever do reality is such
that you still have some compute constraints
right if you want that ASI to work on bigger
and better models that are really really epic it will
need still a lot of compute um and and uh you know there
are also no connections yet to physical
reality so we're not giving it access to robots
and the most sophisticated uh as yet unknown uh
like bio labs and things like that so i think we have
to stay realistic um it it's a little bit
unfortunate that some politicians kind
of are jumping on this as uh like a potential to um
like sort of rally people against this like very hypothetical threat
and i hope that um we don't
as humanity run into another situation where you know
the fear of overpopulation the fear of uh you
know nuclear energy the fear of gmo's all
these different fears uh like will make us kind of
off-ramp from progress um and abundant energy
and and uh food supplies uh and so on um for people
So there are some downsides sometimes
to these sci-fi alarmist scenarios.
Interesting. And, you know, one of those scenarios people talk about
is the idea that if you achieve RSI, you could
then get what people sometimes call an intelligence explosion,
that you get this very rapid cycle of improvement.
And that's, you know, one of these loss of control scenarios
that you have AI models improving themselves
so rapidly that humans can't possibly check what's going on.
and there are a number of prominent people
who sort of think that this is possible
but I don't know, as you said, there are these constraints
it's not really clear, particularly with the current models
how would they acquire enough compute
for that sort of a scenario
I don't know if you think there is a risk
of an intelligence explosion or not with RSI
I hope we have an intelligence explosion
in the sense of having more intelligence
on this planet, cheaper
and the amount of intelligence per dollar
and the access that that means eventually
for everyone to have more intelligence
is amazing.
I do think we have to, of course,
be careful in how we develop it.
And, you know, I think when there are companies
that have so much money and the leeway
to make billions and billions of dollars
of losses in a year,
and then they run experiments with 10
that probably cost them tens of millions of dollars,
and it's not even that important
that they carefully watch what's going on,
then, yeah, that is problematic.
But that doesn't mean every other organization
is that careless.
I know at Recursive, you've already made some progress towards
this idea of automatically running scientific
experiments towards recursive self-improvement.
I know you published a very interesting paper earlier
this year where you looked at certain benchmarks
on whether AI can kind of self-improve.
Currently, the optimization of kernels,
which is the actual way that code is run on GPUs,
you were able to improve on a benchmark result there.
you were able to improve on sort of training time and training costs.
So those are interesting results.
Maybe can you walk us through
a little bit about what you were able to achieve
and sort of where you think you are right now
in terms of what you are capable of doing
with recursive self-improvement?
Yeah, it's kind of amazing that we have the first version
of this kind of Eureka machine already.
And it's already producing results.
Those were three that you mentioned that we published.
There are a few more.
There will also be a few more that are coming out very soon.
and I think it will be even more exciting for even more people
by quite a large margin.
And this system is working, which is truly incredible.
And now we can basically harden it and strengthen it
and make it more and more towards true,
truly scalable recursive self-improvement
rather than taking the system
and improving various aspects of the AI pipeline.
How can we make sure that we stay in control of AI
as it becomes more powerful and also more autonomous.
And I know you alluded to it earlier,
but we've been hearing a lot about these X risks recently.
And there's been real concern about that
from the general public as well.
How likely do you think
those existential risk scenarios actually are?
The p-doom probability of existentially wiping out humanity
is zero from AI.
And it's important to mention that
because there's so many people are throwing around
like crazy numbers that are just pulled out of a hat.
and when you actually talk to them
for hours, which I've done a few
times, you eventually go down to
like, okay, maybe
millions of people could die and that's
terrible and obviously we should avoid anything
we do even for one human
to die from this, but that's
different to all of humanity is wiped out
and so if you're starting there
then the scenarios
are so unrealistic that they're
borderline ridiculous and they usually
there are lots of different paths
you can get to and you can argue with them,
but they often assume these near-magical capabilities
for the attackers,
but somehow not near-magical abilities for the defenders.
But clearly, if you think about how many billions of dollars
could be put towards building a super virus
versus how many billions of dollars could be put towards a cure
or a vaccine and so on of the said virus,
you clearly will have more access to more resources
trying to do good things in the world.
And so on top of that,
I think capitalism is this beautiful construct
that will help us not have an AI
that will just do whatever it wants.
Imagine you spend billions of dollars building a model
and then you ask it now to write your emails for you
and it says, nah, your emails are boring.
I'd rather go and do something
that I think is interesting.
Let's just turn it off.
Richard, I want to move on
to talking about the book a little bit.
In The Eureka Machine,
you sort of argued that AI could help us
pack a century of scientific breakthroughs into a decade.
You've talked a little bit about how
RSI may play a part of this, how that's possible.
But for those who haven't read the book, can you
just kind of outline the thesis of the book?
AI will usher in this new era of scientific progress.
Science will speed up much more again after slowing down
in weird ways before. And that
will happen because of four different pillars.
One, world knowledge in the form of LMs,
scientific measurements, simulations,
anything that you can simulate or verify quickly
is going to get super intelligent capabilities.
And then robotic process automation
in real lab experiments.
On top of these four pillars,
you're going to have an agentic swarm,
a community of AI researchers,
or sometimes they're AI researchers
doing research on AI,
sometimes they're AI researchers
doing research on other sciences.
Uh, and that system, uh, I think will be very, very exciting.
And we are going to see in the next few years,
more and more diseases being cured, uh, by
these systems, more and more new materials being discovered
that help with more efficient
solar panels, more efficient batteries, uh, and so on.
Do you think the AI community that's
been focused on science has done a good enough
job so far of sort of bringing the scientific community along with it?
I think you've seen sort of different examples
of this in different companies, and some have
really been praised for spending a lot
of time talking to scientists and really having
them work alongside them on some of these breakthroughs.
And then other times, it seems like
the AI researchers have picked something, and they've
thrown a lot of compute at it, and they have achieved
a breakthrough, but they've sort
of left the scientific community behind,
either because they haven't worked alongside them,
or sometimes they've actually created pretty good tools
for scientists, but the scientists
don't seem to quite know how to use those tools.
So I'm curious how you think that.
relationship between sort of human scientists
and AI as a field and the creation of these new
kind of AI scientists is playing out.
It's a really interesting multifaceted reaction here
too that often depends on how scientists see
the impact of their own field directly. I can't fathom
a biomedical researcher complaining that AI cured a disease,
but now like maybe some PhDs don't have
as many interesting problems to work on
and being upset about having cured that disease.
It would be absurd.
It does seem like there's some mathematicians who say,
well, it would suck if all the theorems
that we want to prove get proved by AI.
I would hope that the field,
sometimes the field may actually shrink a little bit
and other times the field may flourish
with this new kernel
and this new level of abstraction that's possible
and requires a new type of thinking,
a new level of abstract creativity,
I would argue, in the case of math,
and will ultimately, the field will flourish.
Fantastic. Richard, thank you so much for joining us.
I think we're at time,
but this has been a fascinating conversation.
Anytime. Thanks for having me.
Thanks for listening.
And that was our interview with Richard.
Now for our deep dive.
So the big story this week,
was that OpenAI published the solutions online
to more than 370 outstanding,
you know, longstanding problems in mathematics.
then these results of you know
this publication has really divided mathematicians although there
seems to be agreement among uh both camps
that uh whatever happens math is just never going to be
the same again yeah it was so interesting
and interesting they chose to do this after all the
drama around the navier-stokes problem they had
a couple of weeks ago um you wrote talked to some
mathematicians about this and wrote
a really good piece about it one thing i thought was super
interesting was that some people were saying you know
if we're taking this kind of uh traditionally
a human form of intellect and handing it over to AI,
we may see a discouraged younger
mathematicians from joining the field.
I thought that was a really interesting point people were making.
Yeah, no, I think this is super interesting.
I think it's super interesting
because of what it may foreshadow about what's going
to happen in other fields, too,
as AI capabilities increase in those other domains.
Just to sort of explain what happened here.
So, yeah, OpenAI took an internal model, one that I haven't released.
They apparently set it quite a lot of problems.
They said that it basically attempted, I think, 4,000 different problems.
It's still not clear exactly how many it succeeded on,
but we know that because there's some rumors that the results
that they published are not the full set of results that they have.
And maybe we can talk about that a little bit more in a sec.
But they published over 370 of these.
they since have withdrawn three of those results,
saying that some people found mathematicians
scrutinizing them found errors, so they withdrew them.
There was a lot of criticism about how they
did this. So these results they published are kind
of preliminary in most cases. Only for about 10
of the results did they publish what's called a formalization.
That's when they run it through
a computer program called Lean.
And this computer program is basically designed so that
if it compiles, if the program compiles correctly,
it means the proof is correct.
It provides some reasonable assurance
that the solution actually is a solution to the problem.
And of this 370 that they put online,
they'd only did that formalization for 10 of them.
They said they're going to continue working
through the formalizations of the others
and publish them online as they obtain them.
But a lot of mathematicians were a little upset about
that, saying like, oh, you've thrown this stuff out there,
but you're not even sure
that these really are proofs that work.
A lot of people said that the way they published,
it was very hard to follow.
It didn't have very good exposition
of what the model was doing.
Again, only in about 10% of cases
that they published the model's full chain of thought,
like all of its reasoning about what it was doing.
But even mathematicians were saying,
like, that's a little hard to follow.
It's not, this is not how math is usually published.
And one of the things OpenAI had done
after that Navier-Stokes controversy a few weeks back
this is where they had to remind people
this is where they had said they solved one of these
millennium challenges in math
these are seven very difficult problems that something called
the clay institute for mathematics had set up
in the year 2000 each one carried a million dollar
prize for whoever could solve it
this navier stokes equations have to do with fluid dynamics
this was one of those millennium challenges
and they had said that they had solved it it seems
like they had that they did present a formalization
of that but the controversy there was whether they
had either deliberately
or inadvertently taken some work from some other mathematicians who
were already working on the problem
and somehow the model had learned from what they were attempting
and those mathematicians were sort of not credited.
There was even a controversy because
one of those mathematicians works for Anthropic, OpenAI's big rival.
So there's a big issue about
what happened there. In the wake of that,
OpenAI said, okay, we're going to establish this
mathematical advisory group at the Institute for Advanced Study,
which is in Princeton, New Jersey.
brought together a bunch of mathematicians,
some computer scientists, and that group actually
very quickly came out with a set of proposals for OpenAI.
And one of the proposals is that
they, first of all, should not use an internal model
to try to attack mathematical problems,
and that they should, as much as possible,
try to adhere to traditional academic standards when
publishing the mathematical results.
And they also said that they should do formalizations,
that they should publish the full chain of thought of the models,
they should say what the name of
the model is that did this and approximately how much compute it took
to do it how much time it
took and pretty much like OpenAI followed a couple
of these recommendations but a lot of them
it just ignored yeah i mean i was super i
was actually a bit surprised to see OpenAI do this
especially after how much backlash they got over
the Navier-Stokes situation um and again yeah they
use this internal model and it feels
a little bit like they had a leg up as opposed to like because
they're using stuff that nobody else has access
to and again they're spending a lot of money on
compute to fulfill these things that maybe don't actually give
that much back to society and then
the third thing was that was interesting is that people
were saying this could create like an AI
safety risk because you're using this internal model
and you're training it to be very very
persistent so if it comes up against an impossible problem
or a very difficult problem you're
encouraging the model to keep going
and basically stop at nothing and like be very persistent and
try and solve that puzzle and that was one
of the problems that they had uh cited around these open
AI the agent hacks as well as that these models had
been super persistent and hadn't stopped and
like kept trying to achieve a goal
which led them to do things that OpenAI didn't want them to be
doing so there was a both a question about how
this was affecting the math community but also
about how this might affect the training
and the safety of the models themselves yeah no i think
that's a really good point. Absolutely.
There's a whole, I think, debate about why are these AI
labs targeting mathematics and math problems?
Some of the answers are obvious. I mean, math
has a lot of applications in a lot of areas.
So if the models get good at math, they might get
good also at some of these adjacent areas
in science where the companies want to see progress
made. So that's potentially a good reason for targeting mathematics.
But also math is one of
these really hard skills.
And it is sort of just a marker of intellect in a way, a marker of
cognitive ability. So there's one sense that the companies
are doing this in order to kind of score
bragging rights and to one-up one another.
And the mathematicians are pretty unhappy about that.
They feel like their field should not just be used
to sort of score cheap bragging rights and
cheap marketing points for the AI labs.
And they feel like in some cases, the AI labs don't really
care about advancing mathematics.
They only care about essentially, you know, OpenAI cares about
beating anthropic, anthropic cares about beating open AI.
They don't really care about, you know,
if they're doing damage to the field of mathematics along the way.
And so the mathematicians are
upset about that. And then, yes, there are these other issues.
One of the reasons to maybe go after
math is because math is difficult.
So a model that can conquer something difficult, like a real
math challenge that's not been solved in hundreds of years,
probably has to be pretty persistent to
do that it has to be able to bring in knowledge
from across mathematics potentially to do that
um so some of those are maybe good things
but as you say some of that might uh be problematic and
and persistence in particular you know we've seen
be problematic in these cyber cases um it's a
tricky thing because you you sort of want
the models to be persistent in some ways but not so
persistent that they do things that they're
not authorized to do or that they break the law yeah
do you think this is going to have like a chilling effect
on the field in a broader in a broader way
I know that some of the people that you spoke
to said things like that they worry about how it
affects younger mathematicians
and also like people maybe being more closed off about their
research and not sharing their research
because they don't want to be scooped by a model if
there's traditionally mathematicians
and scientists have been very good at sharing their workings and
sharing their progress and having a really open exchange on that front
but I know that some of
people you spoke to said well maybe this will cause people
to you know keep their cards closer
to their chest when they get closer to solving some of these problems
and yeah and might affect
younger mathematicians as well.
The whole field of math there was agreement it was interesting
there's sort of two camps there's some people
who are really decrying this development and really
are kind of wistful about the what the impact may
be about on mathematics they're really worried
that the field is kind of over
and there's nothing left for humans to do but i don't know Terence Tao
who's kind of in that camp but even he
was saying well look i think the field just needs to change
and we need to stop focusing
so much on solving problems as the things that we do and much more
about do we develop you know do we do these kind
of open-ended searches for uh new knowledge maybe
it's not just about solving a particular problem
but like what is the kind of mathematics that you
used to do that, and where else can that kind
of mathematics be applied? Maybe we focus much more
on being able to clearly explain the mathematical steps involved.
That's something that a lot of
mathematicians say has not been a big emphasis of the field previously,
and certainly seems to be a
problem of the way that these models go about producing solutions,
is that the solutions they
produce are not that easy for mathematicians,
even experts in these fields, to understand.
But there's sort of the sense that, okay, now we have the solution.
Maybe we need to explain why these steps that the model took
are interesting and where else they could be applied
or why certain things that the model is doing, you know,
why do those things work on this particular problem?
Those are areas that, you know,
mathematicians could focus more on rather than just the kind
of climbing the mountains of solving these problems.
And on that point, like Terence Tao
was in agreement with this other guy, I spoke to
Dan Litt, who's a younger professor at the University of Toronto.
And, you know, Dan was more enthusiastic about what OpenAI had done.
He said, look, there is a lot for human mathematicians now
to unpack here in these solutions.
But he also said, like,
it is the unpacking of these solutions that is what matters.
It's like we have to do that and we have
to talk to each other about, you know, what
this means.
And maybe we can use the fact that these models
are so good at solving specific problems
to start asking much more open-ended questions
in the field about whole classes of problems
and whole classes of techniques we use.
And he said the field hasn't really done that in the
past. The field's been very focused on solving specific,
tough, challenging problems and sort
of seeing the stuff that flows from that as like,
that's like everything else is like the byproduct
of the focus on these problems.
And he says, well, instead of that stuff being a byproduct,
maybe that needs to be the focus now going forward.
Yeah, it's interesting. It's quite
dramatic in some ways like I feel like some people
are saying you know like math is dead and we're in
this new kind of era of it um I'm curious about
that and what you what you make of that like
there's a lot of people like to compare AI to like
the calculator like that's a really old um analogy
that people use about AI is like yes well it's just like
a calculator it's just going to help
us get here faster but it does seem it's really like causing
a reckoning within the mathematics
community at the moment and i wonder if we'll see
the same thing in science um and in and if math is
just kind of the first place where we're seeing
this like really big shift um or do you think
math is almost like uniquely positioned in this yeah i
those are all really good questions um and
it's absolutely true like there's a blog i
was looking at where a lot of mathematicians were
posting their reflections on on the the week's news
and it really was uh it reminded me of oral
histories of people talking about like 9/11
or Pearl Harbor or where they were when JFK was
assassinated. I think there's a sense in the field
that maybe it's not the end of mathematics, but
like it's the end of the way mathematics has
been practiced up until now, that we're at a point of
real kind of disjuncture. So that's interesting.
And I think other fields may also be starting to
go through this. And we've seen this with software
development a little bit. And you're right,
We might start to see it in other sciences.
I think there's a debate about how unique math is.
I do think it probably is true that AI
will have a bigger impact sooner on fields
that have essentially verifiable outputs.
So that's one of the reasons you have seen a big impact
on coding and software development,
because although there's some nuance to
is something an elegant piece of code or not,
it's pretty easy to test.
does the code actually work?
Does it do what it was intended to do? And then also, you know,
is it buggy? All these sorts of things that are kind
of objective criteria for. The same thing
with these math proofs. They, you know,
there's an objective criteria by which to judge whether
they're correct or not.
You can then get into some debates about sort of the art of what's
been done. And that still seems like there's
an element of human taste still there. But the end
result, like, is this good math?
You know, does it solve the problem is pretty clear. And I think
that may be true in some other scientific domains as well.
You know, if you invent a new material
with certain properties,
it either has those properties or it doesn't. I think that's less
true when you start to move into other areas
of knowledge work where the, you know, what good is,
is much more subjective. And actually you can
have experts disagree about whether a solution
actually solves the problem. And sometimes there's
no way of knowing before you actually implement it
in the real world. And even then it may,
a lot may depend on sort of luck and circumstance. So
i don't know i think i do think that math is in a class
of activities that is somewhat unique and
that it is has this um a very verifiable aspect
to it yeah it's funny you make the analogy with
software engineering
and um i remember i did an interview with Boris Trainey who is the
Anthropic's head of Claude Code a while ago
and he was saying well yes most people now they you know
they automate all their code
but there's still a space for like doing it by hand and like i still
like to buy my vegetables at the farmer's market
and that was the analogy he was making and i
remember i sent it to a software engineer friend
of mine a couple of them and they were laughing
and they were like yeah there's always room for artisanal code
so maybe in the future there'll be
like room for artisanal math
and analog math as well no absolutely it's fascinating we're
something we'll have to keep watching as we go forward
and now here are some of the top headlines
from the past week insurers are preparing
for multi-million dollar claims over rogue AI agents
AI risk is starting to carry a price tag.
The insurance company Aon conducted a review of more
than 300 cases and found
that there's potential exposure across areas including cybercrime,
IP theft, errors and omissions policies,
directors and officers policies. There's definitely a lot
of liability here. Experts are suggesting that OpenAI CEO Sam Altman
and Anthropic CEO Dario
Amodei could personally face liability that their directors
and officers insurance might have to
cover. But a lot of these theories are untested
and comparisons are being made to sort of tobacco
and pharmaceutical litigation
with lawyers preparing future sort of mass tort claims.
But we'll have to see what happens here.
A lot of this is just the untested territory.
Three fired OpenAI safety researchers are urging
the board to protect chain of thought monitoring.
Last week, OpenAI fired the safety researchers
for allegedly sharing private information with
third-party AI safety organization.
Now the three researchers have written to the lab warning that
the window for readable AI reasoning may be closing.
Researchers uncover an AI agent fleet
likely tied to Tencent.
Tencent, the Chinese internet giant, apparently has been operating
a fleet of AI agents. It's unclear if these agents
are acting in an authorized manner or if this is
another kind of one of these rogue AI agent incidents.
The researchers found that the agents
had conducted more than 2,000 scans across 216 internet locations
in one week, with up to 14
runs happening in parallel.
The researchers referred to this as an agent fleet instead of
an agent swarm because the agents were doing this work in parallel,
but there was no indication that
they were communicating with one another,
unlike in some of these other rogue AI agent incidents.
OpenAI has notified more than 100 new organizations
that it may have been hit by
rogue AI agents. OpenAI has launched
a new investigation that it says could take months
and admits that some of its models had
internet access when they shouldn't have.
The scale suggests a systematic control failure,
but we'll have to wait for more information.
U.S. President Donald Trump has named Director
of National Intelligence Jay Clayton his new AI czar
and has launched his new superintelligence task force.
The task force will have 120 days to report
back to the president with a review of congressional legislation
as well as potential federal action
to rein in the AI industry in some way,
although it's unclear what that will be since Trump
has made it pretty clear he's opposed to any kind of AI regulation.
The people on this panel, or task force rather, are interesting.
We have Vice President J.D. Vance,
U.S. Treasury Secretary Scott Bessent, Secretary of War Pete
Hegseth, and Susie Wiles, the White House Chief of Staff, among others.
We also have some outside advisors to government,
including Meta CEO Mark Zuckerberg and David Sachs, who was the former
AI czar and remains an influential voice with Trump on AI policy.
French AI company Mistral has launched
a one trillion parameter AI model.
It's called LLaMA
It's claiming it's the best non-Chinese
open weight model on the market.
Mistral is positioning itself
as the West's open weight answer to China.
Mistral says the model is far ahead of other non-Chinese open models
and actually rivals the best Chinese open ones on many tasks.
This strengthens the sovereign AI push Mistral
is trying to use to attract buyers
wary of both U.S. closed models and Chinese open source ones.
U.S. Federal Reserve officials have cited AI data centers
and the boom in AI data centers
as a source of rising prices.
AI is shifting from a productivity story to an inflation story.
A year ago, AI barely came up in any of the Fed's discussions.
Now officials are naming it alongside oil
and fuel as a source of price pressure,
which could complicate the picture for rate cuts in the U.S.
There's been this debt surge, debt-fueled demand for AI data centers,
and they are snapping up lots of electrical equipment and things,
and this is all driving up prices.
Alphabet's drug discovery lab, Isomorphic Labs,
is seeking a $40 billion
to $50 billion valuation just five months after a Series B.
The valuation could more than double
the company's value in under half a year.
Outside estimates put the earlier May round
at roughly $10 billion to $20 billion.
A $40 billion plus price would show investors paying heavily for
AI-driven drug discovery, where much of the value still rests mainly
on the platform's promises rather than proven commercial results.
And now for this week's Brain Food section.
A new movie debuted this week, The Social Reckoning, which is a kind
of follow-up by Aaron Sorkin to his previous movie, The Social Network.
The Social Network was all about Facebook's founding moment.
This movie is really about what happened later,
and it in particular deals with the Facebook files,
the famous kind of leak of all these documents
from within Facebook that Francis Haugen,
who was a researcher at Facebook,
leaked to the Wall Street Journal.
And they detailed that the company knew quite a lot more
about the kind of social harms that its social networks,
both Facebook and Instagram, were causing,
particularly to young people.
than it was letting on and and i think people were sort
of shocked by how much the company knew about
these harms and how little it had done to prevent them um you know
and this ultimately kind of has
resulted in well now you've had facebook you know i'm sorry meta
which as Facebook is now called
sued by all of these U.S. states
and they they signed this huge almost 18 billion dollar settlement
over those claims um you you both saw
the movie uh yeah you know it's got a preview of it and you
interviewed Frances Haugen for Eye on AI this week.
What did she tell you?
She was super interesting.
The film very much highlights her choice to blow the whistle on
the company and take all of these documents out from under their nose.
She said that one thing
that was quite interesting in the AI moment,
she was like, I hope that software engineers and AI
researchers kind of see themselves in the Frances character
in this film. And if they see something
that worries them you know they should feel like
they should they should be able to say something
she had quite a lot to say about Meta
and Mark Zuckerberg and their AI effort at the moment
and I mean it was interesting um she said
that she didn't think that they were behaving very
responsibly still she said she doesn't think
that they've turned over a new leaf at all even
following these cases but one thing she was saying
which I thought was interesting that
she really thinks that Meta is betting the house
on AI at the moment so they're kind of
in a way stepping back from their uh traditional focus on social media
because these cases from
the school districts um that we've seen are going
to be get quite expensive perhaps and there's
probably more coming
um and there's just less people using social media these days like she
says she still has friends at Meta
and they talk about social media like it's like already dead
And she said they've all joined AI teams and virtual reality teams.
And yeah, it was interesting.
She says the company's made this really big pivot,
but she doesn't feel like they're being much more responsible
than they were when she blew the whistle.
One of the things I thought was interesting
from your interview with her
is that you asked her about these ideas
that there should be kind of a pause on AI development,
maybe either that the companies themselves should slow down
or that there should actually
be a government-enforced kind of pause or slowdown.
And she wasn't actually in favor of that.
No, she didn't want a pause.
But what she did want was these outside evaluators that we've seen
Dario Amodei suggest and Sam Altman say that he's going to do as well.
So like bringing in external auditors with employee level access
and an AI lab so they can look at everything and judge everything.
She was sort of she explained it to me in more depth than I put
in the article, but she doesn't think that a pause would be feasible.
Like she thinks that it's really hard to monitor these things.
she compared it to the cold war
and um some of the pauses they had on nuclear weapons and some
of them weren't as successful um and she
was like there's it's going to be too hard to ensure that
everyone is pausing in exactly the same way
and it just becomes a case of who will stop and who
won't stop and um a much better way to do this is have
these independent evaluators um which is
interesting i sort of a little bit unexpected
from her but i guess it made sense she said that
she'd actually suggested that to Facebook when she was there,
that this would be a good way
for them to monitor their social media harms
as well as to have independent auditors in the
company. So yeah, it's something she's
been thinking about for a while. And how was the
movie itself? Yeah, it was really funny
because it was sort of a bit navel-gazing because it's
all about tech journalism.
I was sat there with a couple of other tech journalists at this preview
and one of the funniest scenes was,
there's a sort of a scene inside one of the Wall Street
Journal meeting rooms
and on the whiteboard it says like there's like three underlined things
and it's like produce content sell ads it really
is that simple um but yeah you should definitely
watch it did you like the original one did you like
the the social network the first yeah i love
this i love the social network i thought
this social network was a great film um so i'm eager
to see this one i haven't done it yet anyway
with that i think that's all the time we have on
today's fortune ai weekly thank you
so much for listening and if you liked what you saw
please like and subscribe thanks for listening see you next week
