There's a subgenre of video games, mech warrior games. You put yourself in a giant mech suit with guns and rocket launchers and jet packs and go and fight whatever needs fighting.
I never liked mech warrior games. I preferred to be the person doing the fighting, like Master Chief in Halo, rather than doing it in a mech suit.
Which doesn't really make sense. It's all a video game. I'm pressing buttons on a controller either way. And Master Chief is himself wearing an advanced bit of armor. But that's just how I've always felt about it.
As it comes to AI, and specifically all of the developments that have come downstream of large language models, it feels like we've been given access to mech suits fit for quite a few domains – coding/software engineering being the most apparent.
Recently there has been a lot of attention around OpenAI's work on a problem related to the Navier-Stokes Millennium problem. There's plenty of drama around who deserves credit. Some of the mathematicians who had been working on the problem think the model was trained on their conversations about it. Maybe it was, maybe it wasn't. But it's important to note that even the humans on the other side of that drama were heavily AI-assisted.
Which is to say, they were wearing mech suits too.
Our new AI mech suits give us superpowers, and I think that is, in large part, pretty cool. I can make things today that would have taken me months a few years ago. I can wander into areas where I know little and become productive remarkably quickly. There are ideas I simply wouldn't bother pursuing without the mech suit.
But as we get better at using the suits, surely we get a little worse at living without them.
I cannot imagine coding without AI anymore. And surely I'm much worse at it than I was a year ago.
Which I don't like. I have nostalgia about a time where it would make sense to furiously scribble down notes trying to solve a hard problem. But maybe that's ok?
Coding for me has always been a means to an end more than an intellectual pursuit. I want to make the thing. Coding with a mech suit is simply one more level of abstraction. I don't need to know assembly to write Python. I don't need to understand TCP congestion control to build a web app.
Coding with a mech suit could simply be one more level in that tower.
Though there is some difference between abstracting away the machinery underneath your work and abstracting away the work itself.
At some point I'm no longer a programmer using better tools. I'm a guy telling the programmer what I want, and judging the result. I'm not sure exactly where that line is, but I'm pretty sure I crossed it.
Doing things we don't understand
The idea of doing things we don't understand is not new.
General anesthesia has been used for over a century with no accepted mechanism for how it works. Even now, we don't fully understand it. Bread, wine, and cheese predate germ theory by thousands of years. Variolation – intentionally infecting someone with "material" from a smallpox sore to protect them from smallpox – existed centuries before anything resembling modern immunology.
For most of human history, this was basically how we figured things out. We observed the world, noticed patterns, and tried stuff.
Do this and the bread rises.
Do this and you don't get smallpox (hopefully probably).
We knew that something worked before we knew why.
One of the great achievements of science was that we became increasingly capable of going the other direction. We developed theories that explained our observations, and then extended those theories to predict things we'd never observed at all.
So I don't think the interesting thing about AI is that, for the first time, we can discover or do things we don't understand. We've been doing that for a while.
When I drive my car, I benefit from an enormous amount of knowledge that I don't personally possess. Modern civilization is essentially one enormous system for allowing people to benefit from knowledge they don't personally have.
But somewhere, distributed across a lot of people, the understanding exists. Or at least, did at one point in time.
AI introduces the possibility that our collective competence could exceed even our collective understanding.
Imagine an AI designs a new antibiotic. We manufacture it. We test it. It works. Great! Lots of people don't die.
Then we ask why it works. Just like the old days?
Ted Chiang has a short story about this, The Evolution of Human Science. It's a few pages long, written as an editorial in a science journal. In it, a new kind of human, far smarter than the rest of us, does all the real science now. Human scientists are no longer pushing the frontier. Instead, they spend their time trying to understand and translate discoveries made by higher intelligences.
Which is unsettling for me, as someone who has romanticized the inner scientist discoverer/inventor in us all. We've spent centuries building machines to spare our bodies from work. I'm much less sure how I feel about building machines to spare our minds from it.
Our intelligence is what separates us from the animal kingdom. It's what we, as a species, take great pride in.
Maybe that's just vanity.
In 2024, I stopped recommending software engineering as a career to pursue. I think it's a dying art.
There will probably be way more software. But sitting down, understanding a system deeply, and then painstakingly translating that understanding into syntactically precise code? I'm not sure how much of that we'll be doing.
Is the same eventually true of math? Of science?
What's the point of spending years thinking deeply about a math problem if there is no hope of progressing the field?
Bigger monsters
Which brings me back to the mech suit. Maybe what bothered me wasn't really the suit. Master Chief had a suit, too. I think I wanted the suit to make me better at fighting. I didn't want the suit to do the fighting for me. Which is a difference I'm finding difficult to explain precisely (and so relying on video game analogies from my youth).
There's an irony about how often we invent a technology to free ourselves from doing something, only to bring the thing back as a hobby.
We spent thousands of years trying to make food easier to obtain, and now we spend our weekends growing tomatoes in our backyard gardens.
We built machines that can make a table straighter and cheaper than I ever could, and we fill our garages with woodworking equipment.
Maybe this is just what humans do.
We paint a veneer over the real world because our human selves, which have evolved for survival in the wild, are so far removed from that environment. And so we find things to engage with that poke at our ancient systems.
We spent centuries building machines that removed physical effort from our lives, and then had to deliberately add some of it back in. Now we're building machines that can remove cognitive effort too.
And maybe we can replace that with cognitive hobbies just as we replaced carrying water with carrying weights at the gym.
But there's another possibility. I don't think we'll run out of problems to solve. I don't think we're anywhere close.
There are questions that have sat at the edge of science for a very long time. What is consciousness? How did life begin? Why do we age? There are entire areas where it sometimes feels like we've been pushing around the same pieces for decades, waiting for someone to find a new way of looking at them.
Maybe the mech suit gets us there. And this is the best opportunity we could ever ask for to work on these problems.
It's time to fight bigger monsters.
This went out first in issue 22 of The Nine Thirty-Six, the PerThirtySix newsletter I write with Shri Khalpada. His letter in that issue is on the same Chiang story and the open letter mathematicians signed about AI.