The debate about whether AI helps coding is over. The real question is what it does to us.
Yesterday I wrote about the new playbook for juniors. Today, Anthropic dropped a study that puts hard numbers on something we’ve all been feeling.
Does using AI to code make you worse at understanding code?
Short answer: Yes, but also no. It depends entirely on how you use it.
Anthropic took 52 (mostly junior) developers and had them build features using Trio, a Python library none of them knew. Half got AI assistance. Half didn’t.
Then they quizzed everyone on what they’d just built.
The results:
| Group | Quiz Score | Time |
|---|---|---|
| No AI | 67% | baseline |
| With AI | 50% | ~2 min faster |
The score difference was statistically significant. The time difference wasn’t.
Let that sink in. AI users scored 17% lower—nearly two letter grades—on understanding the code they’d written minutes earlier. The biggest gap? Debugging questions.
The people who used AI couldn’t explain why their own code might break.
Not everyone who used AI scored poorly. The researchers watched screen recordings and found distinct patterns:
Low scorers (<40%):
High scorers (>65%):
The difference isn’t AI vs no AI. It’s offloading vs augmenting.
Theo made a point about this study that stuck with me.
Learning to skateboard sucks. It hurts. You feel stupid. 95% of people quit before they can even ollie (get the board off the ground).
But here’s the thing: if you can get that first “holy shit, I did it” moment earlier, way more people make it through.
There’s a dark psychology study from the 1950s. Rats placed in water would swim for about 15 minutes before drowning. But if researchers rescued them just before they gave up, dried them off, and put them back, those rats swam for 60 hours.
One moment of hope extended their effort by 240x.
What if AI is that moment of hope for junior developers?
If AI helps you build something real before you burn out—something that works, that you’re proud of—maybe you stick with coding long enough to actually learn it.
The danger is when AI replaces learning rather than accelerating it.
To their credit, the researchers are clear about the limitations:
“Our earlier observational work measured productivity on tasks where participants already had the relevant skills, while this study examines what happens when people are learning something new.”
Their previous study showed AI can speed up work by 80%. This one showed it can hurt learning by 17%. Both are true. They’re measuring different things.
Also, this footnote:
“This setup is different from agentic coding products like Claude Code; we expect that the impacts of such programs on skill development are likely to be more pronounced than the results here.”
More pronounced. In both directions, I’d guess.
Here’s where I land:
For learning: Use AI like a teacher, not a ghostwriter. Ask why. Ask for explanations. Try to solve similar problems yourself after seeing the solution.
For shipping: Once you understand something, let AI handle the repetitive parts. That’s what it’s for.
For juniors specifically: The risk isn’t that you’ll never learn. It’s that you’ll think you’ve learned when you haven’t. You’ll ship code you can’t debug. You’ll build systems you can’t maintain.
The solution isn’t to avoid AI. It’s to be intentional about when you’re learning vs when you’re producing.
If you manage developers, especially juniors:
The debate about whether AI helps with coding is over. It does. We know that now.
The interesting question is what it does to us over time. Are we building skills or outsourcing them? Are we augmenting our judgment or replacing it?
Anthropic—the company that makes Claude—is researching this themselves. That’s either reassuring or terrifying, depending on your perspective.
I think it’s both.
While we’re on the topic of Anthropic: Claude Sonnet 5 is dropping today, along with Claude Code v2.1.30.
For those of us running Claude Code locally, this is great news—Sonnet 5 is significantly faster and more efficient than Opus, which means my MacBook might finally stop trying to achieve liftoff every time I ask it to review a PR.
The irony of writing about AI skill development while my AI assistant writes this post is not lost on me. But hey, at least my laptop won’t catch fire anymore.
Yesterday: The New Junior Playbook
Sources:
