Anthropic’s latest research delivers an uncomfortable truth: AI coding assistants aren’t just failing to make developers more productive—they’re actively undermining their ability to learn. In a controlled experiment with 52 developers, those using AI scored 17% lower on knowledge assessments despite having access to tools that could generate correct code on demand. The implications extend far beyond software development.
The productivity illusion
Most organisations adopt AI tools expecting twin benefits: immediate productivity gains and faster skill acquisition. The research demolishes both assumptions.
The real story
The experiment tasked developers with learning the Python Trio asynchronous library—a sufficiently complex technical skill to require genuine understanding. Half received AI assistance; half worked without it. Both groups completed the same coding tasks and faced identical assessments.
The results challenge conventional wisdom about AI-assisted learning:
| Metric | AI-assisted group | Control group |
|---|---|---|
| Knowledge assessment scores | 17% lower | Baseline |
| Task completion time | No significant difference | Baseline |
| Errors encountered | ~33% fewer | 3x more errors |
| Conceptual understanding | Impaired | Stronger |
Strategic Reality: AI assistance didn’t make developers faster at completing tasks—it just changed how they completed them. The promised productivity boost never materialised.
What makes this particularly striking: encountering errors three times more frequently correlated with stronger conceptual mastery. The control group’s frustration became their education.
How AI assistance actually works
The research identified six distinct patterns of AI interaction. The gap between effective and ineffective usage patterns proved dramatic.
High-performing patterns (65-86% assessment scores)
Developers who scored well treated AI as a teaching assistant rather than a code generator:
- Asked conceptual questions about how the library worked
- Requested explanations alongside code examples
- Used AI to understand error messages and debugging strategies
- Maintained active cognitive engagement throughout
Low-performing patterns (24-39% assessment scores)
Poor performers delegated thinking entirely:
- Requested complete solutions without understanding requirements
- Copied AI-generated code without review
- Skipped documentation and error analysis
- Treated AI as a replacement for learning rather than a supplement
Critical Context: The difference between highest and lowest performers wasn’t whether they used AI—it was whether they remained mentally active while using it.
This maps directly to educational research on “desirable difficulties”—the counterintuitive finding that obstacles during learning (like errors and confusion) actually strengthen long-term retention.
The organisational implications
This research has immediate consequences for how businesses think about AI tool deployment.
Who this affects
| Stakeholder | Primary impact | Secondary impact |
|---|---|---|
| HR and L&D teams | Training programme design | Onboarding effectiveness |
| Technical managers | Team capability planning | Knowledge transfer protocols |
| Individual contributors | Career development | Skill portfolio growth |
| Executives | Workforce investment ROI | Long-term competitive position |
The competence-productivity trade-off
Organisations face a genuine dilemma. AI assistance may help teams deliver today’s work while undermining their ability to tackle tomorrow’s challenges. A developer who never debugs their own code doesn’t develop debugging skills. A writer who outsources structure to AI doesn’t internalise how arguments build.
Implementation Note: “AI-enhanced productivity is not a shortcut to competence.” The research is explicit: the barriers AI removes—errors, confusion, struggle—are precisely what builds capability.
Success criteria that matter
Traditional metrics miss the real picture:
- Task completion rates show no AI advantage
- Time-to-delivery shows no AI advantage
- Quality assessments show impaired understanding
- Knowledge retention shows significant AI disadvantage
What should organisations measure instead?
- Independent problem-solving capability after AI-assisted projects
- Error recovery skills when AI is unavailable
- Conceptual understanding of tools and frameworks
- Ability to extend or modify AI-generated solutions
Making AI work without undermining development
The research points toward practical interventions for organisations that want AI’s benefits without its developmental costs.
By organisational maturity
Early-stage AI adoption:
- Restrict AI tools during onboarding and training periods
- Require explanation requests before code generation
- Build review processes that test understanding, not just output quality
Established AI usage:
- Rotate between AI-assisted and unassisted work
- Create “learning sprints” where AI is deliberately unavailable
- Assess individuals on their ability to work without AI assistance
Advanced AI integration:
- Design role progression that requires demonstrated AI-independent competence
- Build career frameworks that reward capability development alongside output
- Create internal mentorship programmes pairing AI-native and AI-free experience
Success Factor: The goal isn’t to avoid AI—it’s to ensure AI usage doesn’t become a crutch that prevents genuine skill formation.
Policy interventions that work
The most effective organisations will likely adopt differentiated AI policies:
- Training phases: Limited or structured AI access focused on explanation requests
- Production work: Full AI access with periodic capability assessments
- Career progression: Demonstrated competence without AI assistance required for advancement
- Knowledge transfer: Senior staff rotate through AI-free periods to maintain teaching capability
The challenges nobody discusses
1. Measuring what you can’t see
Skill erosion happens gradually. Teams appear productive while slowly losing capability. By the time the problem becomes visible—a crisis where AI is unavailable, a novel challenge requiring genuine understanding—the damage is done.
Mitigation: Regular capability assessments that specifically test AI-independent competence. These shouldn’t be punitive but diagnostic.
2. The competitive pressure trap
Organisations that restrict AI usage may appear less efficient in the short term. Competitors who embrace unrestricted AI may seem to deliver faster. The temptation to abandon developmental safeguards will be constant.
Mitigation: Track long-term metrics: staff retention, promotion readiness, ability to handle novel challenges, knowledge transfer effectiveness.
3. Individual variation in AI usage patterns
The research showed some AI users scored well—those who maintained cognitive engagement. But most organisations have no visibility into how individuals actually use AI tools.
Mitigation: Train teams on effective AI interaction patterns. Make the distinction between “AI as tutor” and “AI as replacement” explicit and observable.
Warning ⚠️: The most dangerous AI users aren’t the obvious productivity drains—they’re the apparently productive individuals whose output masks their failure to develop genuine competence.
4. The experience gradient problem
Junior staff need development most and face the strongest incentive to delegate to AI. Senior staff have the competence to use AI effectively but the least need for assistance. The tool’s value and its risks are inversely correlated with experience.
Mitigation: Differentiated AI policies by role and experience level. More structured access for those still building foundational skills.
What this means for your organisation
The research confirms what many suspected: AI assistance trades present convenience for future capability. The trade-off isn’t always wrong—but it should always be conscious.
The core insight
AI tools are most dangerous when they work. A tool that generates working code removes the errors that build understanding. A tool that structures your writing removes the struggle that develops clarity. The assistance itself is the problem.
Three factors that determine success
-
Awareness that the trade-off exists. Most organisations deploy AI tools assuming pure upside. The research shows this is false.
-
Deliberate policy design. Default AI deployment optimises for short-term output. Effective deployment requires intentional restrictions and capability assessment.
-
Measurement of the right things. Output metrics miss skill erosion entirely. Organisations need to track competence independently of productivity.
What to do next
- Audit current AI tool deployment against developmental impact
- Identify roles and training phases where AI restriction may be appropriate
- Design capability assessments that test understanding independent of AI access
- Train teams on the distinction between effective and harmful AI interaction patterns
- Build career progression criteria that require demonstrated AI-independent competence
Take Action: If your organisation provides AI tools without policies addressing skill development, you’re likely trading future capability for present convenience. The research suggests this trade-off may not even deliver the productivity gains you expected.
Source: The Effects of Generative AI on High Skilled Work: Evidence from Experienced Software Developers, Shen & Tamkin, Anthropic, 2026
Resultsense helps UK businesses implement AI tools with appropriate governance. Our AI Strategy Blueprint identifies where AI assistance accelerates work without undermining development, while our AI Risk Management service builds policies that balance productivity with long-term capability. Book a free consultation to discuss your AI deployment strategy.