Does AI Assistance Actually Save Time or Just Feel Like It Does?

There has been a lot of discussion about cognitive offloading—the tendency to shift mental work to an external tool (search, calculators, or now AI). The assumption is simple: AI lowers productive struggle (so learning may suffer), but it should increase productivity by helping people finish faster.

A recent randomized experiment by Shen & Tamkin challenges the faster part of that story. In their study, participants who had access to an AI assistant showed lower skill formation, and did not become significantly faster on average largely because time shifted into interacting with the assistant.

What they did

They recruited 52 people with more than one year of Python experience who used Python regularly, and split them into:

· Control: web search + instructions (no AI)

· AI condition: web search + instructions + an AI assistant (base model: GPT-4o)

Participants completed two coding tasks using a new library (Trio), then took a post-task quiz measuring conceptual understanding, code reading, and debugging with no AI allowed during the quiz.

What they found

· Time: In the main study, AI access did not significantly improve completion time on average.

· Learning: The AI group scored lower on the post-task quiz, suggesting lower learning gains.

· Mechanism: Some participants spent substantial time composing and iterating on queries enough to erase average speed gains.

The paper identifies six AI interaction patterns, and the learning outcome depends on how people use AI.

· The lowest-learning patterns relied heavily on AI to generate code or repeatedly debug/verify solutions

· The higher-learning patterns involved more cognitive engagement (asking conceptual questions, or requesting explanations alongside code).

Experience of the task

The control group reported higher self-reported learning, while the AI group reported the task felt easier. Enjoyment was high in both groups.

And this matches something I hear from university students: many of them genuinely want challenge because the reward is not just the answer, but the feeling that the knowledge is now theirs.

So I’m left with the question:
When AI makes a task feel easier, am I actually saving time or just offloading effort in a way that reduces learning?

Reference: Shen JH, Tamkin A. How AI Impacts Skill Formation. arXiv preprint arXiv:260120245. 2026.