A new arXiv study just proved that most AI feedback is basically just a fancy "try again" button — and understanding why changes how you should use AI to learn anything.
The AI Feedback Problem You Didn't Know You Had
Researchers tested thirteen AI models in student-teacher roles across maths, coding, and reasoning tasks. The uncomfortable finding: when an AI gives another AI (or you) feedback, most of the improvement comes from simply having another go — not from the feedback itself.
Self-generated AI feedback, it turns out, adds almost nothing beyond just resampling the problem. The real gains only appeared when an external teacher had genuinely privileged information the student lacked. Generic "here's what you did wrong, try again" advice? Basically noise.
What This Means for AI Feedback Loops in Your Workflow
If you're using AI to review your own AI-generated work — asking ChatGPT to critique a ChatGPT draft, for instance — you're likely getting very little signal. The model is essentially reshuffling the same reasoning it already used.
The study found the real bottleneck isn't the quality of the feedback; it's the student's ability to act on it. In practical terms: if you don't understand the domain well enough to apply the critique, even brilliant feedback is wasted. This flips the script on "just ask AI to review it" as a quality-control strategy.
The practical fix? Use AI feedback most aggressively in areas where you already have baseline knowledge — so you can actually evaluate whether the critique is useful. For genuinely unfamiliar territory, seek feedback from a model or source with demonstrably different training or perspective, not just a reworded echo.
What This Means for Learners
This research is a direct challenge to the "AI as tutor" model that's everywhere right now. If you're using AI to learn a new skill, the feedback loop only works if the AI can give you guidance you genuinely couldn't have generated yourself — and if you have enough context to use it.
The smartest move is to build your own understanding first, then use AI feedback as a check. Our Multi Agent Architecture That Actually Works course digs into exactly how to structure AI agent interactions so feedback is substantive, not circular. And if you want to understand why models behave this way at a deeper level, How Neural Networks Really Work gives you the foundation to stop taking AI output on faith.
Bottom line: AI feedback loops are a powerful productivity tool — but only if you architect them deliberately. Blind iteration is just expensive coin-flipping.
