AI Coding Tool Reviews: What Users Really Think

I've been following the research on AI coding tools pretty closely and the numbers from the last few months paint a really different picture from the marketing.

Quick summary of what the data actually shows:

Anthropic published a randomized controlled trial in January. 52 developers learning a new Python library. The group using AI assistants scored 17% lower on follow-up comprehension tests. And here's the kicker: the productivity gains weren't statistically significant. The developers who used AI for conceptual questions (asking "how does this work?") actually did fine, scoring 65%+. But the ones who just had AI generate the code for them? Below 40%.

Then there's METR's study with experienced open-source contributors. 16 devs, 246 tasks in codebases they'd worked on for years. AI increased completion time by 19%. These devs predicted it would save them 24%. The perception gap is wild.

DeveloperWeek 2026 wrapped this week and the Stack Overflow CPTO made a good point. Off-the-shelf AI models don't understand the internal patterns and conventions of your specific codebase. They generate syntactically correct code that misses the architectural intent. So you spend the time you "saved" on reviews, refactoring, and debugging stuff that doesn't fit.

The other trend I'm watching: junior dev employment has dropped almost 20% since 2022. A Harvard study tracked 62 million workers and found companies that adopt generative AI cut junior developer hiring by 9-10% within six quarters. Senior roles stayed flat. We're essentially removing the bottom rung of the engineering career ladder right when the data says AI actually impairs skill formation.

I still use Claude Code and Cursor daily. They're genuinely useful for boilerplate, tests, and scaffolding. But I've stopped using them for anything where I need to actually learn how the code works, because the research basically confirms what a lot of us already suspected: there's a real tradeoff between speed and understanding.

Curious what you think. Are you seeing the same pattern? And for those of you who hire, has the "AI makes juniors unnecessary" argument actually played out in practice?

The junior nonhiring is creating a lost cohort. This pattern has happened in other industries throughout history and creates a lack of more senior employees later. It likely won't start becoming noticeable for 2-3 more years, but an understanding of this explains why some companies like IBM, github have turned around and started hiring more juniors. If you have the ability to operate on longer timelines it makes sense to plan ahead.

Every mid-level engineer I've worked with in the last 18 months has one of two states. Either they use Cursor or Claude Code or Copilot daily and are shipping 2x faster, or they refuse to use it and are shipping the same rate as before. The productivity story is real for group A.

The part nobody's tracking is what happens six months later when someone else has to touch that code.

The reviewer opens a PR and asks "why did you structure it this way." The author says "the AI suggested it." They don't know why. They can't defend it. They can't say what edge cases it handles or doesn't. They can't say whether the abstraction will hold up when the requirements shift in two months.

Before AI tooling, the answer to "why did you write it this way" was some version of "I considered X and Y and picked this because Z." The reasoning was baked in during the writing. Now the reasoning is somewhere in a chat log, if it's anywhere at all.

The second-order damage:

1. Design docs are getting shorter because "we'll ask the AI when we need to change it" is quietly becoming the fallback for absence-of-thinking.

2. Junior engineers watching mid-levels ship faster are learning that "productive" means "prompted well" not "understood well." Their apprenticeship is training them to be prompters, not engineers.

3. The pipeline of engineers who can actually debug hard problems six months into a large system is not being refilled at the rate it's draining.

4. Mid-level engineers who could previously articulate architectural tradeoffs at a whiteboard are getting reviewed as if that skill still exists, but the review is scoring "code shipped" not "understanding of code shipped."

Nobody in HR or eng leadership is tracking this because none of the metrics catch it. Velocity goes up. Bug count on newer code looks flat because the newer code hasn't hit its post-launch bug tail yet. Compensation is trending up for people who ship features fast, so the incentive is aligned against the thing the profession actually needs.

The correction, if it comes, will come the way most corrections come. Some team six months into an incident will realize nobody understands the module that broke, spend three weeks reverse-engineering their own code, and quietly institute an internal rule about explaining PRs in prose before shipping.

I don't think most orgs will get there before the damage compounds. The tools are too good at hiding the debt.

I honestly didn’t expect AI to blur the line between actual competence and just looking competent this much.

Few days ago I got a PR to review and the code looks like it was put through every AI tool under the sun from start to finish. There’s literally no way the guy who submitted it wrote it himself, or even pieced it together from smaller parts. The task requires a pretty deep understanding of the hardware and the codebase, and I know his work because I’ve reviewed his PRs plenty of times before.

You can also kind of see it in the code. One part is written in a very polished and profesional way, and then a small debugging section uses an incredibly naive solution. The kind of thing I’ve never seen an experienced dev do, but absolutely have seen from juniors writing basic stuff for the first time.

So what am I even supposed to do here??

To make things worse... AI tools are completely banned in this particular repo because of IP concerns...

I could ask him if he used AI, but he’ll probably just deny it. If I put comments on his PR on azure dev ops he can just easily answer them using AI once again... Am I supposed to go through the code line by line and quiz him on what it does? He probably won’t be able to explain half of it but then what?? What does that actually solve? No idea...I’m not his manager either, I’m just another dev on the team. Sure I have more experience but that doesn’t exactly give me the right to interrogate him.

It’s just so frustrating.

A few years ago, when someone was doing advanced work, you could usually assume they actually understood it. You could trust them and respect the skill behind it. Now someone can generate a bunch of impressive-looking code and act like they wrote it themselves.

And to be clear, I use AI too. A lot. I’m not against it at all. But I never submit code I can’t explain or defend. And if a larger part was generated because I couldn’t be bothered to write all the boilerplate by hand, I add a Co-Developed-By note instead of pretending it was all me.

AI definitely made development easier. I REALLY don’t miss writing repetitive boilerplate, it was just so boring... But I feel like we’re only starting to see the downside: people looking more competent than they are, teams losing the ability to tell who actually understands the work, and trust slowly becoming harder to maintain.

Maybe I’m just thinking about this in an old-fashioned way and need to adjust? Guys... feel free to tell me if I’m wrong.

As a programmer it’s increasingly hard to separate “It’s working well for us” and “We’ve lowered our standards to fit a productivity narrative”. LLMs are undoubtedly a good tool for programming, but it’s worth taking into account that the industry is notoriously shortsighted and corrupt in the sense of self-serving.

And here's the kicker

Curious what you think.

Earlier today there was a post asking if AI was useful at coding in real tech companies, and everyone who answered "I am in tech and yes it's used now" was massively downvoted.

I want to say this up front: I do not believe AI is good for society. It's funded by the ruling class in order to liquidate labor and it's not environmentally sustainable. It might not be economically sustainable either, which is dangerous because the entire economy is being gambled on it making money, which it is not.

Anyway here's my lived experience. I am a principal level ML engineer at a fortune 50 company. I was hand coding neural networks for natural language a decade ago. I got it into deep learning because I thought it was fascinating that machines could learn complex patterns from data. I've followed what is now called AI and experimented with it since its infancy, took Stanford classes on the math of transformer networks, etc. This was well before there was a hint it would morph into the monstrosity of capital that it has become.

In 2025 I was a vocal critic of the AI-assisted coding to the point where I pissed off senior execs by voicing my opinion in meetings.

Back then the people hyping AI were either attempting to sell it or they were novice/non-programmers who were blown away that they could get a simulacrum of a web app (riddled with bugs and security issues) without any domain knowledge. I thought it was hilarious, and pathetic.

Through a combination of masochism and curiosity I've exclusively used AI to write all code since November 2025. Which is why I knew that for real project work, AI would run roughshod on your entire codebase, mass producing a Winchester mansion of slop even when given simple tasks. It would take your thoughtful, hard-won abstractions and make 5 competing abstractions rather than leveraging what was there. It'd ignore instructions, break or even delete break core functionality to solve the "bugs" that it created (scare quotes because they weren't even bugs half the time, just incomplete understanding). It would write 1000 lines of slop to fix a problem that a one line change could fix.

It was a huge waste of time and money. I communicated this to management to their chagrin when they looked to me for advice.

In early 2026 improved models came out and the value proposition began to shift towards something somewhat usable. By spring, Claude Opus 4.6 was capable of writing real code, yet it took about as much effort as coding it hand for subpar results. It needed constant handholding and reminders, made a lot of mistakes, and would get looney tunes stupid at the end of a long session and start vandalizing its own progress. I used it as a novelty, but it was really frustrating.

By summer 2026 with the release of Fable and GPT 5.6 Sol, it is actually more than useful, it's good. No, really. It became less like an overzealous and intoxicated intern and more like a practical minded senior engineer. It follows directions, it reads your documentation, it can take notes and reference older notes, and given with connectors to git and internal wikis it can find the answers needed to integrate complicated systems, come up with a plan, independently implement it, write a suite of tests, run review agents, and take in their feedback all in one turn. It takes skill to be able to manage it, and more importantly domain knowledge to ask it for what is needed by stakeholders, but at this point it is much faster than I am. I run 5 sessions at once in different projects and it's probably 5x faster at getting production code than a human in each session.

Anyway, pretty much every engineer I know is using AI now because it works, even old timers who were very much opposed to it. Vibe coding has become the standard, now it's just called coding. Everyone is scrambling to stay relevant, and teams are changing their whole stack around AI agents - implementing code review agents, vibe coding tooling around AI agents, creating connectors and skills for every data resource possible: confluence, jira, outlook, GitHub, databases, etc. Teams are solving months of technical debt in a week, creating boilerplate code 10x faster, etc. Everyone is a bit nervous.

I don't say this because I want you to think AI is good for the world. It's not. The reason I'm writing this is to communicate that the tech has progressed. Even Linus Torvalds, the guy who made git and Linux uses it and accepts AI contributions to the Linux kernel.

There are many good reasons to hate AI and I think they converge in one place: elites are enticed by the prospect that it can obsolete the working class.

Personally, I don't want AI to be good. I don't even want it to exist. I hate that my job is babysitting bots. I miss the creativity of writing code. I don't want to lose my job in a few years when the AI systems we're frantically building are in place and management decides humans are too expensive.

The loom displaced workers and rightly caused a backlash among displaced workers, but it did so while producing cloth. Put your anger in the right place - the power hungry demons who want workers to be obsolete, not the workers who are forced to use it to be competitive in the job market.

It's tempting to think AI for coding is all bullshit and slop as it was until recently used to be; because AI is a net negative for humanity and the planet; because the value prop was non-existent and inflated by hype. However those don't mean that the tech itself is not improving. The Butlerian jihad can't come soon enough.

edit:

thanks to the wisdom of this community it has come to my attention that I am a disingenuous slop peddling brain rotted zombie that has no clue what I'm doing and pushes absolute bug riddled garbage to production and I'm too cowardly to post the code that is owned by my employer. thanks for the epiphany.

maybe it wasn't clear that using this tech effectively isn't easy and it takes a lot of tooling to work around common issues because out of the box it's not reliable enough. it's not worth much without domain experience and putting in the effort and time to figure out how to use it effectively. this isn't a place for sharing software engineering techniques, so I focused on the outcomes which may have oversimplified the technical part.

to the haters, I hope you take some time today to do something you enjoy instead of being miserable to others on the internet.

Is this post AI slop?

E: yes, this is AI slop. 🤣

I have been reading this exact same post basically since Opus dropped. "You think it's bad becase you're out of date. It only got good over the past X weeks."

The harness has come a long way, but senior engineer? I don't think so. It still seems to work well right up until it breaks down with very silly mistakes. I still have to review the code before I can trust it. Contrary to the current hype/ panic cycle, I actually think things have more or less platued. The gap between Sonnet and Opus was much wider than that between Opus and Fable.

The issue is that I've seen pretty much this exact post after every major AI "update". "It used to be bad but now it's good, I promise!"

If it wasn't true after the last one, the only assumption I can make is that it was posted by propaganda bots or people believing the propaganda then.

Occam's razor would suggest it's the same this time.

There has been a huge push by management to speed up velocity and get more done with AI. everyone now uses AI assisted code but it’s been overwhelming for me to read AI generated code where it produces massive lines of code and different files for one feature. how do you even review and understand AI generated code anymore? like I end up having AI explain to me what it is doing instead of reading line by line because it’s super overwhelinng

Thats very strange. I’m a very capable software developer, been working in the field since 2005, we’re in 2026 with tools that we couldn’t even dream about in 2015. On top of that, we can augment our knowledge with LLMs. It’s absolutely amazing! I can literally do much more in 1/10 of the time. And catch errors I wouldn’t otherwise and as quickly.

I am a student working on hobby projects and AI coding has become much harder to afford now.

Few months ago, I could comfortably work on projects using a $20 chatgpt plus subscription for the entire month. Now I keep running into usage limits much faster. For me, 5 hour limit get exhausted quickly in couple of prompts or 30-45 min of basic work and some other AI coding tools like antigravity have become much more restrictive also.

Now it feels like getting a similar level of productivity now requires spending $100+/month, which I simply cannot afford as a student 😞

I am looking for the best coding setup under $20/month that can handle

-Web development (MERN stack)

-Debugging and fixing errors

- Code reviews and refactoring

- Building medium level hobby projects

I would like coding quality close to chatgpt 5.5 without constantly worrying about rate limits.

Yes, probably most will tell that Claude Code is the best, but considering rate limits and price and overall approach of Anthropic towards their users I’m sick of it. Rate limits are cut, their coding agent is not that good comparing to others.

But what are the alternatives?

Cursor eats up the usage in a blink of an eye

Gemini models are bad at coding

I have two projects coming up React Native and Next.js, I need a reliable model and harness that will make the process of developing it fast, secure and overall painless

What are your thoughts? What pair model - harness works for you the best?

There are so many AI coding assistants available today—OpenAI ChatGPT, anthropic.com, github.com, cursor.com, and others. For those who code regularly, which one has improved your productivity the most?

Claude opus is impressive

No one is the best option

• Claude Code (Opus 4.6) for planning and implementation
• Codex (GPT-5.4) as the refiner and code reviewer.

This might just be me, but AI coding tools have made some PRs weirdly harder to review.

The code usually looks clean at first glance, but then you find small logic issues, overcomplicated helpers, weird edge cases, or tests that pass but don’t really prove anything. It feels less like reviewing someone’s thought process and more like trying to guess what the tool assumed.

I still use AI for coding, so this is not an anti-AI rant. It’s just that review feels different now.

Anyone else noticing this? How are you changing your review process?

Of all the Claude models I found Claude Opus 4.5/4.6 great for complex analysis and coding. For debugging purpose I found Claude Sonnet to be good as well. Didn't like Claude Haiku. Other than that, Codex is good for coding/review as well.

Open AI Codex using GPT-5.4 on the $200 a month plan.

I’ve never hit a rate limit, and I think it’s better than Claude.

I use the VS Code plugin.

Honestly, best results come from combining tools. One for planning (Claude), one for execution/refinement (GPT-5.4). No single model nails everything yet.

Been having conversations with engineering leaders recently, and something I keep hearing from them is, AI has gotten incredibly fast at code generation, but code reviews don't seem to be evolving at the same pace. Reviewers are increasingly having to piece together business and code context manually before they can confidently approve changes.

Got me thinking..... is this a fairly universal experience, or is it something that is just affects teams that usually don't prioritise reviews ?