I work on a relatively niche open-source project, where we hardly ever get outside contributions, so we’ve only gotten one AI-generated pull request so far, but that one was just incredible.

It just failed immediately with a compile error, meaning the person who opened it, had never even tried to compile the code. Like, that’s the one unequivocal problem with AI-generated code, that folks will think they can contribute without understanding anything about the project setup or context or really even what they’re supposed to be doing.

I’m pretty sure, they just copy-pasted the issue text we had written into a chatbot and hoped it would cough up the right solution. But the issue text didn’t describe all necessary steps. Anyone looking to actually solve that issue would have had to enter into a discussion with us to work out those details, like a human would’ve probably understood.

Meanwhile, the LLM changed maybe ⅕ of the files, it should’ve changed, and that in ways that made it look plausible, but were far away from solving any of the real complexities or even solving the simple stuff consistently.
It’s just genuinely not even a start to a solution, because the complexity isn’t in producing the code. The complexity lays in thinking through a solution, which is just not something an LLM can do.

The general comments that Ben received were that experienced developers can use AI for coding with positive results because they know what they’re doing. But AI coding gives awful results when it’s used by an inexperienced developer.

I wouldn’t even go so far as to call myself an experienced developer, and this has still been my experience of using AI as a coding assistant. There are definitely situations where you need to bang out a simple script or function and you know that the AI can throw it together faster than you can knock it out yourself, especially when it’s something off to the side of the stuff you’re normally doing, so you’d probably need to read up on some libraries and APIs that you don’t regularly work with. But then you look at the code it spits out, and you’re constantly having to correct minor errors. Not just bugs, but downright bizarre ways of solving problems. In one case ChatGPT wrapped an entire function up as a single threaded future object just to put a timeout on it, even though the function was a subprocess call that already has its own built in timeout option. I run into stuff like that constantly. It also has a really hard time understanding the current state of an evolving language, because in their effort to harvest training data they fed it everything, so when implementing relatively new features like asyncio it tends to flip back and forth between current recommended approaches and outdated methods.

The idea that AI can allow you to code if you’re not already a coder is lunacy. The idea that it could ever successfully do code review is flat out demented. And the problem is that companies will go all in on it anyway, and hire a bunch of junior devs to review and fix the AI code without paying them as much because “Oh, the AI is doing all the work.” They’ll get shit code out, and then they’ll be surprised when it runs badly and breaks constantly.