Matt Garman sees a shift in software development as AI automates coding, telling staff to enhance product-management skills to stay competitive.
Matt Garman sees a shift in software development as AI automates coding, telling staff to enhance product-management skills to stay competitive.
I’m amazed how overstated llm ability to program is. I keep trying, and I’ve yet to have any model output so much as a single function that ran correctly without modification. Beyond that it has made up APIs when I’ve asked about approaches to problems, and I’ve given it code to find bugs and memory issues I think are fairly obvious and it fails every time.
Could it be prompts by devs are different from lay folk? For example, “write a website for selling shoes” would give a more complete result compared to “write a single page app with a postgres back end with TLS encryption” (or whatever), which would add more constraints & reduce the pool of code the AI steals from.
It really depends on the domain. E.g. I wrote a parser and copilot was tremendously useful, presumably because there are a gazillion examples on the internet.
Another case where it saved me literally hours was spawning a subprocess in C++ and capturing stdin/out. It didn’t get it 100% right but it saved me so much time looking up how to do it and the names of functions etc.
Today I’m trying to write a custom image format, and it is pretty useless for that task, presumably because nobody else has done it before.
This makes sense, I’ve largely been trying to use it for things I do regularly, and I’m pretty senior, having been in the industry for some time, so I tend not to be asking the questions that will have a million examples out there. But then again, these are the sorts of things that it will need to be able to do to replace people in industry.
Me too, but this was C++ where there isn’t a strong culture of making high quality libraries available for everything (because it doesn’t have a proper package manager, at least until very recently), so you do end up having to reinvent the wheel a fair bit.
And sometimes you just need things a bit different to what other people have done. So even though there are a gazillion expression parsers out there (so the LLM understood it pretty well) there are hardly any that support 64-bit integers. But that’s a small enough difference that it can deal with it.