Want to wade into the snowy surf of the abyss? Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid: Welcome to the Stubsack, your first port of call for learning fresh Awful youāll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cutānāpaste it into its own post ā thereās no quota for posting and the bar really isnāt that high.
The post Xitter web has spawned soo many āesotericā right wing freaks, but thereās no appropriate sneer-space for them. Iām talking redscare-ish, reality challenged āculture criticsā who write about everything but understand nothing. Iām talking about reply-guys who make the same 6 tweets about the same 3 subjects. Theyāre inescapable at this point, yet I donāt see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldnāt be surgeons because they didnāt believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I canāt escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard for starting this. Merry Christmas, happy Hannukah, and happy holidays in general!)
https://www.windowscentral.com/microsoft/windows-11/my-goal-is-to-eliminate-every-line-of-c-and-c-from-microsoft-by-2030-microsoft-bets-on-ai-to-finally-modernize-windows
wow, *and* algorithms? i didnāt think anyone had gotten that far
I suppose it was inevitable that the insufferable idiocy that software folk inflict on other fields would eventually be turned against their own kind.
https://xkcd.com/1831/
alt text
And xkcd comic.
Long haired woman: or field has been struggling with this problem for years!
Laptop wielding techbro: struggle no more! Iām here to solve it with algorithms.
6 months later:
Techbro: this is really hard Woman: You donāt say.
Q: what kind of algorithms does an AI produce
A: the bubble sort
God damn thatās good.
this made me cackle
very nice
Ah yes, I want to see how they eliminate C++ from the Windows Kernel ā code notoriously so horrific it breaks and reshapes the minds of all who gaze upon it ā with fucking āAIā. Iām sure autoplag will do just fine among the skulls and bones of Those Who Came Before
Before: You were eaten by a grue.
After: Oops, All Grues!
Thereās a lot going on here, but I started by trying to parse this sentence (assuming it wasnāt barfed out by an LLM). Iāve become dissatisfied lately with my own writing being too redundancy-filled and overwrought, showing Iām probably too far out of practice at serious writing, but what is this future Microsoft Fellow even trying to describe here?
at scale
so, ever watched Godzilla? and then did a twofer with a zombie movie? I think thatās essentially the plot here
so what youāre saying is undead kaiju, at scale
They now updated this to say it is just a research project and none of it will be going live. Pinky promise (ok, I added the pinky promise bit).
Not just pinkies, my friend, we are promising with all fingers, at scale!
All twelve fingers? Wow.
So maybe Iām just showing my lack of actual dev experience here, but isnāt āmaking code modifications algorithmically at scaleā kind of definitionally the opposite of good software engineering? Like, Iāll grant that stuff is complicated but if youāre making the same or similar changes at some massive scale doesnāt that suggest that you could save time, energy and mental effort by deduplicating somewhere?
This doesnāt directly answer your question but I guess I had a rant in me so I might as well post it. Oops.
Itās possible to write tools that make point changes or incremental changes with targeted algorithms in a well understood problem space that make safe or probably safe changes that get reviewed by humans.
Stuff like turning pointers into smart pointers, reducing string copying, reducing certain classes of runtime crashes, etc. You can do a lot of stuff if you hand-code C++ AST transformations using the clang / llvm tools.
Of course āletās eliminate 100% of our C code with a chatbotā isā¦ a whole other ballgame and sounds completely infeasible except in the happiest of happy paths.
In my experience even simple LLM changes are wrong somewhere around half the time. Often in disturbingly subtle ways that take an expert to spot. Also in my experience if someone reviews LLM code they also tend to just rubber stamp it. So multiply that across thousands of changes and itās a recipe for disaster.
And what about third party libraries? Corporate code bases are built on mountains of MIT licensed C and C++ code, but surely they wonāt all switch languages. Which means theyāll have a bunch of leaf code in C++ and either need a C++ compatible target language, or have to call all the C++ code via subprocess / C ABI / or cross-language wrappers. The former is fine in theory, but Iām not aware of any suitable languages today. The latter can have a huge impact on performance if too much data needs to be serialized and deserialized across this boundary.
Windows in particular also has decades of baked in behavior that programs depend on. Any change in those assumptions and whoops some of your favorite retro windows games donāt work anymore!
In the worst case theyād end up with a big pile of spaghetti that mostly works as it does today but that introduces some extra bugs, is full of code that no one understands, and is completely impossible to change or maintain.
In the best case theyāre mainly using āAIā for marketing purposes, will try to achieve their goals using more or less conventional means, and will ultimately fall short (hopefully not wreaking too much havoc in the progress) and give up halfway and declare the whole thing a glorious success.
Either way ultimately if any kind of large scale rearchitecting that isnāt seen through to the end will cause the codebase to have layers. Thereās the shiny new approach (never finished), the horrors that lie just beneath (also never finished), and the horrors that lie just beneath the horrors (probably written circa 2003). Any new employees start by being told about the shiny new parts. The company will keep a dwindling cohort of people in some dusty corner of the company who have been around long enough to know how the decades of failed code architecture attempts are duct-taped together.
I just want to add: sailorās reference to āexpertā here is no joke. the amount of wild and subtle UB (undefined behaviour) you get in the C family is extremely high-knowledge stuff. itās the sort of stuff that has in recent years become fashionable to describe as ācursedā, and often with good reason
LLMs being bad at precision and detail is as perfect an antithesis in that picture as I am capable of conceiving. so any thought of a project like this that pairs LLMs (or, more broadly, any of the current generative family of nonsense) as a dependency in itās implementation is just damn wild to me
(and just incase: this post is not an opportunity to quibble about PLT and about what be or become possible.)
Some of the horrors are also going to be load bearing for some fixes people dont properly realize because the space of computers which can run windows is so vast.
Think something like that happend with twitter, when Musk did his impression of a bull in a china store at the stack, they cut out some code which millions of Indians, who used old phones, needed to access the twitter app.
The short answer is no. Outside of this context, Iād say the idea of ācode modifications algorithmically at scaleā is the intersection of code generation and code analysis, all of which are integral parts of modern development. That being said, using LLMs to perform large scale refactors is stupid.
I think Iām with Hauntedās intuition in that I donāt really buy code generation. (As in automatic code generation.) My understanding was you build a thing that takes some config and poops out code that does certain behaviour. But could you not build a thing instead, that does the behaviour directly?
I know people who worked on a system like that, and maybe thereās niches where it makes sense. Just seems like it was a SW architecture fad 20 years ago, and some systems are locked into that know. It doesnāt seem like the pinnacle of engineering to me.
@jaschop
āBut could you not build a thing instead, that does the behaviour directly?ā
Back in the day NeXTās Interface Builder let you connect up and configure āliveā UI objects, and then freeze-dry them to a file, which would be rehydrated at runtime to recreate those objects (or copies of them if you needed more.)
Apple kept this for a while but doesnāt really do it anymore. There were complications with version control, etc.
I have always felt like NeXT/OS X Interface Builder has serious āpath not takenā energy, but the fact that OpenStep/Cocoa failed to become a generalized multiplatform API, as well as the version control issues for the .nib format (never gave much thought to that, but it makes sense) sadly doomed it. And most mobile apps are glorified web pages, each with their own bespoke interface to maintain ābrand identity,ā so it could be argued thereās less than zero demand there for the flexibility (and complexity!) that Interface Builder could enable.
Unfortunately, the terms ācode generationā and āautomatic code generationā are too broad to make any sort of value judgment about their constituents. And I think evaluating software in terms of good or bad engineering is very context-dependent.
To speak to the ideas that have been brought up:
āmaking the same or similar changes at some massive scale [ā¦] suggest[s] that you could save time, energy and mental effort by deduplicating somewhereā
So there are many examples of this in real code bases, ranging everywhere from simple to complex changes.
Giving a complex example here isā¦ difficult. Anyway, I hope Iāve been able to illustrate that sometimes you have to use ācode generationā because itās the right tool for the job.
āMy understanding was you build a thing that takes some config and poops out code that does certain behaviour.ā
This hypothetical is a few degrees too abstract. This describes a compiler, for example, where the āconfigā is source code and ācode that does certain behaviourā is the resulting machine code. Yes, you can directly write machine code, but at that point, you probably arenāt doing software engineering at all.
I know that you probably donāt mean a compiler. But unfortunately, itās compilers all the way down. Software is just layers upon layers of abstraction.
Hereās an example: a web page. (NB I am not a web dev and will get details wrong here) You can write html and javascript by hand, but most of the time you donāt do that. Instead, you rely on a web framework and templates to generate the html/javascript for you. I feel like that fits the config concept youāre describing. In this case, the templates and framework (and common css between pages) double as de-duplication.
This is like the entire fucking genAI-for-coding discourse. Every time someone talks about LLMs in lieu of proper static analysis Iām just likeā¦ Yes, the things you say are of the shape of something real and useful. No, LLMs canāt do it. Have you tried applying your efforts to something that isnāt stupid?
Hmm, sounds like you are suggesting proper static analysis, at scale
If thereās one thing that coding LLMs do āwellā, itās expose the need in frameworks for code generation. All of the enterprise applications I have worked on in modernity were by volume mostly boilerplate and glue. If a statistically significant portion of a code base is boilerplate and glue, then the magical statistical machine will mirror that.
LLMs may simulate filling this need in some cases but of course are spitting out statistically mid code.
Unfortunately, committing engineering effort to write code that generates code in a reliable fashion doesnāt really capture the imagination of money or else we would be doing that instead of feeding GPUs shit and waiting for digital God to spring forth.
Throw in the rust evangelism and you have a techtakes turducken
If you want a warm and fuzzy Christmas contemplation, imagine turducken production at scale