cross-posted from: https://programming.dev/post/36983916
Freund wasn’t looking for a backdoor when he noticed SSH connections to his Debian testing system taking 500 milliseconds longer than usual. As a database engineer benchmarking PostgreSQL performance, he initially dismissed the anomaly. But the engineer’s curiosity persisted.
The backdoor’s technical sophistication was breathtaking. Hidden across multiple stages, from modified build scripts that only activated under specific conditions to obfuscated binary payloads concealed in test files, the attack hijacked SSH authentication through an intricate chain of library dependencies. When triggered, it would grant the attacker complete remote access to any targeted system, bypassing all authentication and leaving no trace in logs.
The backdoored versions 5.6.0 and 5.6.1 had been released in February and March 2024, infiltrating development versions of Fedora, Debian, openSUSE, and Arch Linux. Ubuntu’s upcoming 24.04 LTS release, which would have deployed to millions of production systems, was mere weeks away.
The technical backdoor was merely the final act of a three-year psychological operation that began not with code, but with studying a vulnerable human being.
The takeaway here
Open source has not failed us, we have failed open source. The xz backdoor revealed not a broken development model but a broken economic model, one where we socialize the costs of critical infrastructure while privatizing the benefits. The attack succeeded not because open source is vulnerable, but because we’ve made open source maintainers vulnerable by systematically underfunding the human infrastructure that creates the technical infrastructure we all depend on.
The good part of the backdoor to have been found out is that now it’s much harder to pull that again, be it on the same project, or in projects from people that followed through the issue and/or the news - and with how much noise was made, I’d expect quite a lot more of people than the usual to have seen something about.
There are large areas of open source that don’t rely on volunteer labour because companies with a vested interest pay people to work on them. They tend to be the obvious large projects that are continuously developed and gain new features. The trouble with something like xz is it was mostly “done” (as in it did the thing it was intended to do) but still needed maintenance to address the minor niggles, bug reports and updates to tooling and dependencies.
The foundations could do a better job here of supporting the maintainers. After Heartbleed the Linux Foundation started the Core Infrastructure Initiative to help fund those under recognised projects. I would hope the people running that could be more proactive identifying those critical understaffed components.
Edit I think it’s now called the Open Source Security Foundation: https://openssf.org/
The amount of time needed to do this to an opensource project compared to the time needed to do this to closed source software makes this article almost completely meaningless, especially since it’s a recent article and not when this news was still news. It’s clickbait and useless.
I feel like this is missing a big point of the article.
The vulnerability that the xz backdoor attempt revealed was the developers. The elephant in the room is that for someone capable of writing and maintaining a program so important to modern technical infrastructure, we’re making sure to hang them out to dry. When they burn out because their ‘hobby’ becomes too emotionally draining (either because of a campaign to wear them down intentionally or fully naturally) someone will be waiting to take control. Who can you trust? Here, we see someone attempted (and nearly succeeded) a multi-year effort to establish themselves as a trusted member of the development community who was faking it all along. With the advent of LLMs, it’s going to be even harder to tell if someone is trustworthy, or just a long-running LLM deception campaign.
Maybe, we should treat the people we rely on for these tools a little better for how much they contribute to modern tech infrastructure?
And I’ll point out that’s less aimed at the individuals who use tech, and more at the multi-billion-dollar multi-national tech companies that make money hand over fist using the work others donate.
This talks about one issue. You seem to be confident that this one case is representative of the whole FOSS space? I am not.
Can you elaborate how it would be much easier in closed source software? Because as far as I can see, it’s different. In most cases, you need an actual person instead of an online persona, pass interview and contracting, and then you’re still “the new guy” or Junior in the company or project. It’s not like closed off from public eyes means anyone can do anything without any eyes.
Since paranoia seems to be a virtue in this case, here i go :
While your username is essentially XZ, you are advocating for the community to dismissing an analysis addressing the root causes that made it possible for a backdoor in the xz compression utility to be made.
Hummm … how did you come to choose that username ? … and were there any other articles written previously identifying theses social and funding problems in software infrastructure ?
Probably i am completely wrong since i know close to nothing about this subject.
Thanks for your time and attention.
