This vmalert tool is just an interface to another, even more complicated piece of software.
Not really just an interface. It is a pluggable service that connects to one or more TSDBs, performs periodic queries, and notifies another service when certain thresholds are exceeded. So with all those configuration options, why is the standalone binary expected to have defaults that may sound same on one system but insane in a different one? If the author wants out of the box configuration they could have gotten the helm chart or the operator and then that would be taken care of. But they seem to be deathly allergic to yaml, so I guess that wonāt happen.
Since when do Unix tools output 3,000 word long usage info? Even GNU tools donāt even come closeā¦
HTTP and REST are very strange ways to accomplish IPC or networked communication on Unix when someone would normally accomplish the same thing with signals, POSIX IPC, a simpler protocol over TCP with BSD sockets, or any other thing already in the base system.
Until you need authentication, out of the box libraries, observability instrumentation, interoperabilityā¦ which can be done much more easily with a mature communication protocol like HTTP. And for those chasing the bleeding edge thereās gRPC.
I would hope the filesystems you use are āhigh availabilityā lol
Theyāre not, and Iām disappointed that you think they are. Any individual filesystem is a single point of failure. High availability lets me take down an entire system with zero service disruption because thereās redundancy, load balancing, disaster recoveryā¦
the humble file metaphor can still represent these concepts
They can, and they still doā¦ Inside the container.
Itās not a lack of skill as your comment implies but rather a rejection of this way of doing things.
Which I understand, I honestly do. I rejected containers for a (relatively) long time myself, and the argument that the author is making echoes what I would have said about containers. Which is why I believe myself to be justified in making the argument that I did, because rejecting a way of doing things based on preconception is a lack of flexibility, and in cloud ecosystems that translates to a lack of skill.
You just said that this software was much more complex than Unix tools.
Thatās the problem. The reason Unix became so popular is because it has a highly integrated design and a few very reused abstractions. A lot of simple parts build up in predictable ways to accomplish big things. The complexity is spread out and minimized. The traditional Unix way of doing things is definitely very outdated though. A modern Unix system is like a 100 story skyscraper with the bottom 20 floors nearly abandoned.
Kubernetes and its users would probably be happier if it was used to manage a completely different operating system. In the end, Kubernetes is trying to impose a semi-distributed model of computation on a very NOT distributed operating system to the detriment of system complexity, maintainability, and security.
Until you need authentication, out of the box libraries, observability instrumentation, interoperabilityā¦ which can be done much more easily with a mature communication protocol like HTTP.
I agree that universal protocols capable of handling these things are definitely useful. This is why the authors of Unix moved away from communication and protocols that only function on a single system when they were developing Plan 9 and developed the Plan 9 Filesystem Protocol as the universal system ābusā protocol capable of working over networks and on the same physical system. I donāt bring this up to be an evangelist. I just want to emphasize that there are alternative ways of doing things. 9P is much simpler and more elegant than HTTP. Also, many of the people who worked on Plan 9 ended up working for Google and having some influence over the design of things there.
Theyāre not, and Iām disappointed that you think they are. Any individual filesystem is a single point of failure. High availability lets me take down an entire system with zero service disruption because thereās redundancy, load balancing, disaster recoveryā¦
A filesystem does not exclusively mean an on-disk representation of a tree of files with a single physical point of origin. A filesystem can be just as āhighly availableā and distributed as any other way of representing resources of a system if not more so because of its abstractness. Also, youāre ādisappointedā in me? Lmao
They can, and they still doā¦ Inside the container.
And how do you manage containers? With bespoke tools and infrastructure removed from the file abstraction. Which is another way Kubernetes is removed from the Unix way of doing things. Unless Iām mistaken, itās been a long time since I touched Kubernetes.
because rejecting a way of doing things based on preconception is a lack of flexibility
Itās not a preconception. They engaged with your way of doing things and didnāt like it.
in cloud ecosystems that translates to a lack of skill.
By what standard? The standard of you and your employer? In general, you seem to be under the impression that the conventional hegemonic corporate ācloudā way of doing things is the only correct way and that everyone else is unskilled and not flexible.
Iām not saying that this approach doesnāt have merits, just that you should be more open-minded and not judge everyone else seeking a different path to the conventional model of cloud/distributed computing as naive, unskilled people making ābad-faith argumentsā.
A lot of simple parts build up in predictable ways to accomplish big things. The complexity is spread out and minimized.
This has always felt untrue to me. The command line has always been simple parts. However we cannot argue that this applies to all Unix-like systems: The monolithic Linux kernel, Kerberos, httpd, SAMBA, X windowing, heck even OpenSSL. Thereās many examples of tooling built on top of Unix systems that donāt follow that philosophy.
The traditional Unix way of doing things is definitely very outdated though.
Depends on what you mean. āEverything is a fileā? Sure, that metaphor can be put to rest. āLow coupling, high cohesionā? Thatās even more valid now for cloud architectures. You cannot scale a monolith efficiently these days.
In the end, Kubernetes is trying to impose a semi-distributed model of computation on a very NOT distributed operating system to the detriment of system complexity, maintainability, and security.
Kubernetes is more complex than a single Unix system. It is less complex than manually configuring multiple systems to give the same benefits of Kubernetes in terms of automatic reconciliation, failure recovery, and declarative configuration. This is because those three are first class citizens in Kubernetes, whereas theyāre just afterthoughts in traditional systems. This also makes Kubernetes much more maintainable and secure. Every workload is containerized, every workload has predeclared conditions under which it should run. If it drifts out of those parameters Kubernetes automatically corrects that (when it comes to reconciliation) and/or blocks the undesirable behaviour (security). And Kubernetes keeps an audit trail for its actions, something that again in Unix land is an optional feature.
If you work with the Kubernetes model then you spend 10% more time setting things up and 90% less time maintaining things.
9P is much simpler and more elegant than HTTP
It also has negligible adoption compared to HTTP. And unless it provides an order of magnitude advantage over HTTP, then itās going to be unlikely that developers will use it. Consider git vs mercurial. Is the latter better than git? Almost certainly. Is it 10x better? No, and thatās why it finds it hard to gain traction against git.
A filesystem does not exclusively mean an on-disk representation of a tree of files with a single physical point of origin. A filesystem can be just as āhighly availableā and distributed as any other way of representing resources of a system if not more so because of its abstractness.
Even an online filesystem does not guarantee high availability. If I want highly available data I still need to have replication, leader election, load balancing, failure detection, traffic routing, and geographic distribution. You donāt do those in the filesystem layer, you do them in the application layer.
Also, youāre ādisappointedā in me? Lmao
Nice ad hominem. I guess itās rules for thee, but not for me.
And how do you manage containers? With bespoke tools and infrastructure removed from the file abstraction. Which is another way Kubernetes is removed from the Unix way of doing things. Unless Iām mistaken, itās been a long time since I touched Kubernetes.
So whatās the problem? Didnāt you just say that the Unix way of doing things is outdated? Let the CSI plugin handle the filesystem side if things, and let Kubernetes focus on the workload scheduling and reconciliation.
Itās not a preconception. They engaged with your way of doing things and didnāt like it.
Dismissal based on flawed anecdote is preconception.
By what standard? The standard of you and your employer? In general, you seem to be under the impression that the conventional hegemonic corporate ācloudā way of doing things is the only correct way and that everyone else is unskilled and not flexible.
No. Iām not married to the ācloudā way of doing things. But if someone comes to me and says āHey boblin, we want to implement something on system foo, can you help us?ā and I am not used to doing things the foo way I will say āIām not familiar with it but letās talk about your requirements, and why you chose fooā instead of āfoo is for bureaucrats, I donāt want to use itā. Iād rather hire an open-mined junior than a gray-bearded Unix wizard that dismisses anything unfamilar. And I will also be the first person to reject use cases for Kubernetes when they do not make sense.
just that you should be more open-minded and not judge everyone else seeking a different path to the conventional model of cloud/distributed computing as naive, unskilled people making ābad-faith argumentsā.
There are scenarios where cloud compute just does not make sense, like HPC. If the author had led with something like that, then they would have made a better argument. But instead they went for
cloud-native tooling feels like itās meant for bureaucrats in well-paid jobs,
,
In the 90s my school taught us files and folders when we were 8 years old
, and
When you finally specify all those flags, neatly namespaced with . to make it feel all so very organised, you feel like youāve achieved something. Sunk-cost fallacy kicks in: look at all those flags that Iāve tuned just so - it must be robust and performant!
This has always felt untrue to me. The command line has always been simple parts. However we cannot argue that this applies to all Unix-like systems: The monolithic Linux kernel, Kerberos, httpd, SAMBA, X windowing, heck even OpenSSL. Thereās many examples of tooling built on top of Unix systems that donāt follow that philosophy.
I can see why you would come to think that if all youāve been exposed to is Linux and its orbiting ecosystem. I agree with you that modern Unix has failed to live up to its ideals. Even its creators began to see its limitations in the late 80s and began to develop a whole new system from scratch.
Depends on what you mean. āEverything is a fileā? Sure, that metaphor can be put to rest.
That was never true in the first place. Very few things under Unix are actually represented as files (though credit to Linux for pursuing this idea in kernel-space more than most). But Plan 9 shows us this metaphor is worth expanding and exploring in how it can accomplish being a reliable, performant distributed operating system with a fraction of the code required by other systems.
Kubernetes is more complex than a single Unix system. It is less complex than manually configuring multiple systems to give the same benefits of Kubernetes in terms of automatic reconciliation, failure recovery, and declarative configuration. This is because those three are first class citizens in Kubernetes, whereas theyāre just afterthoughts in traditional systems. This also makes Kubernetes much more maintainable and secure. Every workload is containerized, every workload has predeclared conditions under which it should run. If it drifts out of those parameters Kubernetes automatically corrects that (when it comes to reconciliation) and/or blocks the undesirable behaviour (security). And Kubernetes keeps an audit trail for its actions, something that again in Unix land is an optional feature.
My point is Kubernetes is a hack (a useful hack!) to synchronize multiple separate, different systems in certain ways. It cannot provide anything close to something like a single system image and it canāt bridge the discrete model of computation that Unix assumes.
This also makes Kubernetes much more maintainable and secure. Every workload is containerized, every workload has predeclared conditions under which it should run. If it drifts out of those parameters Kubernetes automatically corrects that (when it comes to reconciliation) and/or blocks the undesirable behaviour (security). And Kubernetes keeps an audit trail for its actions, something that again in Unix land is an optional feature.
All these features require a lot of code and complexity to maintain (latest info I can find is almost 2 million as of 2018). Ideally, Kubernetes is capable of what you said, in the same way that ideally programs canāt violate Unix filesystem DAC or other user permissions but in practice every line of code is another opportunity for something to go wrongā¦
Just because something has more security features doesnāt mean itās actually secure. Or that itās maintainable without a company with thousands of engineers and tons of money maintaining for you. Keeping you in a dependent relationship.
It also has negligible adoption compared to HTTP. And unless it provides an order of magnitude advantage over HTTP, then itās going to be unlikely that developers will use it. Consider git vs mercurial. Is the latter better than git? Almost certainly. Is it 10x better? No, and thatās why it finds it hard to gain traction against git.
So? I donāt expect many of these ideas will be adopted in the mainstream under the monopoly-capitalist market system. Itās way more profitable to keep selling support to manage sprawling and complex systems that require armies of software engineers to upkeep. I think if state investment or public research in general becomes relevant again maybe these ideas will be investigated and adopted for their technical merit.
Even an online filesystem does not guarantee high availability. If I want highly available data I still need to have replication, leader election, load balancing, failure detection, traffic routing, and geographic distribution. You donāt do those in the filesystem layer, you do them in the application layer.
āHighly availableā is carrying a lot of weight there lol. If we can move some of these qualities into a filesystem layer (which is a userspace application on some systems) and get these benefits for free for all data, why shouldnāt we? The filesystem layer and application layer are not 2 fundamentally separate unrelated parts of a whole.
Nice ad hominem. I guess itās rules for thee, but not for me.
Lol, stop being condescending and I wonāt respond in kind.
So whatās the problem? Didnāt you just say that the Unix way of doing things is outdated?
I think the reason the Unix way of doing things is outdated is cuz it didnāt go far enough!
Dismissal based on flawed anecdote is preconception.
What? lol
Itās not a flawed anecdote or a preconception. They had their own personal experience with a cloud tool and didnāt like it.
You canāt someone into liking something.
Iād rather hire an open-mined junior than a gray-bearded Unix wizard that dismisses anything unfamilar.
Iām not a gray-bearded Unix wizard and Iām not dismissing these tools because theyāre unfamiliar. I have technical criticism of them and their approach. I think the OP feels the same way.
The assumption among certain computer touchers is that you canāt use Kubernetes or ācloudā tools and not come away loving them. So if someone doesnāt like them they must not really understand them!
Itās hard to not take that as bad faith.
They probably couldāve said it nicer. Itās still no excuse to dismiss criticism because you didnāt like the tone.
I think Kubernetes has its uses, for now. But itās still a fundamentally limited and harmful (because of its monopolistic maintainers/creators) way to do a kind of distributed computing. I donāt think anyone is coming for you to take your Kubernetes thoughā¦
My point is Kubernetes is a hack (a useful hack!) to synchronize multiple separate, different systems in certain ways. It cannot provide anything close to something like a single system image and it canāt bridge the discrete model of computation that Unix assumes.
Kubernetes is not intended to provide anything like a single system image. Itās a workload orchestration system, not an operating system. Given a compatible interface (a runtime) Kubernetes can in theory distribute workloads to any OS.
All these features require a lot of code and complexity to maintain (latest info I can find is almost 2 million as of 2018). Ideally, Kubernetes is capable of what you said, in the same way that ideally programs canāt violate Unix filesystem DAC or other user permissions but in practice every line of code is another opportunity for something to go wrongā¦
Just because something has more security features doesnāt mean itās actually secure. Or that itās maintainable without a company with thousands of engineers and tons of money maintaining for you. Keeping you in a dependent relationship.
Iām not going to argue that Kubernetes is not complex. But as I stated previously Kubernetes as a bespoke ecosystem is less complex than configuring the same features with decoupled systems. The requirements for an orchestrator and the challenges (technical, security, human, etc) to manage said orchestrator are higher. All else being equal, Kubernetes has implemented this in a very lean way, delegating networking, storage, and runtime to pluggable providers on the left, and delegating non-basic workload aspects to operators on the right. Itās this extensibility that makes it both popular with operators and makes it appear daunting to a layperson. And going back to security, is has provably shown to have a reduced attack surface when managed by a competent operator.
So? I donāt expect many of these ideas will be adopted in the mainstream under the monopoly-capitalist market system. Itās way more profitable to keep selling support to manage sprawling and complex systems that require armies of software engineers to upkeep. I think if state investment or public research in general becomes relevant again maybe these ideas will be investigated and adopted for their technical merit.
So youāreā¦ what, dismissing HTTP because it has been adopted by capitalist market systems? Are you going to dismiss the Fediverse for using HTTP? What about widely adopted protocols? DNS, BGP, IPv4/6, etc?
How about we bring this part of the discussion back to the roots? You said that HTTP and REST as communication protocols seemed strange to you because Unix has other primitives. I pointed out that those primitives do not address many modern client-server communication requirements. You did not refute that, but you said, and I paraphrase ā9P did it betterā. I refrain from commenting on that because thereās no comparative implementation of complex Internet-based systems in 9P. I did state though that even if 9P is superior, as you claim, it did not win out in the end. Thereās plenty of precedents for this: Betamax-VHS, git-mercurial, etc.
āHighly availableā is carrying a lot of weight there lol. If we can move some of these qualities into a filesystem layer (which is a userspace application on some systems) and get these benefits for free for all data, why shouldnāt we? The filesystem layer and application layer are not 2 fundamentally separate unrelated parts of a whole.
(My emphasis) Itās not free though. Thereās an overhead for doing this, and you end up doing things in-filesystem that have no business being there.
Itās not a flawed anecdote or a preconception. They had their own personal experience with a cloud tool and didnāt like it.
*Ahem*:
āNobody really uses Kubernetes for day-to-day work, and it shows.ā
That is not an experience, itās a provably wrong statement.
The assumption among certain computer touchers is that you canāt use Kubernetes or ācloudā tools and not come away loving them. So if someone doesnāt like them they must not really understand them!
Thatās a very weird assumption, and itās the first time Iāve heard it. Can you provide a source? Because in my experience the opposite is the case - thereās no community more critical of Kubernetesā flaws than their developers/users themselves.
They probably couldāve said it nicer. Itās still no excuse to dismiss criticism because you didnāt like the tone.
I dismissed the criticism because it makes an appeal to pathos, not to logos. Like I said, thereās plenty of valid technical criticisms of Kubernetes, and even an argument on the basis of ethics (like youāre making) is more engaging.
I think Kubernetes has its uses, for now. But itās still a fundamentally limited and harmful (because of its monopolistic maintainers/creators) way to do a kind of distributed computing. I donāt think anyone is coming for you to take your Kubernetes thoughā¦
No my Kubernetes. I use it because itās academically interesting, and because it does the tasks it is meant to do better than most alternatives. But if CNCF were to implode today and Kubernetes became no longer practical to use then I would just pivot to another system.
Iām not going to argue whether itās a harmful way of doing distributed computing based on their maintainers/pedrigee. Thatās a longer philosophical discussion than I suspect neither you or I have time for.
You just said that this software was much more complex than Unix tools
Probably need to keep in mind incidental versus essential complexity here.
So with all those configuration options, why is the standalone binary expected to have defaults that may sound same on one system but insane in a different one?
Because this is how much of what we use already is implemented.
Significant effort goes in to portability, interoperability and
balancing compromises. When Iām doing software development e.g.
writing HTTP APIs (of which I apparently know nothing about ;) ) - I
feel like Iāve got a responsibility to carefully balance what I expose
as some user-configurable thing versus something managed internally by
the application. Sometimes, thankfully, the application doesnāt even
have to think about it al all - like what TCP flags to set when I dial
some service.
You bring up containers which is a great example of some cool features
provided by the Linux kernel to solve interesting problems. If youāre
interested, have a look at FreeBSDās Jails, Plan 9 and LXC. Compare
the interface to all these systems, both at the library level and
userspace, and compare the applications developed using those systems.
How easy is it to get going? How much do I need to keep in my head
when using these features? Docker, Kubernetes, and the rest all have made different tradeoffs and compromises.
Another one I think about is SQLite. Some seriously clever smarts.
Huge numbers of people donāt know anything about for-loops, C, or
B-Trees but can read & write SQL. Thatās technology at its best.
Consider how difficult it could be to, say, start a car in all the
different operating conditions it is expected to be used in. But we
never think about it.
We as tech people pride ourselves on familiarity with esoteric detail,
but it doesnāt need to be like this. Nor does memorising it all have
anything to do with āskillā.
What Iām struggling with are thoughts of significant vested commercial
interest in exposing this kind of detail, fuelling multi-billion
dollar service industries. Feelings of being an outsider despite
understanding how it all fits together.
It is a pluggable service that connects to one or more TSDBs, performs periodic queries, and notifies another service when certain thresholds are exceeded.
Have you ever written this kind of software before?
It sounds like you are comfortable with the status quo of this part of
the software industry, and Iām truly jealous! If youāve got any tips on dealing with this kind of stuff you can find my email at https://www.olowe.co/about.html
Thanks :)
Probably need to keep in mind incidental versus essential complexity here.
Go onā¦
Because this is how much of what we use already is implemented. Significant effort goes in to portability, interoperability and balancing compromises. When Iām doing software development e.g. writing HTTP APIs (of which I apparently know nothing about ;) ) - I feel like Iāve got a responsibility to carefully balance what I expose as some user-configurable thing versus something managed internally by the application. Sometimes, thankfully, the application doesnāt even have to think about it al all - like what TCP flags to set when I dial some service.
In the case of vmalert, the binary makes no assumptions as to default behaviour because it was not meant to be run standalone. It comes as part of a container with specific environment variables, which in turn is packaged as a Helm chart which has sane configurations. Taking the vmalert binary by itself is like taking a kerberos server binary without its libraries and config files in /etc files and complaining that itās not working.
You bring up containers which is a great example of some cool features provided by the Linux kernel to solve interesting problems. If youāre interested, have a look at FreeBSDās Jails, Plan 9 and LXC. Compare the interface to all these systems, both at the library level and userspace, and compare the applications developed using those systems. How easy is it to get going? How much do I need to keep in my head when using these features? Docker, Kubernetes, and the rest all have made different tradeoffs and compromises.
I am very well versed in jails, chroot, openvz, LXC, etc. OCI containers are in a different class - donāt think of them as an OS-like environment, think of them as a self-contained, packaged service. Docker is then one example of a runtime runtime on which those services run, and Kubernetes is an orchestrator that managed containers in runtimes. And yes, there are some tradeoffs and compromises, but those are well within the bounds of the Pareto principle - remove the 10% long tail of features on the host, reduce user-facing complexity by 90%.
Another one I think about is SQLite. Some seriously clever smarts. Huge numbers of people donāt know anything about for-loops, C, or B-Trees but can read & write SQL. Thatās technology at its best.
Are you arguing that Kubernetes doesnāt do that for you? Because with Kubernetes I can say ārun the service in this container with these settings and so many replicasā, attach some conditions like āstop sending traffic to any one container that takes longer than N seconds to respondā and ārestart the container if a certain command returns an errorā, and just let it run. I can do a rolling upgrade of the nodes and Kubernetes will reschedule the containers on any other available node, it can load balance traffic, I can update the spec of a deployment and Kubernetes will do a zero-downtime upgrade for me. Try implementing the same on a Unix system. Youād need a way to push configs (Ansible, Puppet, etc?). You need load balancing and leader election (Keepalived?). You need error detection. You need DNS. You need to run the services. You need to ensure thereās no library conflict. Thereās a LOT of complexity that a Kubernetes user does not need to worry about any more. Tell me thatās not serious smarts and technology at its best.
What Iām struggling with are thoughts of significant vested commercial interest in exposing this kind of detail, fuelling multi-billion dollar service industries. Feelings of being an outsider despite understanding how it all fits together.
You seem to be conflating Kubernetes and cloud services. Being a cloud native technology does not mean it has to run on a managed cloud service. It just means that it has certain expectations as to how workloads run on it, and if those expectations are met then it makes certain promises about how it will behave.
Have you ever written this kind of software before?
I have contributed to several similar open source projects, yes. What about it?
It sounds like you are comfortable with the status quo of this part of the software industry, and Iām truly jealous!
I am comfortable with my knowledge of this part of the software industry. There is no status quo - thereās currently an equilibrium, yes, but it is a tenuous one. I know the tools I use today will likely not be the same tools I will be using a decade from now. But I also know that the concepts and architectures I learn from managing these tools will still be applicable then, and I can stay agile enough to adapt and become comfortable in a new ecosystem. I would urge you to consider the same approach for yourself.
Not really just an interface. It is a pluggable service that connects to one or more TSDBs, performs periodic queries, and notifies another service when certain thresholds are exceeded. So with all those configuration options, why is the standalone binary expected to have defaults that may sound same on one system but insane in a different one? If the author wants out of the box configuration they could have gotten the helm chart or the operator and then that would be taken care of. But they seem to be deathly allergic to yaml, so I guess that wonāt happen.
You just said that this software was much more complex than Unix tools. Also if only there were alternate documentation formatsā¦.
Until you need authentication, out of the box libraries, observability instrumentation, interoperabilityā¦ which can be done much more easily with a mature communication protocol like HTTP. And for those chasing the bleeding edge thereās gRPC.
Theyāre not, and Iām disappointed that you think they are. Any individual filesystem is a single point of failure. High availability lets me take down an entire system with zero service disruption because thereās redundancy, load balancing, disaster recoveryā¦
They can, and they still doā¦ Inside the container.
Which I understand, I honestly do. I rejected containers for a (relatively) long time myself, and the argument that the author is making echoes what I would have said about containers. Which is why I believe myself to be justified in making the argument that I did, because rejecting a way of doing things based on preconception is a lack of flexibility, and in cloud ecosystems that translates to a lack of skill.
Thatās the problem. The reason Unix became so popular is because it has a highly integrated design and a few very reused abstractions. A lot of simple parts build up in predictable ways to accomplish big things. The complexity is spread out and minimized. The traditional Unix way of doing things is definitely very outdated though. A modern Unix system is like a 100 story skyscraper with the bottom 20 floors nearly abandoned.
Kubernetes and its users would probably be happier if it was used to manage a completely different operating system. In the end, Kubernetes is trying to impose a semi-distributed model of computation on a very NOT distributed operating system to the detriment of system complexity, maintainability, and security.
I agree that universal protocols capable of handling these things are definitely useful. This is why the authors of Unix moved away from communication and protocols that only function on a single system when they were developing Plan 9 and developed the Plan 9 Filesystem Protocol as the universal system ābusā protocol capable of working over networks and on the same physical system. I donāt bring this up to be an evangelist. I just want to emphasize that there are alternative ways of doing things. 9P is much simpler and more elegant than HTTP. Also, many of the people who worked on Plan 9 ended up working for Google and having some influence over the design of things there.
A filesystem does not exclusively mean an on-disk representation of a tree of files with a single physical point of origin. A filesystem can be just as āhighly availableā and distributed as any other way of representing resources of a system if not more so because of its abstractness. Also, youāre ādisappointedā in me? Lmao
And how do you manage containers? With bespoke tools and infrastructure removed from the file abstraction. Which is another way Kubernetes is removed from the Unix way of doing things. Unless Iām mistaken, itās been a long time since I touched Kubernetes.
Itās not a preconception. They engaged with your way of doing things and didnāt like it.
By what standard? The standard of you and your employer? In general, you seem to be under the impression that the conventional hegemonic corporate ācloudā way of doing things is the only correct way and that everyone else is unskilled and not flexible.
Iām not saying that this approach doesnāt have merits, just that you should be more open-minded and not judge everyone else seeking a different path to the conventional model of cloud/distributed computing as naive, unskilled people making ābad-faith argumentsā.
This has always felt untrue to me. The command line has always been simple parts. However we cannot argue that this applies to all Unix-like systems: The monolithic Linux kernel, Kerberos, httpd, SAMBA, X windowing, heck even OpenSSL. Thereās many examples of tooling built on top of Unix systems that donāt follow that philosophy.
Depends on what you mean. āEverything is a fileā? Sure, that metaphor can be put to rest. āLow coupling, high cohesionā? Thatās even more valid now for cloud architectures. You cannot scale a monolith efficiently these days.
Kubernetes is more complex than a single Unix system. It is less complex than manually configuring multiple systems to give the same benefits of Kubernetes in terms of automatic reconciliation, failure recovery, and declarative configuration. This is because those three are first class citizens in Kubernetes, whereas theyāre just afterthoughts in traditional systems. This also makes Kubernetes much more maintainable and secure. Every workload is containerized, every workload has predeclared conditions under which it should run. If it drifts out of those parameters Kubernetes automatically corrects that (when it comes to reconciliation) and/or blocks the undesirable behaviour (security). And Kubernetes keeps an audit trail for its actions, something that again in Unix land is an optional feature.
If you work with the Kubernetes model then you spend 10% more time setting things up and 90% less time maintaining things.
It also has negligible adoption compared to HTTP. And unless it provides an order of magnitude advantage over HTTP, then itās going to be unlikely that developers will use it. Consider git vs mercurial. Is the latter better than git? Almost certainly. Is it 10x better? No, and thatās why it finds it hard to gain traction against git.
Even an online filesystem does not guarantee high availability. If I want highly available data I still need to have replication, leader election, load balancing, failure detection, traffic routing, and geographic distribution. You donāt do those in the filesystem layer, you do them in the application layer.
Nice ad hominem. I guess itās rules for thee, but not for me.
So whatās the problem? Didnāt you just say that the Unix way of doing things is outdated? Let the CSI plugin handle the filesystem side if things, and let Kubernetes focus on the workload scheduling and reconciliation.
Dismissal based on flawed anecdote is preconception.
No. Iām not married to the ācloudā way of doing things. But if someone comes to me and says āHey boblin, we want to implement something on system foo, can you help us?ā and I am not used to doing things the foo way I will say āIām not familiar with it but letās talk about your requirements, and why you chose fooā instead of āfoo is for bureaucrats, I donāt want to use itā. Iād rather hire an open-mined junior than a gray-bearded Unix wizard that dismisses anything unfamilar. And I will also be the first person to reject use cases for Kubernetes when they do not make sense.
There are scenarios where cloud compute just does not make sense, like HPC. If the author had led with something like that, then they would have made a better argument. But instead they went for
,
, and
Itās hard to not take that as bad faith.
I can see why you would come to think that if all youāve been exposed to is Linux and its orbiting ecosystem. I agree with you that modern Unix has failed to live up to its ideals. Even its creators began to see its limitations in the late 80s and began to develop a whole new system from scratch.
That was never true in the first place. Very few things under Unix are actually represented as files (though credit to Linux for pursuing this idea in kernel-space more than most). But Plan 9 shows us this metaphor is worth expanding and exploring in how it can accomplish being a reliable, performant distributed operating system with a fraction of the code required by other systems.
My point is Kubernetes is a hack (a useful hack!) to synchronize multiple separate, different systems in certain ways. It cannot provide anything close to something like a single system image and it canāt bridge the discrete model of computation that Unix assumes.
All these features require a lot of code and complexity to maintain (latest info I can find is almost 2 million as of 2018). Ideally, Kubernetes is capable of what you said, in the same way that ideally programs canāt violate Unix filesystem DAC or other user permissions but in practice every line of code is another opportunity for something to go wrongā¦
Just because something has more security features doesnāt mean itās actually secure. Or that itās maintainable without a company with thousands of engineers and tons of money maintaining for you. Keeping you in a dependent relationship.
So? I donāt expect many of these ideas will be adopted in the mainstream under the monopoly-capitalist market system. Itās way more profitable to keep selling support to manage sprawling and complex systems that require armies of software engineers to upkeep. I think if state investment or public research in general becomes relevant again maybe these ideas will be investigated and adopted for their technical merit.
āHighly availableā is carrying a lot of weight there lol. If we can move some of these qualities into a filesystem layer (which is a userspace application on some systems) and get these benefits for free for all data, why shouldnāt we? The filesystem layer and application layer are not 2 fundamentally separate unrelated parts of a whole.
Lol, stop being condescending and I wonāt respond in kind.
I think the reason the Unix way of doing things is outdated is cuz it didnāt go far enough!
What? lol
Itās not a flawed anecdote or a preconception. They had their own personal experience with a cloud tool and didnāt like it.
You canāt
someone into liking something.
Iām not a gray-bearded Unix wizard and Iām not dismissing these tools because theyāre unfamiliar. I have technical criticism of them and their approach. I think the OP feels the same way.
The assumption among certain computer touchers is that you canāt use Kubernetes or ācloudā tools and not come away loving them. So if someone doesnāt like them they must not really understand them!
They probably couldāve said it nicer. Itās still no excuse to dismiss criticism because you didnāt like the tone.
I think Kubernetes has its uses, for now. But itās still a fundamentally limited and harmful (because of its monopolistic maintainers/creators) way to do a kind of distributed computing. I donāt think anyone is coming for you to take your Kubernetes thoughā¦
Kubernetes is not intended to provide anything like a single system image. Itās a workload orchestration system, not an operating system. Given a compatible interface (a runtime) Kubernetes can in theory distribute workloads to any OS.
Iām not going to argue that Kubernetes is not complex. But as I stated previously Kubernetes as a bespoke ecosystem is less complex than configuring the same features with decoupled systems. The requirements for an orchestrator and the challenges (technical, security, human, etc) to manage said orchestrator are higher. All else being equal, Kubernetes has implemented this in a very lean way, delegating networking, storage, and runtime to pluggable providers on the left, and delegating non-basic workload aspects to operators on the right. Itās this extensibility that makes it both popular with operators and makes it appear daunting to a layperson. And going back to security, is has provably shown to have a reduced attack surface when managed by a competent operator.
So youāreā¦ what, dismissing HTTP because it has been adopted by capitalist market systems? Are you going to dismiss the Fediverse for using HTTP? What about widely adopted protocols? DNS, BGP, IPv4/6, etc?
How about we bring this part of the discussion back to the roots? You said that HTTP and REST as communication protocols seemed strange to you because Unix has other primitives. I pointed out that those primitives do not address many modern client-server communication requirements. You did not refute that, but you said, and I paraphrase ā9P did it betterā. I refrain from commenting on that because thereās no comparative implementation of complex Internet-based systems in 9P. I did state though that even if 9P is superior, as you claim, it did not win out in the end. Thereās plenty of precedents for this: Betamax-VHS, git-mercurial, etc.
(My emphasis) Itās not free though. Thereās an overhead for doing this, and you end up doing things in-filesystem that have no business being there.
*Ahem*:
That is not an experience, itās a provably wrong statement.
Thatās a very weird assumption, and itās the first time Iāve heard it. Can you provide a source? Because in my experience the opposite is the case - thereās no community more critical of Kubernetesā flaws than their developers/users themselves.
I dismissed the criticism because it makes an appeal to pathos, not to logos. Like I said, thereās plenty of valid technical criticisms of Kubernetes, and even an argument on the basis of ethics (like youāre making) is more engaging.
No my Kubernetes. I use it because itās academically interesting, and because it does the tasks it is meant to do better than most alternatives. But if CNCF were to implode today and Kubernetes became no longer practical to use then I would just pivot to another system.
Iām not going to argue whether itās a harmful way of doing distributed computing based on their maintainers/pedrigee. Thatās a longer philosophical discussion than I suspect neither you or I have time for.
Probably need to keep in mind incidental versus essential complexity here.
Because this is how much of what we use already is implemented. Significant effort goes in to portability, interoperability and balancing compromises. When Iām doing software development e.g. writing HTTP APIs (of which I apparently know nothing about ;) ) - I feel like Iāve got a responsibility to carefully balance what I expose as some user-configurable thing versus something managed internally by the application. Sometimes, thankfully, the application doesnāt even have to think about it al all - like what TCP flags to set when I dial some service.
You bring up containers which is a great example of some cool features provided by the Linux kernel to solve interesting problems. If youāre interested, have a look at FreeBSDās Jails, Plan 9 and LXC. Compare the interface to all these systems, both at the library level and userspace, and compare the applications developed using those systems. How easy is it to get going? How much do I need to keep in my head when using these features? Docker, Kubernetes, and the rest all have made different tradeoffs and compromises.
Another one I think about is SQLite. Some seriously clever smarts. Huge numbers of people donāt know anything about for-loops, C, or B-Trees but can read & write SQL. Thatās technology at its best.
Consider how difficult it could be to, say, start a car in all the different operating conditions it is expected to be used in. But we never think about it.
We as tech people pride ourselves on familiarity with esoteric detail, but it doesnāt need to be like this. Nor does memorising it all have anything to do with āskillā.
What Iām struggling with are thoughts of significant vested commercial interest in exposing this kind of detail, fuelling multi-billion dollar service industries. Feelings of being an outsider despite understanding how it all fits together.
Have you ever written this kind of software before?
It sounds like you are comfortable with the status quo of this part of the software industry, and Iām truly jealous! If youāve got any tips on dealing with this kind of stuff you can find my email at https://www.olowe.co/about.html Thanks :)
Go onā¦
In the case of
vmalert, the binary makes no assumptions as to default behaviour because it was not meant to be run standalone. It comes as part of a container with specific environment variables, which in turn is packaged as a Helm chart which has sane configurations. Taking thevmalertbinary by itself is like taking a kerberos server binary without its libraries and config files in/etcfiles and complaining that itās not working.I am very well versed in jails, chroot, openvz, LXC, etc. OCI containers are in a different class - donāt think of them as an OS-like environment, think of them as a self-contained, packaged service. Docker is then one example of a runtime runtime on which those services run, and Kubernetes is an orchestrator that managed containers in runtimes. And yes, there are some tradeoffs and compromises, but those are well within the bounds of the Pareto principle - remove the 10% long tail of features on the host, reduce user-facing complexity by 90%.
Are you arguing that Kubernetes doesnāt do that for you? Because with Kubernetes I can say ārun the service in this container with these settings and so many replicasā, attach some conditions like āstop sending traffic to any one container that takes longer than N seconds to respondā and ārestart the container if a certain command returns an errorā, and just let it run. I can do a rolling upgrade of the nodes and Kubernetes will reschedule the containers on any other available node, it can load balance traffic, I can update the spec of a deployment and Kubernetes will do a zero-downtime upgrade for me. Try implementing the same on a Unix system. Youād need a way to push configs (Ansible, Puppet, etc?). You need load balancing and leader election (Keepalived?). You need error detection. You need DNS. You need to run the services. You need to ensure thereās no library conflict. Thereās a LOT of complexity that a Kubernetes user does not need to worry about any more. Tell me thatās not serious smarts and technology at its best.
You seem to be conflating Kubernetes and cloud services. Being a cloud native technology does not mean it has to run on a managed cloud service. It just means that it has certain expectations as to how workloads run on it, and if those expectations are met then it makes certain promises about how it will behave.
I have contributed to several similar open source projects, yes. What about it?
I am comfortable with my knowledge of this part of the software industry. There is no status quo - thereās currently an equilibrium, yes, but it is a tenuous one. I know the tools I use today will likely not be the same tools I will be using a decade from now. But I also know that the concepts and architectures I learn from managing these tools will still be applicable then, and I can stay agile enough to adapt and become comfortable in a new ecosystem. I would urge you to consider the same approach for yourself.