I’ve recently discovered this project, which assuming it works as advertised (which I think wasn’t really tested yet, since it seems to be a pretty new repo) sounds like a pretty good library to add into your toolbox.
For those that do not know, LINQ is basically a query language over collections in C#, that allows you (from the top of my head) to do stuff like
someList.Where(x => x.value < 10).OrderBy(x => x.priority).Select(x => x.name)
which would give you a IEnumerable list with names of elements where value is smaller than 10, ordered by priority.
However, using LINQ in performance critical code, such as per-frame Updates, is not really a good idea because it unfortunately does generate a lot of garbage (allocations for GC to collect). Having a version that doesn’t allocate anything sounds awesome, assuming you are a fan of LINQ.
What are your thoughts? For me, it sounds like something really useful. While it’s not really that difficult to avoid LINQ, I’m a fan of the simplicity and descriptive nature of the syntax, and not having to avoid it would be great. It does seem there are quite a few issues starting to pop up, but it’s definitely a project that could be worth it to follow.
CySharp keeps putting out this zero allocation things that are awesome. Zero allocations in game dev it’s real shit.
What’s the advantage?
https://cysharp.co.jp/en/ - https://github.com/Cysharp
- MagicOnion: Unified Realtime/API framework for .NET platform and Unity.
- UniTask: Provides an efficient async/await integration for Unity.
- MemoryPack: Extreme performance binary serializer for C# and Unity.
- ZString: Zero Allocation StringBuilder for .NET and Unity.
- MasterMemory: Embedded Typed Readonly In-Memory Document Database for .NET and Unity.
- ConsoleAppFramework: Micro-framework for console applications to building CLI tools for .NET.
I’ve never heard of them, but a colleague told me he recognized the name and thought that they do middleware that’s extremely expensive. Haven’t looked into it, but their FOSS stuff looks nice.
Have you worked with their other/older projects? I’d be interrested in someones experience with this developer, if it’s something that can be reasonably trusted.
Kissaki provided a neat list. Of those I’ve used UniTask extensively and is a game changer. It’s a must have if you are using threading, as it forces single threads on platforms where is unavailable, without compromising performance in the others. We have a game on iOS and Android, with a lot of threading work involved to load bazillions of things, but webgl doesn’t support threads, so adding it as a platform presented an interesting challenge. Switching our threading to UniTask instead of native Tasks makes it possible to not change our code or having weird #if UNITY_WEBGL flags scattered around virtually everywhere to toggle threading. And the zero allocation tasks are awesome. And it has a lot of interop and extensions to neatly integrate with Unity better than the native tasks, while not breaking the native interface.
It’s/The technical readme desc is certainly interesting.
I’m using linq (to mssql) extensively at work. I also have parsing use cases where we work with spans.
I’ll look more into this lib, keep it in mind, and potentially experiment with it.
If you’re using Entity Framework for the mssql, I doubt that this library would work as a substitute.
Because that linq gets parsed into expression trees and then send to the underlying provider (mssql/mysql etc) to be converted into sql. So if you you some non-standard library those providers won’t be able to convert that linq to sql
Yeah, that was ma understanding/assessment as well. It could be useful for my other use case of reading from linear byte data spans though. I suspect the dependency overhead won’t be worth for what it provides though when I can go over the span myself. We don’t have many query-condition-like aspects to reading it.
Readers note: link title and teaser, through repo desc, says SIMD, but repo readme only says “wip” about it. So looks planned rather than covered.
Honestly, I don’t really see a reason why anyone would need to be running LINQ every single frame, even if it performed better. I like the idea that this performs better, but that specific use case I don’t see a benefit over using the builtin solution, since IMO you shouldn’t really ever need to do that anyways.
I can imagine a few use cases, sorting or selecting or filtering a list is something that does pop up from time to time. It’s nothing major, and definitely not a must have.
From the top of my head it can be finding a closest enemy to the player, while it doesnt have to be done every frame, you’d want to do it often enough.
Sure, you can just for loop it (as with any LINQ calls), but I’d say that LINQ is more readable.
because it unfortunately does generate a lot of garbage
What? Can you be more specific?
Some of LINQ functions do memory allocations in the background, which are then thrown away and left for the garbage collector to handle.
Here is an example what I mean - https://www.jacksondunstan.com/articles/4840
Scripting Runtime Version: .NET 4.x Equivalent
net 4 (framework) is very old.
MS has done a lot of optimizations across net 5 6 7 8 9, especially around using spans and allocations as well.
Is Unity still stuck on net 4?
I’ve picked the first example article I found, which was really old (Unity 2018), since I wasn’t sure if the question was about what’s garbage or how much, so I just wanted to illistrate the concept more than concrete numbers.
The ZLINQ repo does have a benchmark screenshot in readme and it does shows that LINQ still does allocations. It’s not clear what the benchmark was ran on, but it shows mean of 200b of allocations, so LINQ probably still does them in some capacity.
Without knowing more, my interpretation and broad assessment is that LINQ is a generalized API/framework with providers implementing translations, for example to SQL.
ZLINQ seems to be for reading over spans. It can’t do translation to SQL.
Saying LINQ produces garbage is uncalled for when it’s a different use case and supports other or more use cases.
I didn’t know that LINQ had more usages than just me being lazy to write a for loop, I have something to look into, thanks. Judging by the first documentation page I found, I wouldn’t even recognize the syntax as the LINQ I’m used to. I really need to catch up on new C# stuff.
Saying LINQ produces garbage is uncalled for when it’s a different use case and supports other or more use cases.
I don’t think I understand why would it be uncalled for, though. At least in the context of game development, where even small allocations can be a problem, it feels ok to generalize, especially if most people probably only encounter LINQ in it’s basic form instead of the other use cases. Mostly for the sake of new programmers, who may fall into a trap of over-using it.
Unless you are talking about using the term garbage, which now I realize may sound degradatory. That wasn’t my intention, and I don’t have any negative connotations with that word, so it was not meant in a negative way - I though it’s the correct terminology for allocations that need to be collected by garbage collector later, which is an issue in performance critical applications.
Judging by the first documentation page I found, I wouldn’t even recognize the syntax as the LINQ I’m used to.
Personally, I hate the expression syntax. I would stick with the method syntax. (That you know and use and that ZLINQ is compatible to.)
The good thing about method syntax is that you have a clearly scoped context and state flow. You type
.Where, add a condition that is limited to the Where precondition parameter expression body, and afterwards have a clear state/result again.That is not the case in expression syntax. You can have variables, the operator order and aliasing is confusing, it’s SQL but worse.
I agree that in the context of gamedev you could generalize and call it such, but even then, saying “LINQ produces garbage” is over-generalized and missing context. It’s the wrong tool for the job/gamedev. Doesn’t mean it produces garbage/junk. In my eyes anyway. I feel like that context is important.
For me, garbage implies no use. What it allocates it makes use of. It’s just the wrong approach alltogether for gamedev.
Do you know how Zlinq does avoid allocations? I do understand that if you map, let’s say from Integer to Integer, that you could do this in-place. But what about a mapping Integer to String?
My interpretation, without having used ZLINQ or actually analyzed it in-depth:
Zero allocations must be for the byte-/data-span traversal. When you transform them to deserialized interpreted data you can’t avoid allocations.
It’s useful when you’re reading / querying from linear data, avoiding the whole LINQ provider accumulation and translation phase.
