For me, the most useful part of LINQ is neither the IQueryable syntax tree based extension mechanism, nor the language integrated part (which I dislike), but simply the IEnumerable extensions. Originally known somewhat confusingly as linq to objects. Those allow me to write c# in functional style, keeping the code concise.
The post I submitted refers mostly to optimizations to those extension methods.
This clicked for me after having learned Haskell. It also shares some of Haskell's features/pitfalls, such as laziness.
There are pitfalls, sure, and honestly I wouldn't advise a team having no one somewhat experienced with basic functional idioms (including laziness) to use it. It can lead to obtuse and slow code if used indiscriminately. I try to lead by example myself.
This is how I use LINQ as well. With some non-standard names, it has everything you need! Eric Lippert wrote a great series on monads tying it to LINQ:
Those attributes correspond to the benchmarking library used in the article. Unit testing & benchmarking code does typically look kind of like a plate of spaghetti.
That said, you would never get a PR through me that does this in the actual business logic. You can use things like AspNetCore without touching a single attribute if you really don't want to.
Whenever I work in other languages/ecosystems LINQ is the one thing that I really miss. It's just such a nice capability to have available in the standard library. Beautifully designed given the constraints it had to work within.
"LINQ Performance improvements" really should read: "our own List<T> implementation performance improvements". Microsoft seem to spend their time improving what they need, rather than spending time on general improvements. LINQ (the syntax, not the method extensions) needs investment. Primarily around lambda allocation and potentially on compile-time reduction of lambdas.
It's time to have value-type local lambdas, or a strategy that doesn't make lambda allocation the overhead that it is. And also there really should be wildcard support (`_`) for LINQ variables by now. Which was been completely ignored when they were brought in for lambdas. It should also be possible to use a lifted-type (like IEnumerable<T>, Option<T>, etc.) as the final item in a LINQ expression, rather than `select ...`. The select adds overhead that isn't needed for certain use-cases and limits things like tail-recursive LINQ expressions.
Libraries like mine that go all-in on LINQ [1], but aren't using `IEnumerable` or `IQueryable`, or any of the LINQ extensions, continually get ignored because MS are focusing purely on improving the performance of their own projects.
A good example is the improved lambda inference. It was only brought forward because ASP.NET Core needed it for its minimal API. It seems like many of the features of the language/framework are driven by their own needs rather than those of the community. The absolute worst thing is the ever expanding set of 'magic methods' (like the LINQ extensions Select, SelectMany, and Where, but also GetAwaiter, and the rest). MS are adding capabilities for themselves (the compiler mostly) rather than do what is really needed and add proper higher-kinded traits to resolve the magic. So everything is weakly typed and only vaguely discoverable by the compiler :/
LINQ is one of those key differentiators between languages, yet it's been wallowing, pretty much untouched since C# 3. I think it's a crying shame that it has basically been ignored since then and, even now, they think LINQ is only useful for iterating lists. And primarily only their own list implementations.
/rant
Don't get me wrong, I appreciate all performance improvements, I'm sure it will help plenty of users. But the focus always seems to be narrowly targeted, which limits the potential.
Anyone knows of a comprehensive book/tutorial to learn end to end web development in dotnet. Most I have found are elementary, outdated or poor quality.
That is because all the hot-and-new in .NET web development is Blazor, and it is not really popular outside of Microsoft's Blogosphere (and IMHO never will be and go down the same way as Silverlight did). The "venerable" technologies are still present in .NET 9 and still work, get maintained etc.
Doing web dev in .NET nowadays for me is mostly creating HTTP/JSON/REST APIs and using whatever FE framework you like to interface it (for me that is React or NextJS). The keyword you want to search for is "ASP.NET WebApi" or - more modern - "ASP.NET minimal API".
You could still do .NET MVC server-side rendering using "Razor" (the markup language in ASP.NET MVC - search for "ASP.NET MVC Razor".
Blazor for better and worse can't have the same ending as Silverlight did. In Server Mode it is just Razor + SignalR for HTMX-style frontends that sort of live update/"magic RPC". In Client Mode it is just Razor as a virtual DOM-ish framework running directly in Web Assembly (wasm) in the Browser. At the end of the day Client Blazor is much more like writing front end websites with Elm than writing websites for the Silverlight plugin.
(Hybrid Blazor where you use both server and client possibly in the exact same assembly/project is where most of the worst Blazor mistakes lie, but those aren't the mistakes of Silverlight, those are the mistakes of ASP Classic before .NET where you can be easily confused by new versions of the which bit of this is `runat="server"` and which bit is `runat="client"` and easily and accidentally create wild RPC ravioli/spaghetti code.)
Certainly I also can't imagine how you would get strong performance out of Server Blazor in real world/Enterprise conditions. RPC busses are going to RPC and roundtrips are still roundtrips even when SignalR-backed web sockets. Flooding UDP web sockets with HTML chunks isn't necessarily a great idea, even if HTTP3 (QUIC) kind of/mostly succeeds at it, but SignalR as good as it is certainly isn't HTTP3.
On the other hand, Client Blazor has huge initial overhead (it compiles a form of the entire .NET platform in WASM; you can golf it down a bit, but it's still an entire runtime and GC and most of a standard lib depending on what your app does, and its dependencies), but once it is loaded it can be as performant as just about any other Virtual DOM-like (modulo the overhead between DOM/JS and WASM, but that overhead is surprisingly small in current browsers and seems to only get smaller as browsers continue to optimize WASM scenarios).
I don't think I'd recommend it for most consumer-facing websites any time soon, but in my experience Client Blazor is still much better than Angular for performance in an Enterprise/Internal App scenario.
It's the slowness/sluggishness, not the looks. Most of the time even a simple interaction like clicking a checkbox takes ~1-2 seconds, which probably comes from tracking the state on server.
(I don't mind outdated for .NET stuff, nearly everything from enterprise vendors looks like it just jumped out of WinXP anyway.)
I've recently taken an interest in web development using C#. It seems to me that ASP.NET is THE way for creating web applications in .NET, for better or worse ("for worse" since lack of alternatives sounds a bit suspicious to me...).
Anyway, I came across a podcast featuring the author of "ASP.NET Core in Action", Andrew Lock, and he seems to know his stuff. I haven't read the book yet, but maybe this is the book you are looking for.
There are alternatives, but most of them have been "consumed" in that ASP.NET does what they do better (most recently "ASP.NET Minimal API" removed a lot of the appeal for many of the smaller alternatives), or "consumed" because "ASP.NET" the brand also includes the low level HTTP machinery at this point and alternatives are increasingly "ASP.NET + some extra framework" which doesn't sound like an alternative, or are focused on small niches and other languages. There's a lot of fun alternatives in F# (Giraffe is the first one to my mind), but obviously that is no longer web development in C#.
I really wish suave wasn’t abandoned. We could have such nice things in F#-land but all of our the cool stuff gets dropped in favor of veneers over ms frameworks.
I agree, "Minimal API" owes a lot to Nancy, specifically. There were a few others in that space, but Nancy was definitely the big one that pushed ASP.NET the most.
For server-rendered UI, look for resources that uses Razor, ignore Blazor stuff at the beginning.
For .NET Backend/JS Frontend, look for resources that uses minimal APIs. MVC is also good but has a lot of backwards compatibility baggage that led to the creation of minimal APIs.
An architecture I've been using lately is writing a functional core in F# and importing the library in both C# API backends as well as React frontends. As you know, Fable can compile to TS which you can import into your node application.
This means that you can have a team of C# developers writing in a language they are familiar with, a team of node/TS developers writing React and a team of F# developers working on a pure functional core with all of the business logic. Write your validators in F# can you can share the same logic for a form in the UI and an API endpoint on the backend.
In my opinion having type inference, discriminated unions, computation expressions, et al., makes for a very concise and expressive way to model the core logic of an application.
Books:
C# 12 and .NET 8 - Modern Cross-Platform Development Fundamentals - Eighth Edition: Start building websites and services with ASP.NET Core 8, Blazor, and EF Core 8 by Mark J Price
Web API Development with ASP.NET Core 8: Learn techniques, patterns, and tools for building high-performance, robust, and scalable web APIs by Xiaodi Yan
Turorials: series of tutorials on YouTube by IAmTimCorey and Shawn Wildermuth.
It’s been consistently one of my favorite tools for a good decade. I don’t know how much Microsoft is paying them to keep it Windows-only because that’s the only reason I keep windows around, and I’m sure that goes for a number of important developers.
> Some more optimizations can happen when the chain ends up with methods like Count(), First(), Last(), ElementAt() or Sum(). For instance, OrderBy(criteria).First() can be optimized to execute as Min(criteria).
As useful as that might be, we should really be writing better code. This is interesting for dynamically generated chains, I suppose. But if these operations are done on bespoke code, this is kind of perverse positive reinforcement. The library itself is recognizing a sub-optimal pattern and correcting for it. I hope there's feedback, at least, that suggests improvements to the underlying code.
Maybe you've been unlucky but LINQ didn't really seem all that interesting to me using it.
It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database and I am the type of person who usually does go and find that kind of thing out before using something for a "serious" project.
Maybe it's just a microcosm for C# as a whole - very productive language, good design team, quite hobbled by it's weird/philistine upbringings: Bjarne said within C++ there is a simple language trying to escape, in C# you basically have a simple language buried in nouns.
> It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable
There's a lot hidden in there, but basically they expect you to use EF. Writing an IQueryable is a similar amount of work to writing a SQL query planner. You get passed a tree of Expression objects.
Back when I was primarily a C# dev, I used OSS lightweight ORMs which had LINQ interfaces. I also frequently used LINQ on in-memory structures. It's fantastic, and I never felt any need to use EF.
That said, C# / .NET shops did have a tendency to mindlessly buy into all sorts of terrible Microsoft enterprisey stuff. That drove me crazy and ultimately is what made me head out for greener pastures.
I was discussing IQueryable with someone else to whom it was important. In reply to
> I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database
And "for some database" the default answer is "use EF" as the intermediary between LINQ queries and the database itself, rather than delving into IQueryable.
LINQ-to-objects is very important and useful but I was talking about something else.
It is not rocket science to implement IQueryable but it is not trivial either since the API is optimized towards ease of use rather then ease of implementation. The Select and Where methods support arbitrary C# expression trees, so the implementation have to traverse the tree and throw errors if some expression cannot be translated to the underlying engine.
Yes and no, the LINQ syntax being coherent between IEnumerable<> and IQueryable<> hides a lot of good magic.
IEnumerable<> is regular in-memory lambdas/streams, same what you find in many places.
IQueryable<> relies on the LINQ expressions, those CAN be JIT compiled for direct execution, but the fact that they are data-objects is what allows the translation to SQL and execution on the server rather than locally and can give massive gains since processing can be done where the data lives.
In C# it'd look identical, but in JS or Java this would be achieved via proxy-object hacks (the .lt() function in the filter instead of the < operator and the .id property getter for the user map to send a flag under the hood for the mapping function).
One difference with LINQ is its ubiquity. It works with database, in memory data structures, on disk files. You can use your skills/code across all the system.
Takes a lot more than that, LINQ providers work by accepting a LINQ Expression Syntax tree instead of an opaque function, which allows providers to inspect and traverse the Expression's AST and translate it into the data source it's implementing.
This Expression AST is constructed by the compiler, not something that can be tacked on by a library later.
Yes, but I think the point is practically every high level language can already do this pretty trivially.
If it's scripted you can typically just get a string representation of the function.
If it's Java, JAR inspection/dynamics have been a thing for a long time. And in other languages, they usually directly support metaprogramming (like Rust) and plugging code into the compilation logic.
If it were trivial you'd see LINQ-like providers implemented in "practically every high level language".
Source code of the function means you have to implement the parser/lexer to convert it into a usable AST which is bad for both runtime performance and library size.
Very much doubt this is available in Java, which Java ORM lets you use native Java language expression syntax to query a database?
You're replying to a thread about what it takes to implement a LINQ provider, which was dismissed as every high level language implements it with iterables, then proceed to give non-equivalent examples.
Wait what? Am I gonna include a source code parser and AST analyser to my JavaScript library for example, to examine the provided expression source and do this? This reads like the infamous Dropbox comment from when it first got released.
Having used it since its inception, I've come to the conclusion that the SQL translator is kind of a misfeature. It creates so many weird bugs and edge-cases and tedium.
I love LINQ, I love having a typesafe ORM as a standard feature of C#, but the convenience of being able to reuse my Pocos and some expressions for both in-memory and in-SQL don't outweigh the downsides.
If I were designing SQL/LINQ today, I'd keep the in-memory record classes and in-database record classes distinct and use some kind of codegen/automapping framework for keeping them synched up. Maybe allow predicate operators to return things other than booleans so we could make `a == b` return some kind of expression tree node.
For ad-hoc queries using anonymous classes? Support defining an interface inline in a generic so you can say
public T MyQuery<interface {string Firstname{get;set;}; string Lastname{get;set:}} T>();
Like, to elaborate, if you were doing some kind of JSON-based codegen (alternately you could do something where you have a separate hand-written POCO Model assembly and use reflection against it to generate your DbModel classes so it's still Code First). Yes, I know MS tried and abandoned this approach, I used LinqToSQL and EF3.5 and whatnot and suffered all that pain.
like, your master datatable file would be something like
```cs
public class DataRecordsNamespace.DbPerson : DbRecord {
public DbPerson() { throw ThisIsAFakeClassException(); }
public DbInt PKID{
get => throw ThisIsAFakeClassException();
set => throw ThisIsAFakeClassException();
}
public DbNVarChar {
get => throw ThisIsAFakeClassException();
set => throw ThisIsAFakeClassException();
}
}
public partial class PocosNamespace.Person {
public AutoGenerated<int> PKID{ get; init; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
public class MyDbModel : DbModel {
public DbTable<DbPerson> Persons => DoSomeLazyStuff();
}
public static class MyDbContextExtensions {
public static List<Person> Resolve(this DbQuery<DbPerson> dbPersons)
{
//call code to execute the actual query.
}
}
```
Am I making sense? Then you wouldn't have the problem of "oops I used an untranslateable method or member of Person", because MyDbModel can't have any of those. You'd lose the ability to to switch from whether a query is in-memory or in-database just by removing the ToList(), but I'd argue that's a misfeature, and better-handled by having some kind of InMemory implementation. Like, having DbQuery have a simple `.ToLocalMemory()` function that is a hint that the next part should be done locally instead of in the database would be a better way to do that. Then you could still do
Saw EF now supports custom SQL queries, so been considering that once we've moved to MSSQL (old db server isn't supported by EF).
We're quite accustomed to writing our own SQL select statements and would like to continue doing that to have known performance, but the update, insert and delete statements are a chore to do manually, especially for once you're 4-5 parent child levels deep.
Guess everyone has their preferred style, I personally avoid code-gen data models like the plague and much prefer code-first libraries.
Here's how you'd do something similar in our OrmLite ORM [1]:
public class Person
{
[AutoIncrement]
public int Id { get; set; }
public string? FirstName { get; set; }
[Required]
public string LastName { get; set; }
}
Create Table:
var db = dbFactory.Open(); // Resolve ADO.NET IDbConnection
db.CreateTable<Person>(); // Create RDBMS Table from POCO definition
Execute Query:
// Performs SQL Query on Server that's returned in a List<Person>
var results = db.Select<Person>(x => x.FirstName.StartsWith("A") && x.LastName == "B");
// Use LINQ to further transform an In Memory collection
var to = results.Where(MemoryFilter).OrderBy(MemorySort).ToList();
Everything works off the POCO, no other external tools, manual configuration mapping, or code gen needed.
This would fail at run-time instead of compile-time.
That's why I'd rather see the DB classes auto-generated with a mapper to convert them. Having the "master" be POCOs instead of JSON/XML/YAML/whatever isn't something I'm convinced on in either direction, but imho the in-database classes being not real POCOs is the important part because it reduces the the problem of somebody writing Person.MyMethod() and then blowing up because it's not a SQL function.
How would you perform this regex query with your code generated solution? What would have to be code generated and what would the developer have to write?
As there's a lot more features available in different RDBMS's than what's available in C# expression syntax, you can use SQL Fragments whenever you need to:
I've found [0] for clojure, which maps the best IMO, but it also contains links to the same LINQ examples in other languages (java, kotlin, swift, elixir, python, ...).
The difference is many of those are dynamically typed languages. It's still useful, but a lot of the a beauty of LINQ comes from the fact that it is within a statically typed language.
You can do functionality similar to LINQ with chaining as long as you don’t need to call a method with a generic different from the one defined. If you do need that, you’re going to have to do it without chaining. You can still do something similar but it’ll be a lot less elegant than how C# does it.
It’s part of the design philosophy of Go though. They don’t want any magic. It’s similar to why they enforce explicit error handling instead of allowing you to chose between explicit and implicit. They want you to write everything near where it happens and not rely on things you can’t see.
It’s probably the primary reason that Go is either hated or loved. I think it’s philosophy is great, a lot of people don’t. I have written a lot of C# over the years, so I’m a little atypical in that regard, I think most C# developers think Go is fairly inferior and in many regards they are correct. Just not in the ones that matter (come at me!). To elaborate a little on that, Go protects developers from themselves. C# is awesome when it’s written by people who know how it works, when it’s not you’ll get LINQ that runs in memory when it really shouldn’t and so on.
Those are linq expressions. They are indeed wonderful. You get an abstract tree from which you can create SQL or API commands to access the data source. I remember in the early days (.NET 3.5?) there were multiple examples of LINQ2X like Linq2Csv, Linq2Rss Linq2Drobox (I'm paraphrasing, I don't remember actual examples, but it was wild).
There is also relinq library which transforms linq expressions into expressions which are easier to understand/use.
There are easy ways to do some subset of what LINQ does in Java (using annotation processors), in Go (using generators), in Python (using double-underscore methods to capture all the operations the expression is working with at runtime, see SQLAlchemy) and in Ruby.
There isn't a seamless way to do what LINQ does in any of those languages. But if the runtime supports a LISP then you can do more than what LINQ does (Clojure for the JVM, something like zygomys for Go, Hy for Python, and ... well, Ruby for Ruby).
"Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language." With LINQ, a query is a first-class language construct, just like classes, methods, and events.
doing this in java is not LINQ imo
List<Integer> lowNums = filter(toList(numbers), new
Predicate<Integer>() {
@Override
public boolean apply(Integer n) {
return n < 5;
}
});
I guess the very fact that LINQ is so many things makes such discussions a bit annoying at times because everyone refers to a different set of features.
Is it the SQL-like query syntax? LINQ to objects? LINQ to SQL? Expression trees in general?
Expression trees and LINQ to SQL/EF/etc. are hard to find elsewhere. The query syntax often doesn't seem to be that big of a deal, especially since not all methods are available there, so pure query syntax often doesn't work anyway.
One of the unfortunate pieces missing from Java's streams is the ability to easily extend them with additional operators. In C# they are all extension methods for the IEnumerable interface so you can add your own methods into the chain but that's not possible in Java.
F# is even better. I only wish performance wasn't hit or miss around sequences/iterator expressions. Hopefully one day it will reach parity with C# through compiler improvements and/or detaching FSharp.Core from conservatively targeting NS2.0 and modernizing it.
I never really got over the order-dependent and fairly slow compile times, but it's been like 10 years since I used it for anything even slightly complex. Is F# better in this regard now, or are there accessible patterns to help deal with that?
Order-dependency is a feature, not a bug, but it does take some getting used to. If you really want to avoid it, you can now declare all functions in a module to be mutually recursive, so you can put them in any order. Cyclic dependencies between files are still prohibited, for good reason. See https://fsharpforfunandprofit.com/posts/cyclic-dependencies/.
The F# compiler is slower than the C# compiler, but it's still more than fast enough for building large applications.
It's just an API for JIT, basically metaprogramming. It's cool but you can definitely do a similar thing in pretty much every high level language.
With scripting languages, it's all JIT :)
The C# teams progress on this has been slow. Keep in mind the CIL bytecode has had such capabilities for at least 20 years now and only in the past like decade are we seeing more features and optimizations around LINQ and System.Reflection.Emit.
Dynamics were extremely slow in C# and if you look at the CIL generated you see why. It's possible for example to use something like a Haxe anonymous types[1] to optimize Dynamics so that CallSite caching is way more performant.
I am pretty sure in C# the only way to accept an anonymous type is as a dynamic value, so even though the type of the structure is well-defined at compile-time, it will still rely heavily on runtime reflection/DLR with no additional caching beyond what DLR does for any other dynamic type.
Anyways, this leads to niche libraries being built for handling dynamic data like JSON performantly.
Which leads to annoying things like .NET libraries/apps being incompatible (without some adapter) if they use for example, different JSON libraries under the hood. (See [2]).
Problems like these (the lack of actually good JIT/dynamic code support) in my opinion significantly slow down the .NET ecosystems development, that's why it always feels like .NET is just catching up with features other popular languages have.
To be fair though, much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono).
> It's cool but you can definitely do a similar thing in pretty much every high level language.
No. When it was release (circa 2007), very few mainstream languages embraced "Code as Data" the way C# did. In Java, there was no way to pass an expression (as an AST) to an SQL library. Which is why LINQ is so much more ergonomic than Hibernate. In C#, you could use language features you're already familiar with (such as "order.id > 100 && order.id < 200") in your queries, whereas Hibernate made you learn the framework's specific patterns (add Criteria etc etc, I don't recall now). Java just wasn't expressive enough for this.
In fact, you couldn't do this even today in a language like say Python or JS. I mean, not without running it through something like babel to get an AST, and having arbitrary rules on what's code and what's data. C# had this in the spec; based on whether it was IQueryable.
> Almost everything good that came out of .NET came from open source/non MS teams (like Mono).
My team adopted Mono very early - like in 2005. Your statement is not true at all. C# and the Framework was a very good spec irrespective of what Open Source / Mono did, and while Mono existing might have accelerated .Net's transition into Open Source, it would have happened anyway due to the definitive swing towards Open Source in the 2000s. Linq-to-SQL, Asp.Net MVC, EF etc didn't come out of Mono.
This is really the thing with the entire .NET stack that’s very hard to communicate. The standard library and framework design are so well thought out relative to anything else out there. More than that, the support within VS is beyond any other dev tool that exists for any other language - it’s not even particularly close. Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload, and is wildly more productive and useful.
I remember back close to 20 years ago DHH was espousing Ruby/Rails and that the concept of types at all were a code smell, and thinking “you’re just very wrong, and clearly aren’t familiar with what else is out there”. Eventually a lot of that crowd moved to Node, then to typescript, and came around.
VS Enterprise (expensive as it is) had features 15 years ago that still seem magical when I show them to JS/TS folks now. IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked.
The big problems with the ecosystem are that the docs are exhaustive but terribly boring, and not well explained from a learning-something-for-the-first-time perspective. They also really expect that everything you do is the Microsoft way, so if you’re trying to interface your code with something like an Avalonia UI, you’re on your own.
The language is absolutely wonderful though, even when used with Rider. The productivity relative to node/typescript is better enough that it crushes my soul having to go back to wrestling tsconfig and imports after working with .NET references for even small changes. So many of the little things I used to take for granted really just work, and work well. It’s just a wonderful piece of work executed over decades, held back by poor community outreach and badly written documentation.
Agree with all of this. I think the biggest problem with popularity around these tools is that they're too good.
When everything just works, you have a lot more time to focus on your problem. Once you can focus on your problem, you may quickly find out you don't actually care about it. Focusing on tools is a great way to hide from this reality.
> IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked.
LINQ is a veritable footgun in any large team I find. While it's extremely powerful and really nice, it's so so so easy to blow your toes off if you don't know what you are doing. Some of my favourite screw ups I saw:
* Not understanding when something is evaluated.
* Not understanding the computational complexity of multiple chained operations/aggregates.
* Not understanding the expectation that Single() requires exactly one of something.
* Not understanding how damn hard it is to test LINQ stuff.
Single gives you some guarantees about the returned value. Use First/FirstOrDefault if you don't need those guarantees. You can also provide predicate for FirstOrDefault to select First element that matches your predicate.
> Enumerable.Single Method - Returns a single, specific element of a sequence.
Some overload descriptions:
- Returns the only element of a sequence that satisfies a specified condition, and throws an exception if more than one such element exists.
- Returns the only element of a sequence, and throws an exception if there is not exactly one element in the sequence.
I sort of agree. I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting.
> Not understanding how damn hard it is to test LINQ stuff.
I disagree with this. Just run the LINQ query on a compatible iterable.
> I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting.
You can write hard to read code with any framework. Yes it takes effort sometimes to make linq code clear, but you should not give up on it.
> * Not understanding when something is evaluated.
Linq is lazy. .ToList() reifies. there, that's the gist of what you need to know. Not hard.
> Not understanding the expectation that Single() requires exactly one of something.
eh? There are a bunch of these methods, Single, SingleOrDefault, First, FirstOrDefault, Last, LastOrDefault and you can look up and grasp how they differ. It's fairly simple. I don't know what the problem is, outside of learning it.
> Not understanding how damn hard it is to test LINQ stuff.
Hard disagree. LInq chains can be unit tested, unless your Db access is mixed in, which is not a LINQ issue at all, it is a database query testing issue. LINQ code, in itself, is easily unit testable.
OK, so since I've never actually bothered to look at what LINQ actually is.. there may be others in the same boat: "LINQ allows you to query any enumerable collections such as List<T>, Array, or Dictionary<TKey,TValue>.". Got to admit I still find the LINQ to DB passtru rather opaque even after reading up on it: https://stackoverflow.com/questions/30716776/passing-a-query...
Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway). Does anything library wise come close to this within the Golang ecosystem?
While LINQ does include a library of extension methods for functional programming with .NET collections (which is great), it also includes "Expression Classes". In a nutshell, this allows a user to pass a single expression lambda to a function, and the function implementor receives the abstract syntax tree for the lambda, not the lambda itself. You can then not only receive and analyze these trees, you can also manually build and compile them. This effectively allows a limited set of runtime macros within .NET.
> Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL.
Pretty much. There's the "language integrated" version which looks a lot like SQL:
var foo = new List<int> { 1, 2, 3, };
var response = from x in foo where x > 1 select x.ToString();
But that just translates to the method-orientated one which many people prefer
response = foo.Where(x => x > 1).Select(x => x.ToString());
If instead of querying a List or Dictionary you query a database, using an ORM (usually Entity Framework), that will actually get converted to SQL and run on the DB.
It's a shame, actually, that .NET performance improvements of up to x1000 could still be found after two decades and hundreds of millions spent on development.
Most of the time, it is not because there were too many slow things to be improved, it is mostly because they are adding more facilities to the runtime, enabling other performance improvements.
For example, the ReadOnlySpan type is a recent addition to the runtime and it will allow faster iterations when used. They simply enabled Linq to use it now.
ReadOnlySpan is a breakthrough innovative data structure, consisting of a pointer and a _length, that took Microsoft Corporation two decades to invent.
other languages do not have special pointers that can point to GC object interiors, be transparently addressed with arithmetics and cooperatively updated by GC without pinning, while also allowing to point to stack and unmanaged memory
But those pointers were around since .NET 1.0. Not only that, but things like e.g. ArgIterator were also there! Span could have been there too; it was really a matter of design rather than technical capability.
I think the main reason why C# didn't have this (and other low-level features like plain function pointers) for so long is because the original vision for .NET was that you'd mix and match different languages in your project as needed. So if you needed that kind of stuff, you'd reach out for managed C++ (and later, C++/CLI), while C# was kept deliberately more high-level.
And so once that approach was abandoned, C# needed to catch up to be able to play the "one language that can do it all" role.
For me, the most useful part of LINQ is neither the IQueryable syntax tree based extension mechanism, nor the language integrated part (which I dislike), but simply the IEnumerable extensions. Originally known somewhat confusingly as linq to objects. Those allow me to write c# in functional style, keeping the code concise.
The post I submitted refers mostly to optimizations to those extension methods.
This clicked for me after having learned Haskell. It also shares some of Haskell's features/pitfalls, such as laziness.
There are pitfalls, sure, and honestly I wouldn't advise a team having no one somewhat experienced with basic functional idioms (including laziness) to use it. It can lead to obtuse and slow code if used indiscriminately. I try to lead by example myself.
This is how I use LINQ as well. With some non-standard names, it has everything you need! Eric Lippert wrote a great series on monads tying it to LINQ:
https://ericlippert.com/2013/04/02/monads-part-twelve/
C# has quite a few easy to use things that take a while to understand. In some ways it is a bit of an "experts only" language.
There has to be a better way to do things than this annotation noodle soup. My eyes are bleeding every time I look at modern Net code.
Those attributes correspond to the benchmarking library used in the article. Unit testing & benchmarking code does typically look kind of like a plate of spaghetti.
That said, you would never get a PR through me that does this in the actual business logic. You can use things like AspNetCore without touching a single attribute if you really don't want to.
The more C# borrows from F#, the happier I am. I am awaiting for discriminated unions to finally reach C# so I can do domain modelling like a boss. :)
I'd kill for units of measure too. It makes maintaining any kind of engineering or scientific code so much easier.
You can largely already pretty easily incorporate DUs using OneOf[0] and Dunet[1].
A practical example of using this: https://chrlschn.dev/blog/2024/07/csharp-discriminated-union...
[0] https://github.com/mcintyre321/OneOf
[1] https://github.com/domn1995/dunet
Whenever I work in other languages/ecosystems LINQ is the one thing that I really miss. It's just such a nice capability to have available in the standard library. Beautifully designed given the constraints it had to work within.
"LINQ Performance improvements" really should read: "our own List<T> implementation performance improvements". Microsoft seem to spend their time improving what they need, rather than spending time on general improvements. LINQ (the syntax, not the method extensions) needs investment. Primarily around lambda allocation and potentially on compile-time reduction of lambdas.
It's time to have value-type local lambdas, or a strategy that doesn't make lambda allocation the overhead that it is. And also there really should be wildcard support (`_`) for LINQ variables by now. Which was been completely ignored when they were brought in for lambdas. It should also be possible to use a lifted-type (like IEnumerable<T>, Option<T>, etc.) as the final item in a LINQ expression, rather than `select ...`. The select adds overhead that isn't needed for certain use-cases and limits things like tail-recursive LINQ expressions.
Libraries like mine that go all-in on LINQ [1], but aren't using `IEnumerable` or `IQueryable`, or any of the LINQ extensions, continually get ignored because MS are focusing purely on improving the performance of their own projects.
A good example is the improved lambda inference. It was only brought forward because ASP.NET Core needed it for its minimal API. It seems like many of the features of the language/framework are driven by their own needs rather than those of the community. The absolute worst thing is the ever expanding set of 'magic methods' (like the LINQ extensions Select, SelectMany, and Where, but also GetAwaiter, and the rest). MS are adding capabilities for themselves (the compiler mostly) rather than do what is really needed and add proper higher-kinded traits to resolve the magic. So everything is weakly typed and only vaguely discoverable by the compiler :/
LINQ is one of those key differentiators between languages, yet it's been wallowing, pretty much untouched since C# 3. I think it's a crying shame that it has basically been ignored since then and, even now, they think LINQ is only useful for iterating lists. And primarily only their own list implementations.
/rant
Don't get me wrong, I appreciate all performance improvements, I'm sure it will help plenty of users. But the focus always seems to be narrowly targeted, which limits the potential.
[1] https://github.com/louthy/language-ext/
Anyone knows of a comprehensive book/tutorial to learn end to end web development in dotnet. Most I have found are elementary, outdated or poor quality.
That is because all the hot-and-new in .NET web development is Blazor, and it is not really popular outside of Microsoft's Blogosphere (and IMHO never will be and go down the same way as Silverlight did). The "venerable" technologies are still present in .NET 9 and still work, get maintained etc.
Doing web dev in .NET nowadays for me is mostly creating HTTP/JSON/REST APIs and using whatever FE framework you like to interface it (for me that is React or NextJS). The keyword you want to search for is "ASP.NET WebApi" or - more modern - "ASP.NET minimal API".
You could still do .NET MVC server-side rendering using "Razor" (the markup language in ASP.NET MVC - search for "ASP.NET MVC Razor".
Blazor for better and worse can't have the same ending as Silverlight did. In Server Mode it is just Razor + SignalR for HTMX-style frontends that sort of live update/"magic RPC". In Client Mode it is just Razor as a virtual DOM-ish framework running directly in Web Assembly (wasm) in the Browser. At the end of the day Client Blazor is much more like writing front end websites with Elm than writing websites for the Silverlight plugin.
(Hybrid Blazor where you use both server and client possibly in the exact same assembly/project is where most of the worst Blazor mistakes lie, but those aren't the mistakes of Silverlight, those are the mistakes of ASP Classic before .NET where you can be easily confused by new versions of the which bit of this is `runat="server"` and which bit is `runat="client"` and easily and accidentally create wild RPC ravioli/spaghetti code.)
Do you know of any performant, quality examples of Blazor out in the wild? All the examples I've seen have unacceptable UX by modern standards.
Certainly I also can't imagine how you would get strong performance out of Server Blazor in real world/Enterprise conditions. RPC busses are going to RPC and roundtrips are still roundtrips even when SignalR-backed web sockets. Flooding UDP web sockets with HTML chunks isn't necessarily a great idea, even if HTTP3 (QUIC) kind of/mostly succeeds at it, but SignalR as good as it is certainly isn't HTTP3.
On the other hand, Client Blazor has huge initial overhead (it compiles a form of the entire .NET platform in WASM; you can golf it down a bit, but it's still an entire runtime and GC and most of a standard lib depending on what your app does, and its dependencies), but once it is loaded it can be as performant as just about any other Virtual DOM-like (modulo the overhead between DOM/JS and WASM, but that overhead is surprisingly small in current browsers and seems to only get smaller as browsers continue to optimize WASM scenarios).
I don't think I'd recommend it for most consumer-facing websites any time soon, but in my experience Client Blazor is still much better than Angular for performance in an Enterprise/Internal App scenario.
https://github.com/Webreaper/Damselfly
> All the examples I've seen have unacceptable UX by modern standards.
What does that have to do with Blazor?
One could easily write outdated UX in React if they so choose.
It's the slowness/sluggishness, not the looks. Most of the time even a simple interaction like clicking a checkbox takes ~1-2 seconds, which probably comes from tracking the state on server.
(I don't mind outdated for .NET stuff, nearly everything from enterprise vendors looks like it just jumped out of WinXP anyway.)
I've recently taken an interest in web development using C#. It seems to me that ASP.NET is THE way for creating web applications in .NET, for better or worse ("for worse" since lack of alternatives sounds a bit suspicious to me...).
Anyway, I came across a podcast featuring the author of "ASP.NET Core in Action", Andrew Lock, and he seems to know his stuff. I haven't read the book yet, but maybe this is the book you are looking for.
--
1: https://dotnetcore.show/season-6/navigating-the-aspnet-core-...
2: https://www.manning.com/books/asp-net-core-in-action-third-e...
There are alternatives, but most of them have been "consumed" in that ASP.NET does what they do better (most recently "ASP.NET Minimal API" removed a lot of the appeal for many of the smaller alternatives), or "consumed" because "ASP.NET" the brand also includes the low level HTTP machinery at this point and alternatives are increasingly "ASP.NET + some extra framework" which doesn't sound like an alternative, or are focused on small niches and other languages. There's a lot of fun alternatives in F# (Giraffe is the first one to my mind), but obviously that is no longer web development in C#.
I really wish suave wasn’t abandoned. We could have such nice things in F#-land but all of our the cool stuff gets dropped in favor of veneers over ms frameworks.
Modern Asp.net stole a lot from a little web framework called NancyFx. It really shaped what Asp.net Core is today.
I agree, "Minimal API" owes a lot to Nancy, specifically. There were a few others in that space, but Nancy was definitely the big one that pushed ASP.NET the most.
For server-rendered UI, look for resources that uses Razor, ignore Blazor stuff at the beginning.
For .NET Backend/JS Frontend, look for resources that uses minimal APIs. MVC is also good but has a lot of backwards compatibility baggage that led to the creation of minimal APIs.
A bit of the beaten path, but F# with Fable is a very powerful combination.
On the server you can run ASP.Net using Giraffe, which is a Functional Programming layer with comparable performance to C#.
On the front-end, you can write React in an true Functional Programming language.
And of course you can share F# code between frontend and backend.
An architecture I've been using lately is writing a functional core in F# and importing the library in both C# API backends as well as React frontends. As you know, Fable can compile to TS which you can import into your node application.
This means that you can have a team of C# developers writing in a language they are familiar with, a team of node/TS developers writing React and a team of F# developers working on a pure functional core with all of the business logic. Write your validators in F# can you can share the same logic for a form in the UI and an API endpoint on the backend.
In my opinion having type inference, discriminated unions, computation expressions, et al., makes for a very concise and expressive way to model the core logic of an application.
I've learned by doing, but here you go.
Books: C# 12 and .NET 8 - Modern Cross-Platform Development Fundamentals - Eighth Edition: Start building websites and services with ASP.NET Core 8, Blazor, and EF Core 8 by Mark J Price
Web API Development with ASP.NET Core 8: Learn techniques, patterns, and tools for building high-performance, robust, and scalable web APIs by Xiaodi Yan
Turorials: series of tutorials on YouTube by IAmTimCorey and Shawn Wildermuth.
Beware, once you get used to using LINQ and usually work in a domain that it shines, you won't want to go back to using anything else.
Do not, my friends, become addicted to LINQ. It will take hold of you, and you will resent its absence.
I have fond memories of using LINQPad
That's why I guess today I work in Clojure :)
It’s been consistently one of my favorite tools for a good decade. I don’t know how much Microsoft is paying them to keep it Windows-only because that’s the only reason I keep windows around, and I’m sure that goes for a number of important developers.
> Some more optimizations can happen when the chain ends up with methods like Count(), First(), Last(), ElementAt() or Sum(). For instance, OrderBy(criteria).First() can be optimized to execute as Min(criteria).
As useful as that might be, we should really be writing better code. This is interesting for dynamically generated chains, I suppose. But if these operations are done on bespoke code, this is kind of perverse positive reinforcement. The library itself is recognizing a sub-optimal pattern and correcting for it. I hope there's feedback, at least, that suggests improvements to the underlying code.
Relevant section in the annual book-sized post on all performance improvements in .NET 9:
https://devblogs.microsoft.com/dotnet/performance-improvemen...
(for some reason HN would not allow to resubmit it, and the post was left by wayside not reaching frontpage)
C# is Pragmatic [0].
[0] https://news.ycombinator.com/item?id=41761346
LINQ is so fucking useful and well designed feature of .NET ecosystem that it is unreal when you gotta use lang which doesnt have such a thing.
C# design team is/was unparalleled
Maybe you've been unlucky but LINQ didn't really seem all that interesting to me using it.
It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database and I am the type of person who usually does go and find that kind of thing out before using something for a "serious" project.
Maybe it's just a microcosm for C# as a whole - very productive language, good design team, quite hobbled by it's weird/philistine upbringings: Bjarne said within C++ there is a simple language trying to escape, in C# you basically have a simple language buried in nouns.
> It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable
There's a lot hidden in there, but basically they expect you to use EF. Writing an IQueryable is a similar amount of work to writing a SQL query planner. You get passed a tree of Expression objects.
https://learn.microsoft.com/en-us/archive/blogs/mattwar/linq...
Back when I was primarily a C# dev, I used OSS lightweight ORMs which had LINQ interfaces. I also frequently used LINQ on in-memory structures. It's fantastic, and I never felt any need to use EF.
That said, C# / .NET shops did have a tendency to mindlessly buy into all sorts of terrible Microsoft enterprisey stuff. That drove me crazy and ultimately is what made me head out for greener pastures.
> basically they expect you to use EF. Writing an IQueryable...
I don't agree. I don't feel any expectation to use EF. It would not be relevant anyway to our code.
LINQ is not EF and EF is not LINQ. EF uses LINQ but not vice versa. LINQ is useful without EF.
The LINQ extension methods that we use constantly are on IEnumerable<T> so EF and IQueryable is of no importance to us, but LINQ is used everywhere.
> IQueryable is of no importance to us
I was discussing IQueryable with someone else to whom it was important. In reply to
> I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database
And "for some database" the default answer is "use EF" as the intermediary between LINQ queries and the database itself, rather than delving into IQueryable.
LINQ-to-objects is very important and useful but I was talking about something else.
> Maybe you've been unlucky but LINQ didn't really seem all that interesting to me using it.
Getting good use out of a tool you do not find interesting would mean a person was… unlucky?
It is not rocket science to implement IQueryable but it is not trivial either since the API is optimized towards ease of use rather then ease of implementation. The Select and Where methods support arbitrary C# expression trees, so the implementation have to traverse the tree and throw errors if some expression cannot be translated to the underlying engine.
> I could not tell you now how to implement a custom IQueryable
So what? I see LINQ used all the time, and it is almost entirely (extension) methods IEnumerable<T>
Could I implement IEnumerable<T>? I think I did once, as an exercise. It's not that complex. Not that interesting to be able to do it either.
LINQ is useful without EF. LINQ is not EF and EF is not LINQ.
LINQ is largely based on FP stuff, also how Smalltalk collections work.
It is relatively easy to find similar capabilities in most languages nowadays, unless one is stuck on Go, C and similar.
Yes and no, the LINQ syntax being coherent between IEnumerable<> and IQueryable<> hides a lot of good magic.
IEnumerable<> is regular in-memory lambdas/streams, same what you find in many places.
IQueryable<> relies on the LINQ expressions, those CAN be JIT compiled for direct execution, but the fact that they are data-objects is what allows the translation to SQL and execution on the server rather than locally and can give massive gains since processing can be done where the data lives.
For reference, to achieve what IQueryable does with 100% normal code in JavaScript you need something like Qustar that was posted here a month ago.
Regular transform code in JS (Like IEnumerable)
const ids = users.filter(user => user.age<18).map(user => user.id);
IQueryable like to be transformed to the server:
const ids = users.filter(user => user.age.lt(18)).map(user => user.id);
In C# it'd look identical, but in JS or Java this would be achieved via proxy-object hacks (the .lt() function in the filter instead of the < operator and the .id property getter for the user map to send a flag under the hood for the mapping function).
https://github.com/tilyupo/qustar
One difference with LINQ is its ubiquity. It works with database, in memory data structures, on disk files. You can use your skills/code across all the system.
It's just built on top of anything that is Iterable. If a language has first class iterator support, they could do something similar.
Takes a lot more than that, LINQ providers work by accepting a LINQ Expression Syntax tree instead of an opaque function, which allows providers to inspect and traverse the Expression's AST and translate it into the data source it's implementing.
This Expression AST is constructed by the compiler, not something that can be tacked on by a library later.
Yes, but I think the point is practically every high level language can already do this pretty trivially.
If it's scripted you can typically just get a string representation of the function.
If it's Java, JAR inspection/dynamics have been a thing for a long time. And in other languages, they usually directly support metaprogramming (like Rust) and plugging code into the compilation logic.
If it were trivial you'd see LINQ-like providers implemented in "practically every high level language".
Source code of the function means you have to implement the parser/lexer to convert it into a usable AST which is bad for both runtime performance and library size.
Very much doubt this is available in Java, which Java ORM lets you use native Java language expression syntax to query a database?
jOOQ would be one such example, https://www.jooq.org/
Not that I use this, I am a myBatis person in what concerns database access in Java, and Dapper in .NET for that matter, not a big ORM fan.
And case in point most people use LINQ for in-memory datastructures, not the database part.
This is a custom expression language to work within the expressive limitations of the language:
If Java supported LINQ you'd be able to use a more intuitive and natural Java expression syntax instead:Java streams are what you're looking for.
If you insist in telling LINQ === EF, well that isn't what most folks in .NET use System.Linq for.
And back to the ORM thing, jOOQ is one way, there are others, and even if it isn't 1:1 to "from x select whatever" the approach exists.
> If you insist in telling LINQ === EF
I don't use EF, nor have I ever mentioned it.
You're replying to a thread about what it takes to implement a LINQ provider, which was dismissed as every high level language implements it with iterables, then proceed to give non-equivalent examples.
Wait what? Am I gonna include a source code parser and AST analyser to my JavaScript library for example, to examine the provided expression source and do this? This reads like the infamous Dropbox comment from when it first got released.
Not that I agree it's trivial but even if it was, so what?
This just feels like sour grapes.
Having used it since its inception, I've come to the conclusion that the SQL translator is kind of a misfeature. It creates so many weird bugs and edge-cases and tedium.
I love LINQ, I love having a typesafe ORM as a standard feature of C#, but the convenience of being able to reuse my Pocos and some expressions for both in-memory and in-SQL don't outweigh the downsides.
If I were designing SQL/LINQ today, I'd keep the in-memory record classes and in-database record classes distinct and use some kind of codegen/automapping framework for keeping them synched up. Maybe allow predicate operators to return things other than booleans so we could make `a == b` return some kind of expression tree node.
For ad-hoc queries using anonymous classes? Support defining an interface inline in a generic so you can say
Like, to elaborate, if you were doing some kind of JSON-based codegen (alternately you could do something where you have a separate hand-written POCO Model assembly and use reflection against it to generate your DbModel classes so it's still Code First). Yes, I know MS tried and abandoned this approach, I used LinqToSQL and EF3.5 and whatnot and suffered all that pain.like, your master datatable file would be something like
which would generates something like Am I making sense? Then you wouldn't have the problem of "oops I used an untranslateable method or member of Person", because MyDbModel can't have any of those. You'd lose the ability to to switch from whether a query is in-memory or in-database just by removing the ToList(), but I'd argue that's a misfeature, and better-handled by having some kind of InMemory implementation. Like, having DbQuery have a simple `.ToLocalMemory()` function that is a hint that the next part should be done locally instead of in the database would be a better way to do that. Then you could still do edits: fix some of the HN pseudomarkdownSaw EF now supports custom SQL queries, so been considering that once we've moved to MSSQL (old db server isn't supported by EF).
We're quite accustomed to writing our own SQL select statements and would like to continue doing that to have known performance, but the update, insert and delete statements are a chore to do manually, especially for once you're 4-5 parent child levels deep.
Guess everyone has their preferred style, I personally avoid code-gen data models like the plague and much prefer code-first libraries.
Here's how you'd do something similar in our OrmLite ORM [1]:
Create Table: Execute Query: Everything works off the POCO, no other external tools, manual configuration mapping, or code gen needed.[1] https://docs.servicestack.net/ormlite/
My problem with this approach is that this falls apart if you write:
This would fail at run-time instead of compile-time.That's why I'd rather see the DB classes auto-generated with a mapper to convert them. Having the "master" be POCOs instead of JSON/XML/YAML/whatever isn't something I'm convinced on in either direction, but imho the in-database classes being not real POCOs is the important part because it reduces the the problem of somebody writing Person.MyMethod() and then blowing up because it's not a SQL function.
Isn't this just `.StartsWith("A")`?
How would you perform this regex query with your code generated solution? What would have to be code generated and what would the developer have to write?
As there's a lot more features available in different RDBMS's than what's available in C# expression syntax, you can use SQL Fragments whenever you need to:
Mainly first-class functions I think. If you have those, you can just use fold in the core combinators.
I've found [0] for clojure, which maps the best IMO, but it also contains links to the same LINQ examples in other languages (java, kotlin, swift, elixir, python, ...).
[0]: https://github.com/mythz/clojure-linq-examples
The difference is many of those are dynamically typed languages. It's still useful, but a lot of the a beauty of LINQ comes from the fact that it is within a statically typed language.
It's surprising that Go didn't ship with it, but given that they just added iterators, it's coming.
Rust has combinators, which is the same thing.
Most new languages are recognizing that functional support (even if they don't consider themselves FP languages) is necessary.
You can do functionality similar to LINQ with chaining as long as you don’t need to call a method with a generic different from the one defined. If you do need that, you’re going to have to do it without chaining. You can still do something similar but it’ll be a lot less elegant than how C# does it.
It’s part of the design philosophy of Go though. They don’t want any magic. It’s similar to why they enforce explicit error handling instead of allowing you to chose between explicit and implicit. They want you to write everything near where it happens and not rely on things you can’t see.
It’s probably the primary reason that Go is either hated or loved. I think it’s philosophy is great, a lot of people don’t. I have written a lot of C# over the years, so I’m a little atypical in that regard, I think most C# developers think Go is fairly inferior and in many regards they are correct. Just not in the ones that matter (come at me!). To elaborate a little on that, Go protects developers from themselves. C# is awesome when it’s written by people who know how it works, when it’s not you’ll get LINQ that runs in memory when it really shouldn’t and so on.
nah it's not coming, functions like maps and filters won't come to go by design, iterators are not only about FP stuff
Go culture is quite clearly against this kind of niceties.
Do you know an equivalent for Linw to EF in kotlin or Java because I have not found it.
Those are linq expressions. They are indeed wonderful. You get an abstract tree from which you can create SQL or API commands to access the data source. I remember in the early days (.NET 3.5?) there were multiple examples of LINQ2X like Linq2Csv, Linq2Rss Linq2Drobox (I'm paraphrasing, I don't remember actual examples, but it was wild).
There is also relinq library which transforms linq expressions into expressions which are easier to understand/use.
Rather quick search, as I am more of a myBatis person,
Java: https://www.jooq.org/
Kotlin: https://www.ktorm.org
Any pointers to such libraries in python?
itertools would be the starting point, unfortunelly Python is rather limited due to the way it only supports one line lambdas.
No it isn't easy to find similar capabitites in java, go, python, ruby.
Maybe you do simulate some of this using meta programming in ruby but its certainly not 'easy to find'.
There are easy ways to do some subset of what LINQ does in Java (using annotation processors), in Go (using generators), in Python (using double-underscore methods to capture all the operations the expression is working with at runtime, see SQLAlchemy) and in Ruby.
There isn't a seamless way to do what LINQ does in any of those languages. But if the runtime supports a LISP then you can do more than what LINQ does (Clojure for the JVM, something like zygomys for Go, Hy for Python, and ... well, Ruby for Ruby).
It certainly is, unless you are talking about SQL like syntax, which is basically syntax sugar for classical FP.
And I explicitly left Go out of my list.
>unless you are talking about SQL like syntax
yes thats what linq is?
https://learn.microsoft.com/en-us/dotnet/csharp/linq/
"Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language." With LINQ, a query is a first-class language construct, just like classes, methods, and events.
doing this in java is not LINQ imo
I guess the very fact that LINQ is so many things makes such discussions a bit annoying at times because everyone refers to a different set of features.
Is it the SQL-like query syntax? LINQ to objects? LINQ to SQL? Expression trees in general?
Expression trees and LINQ to SQL/EF/etc. are hard to find elsewhere. The query syntax often doesn't seem to be that big of a deal, especially since not all methods are available there, so pure query syntax often doesn't work anyway.
I discovered someone stuff in Java pre-history days.
Because I am feeling nice,
These days it would be
2024's Java is also quite a bit better than 2013's Java.Which still isn't as nice as LINQ, but this way we've painted the alternative in its best light, not in the light that makes C# look the best.
One of the unfortunate pieces missing from Java's streams is the ability to easily extend them with additional operators. In C# they are all extension methods for the IEnumerable interface so you can add your own methods into the chain but that's not possible in Java.
It is coming, it is called Gatherers, another approach, same result.
I'm a huge fan of Gatherers, lovely API!
i posted that example from a comment that linked to this repo https://github.com/mythz/clojure-linq-examples
my point was that laguange support for sql like sytax is part of what makes LINQ linq. Java niceties is not relevant.
> yes thats what linq is?
The link that you gave says "LINQ is the name for a set of technologies" which includes the "SQL like syntax".
Includes is not the same as "is".
It isn't the most often used part of LINQ.
sure but you cannot just remove 'sql like syntax' and claim you can do linq in any language.
To be fair, 99% of linq usage is method chaining syntax, not query syntax
LINQ = Language Integrated Query
Its even in the name. What do you mean by "method chaining is linq" ?
Considering the methods live in the System.Linq namespace, I think the extension methods may also be called LINQ.
"var results = list.Where(x => x.Salary > 12345).Select(x => x.Name).ToList())"
Giant majority of ppl refers to this when talking about LINQ.
But yea, it is LINQ method chaining.
SQL like syntax is LINQ query syntax
> But yea, it is LINQ method chaining.
You mean like fluent interface? https://en.wikipedia.org/wiki/Fluent_interface
What does this have to do with LINQ or C#. I remember doing 'method chaining' in 1990s .
>fluent interface
Various names, same concept.
"fluent interface is an object-oriented API whose design relies extensively on method chaining."
>What does this have to do with LINQ or C#.
Check the name of the namespace where all those APIs like Where, GroupBy, etc. are implemented, it is "System.Linq"
So thats why majority of ppl think about them when talking about LINQ.
Query syntax has like less than 1% of the "market share" versus method chaining style
If you like LINQ, you should really give F# a try.
F# is even better. I only wish performance wasn't hit or miss around sequences/iterator expressions. Hopefully one day it will reach parity with C# through compiler improvements and/or detaching FSharp.Core from conservatively targeting NS2.0 and modernizing it.
I never really got over the order-dependent and fairly slow compile times, but it's been like 10 years since I used it for anything even slightly complex. Is F# better in this regard now, or are there accessible patterns to help deal with that?
Order-dependency is a feature, not a bug, but it does take some getting used to. If you really want to avoid it, you can now declare all functions in a module to be mutually recursive, so you can put them in any order. Cyclic dependencies between files are still prohibited, for good reason. See https://fsharpforfunandprofit.com/posts/cyclic-dependencies/.
The F# compiler is slower than the C# compiler, but it's still more than fast enough for building large applications.
It's just an API for JIT, basically metaprogramming. It's cool but you can definitely do a similar thing in pretty much every high level language.
With scripting languages, it's all JIT :)
The C# teams progress on this has been slow. Keep in mind the CIL bytecode has had such capabilities for at least 20 years now and only in the past like decade are we seeing more features and optimizations around LINQ and System.Reflection.Emit.
Dynamics were extremely slow in C# and if you look at the CIL generated you see why. It's possible for example to use something like a Haxe anonymous types[1] to optimize Dynamics so that CallSite caching is way more performant.
I am pretty sure in C# the only way to accept an anonymous type is as a dynamic value, so even though the type of the structure is well-defined at compile-time, it will still rely heavily on runtime reflection/DLR with no additional caching beyond what DLR does for any other dynamic type.
Anyways, this leads to niche libraries being built for handling dynamic data like JSON performantly.
Which leads to annoying things like .NET libraries/apps being incompatible (without some adapter) if they use for example, different JSON libraries under the hood. (See [2]).
Problems like these (the lack of actually good JIT/dynamic code support) in my opinion significantly slow down the .NET ecosystems development, that's why it always feels like .NET is just catching up with features other popular languages have.
To be fair though, much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono).
[1] - https://haxe.org/manual/types-anonymous-structure.html
[2] - https://learn.microsoft.com/en-us/dotnet/standard/serializat...
C# generics will handle anonymous types just fine. That's what lets you write stuff like `from ... select new { ... } where ...`.
> It's cool but you can definitely do a similar thing in pretty much every high level language.
No. When it was release (circa 2007), very few mainstream languages embraced "Code as Data" the way C# did. In Java, there was no way to pass an expression (as an AST) to an SQL library. Which is why LINQ is so much more ergonomic than Hibernate. In C#, you could use language features you're already familiar with (such as "order.id > 100 && order.id < 200") in your queries, whereas Hibernate made you learn the framework's specific patterns (add Criteria etc etc, I don't recall now). Java just wasn't expressive enough for this.
In fact, you couldn't do this even today in a language like say Python or JS. I mean, not without running it through something like babel to get an AST, and having arbitrary rules on what's code and what's data. C# had this in the spec; based on whether it was IQueryable.
> Almost everything good that came out of .NET came from open source/non MS teams (like Mono).
My team adopted Mono very early - like in 2005. Your statement is not true at all. C# and the Framework was a very good spec irrespective of what Open Source / Mono did, and while Mono existing might have accelerated .Net's transition into Open Source, it would have happened anyway due to the definitive swing towards Open Source in the 2000s. Linq-to-SQL, Asp.Net MVC, EF etc didn't come out of Mono.
I’ll get started then end up on a rant but..
This is really the thing with the entire .NET stack that’s very hard to communicate. The standard library and framework design are so well thought out relative to anything else out there. More than that, the support within VS is beyond any other dev tool that exists for any other language - it’s not even particularly close. Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload, and is wildly more productive and useful.
I remember back close to 20 years ago DHH was espousing Ruby/Rails and that the concept of types at all were a code smell, and thinking “you’re just very wrong, and clearly aren’t familiar with what else is out there”. Eventually a lot of that crowd moved to Node, then to typescript, and came around.
VS Enterprise (expensive as it is) had features 15 years ago that still seem magical when I show them to JS/TS folks now. IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked.
The big problems with the ecosystem are that the docs are exhaustive but terribly boring, and not well explained from a learning-something-for-the-first-time perspective. They also really expect that everything you do is the Microsoft way, so if you’re trying to interface your code with something like an Avalonia UI, you’re on your own.
The language is absolutely wonderful though, even when used with Rider. The productivity relative to node/typescript is better enough that it crushes my soul having to go back to wrestling tsconfig and imports after working with .NET references for even small changes. So many of the little things I used to take for granted really just work, and work well. It’s just a wonderful piece of work executed over decades, held back by poor community outreach and badly written documentation.
> Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload
I'm certainly guilty of this! What's the difference?
Your experiences are coherent with mine.
Developer experience is far ahead any other technology out there.
Std lib and its API design is world class, I wish cpp had as good stdlib. Tooling is strong, especially debugger
Agree with all of this. I think the biggest problem with popularity around these tools is that they're too good.
When everything just works, you have a lot more time to focus on your problem. Once you can focus on your problem, you may quickly find out you don't actually care about it. Focusing on tools is a great way to hide from this reality.
> IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked.
https://rr-project.org/
LINQ is a veritable footgun in any large team I find. While it's extremely powerful and really nice, it's so so so easy to blow your toes off if you don't know what you are doing. Some of my favourite screw ups I saw:
* Not understanding when something is evaluated.
* Not understanding the computational complexity of multiple chained operations/aggregates.
* Not understanding the expectation that Single() requires exactly one of something.
* Not understanding how damn hard it is to test LINQ stuff.
How much of that is LINQ-specific and how much is just the cost of using an ORM to build queries rather than typing them out as SQL?
I've never encountered testing problems with LINQ-to-objects.
Single gives you some guarantees about the returned value. Use First/FirstOrDefault if you don't need those guarantees. You can also provide predicate for FirstOrDefault to select First element that matches your predicate.
> Enumerable.Single Method - Returns a single, specific element of a sequence.
Some overload descriptions:
- Returns the only element of a sequence that satisfies a specified condition, and throws an exception if more than one such element exists.
- Returns the only element of a sequence, and throws an exception if there is not exactly one element in the sequence.
https://learn.microsoft.com/en-us/dotnet/api/system.linq.enu...
> Enumerable.FirstOrDefault Method - Returns the first element of a sequence, or a default value if no element is found.
https://learn.microsoft.com/en-us/dotnet/api/system.linq.enu...
>Not understanding the expectation that Single() requires exactly one of something.
Sorry, but idk how it is footgun of LINQ. It is like complaining about 0 or 1 based indexing
>Not understanding how damn hard it is to test LINQ stuff.
Any examples? Because I struggle to see such
I sort of agree. I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting.
> Not understanding how damn hard it is to test LINQ stuff.
I disagree with this. Just run the LINQ query on a compatible iterable.
Regarding the testing, it's more the sheer multiplicative number of cases you have to consider on a LINQ expression.
> I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting.
You can write hard to read code with any framework. Yes it takes effort sometimes to make linq code clear, but you should not give up on it.
> * Not understanding when something is evaluated.
Linq is lazy. .ToList() reifies. there, that's the gist of what you need to know. Not hard.
> Not understanding the expectation that Single() requires exactly one of something.
eh? There are a bunch of these methods, Single, SingleOrDefault, First, FirstOrDefault, Last, LastOrDefault and you can look up and grasp how they differ. It's fairly simple. I don't know what the problem is, outside of learning it.
> Not understanding how damn hard it is to test LINQ stuff.
Hard disagree. LInq chains can be unit tested, unless your Db access is mixed in, which is not a LINQ issue at all, it is a database query testing issue. LINQ code, in itself, is easily unit testable.
OK, so since I've never actually bothered to look at what LINQ actually is.. there may be others in the same boat: "LINQ allows you to query any enumerable collections such as List<T>, Array, or Dictionary<TKey,TValue>.". Got to admit I still find the LINQ to DB passtru rather opaque even after reading up on it: https://stackoverflow.com/questions/30716776/passing-a-query...
Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway). Does anything library wise come close to this within the Golang ecosystem?
While LINQ does include a library of extension methods for functional programming with .NET collections (which is great), it also includes "Expression Classes". In a nutshell, this allows a user to pass a single expression lambda to a function, and the function implementor receives the abstract syntax tree for the lambda, not the lambda itself. You can then not only receive and analyze these trees, you can also manually build and compile them. This effectively allows a limited set of runtime macros within .NET.
> Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL.
Pretty much. There's the "language integrated" version which looks a lot like SQL:
But that just translates to the method-orientated one which many people prefer If instead of querying a List or Dictionary you query a database, using an ORM (usually Entity Framework), that will actually get converted to SQL and run on the DB.> Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway)
It's more general and reusable than SQL, so you can map a subset of it to SQL, which is what some object-relational mappers do.
It's a shame, actually, that .NET performance improvements of up to x1000 could still be found after two decades and hundreds of millions spent on development.
Most of the time, it is not because there were too many slow things to be improved, it is mostly because they are adding more facilities to the runtime, enabling other performance improvements.
For example, the ReadOnlySpan type is a recent addition to the runtime and it will allow faster iterations when used. They simply enabled Linq to use it now.
ReadOnlySpan is a breakthrough innovative data structure, consisting of a pointer and a _length, that took Microsoft Corporation two decades to invent.
Well, better late than never.
other languages do not have special pointers that can point to GC object interiors, be transparently addressed with arithmetics and cooperatively updated by GC without pinning, while also allowing to point to stack and unmanaged memory
But those pointers were around since .NET 1.0. Not only that, but things like e.g. ArgIterator were also there! Span could have been there too; it was really a matter of design rather than technical capability.
I think the main reason why C# didn't have this (and other low-level features like plain function pointers) for so long is because the original vision for .NET was that you'd mix and match different languages in your project as needed. So if you needed that kind of stuff, you'd reach out for managed C++ (and later, C++/CLI), while C# was kept deliberately more high-level.
And so once that approach was abandoned, C# needed to catch up to be able to play the "one language that can do it all" role.
I see, thanks. Still, x1000 performance improvement is called a bug fix.