I wrote a blog post some time ago which contrasts the "Schwartzian transform" with the slightly clearer alternative used by most APL-family languages and (in my opinion) an even cleaner approach which is possible in languages with a first-class table type: https://beyondloom.com/blog/rankingoffruits.html
That was a lovely vignette, thanks. Much earlier in my career I had exactly the sense suggested that “something’s off” about the Schwartzian, even though it became a useful and idiomatic tool at the time. I only really understood the disquiet much later, when working with columnar data, and realising the positive qualities of declaring structs-of-arrays rather than arrays-of-structs, with subsequently excellent mechanical sympathy, and so on from there.
I find it interesting that the transform was controversial in the '90s.
Today, it seems like a normal solution to the problem to me, and the controversy seems silly. I have much experience with the map function from Javascript. It is too simple to be objectionable.
But in the '90s, I would also have had trouble understanding the transform. Lambdas/closures were unheard of to everyone except Lisp dweebs. Once I figured out what the code was doing, I would be suspicious of its performance and memory consumption. This was 1994! Kilobytes mattered and optimal algorithmic complexity was necessary for anything to be usable. Much safer to use a well understood for loop. I have plenty of experience making those fast, and that's what map() must be doing under the hood anyway.
But I would have been wrong! map() isn't doing anything superfluous and I can't do it faster myself. The memory consumption of the temporary decorated array is worth it to parse the last word N times instead of N log N times. Lisp is certainly a slow language compared to C, but that's not because of its lambdas!
I believe I first came across this in the early xslt days, when I had a manager that wanted to write all software with an XML requirements spec, then use xslt to simultaneously generate code AND documentations.
1. It did not work because of course it didn't work
2. This meant that all data output of the system would done in XML and transformed, so I got to know xslt quite well
3. It did give me a fun moment at a conference where a speaker asked "Who knows the Schwartzian transform or has used it?" and only me and my coworkers raised their hands
Obviously, the Schwartzian Transform can be manually implemented in any modern language elegantly, but it's interesting that only C# uses it implicitly!
PS: The fancy solution would be a temporary "Hashset<T,K>" used to map values to the transformed sort keys, so that the mapping Foo(T t): K would only be called once for each unique input value 't'. That way the transformation could be invoked less than O(n) times!
Inspired by your behavior, I looked up the Wikipedia article on Schwartzian transform[1]. The article mentioned that Rust not only provides sort_by_key, but also sort_by_cached_key. :D
To be fair, this isn't what normal Perl usually looks like. Perl allows you to write like this but most people don't. This is more like Duff's Device in that it works and it's clever and you'd can anyone who put it in production code.
That's not to say that idiomatic Perl doesn't have its quirks and oddities - Perl idioms do throw off people who aren't used to them. But this is a bit far even for Perl.
I wrote a blog post some time ago which contrasts the "Schwartzian transform" with the slightly clearer alternative used by most APL-family languages and (in my opinion) an even cleaner approach which is possible in languages with a first-class table type: https://beyondloom.com/blog/rankingoffruits.html
Unfortunately, the only such language to which most workaday devs have access is some dialect of SQL. Fortunately, at least there's that.
That was a lovely vignette, thanks. Much earlier in my career I had exactly the sense suggested that “something’s off” about the Schwartzian, even though it became a useful and idiomatic tool at the time. I only really understood the disquiet much later, when working with columnar data, and realising the positive qualities of declaring structs-of-arrays rather than arrays-of-structs, with subsequently excellent mechanical sympathy, and so on from there.
I find it interesting that the transform was controversial in the '90s.
Today, it seems like a normal solution to the problem to me, and the controversy seems silly. I have much experience with the map function from Javascript. It is too simple to be objectionable.
But in the '90s, I would also have had trouble understanding the transform. Lambdas/closures were unheard of to everyone except Lisp dweebs. Once I figured out what the code was doing, I would be suspicious of its performance and memory consumption. This was 1994! Kilobytes mattered and optimal algorithmic complexity was necessary for anything to be usable. Much safer to use a well understood for loop. I have plenty of experience making those fast, and that's what map() must be doing under the hood anyway.
But I would have been wrong! map() isn't doing anything superfluous and I can't do it faster myself. The memory consumption of the temporary decorated array is worth it to parse the last word N times instead of N log N times. Lisp is certainly a slow language compared to C, but that's not because of its lambdas!
I believe I first came across this in the early xslt days, when I had a manager that wanted to write all software with an XML requirements spec, then use xslt to simultaneously generate code AND documentations.
1. It did not work because of course it didn't work 2. This meant that all data output of the system would done in XML and transformed, so I got to know xslt quite well 3. It did give me a fun moment at a conference where a speaker asked "Who knows the Schwartzian transform or has used it?" and only me and my coworkers raised their hands
Python's key-functions nicely encapsulate the whole process.
Thank you for posting! I always felt like such a badass in the late 1990s when I used this.
I was curious to see what other languages do by default (or not), so I quickly tested a few:
C# evaluates the lambda only once per item, in input order: https://dotnetfiddle.net/oyG4Cv
Java uses "Comparators", which are called repeatedly: https://www.jdoodle.com/ia/1IO5
Rust has sort_by_key() which is encouraging... but also calls the key transformation many times: https://play.rust-lang.org/?version=stable&mode=debug&editio...
C++20 with std::range also calls the lambda repeatedly: https://godbolt.org/z/raa1766PG
Obviously, the Schwartzian Transform can be manually implemented in any modern language elegantly, but it's interesting that only C# uses it implicitly!
PS: The fancy solution would be a temporary "Hashset<T,K>" used to map values to the transformed sort keys, so that the mapping Foo(T t): K would only be called once for each unique input value 't'. That way the transformation could be invoked less than O(n) times!
Inspired by your behavior, I looked up the Wikipedia article on Schwartzian transform[1]. The article mentioned that Rust not only provides sort_by_key, but also sort_by_cached_key. :D
[1]: https://en.wikipedia.org/wiki/Schwartzian_transform
I'm so grateful for the Lisp translation. Perl is such a write-only language, my hair curls up just from looking at it.
Eh. It's not worse than Ruby, and vice versa.
To be fair, this isn't what normal Perl usually looks like. Perl allows you to write like this but most people don't. This is more like Duff's Device in that it works and it's clever and you'd can anyone who put it in production code.
That's not to say that idiomatic Perl doesn't have its quirks and oddities - Perl idioms do throw off people who aren't used to them. But this is a bit far even for Perl.
I've definitely seen it in production code, but it was a good deal after 1994.
By then it was a commonly known technique, so it was a pattern a Perl dev would be expected to recognize.