I've been thinking recently about how things like Project Euler, LeetCode, and to a bit less of an extent, Advent of Code, are so heavily focused on making clever use of math, data structures and algorithms, that it makes them suboptimal as a tools for getting familiar with a new programming language.
I know that that critique isn't new to anyone but it makes me think about how it would be cool if there were a code puzzler site that is specifically geared towards little self-contained tasks that are more to do with forcing you to get familiar with the common everyday tasks of software development.
Some example puzzlers could be like:
- open an http server on port 80
- open a file and write this data to it
- write temporary files to a location, deleting them when process exits
- query a database
- deal with such and such error scenario
- find a way to test this component
- bundle this code as an executable
- sanitize user input here
- make this behavior configurable
- take the config from environment variable variable and/or config file and/or arguments
- parse this data file
You do get a bit of parsing and file handling with Advent of Code but imagine a series of dozens of small problems that grill you on every corner of the python filesystem api. Would be a lot less dry than reading docs cover to cover.
> Advent of Code, are so heavily focused on making clever use of math, data structures and algorithms
I've done a fair amount of Advent of Code and I wouldn't say it's at all "focused" on this. The vast majority of the questions use hash tables and graph traversal as the full extent of their use of math/DS/algos.
There's always one or two puzzles every year that require some particular math/CS insight but most of them just need you to know BFS and/or how to write a parser.
Your examples are also not bad, but they seem to be primarily concerned with "getting familiar with a new programming language" in the context of writing a web server, which is one of the parts of programming I try to stay away from. Most of your examples require less familiarity with the language's features and more with libraries you might use, which is less interesting to me personally (then again, I'm a PL fan and I write compilers for a living).
Meanwhile, I like AoC because I've used language features to take the fairly straightforward potential implementations and write them more elegantly in the language I choose. e.g. I use Monads in Haskell, or use Rust's easy threading to parallelize solutions, etc.
For me, learning a new programming language is largely uninteresting unless it changes the fundamental "shape" I think in, rather than what the exact names of the libraries I use change to. e.g. I already know Java so I'm not really going to bother "learning" C#. I already know Python so I don't really bother diving deep into Ruby, etc. However, I learn Haskell, Rust, Forth, Erlang, Scheme, etc.
AoC is still is algorithms and data structures: there's minimal interaction with the outside world, just solving the problem for the input data. It's just about coming up with the algorithm yourself instead of applying fancy well-known ones.
I think a lot of people here would be surprised how much of a step up this is from classic leetcode or DSA stuff. I have been involved in introducing people to AoC and helping them and the amount of people who have basic knowledge of algorithms but struggled to parse the input from a file was a little shocking to me at first. Of course, I do not blame anyone for not knowing something, classic academic courses can be misguided sometimes.
This doesn't negate the fact that there is somewhat of a lack in problems with more outside world interaction and it would be cool to see more of that.
Exercism.io does what you want? It has language tracks and each track has questions geared to seal your understanding of some language concept. It also has it gamified by building a community around it and folks comparing their solutions.
This might be a little bit nearer to what you look for.
---
> it would be cool if there were a code puzzler site that is specifically geared towards little self-contained tasks
In my opinion the tasks on Project Euler or LeetCode are much more self-contained than what you suggest as alternatives: many of your suggestions are deeply intertwined with libraries (and understanding them) for doing the respective task instead of being self-contained programming tasks.
The best recommendation I have for anyone trying to learn a new programming language is to try and program a board game.
There will be clear rules (business logic), UI, etc.
It's a confined enough problem that you can implement it without too much effort but deep enough that you can get a feel for how that programming language, framework, whatever works.
Plus there's a near endless set to choose from and it's easily scalable to the level of complexity you want. If it works add AI players, network play, etc.
I think we are a bit alike in our views, but I have a slightly different take on it. I consider coding something like a Chip-8 emulator to be more fun and optimal. It gives a holistic view of the language - you get to work with simple graphics, sound effects, and gain a feel for memory operations and data structures, as well as control structures like conditionals, looping, and exception handling. If that’s not all - for beginners, it provides an introduction to virtualizing CPUs with registers, stacks, opcode handling, memory units, arithmetic/bitwise operations, and more. You’ll even learn a bit about concurrency and synchronization, and by extension, threading. Also, performance optimization.
I suppose a decent game project could achieve these things too, but the real fun of Chip-8 is in throwing different ROMs at it and debugging the issues until it’s perfect enough to play all your favorite games!
to be fair, PE is not designed or meant for helping people learn a language. that isn't the project's intent.
people do like to say they use PE for learning new languages, but I doubt that is a useful exercise beyond maybe the first dozen problems or so. And even then, if the solution isn't obvious to you, you're doing two things at once - learning a language and solving a math puzzle. I don't see why people would sign up to get frustrated like that.
> I don't see why people would sign up to get frustrated like that.
I actually use this as a learning trick. Pick two or three things to learn simultaneously, then when I get stuck on one aspect, switch to another. When I finally switch back, I often find the background time I gave my brain to process the problem means I'll now be much faster to get unstuck on the original issue.
There's definitely ways this can go sideways, and it's not for everybody, but I find it pretty effective.
I guess the trouble is that in the case of LeetCode/PE, people actually just want the one thing (programming language proficiency), and not the other (mathematics). Learning 2 things at once can be very useful, but a lot of people would probably choose something else for their second skill. Most relevant for developers might be, networking, OS's, version control, databases, testing, CI/CD, security, concurrency.
Oh yeah totally, it's not a criticism of PE, that's not what it's meant for. People just use PE and LC and AoC because that's the closest thing, but I think there is space in the market for a product I describe that really drills down on getting you familiar with the common tasks and stdlib of various languages.
Codewars can do this, they keep how problems was solved. If a problem was solved using some string by most answers than it is a good problem to learn that part of the language. With documentation you have really good way to cover all parts of a programming language.
I really, really like this list. I've been wondering for the last year what the "optimal" problem is to learn at least the syntax of a language. After learning to run something, how to print to the console, I like using heapsort to start learning syntax of a new language, then reading/writing to a file, then building a small TodoList server.
Most of these wouldn't qualify as "puzzles", would they?
I find it nice to learn new languages via data structure puzzles, because to me the data structures of a language feel like the grammar and once I have that down everything else falls into place
I disagree. Yes, you have to learn how to work with the basic data structures of a language, but 90% of programming, for most people, is not that. It's IO, error handling, db querying, logging, input parsing, parameterization, business logic, preserving backwards compatibility, persistence, state management, testing, mocking, benchmarking, build design (for lack of better term -- futzing around with Make/Gradle/Npm, Dockerfiles). All of that doesn't just fall out of learning DS/Alg's, it takes time to become familiar and fluent in how all these are done in your ecosystem.
When employers or team mates ask you if you "know" or "are competent in" Java they don't care if you know how to work with lists, arrays, loops, hashmaps and sets. Well, I mean, that's table stakes. They're asking if you're familiar with the idiosyncrasies of the language with respect to those above concerns.
Imo DS/Algs to an extent are a good prerequisite.
Once you know them in one language that knowledge is portable enough to get you up-to-speed in $lang.
Then, and no earlier will I start worrying about doing real things in my programs and the "idiosyncracies"
Exercism.io does what you want? It has language tracks and each track has questions geared to seal your understanding of some language concept. It also has it gamified by building a community around it and folks comparing their solutions.
Something I'd love to see is "AoC hard mode": the exact same problems but the input data set is ~10 GB, and/or similarly "scaled" such that naive solutions fail outright.
Other scaling-of-inputs could include: Text with line-lengths over 2 GB, numbers above 2^60, data designed such that naive nested-loop solutions (quadratic scaling) take over a year to compute the answer, etc...
Basically, force developers to solve the problem robustly with: streaming, parallelism, efficient algorithms with good big-O properties, correct data type choice (including intermediate accumulator values!), and so forth.
It could be a three-star challenge feature added to the current version. It wouldn't even require large downloads: a Python script or something similar could be used to generate arbitrarily large inputs. (Alternatively, a common CDN-cacheable prefix with a distinct suffix per competitor.)
That's exactly what Project Euler problems often do, especially once you get past the first hundred or two. Problems are scaled so a brute-force often means hours to days of compute time, or worse.
You get to recognize the effect - if I see a problem that's clearly number-theory related and with a limit of 10^12, I know they're looking for a sublinear algorithm, probably O(n^(2/3)) thanks to various multiplicative function ideas that appear over and over.
1. Yes, solving these puzzles would mean often mean using a library.
2. However, for most of the things I listed above, in most languages, a competent software developer should be able to, and most likely would just use the standard library.
3. Why not both? I can imagine a catalog with thousands of problem sets. Some may challenge you to (re-)implement some existing functionality yourself (as a super basic example, re-implement the java Optional::flatMap method or something). Others could challenge you to make use of existing implementations. Learning to make use of stdlib and other libraries and tools in the ecosystem is part of one's growth, and also so is working through those tools, tinkering, trying to think how you would have implemented them, and getting a better understanding of their internals (or at least, an understanding of how their internals MIGHT work)
During the solving of a problem on Project Euler, I learned that compilers are smarter than me.
I don't remember the problem number or its title, but it involved starting from the top-left corner of a 2D grid and finding how many possible paths there are to get to the bottom-right corner while only moving either down or right.
My naive solution was a brute-force depth-first recursive search. On my CPU at the time, it took about 3 minutes to solve. I thought, the logic of this is incredibly simple, why not do it in assembly?
My assembly solution took 5 minutes.
I decompiled the compiled C code to see what it had done, but I couldn't understand it. My assembly knowledge was too basic.
Thinking on it now, I wonder if a modern compiler would solve the entire problem at compile-time and just hard-code the answer to be output at runtime.
> Starting in the top left corner of a 2x2 grid, and only being able to move to the right and down, there are exactly 6 routes to the bottom right corner.
ASCII art by hand (⌐■_■) it only took me 5 minutes or so … That said, I think someone could have done it faster with a tool like Monodraw (https://monodraw.helftone.com/) or something similar.
Oops! Thanks, just distraction... It's that first I wanted to see the general solution to those kind of problems (and I gave the solution for a wrong one in the class), then I wanted to verify the C implementation of the solution with POPCNT like the OP seems to have done (I am writing the code)...
Edit: ...and yes, it seems that brute-forcing (counting to one trillion) takes more time than I expected.
Now I see it. Twenty items (R operations) in 40 slots: 40!/(20!) and the order of the items does not matter, so again divide by 20!. The remaining slots are D operations.
The answer actually came to me in the shower this morning.
Something this illustrates, though - once you solve a problem, read the forum thread (the link will be at the bottom of the problem page when solved).
I learned so much poring over the solutions of earlier solvers. Without that knowledge, there's no way I could have gotten lots of later problems.
Sure, problem #15 is very early and I happened to know the answer right off, back the first day I started solving PE problems. But even so, there are some gems in the posts from other solvers.
On second look I think it can be further generalized to N+M nCr M for all grids NxM where N >= M since the number of ways should depend on combinations of the smaller number.
I'm a bit disappointed I didn't see this quicker back when I did this problem like 15 years ago or whatever it is now. My solution was to use matrix exponentiation on the adjacency matrix of the path graph, which is a heck of a lot better than brute force enumeration of all paths, but still roughly cubic rather than linear like computing a bunch of factorials. Even as a math major, I'd done a lot more linear algebra than combinatorics.
That is mind blowing, but it’s not immediately obvious to me that it’s equivalent for n > sqrt(INT_MAX). Is it? And if so, is the compiler somehow smart enough to know that?
We all have our skills and super powers and yours do not involve optimizing for time by switching from C to ASM. Fuck that! I bet you have super powers of some sort.
Compilers are not smarter than you - that's daft. The nutters that program the compilers and tweak and twiddle them, are better informed about how to deliver faster machine code for a given task.
One of your super powers is to know when to say: "fuck it, I'm trotting off and getting by with a five minutes runtime".
Project Euler is a lot of fun if you like a dash of math in your programming. The problems generally won't apply to your job or even to interviews, so don't go in expecting that.
My favorite is https://projecteuler.net/problem=113, "Non-Bouncy Numbers." It takes some clever tricks to figure out but doesn't require any significant background knowledge, and the optimizations required to get it to run within 60 seconds (at least for my approach) all felt reasonable.
Well said, that is always how I saw it as well. The sort of math problem solving we did for fun in school but all the problems require programmatic thinking and usually eventually an algorithm. I learned so much from doing Eulers. Even basic stuff I thought I would know like the best way to get GCD
That’s mostly right, but some of them don’t require a computer. I’ve never solved one with just pencil and paper, but in some of the solution threads you will actually find pencil and paper solutions. I’m not sure if any of the later problems work like that, though.
On a side note, I remember when the website got hacked [1][2].
Many people, including myself, migrated to other platforms, but Project Euler problems always remained math-focused compared to the myriad of other websites like LeetCode and HackerRank, among others, listing programming-focused problems, which eventually popularized the use in modern tech interviews.
I wouldn't. The majority of problems there have mathematical optimizations which real problems never do. Worse if you start thinking in the way those problems encourage you to your programs will be completely invalidated even with a tiny change in the spec.
A good set of questions would be something between the advent of code - where the problems are hard because the spec is so bad on purpose - and project Euler - where the spec is so exact you don't really need a computer to solve the problems with enough thinking.
Something like 'plot the histogram of collatz sequence lengths of the first 100,000 numbers'.
The text always clearly states how your code has to behave, albeit it doesn't spell out every edge case you might overlook. Real world specs on the other hand is often contradictory and impossible to fullfil.
Agreed about learning a new language starting with PE.
After that, I like to invent a big gnarly scenario that I don't have to completely solve, but one that takes me well out of the range of typical cookie-cutter tutorials. I want to find all the sharp edges on a new language/library/framework.
Chatgpt can write a solution for this problem if we go up to 10^10. 10^16 is too hard!
I don’t understand this problem (I didn’t tackle it myself) but wanted to see how quickly chatgpt could solve it and how far along it would go.
First it made a naive solution that would work until 10^6. Then we used that output to verify improved versions.
And we managed to improve it until 10^10 only. (Staying within a minute timeout.)
It did a DP approach (it suggested itself). I suggested to try numba and numpy. And with that it managed until 10^10. I think it’s still brute forcing it and one might leverage much better techniques in order to reach 10^16.
It hadn’t occurred to me that PE is a pretty good test for ChatGPT or SOTA LLM’s in general. Even if explicit solutions for earlier problems do appear somewhere on the web, it’s probably a fairly safe bet that the latest version of ChatGPT did not include the latest Project Euler problem in its training data.
For those who enjoy regular streams of programming puzzles similar to Project Euler, The Weekly Challenge is another fun resource: https://theweeklychallenge.org
I don't know that I'd gauge anything by popularity with the HN crowd. It's also a diverse group. I'm closing in on a hundred problems solved, in a not very completion-ist fashion (I've got a whole bunch of skips and random choices of puzzles).
Maths isn't my strongest suit, and I have no academic comp-sci background, so there's been a number of these I sort of brute force and then go read the answers in the thread; or I brute force the first few integers in the sequence and then try and wrap my head around what https://oeis.org/ is attempting to tell me about them.
It has challenged me a bit on some of my fundamentals with programming, really making me think about efficiency etc.
While I've done most of the problems in rust so far, I've been having to refresh my knowledge of Go recently, so I've started porting answers between the two languages, and it's definitely helping there a bunch.
It’s my game plan for keeping my brain active in retirement. Been heavily involved for the past 7 years. Been at the 100% solved level since summer 2023, though I’m back to one away the past couple weeks - PE910 is _hard_
I've got a bachelor's in math, but that's 40+ years ago. I had intended to go on for a PhD in math, but fell into computers instead - programming was easier and way more lucrative, even in the early 80s. Once I was retired and found my way to Project Euler, it became an obsession, tickling that desire to go deeper into math that I had in my college days.
I attacked roughly the first 250 problems in order. The early problems build on each other to introduce new topics. I also got good at figuring out the right search term to find some random paper in number theory, combinatorics, probability, whatever.
Later problems introduced new, more niche areas, like chromatic polynomials and impartial & partisan game theory. But by then, I found it much easier to figure out what part of math a problem was based on and how to find relevant literature.
It helps to be really really stubborn, and to have the patience to let a problem stew in my brain, sometimes for weeks at a time. That seems to help lead to that Eureka moment.
One other thing - can't believe I forgot to mention this: once you solve a problem, read the solvers' thread in the forum! I learned so much by doing that, which fed into success on later problems. The link will be at the bottom of the problem page once you've solved it.
There are some much later problems where some obscure technique gets mentioned, even though the problem is doable without that technique. But then later on, there are other problems where that technique is practically required. I can think of multiple 100% difficulty problems which were actually much easier than that for me, because I had already seen and tried out the techniques that enable a fast solution.
And sorry, not going to mention any of those techniques. A lot of the fun I have in solving PE problems is that incremental increase in knowledge as time goes on.
As a class of problems, I'd say the combinatorial game theory ones are my favorites. There are a lot of impartial game theory problems - look for problems mentioning Nim or stone games. They build on each other nicely, from the mid 300s on. The site has been getting into partisan game theory problems in the past year, which finally got me to buy "Winning Ways For Your Mathematical Plays", vol 1, and "Lessons In Play". I find pretty much any problem with John Conway's influence fun to do.
As for a single problem, I'm fond of PE589, "Poohsticks Marathon". That was my 501st solution, two years after first attempting it (solved 5 years ago, yikes). I like it because it's a problem with a 95% difficulty rating, so very tough, but the development team slotted it in as an easy problem (problems normally get scheduled in batches of 6 with a cadence of medium/easy/medium/easy/medium/hard). Once I solved it, I agreed that it was relatively easy, in that it uses techniques introduced by early PE problems, but something about it makes using those techniques unexpectedly difficult.
I have been working on PE problems for most of 10 years. One thing I would sort of like to do is make a library (mostly I have been using python) for some of the more common functions. Do you have anything like that?
I do, but there's nothing too obscure in it. Efficient prime number sieve, prime factorization using trial division, generating list of divisors from the prime factorization, modular inverse via Euclidean algorithm, Chinese Remainder Theorem.
I also have some stand-alone modules, one to solve generalized Pell equations, another to find a polynomial given a sequence via the differences (e.g. 2, 5, 10, 17, first differences 3, 5, 7, second 2, 2 is enough to find n^2+1). There's another to find the closed form for a sequence as a linear recurrence.
Some solvers have much more extensive libraries, but I tend to grab bits of code from old solutions to reuse on the fly.
It’s been discussed quite a bit in the past. I love Project Euler, but haven’t been active for a few years. Even then I felt like the social aspect of it wasn’t designed for newcomers; a lot of activity on new problems happens as soon as it becomes available and then it mostly freezes. The problems get pretty hard on average once you’re past the first hundred, one reason I personally found it hard to keep going.
There's certainly a strong element of computational mathematics. A lot of the problems deal with things like Mersenne Primes or Collatz sequences. On the other hand, there are problems like "Maximum Path Sum" which definitely make use of DSA-style tricks (https://projecteuler.net/problem=67).
Has here been any work done to find out about the average HN user? It seems like it would be a hard thing to do but it might be interesting to see. I do not even know what sort of metrics would be useful to measure.
I have great memories of competing with a college friend to solve these problems. Since I last seriously did the problems, probably 15 years ago, they have added a lot more.
o1 thought for 105 seconds, cycling through many relevant-sounding status messages like "looking for patterns," before writing a collection of thematic but flawed thoughts. The "Calculation Steps" approach is incorrect, but correctly implemented by the code.
It flubs a basic calculation that it correctly implements in python:
"10^16 mod (10^9 + 7) = 49" (it's actually 930000007)
but succeeds in a seemingly harder calculation:
"the modular inverse of 12 modulo 10^9 + 7 is 83333334"
Finally, o1 claims the code prints "0" when it actually prints "982790507" (both wrong answers).
Note: input was copied from the html-only Project Euler url since the formulas in the human-optimized url are not copyable: https://projecteuler.net/minimal=912
I've been thinking recently about how things like Project Euler, LeetCode, and to a bit less of an extent, Advent of Code, are so heavily focused on making clever use of math, data structures and algorithms, that it makes them suboptimal as a tools for getting familiar with a new programming language.
I know that that critique isn't new to anyone but it makes me think about how it would be cool if there were a code puzzler site that is specifically geared towards little self-contained tasks that are more to do with forcing you to get familiar with the common everyday tasks of software development.
Some example puzzlers could be like:
- open an http server on port 80
- open a file and write this data to it
- write temporary files to a location, deleting them when process exits
- query a database
- deal with such and such error scenario
- find a way to test this component
- bundle this code as an executable
- sanitize user input here
- make this behavior configurable
- take the config from environment variable variable and/or config file and/or arguments
- parse this data file
You do get a bit of parsing and file handling with Advent of Code but imagine a series of dozens of small problems that grill you on every corner of the python filesystem api. Would be a lot less dry than reading docs cover to cover.
> Advent of Code, are so heavily focused on making clever use of math, data structures and algorithms
I've done a fair amount of Advent of Code and I wouldn't say it's at all "focused" on this. The vast majority of the questions use hash tables and graph traversal as the full extent of their use of math/DS/algos.
There's always one or two puzzles every year that require some particular math/CS insight but most of them just need you to know BFS and/or how to write a parser.
Your examples are also not bad, but they seem to be primarily concerned with "getting familiar with a new programming language" in the context of writing a web server, which is one of the parts of programming I try to stay away from. Most of your examples require less familiarity with the language's features and more with libraries you might use, which is less interesting to me personally (then again, I'm a PL fan and I write compilers for a living).
Meanwhile, I like AoC because I've used language features to take the fairly straightforward potential implementations and write them more elegantly in the language I choose. e.g. I use Monads in Haskell, or use Rust's easy threading to parallelize solutions, etc.
For me, learning a new programming language is largely uninteresting unless it changes the fundamental "shape" I think in, rather than what the exact names of the libraries I use change to. e.g. I already know Java so I'm not really going to bother "learning" C#. I already know Python so I don't really bother diving deep into Ruby, etc. However, I learn Haskell, Rust, Forth, Erlang, Scheme, etc.
AoC is still is algorithms and data structures: there's minimal interaction with the outside world, just solving the problem for the input data. It's just about coming up with the algorithm yourself instead of applying fancy well-known ones.
> just solving the problem for the input data
I think a lot of people here would be surprised how much of a step up this is from classic leetcode or DSA stuff. I have been involved in introducing people to AoC and helping them and the amount of people who have basic knowledge of algorithms but struggled to parse the input from a file was a little shocking to me at first. Of course, I do not blame anyone for not knowing something, classic academic courses can be misguided sometimes.
This doesn't negate the fact that there is somewhat of a lack in problems with more outside world interaction and it would be cool to see more of that.
> It's just about coming up with the algorithm yourself instead of applying fancy well-known ones.
Isn’t that the whole fun?
Exercism.io does what you want? It has language tracks and each track has questions geared to seal your understanding of some language concept. It also has it gamified by building a community around it and folks comparing their solutions.
Have a look at Rosetta Code
> https://rosettacode.org/wiki/Rosetta_Code
This might be a little bit nearer to what you look for.
---
> it would be cool if there were a code puzzler site that is specifically geared towards little self-contained tasks
In my opinion the tasks on Project Euler or LeetCode are much more self-contained than what you suggest as alternatives: many of your suggestions are deeply intertwined with libraries (and understanding them) for doing the respective task instead of being self-contained programming tasks.
The best recommendation I have for anyone trying to learn a new programming language is to try and program a board game.
There will be clear rules (business logic), UI, etc.
It's a confined enough problem that you can implement it without too much effort but deep enough that you can get a feel for how that programming language, framework, whatever works.
Plus there's a near endless set to choose from and it's easily scalable to the level of complexity you want. If it works add AI players, network play, etc.
I think we are a bit alike in our views, but I have a slightly different take on it. I consider coding something like a Chip-8 emulator to be more fun and optimal. It gives a holistic view of the language - you get to work with simple graphics, sound effects, and gain a feel for memory operations and data structures, as well as control structures like conditionals, looping, and exception handling. If that’s not all - for beginners, it provides an introduction to virtualizing CPUs with registers, stacks, opcode handling, memory units, arithmetic/bitwise operations, and more. You’ll even learn a bit about concurrency and synchronization, and by extension, threading. Also, performance optimization.
I suppose a decent game project could achieve these things too, but the real fun of Chip-8 is in throwing different ROMs at it and debugging the issues until it’s perfect enough to play all your favorite games!
to be fair, PE is not designed or meant for helping people learn a language. that isn't the project's intent.
people do like to say they use PE for learning new languages, but I doubt that is a useful exercise beyond maybe the first dozen problems or so. And even then, if the solution isn't obvious to you, you're doing two things at once - learning a language and solving a math puzzle. I don't see why people would sign up to get frustrated like that.
> I don't see why people would sign up to get frustrated like that.
I actually use this as a learning trick. Pick two or three things to learn simultaneously, then when I get stuck on one aspect, switch to another. When I finally switch back, I often find the background time I gave my brain to process the problem means I'll now be much faster to get unstuck on the original issue.
There's definitely ways this can go sideways, and it's not for everybody, but I find it pretty effective.
I guess the trouble is that in the case of LeetCode/PE, people actually just want the one thing (programming language proficiency), and not the other (mathematics). Learning 2 things at once can be very useful, but a lot of people would probably choose something else for their second skill. Most relevant for developers might be, networking, OS's, version control, databases, testing, CI/CD, security, concurrency.
Oh yeah totally, it's not a criticism of PE, that's not what it's meant for. People just use PE and LC and AoC because that's the closest thing, but I think there is space in the market for a product I describe that really drills down on getting you familiar with the common tasks and stdlib of various languages.
I really think you'd like these two project pages, they were posted to HN some time ago but the original links have been slain by bitrot.
https://austinhenley.com/blog/challengingprojects.html https://austinhenley.com/blog/morechallengingprojects.html
That’s just a job. Might as well get paid while you do that kinda stuff
Codewars can do this, they keep how problems was solved. If a problem was solved using some string by most answers than it is a good problem to learn that part of the language. With documentation you have really good way to cover all parts of a programming language.
I really, really like this list. I've been wondering for the last year what the "optimal" problem is to learn at least the syntax of a language. After learning to run something, how to print to the console, I like using heapsort to start learning syntax of a new language, then reading/writing to a file, then building a small TodoList server.
Most of these wouldn't qualify as "puzzles", would they?
I find it nice to learn new languages via data structure puzzles, because to me the data structures of a language feel like the grammar and once I have that down everything else falls into place
I disagree. Yes, you have to learn how to work with the basic data structures of a language, but 90% of programming, for most people, is not that. It's IO, error handling, db querying, logging, input parsing, parameterization, business logic, preserving backwards compatibility, persistence, state management, testing, mocking, benchmarking, build design (for lack of better term -- futzing around with Make/Gradle/Npm, Dockerfiles). All of that doesn't just fall out of learning DS/Alg's, it takes time to become familiar and fluent in how all these are done in your ecosystem.
When employers or team mates ask you if you "know" or "are competent in" Java they don't care if you know how to work with lists, arrays, loops, hashmaps and sets. Well, I mean, that's table stakes. They're asking if you're familiar with the idiosyncrasies of the language with respect to those above concerns.
Imo DS/Algs to an extent are a good prerequisite. Once you know them in one language that knowledge is portable enough to get you up-to-speed in $lang. Then, and no earlier will I start worrying about doing real things in my programs and the "idiosyncracies"
I think you overestimate the proportion of programmers who primarily work on corporate stuff.
One that I like to suggest, which covers more terminal and Linux skills then programming, is https://overthewire.org/wargames/
Exercism.io does what you want? It has language tracks and each track has questions geared to seal your understanding of some language concept. It also has it gamified by building a community around it and folks comparing their solutions.
Thanks, I'll take a look!
Something I'd love to see is "AoC hard mode": the exact same problems but the input data set is ~10 GB, and/or similarly "scaled" such that naive solutions fail outright.
Other scaling-of-inputs could include: Text with line-lengths over 2 GB, numbers above 2^60, data designed such that naive nested-loop solutions (quadratic scaling) take over a year to compute the answer, etc...
Basically, force developers to solve the problem robustly with: streaming, parallelism, efficient algorithms with good big-O properties, correct data type choice (including intermediate accumulator values!), and so forth.
It could be a three-star challenge feature added to the current version. It wouldn't even require large downloads: a Python script or something similar could be used to generate arbitrarily large inputs. (Alternatively, a common CDN-cacheable prefix with a distinct suffix per competitor.)
That's exactly what Project Euler problems often do, especially once you get past the first hundred or two. Problems are scaled so a brute-force often means hours to days of compute time, or worse.
You get to recognize the effect - if I see a problem that's clearly number-theory related and with a limit of 10^12, I know they're looking for a sublinear algorithm, probably O(n^(2/3)) thanks to various multiplicative function ideas that appear over and over.
There are a couple posters on 4chan's /g/ threads on AoC that create "Big Boy" inputs, which is what you're looking for. It's unofficial, though.
If you're trying to model those "puzzlers" on actual dev work, then doing any of those things without a library/framework is a wrong answer.
Or is that your point? That coding like a pro means gluing those things together?
A few different things:
1. Yes, solving these puzzles would mean often mean using a library.
2. However, for most of the things I listed above, in most languages, a competent software developer should be able to, and most likely would just use the standard library.
3. Why not both? I can imagine a catalog with thousands of problem sets. Some may challenge you to (re-)implement some existing functionality yourself (as a super basic example, re-implement the java Optional::flatMap method or something). Others could challenge you to make use of existing implementations. Learning to make use of stdlib and other libraries and tools in the ecosystem is part of one's growth, and also so is working through those tools, tinkering, trying to think how you would have implemented them, and getting a better understanding of their internals (or at least, an understanding of how their internals MIGHT work)
During the solving of a problem on Project Euler, I learned that compilers are smarter than me.
I don't remember the problem number or its title, but it involved starting from the top-left corner of a 2D grid and finding how many possible paths there are to get to the bottom-right corner while only moving either down or right.
My naive solution was a brute-force depth-first recursive search. On my CPU at the time, it took about 3 minutes to solve. I thought, the logic of this is incredibly simple, why not do it in assembly?
My assembly solution took 5 minutes.
I decompiled the compiled C code to see what it had done, but I couldn't understand it. My assembly knowledge was too basic.
Thinking on it now, I wonder if a modern compiler would solve the entire problem at compile-time and just hard-code the answer to be output at runtime.
Lattice Paths — https://projecteuler.net/problem=15
> Starting in the top left corner of a 2x2 grid, and only being able to move to the right and down, there are exactly 6 routes to the bottom right corner.
> How many such routes are there through a 20x20 grid?Yup! That was it!
Good job on the ASCII art, btw.
Did you just whip out that amazing ASCII art or do you use a tool?
ASCII art by hand (⌐■_■) it only took me 5 minutes or so … That said, I think someone could have done it faster with a tool like Monodraw (https://monodraw.helftone.com/) or something similar.
> How many such routes are there through a 20x20 grid?
Is it 21! ?
No. It's 20!/(10!x10!)
You have 20 slots to be filled with 10 items vs other 10 items.
Close, it's 40!/(20!*20!)
20 Rs, 20 Ds in a 20x20 grid.
Example pattern: RRDDDR...D (40 letters)
Basically the number of permutations, with repetition, of 20 Rs and 20 Ds.
Oops! Thanks, just distraction... It's that first I wanted to see the general solution to those kind of problems (and I gave the solution for a wrong one in the class), then I wanted to verify the C implementation of the solution with POPCNT like the OP seems to have done (I am writing the code)...
Edit: ...and yes, it seems that brute-forcing (counting to one trillion) takes more time than I expected.
Now I see it. Twenty items (R operations) in 40 slots: 40!/(20!) and the order of the items does not matter, so again divide by 20!. The remaining slots are D operations.
The answer actually came to me in the shower this morning.
Nope, and only off by a few orders of magnitude.
If it is any comfort, my answer produced a non-negative integer - just like the true answer ))
Am I missing something here or would that be as simple as 2N nCr N for an NxN grid?
You’re not missing anything. This is an easy problem aimed at people who are not familiar with basic combinatorics.
Something this illustrates, though - once you solve a problem, read the forum thread (the link will be at the bottom of the problem page when solved).
I learned so much poring over the solutions of earlier solvers. Without that knowledge, there's no way I could have gotten lots of later problems.
Sure, problem #15 is very early and I happened to know the answer right off, back the first day I started solving PE problems. But even so, there are some gems in the posts from other solvers.
On second look I think it can be further generalized to N+M nCr M for all grids NxM where N >= M since the number of ways should depend on combinations of the smaller number.
I'm a bit disappointed I didn't see this quicker back when I did this problem like 15 years ago or whatever it is now. My solution was to use matrix exponentiation on the adjacency matrix of the path graph, which is a heck of a lot better than brute force enumeration of all paths, but still roughly cubic rather than linear like computing a bunch of factorials. Even as a math major, I'd done a lot more linear algebra than combinatorics.
So fun fact, if you compile
clang, with -O2, will turn this into the polynomial (n+1)*n//2. It can also do similar transformations for multiple loops.https://godbolt.org/z/so6neac33
So if you do a brute force solution which could have been reduced to a polyomial, clang has a shot of doing just that.
That is mind blowing, but it’s not immediately obvious to me that it’s equivalent for n > sqrt(INT_MAX). Is it? And if so, is the compiler somehow smart enough to know that?
Integer overflow is actually undefined behaviour thus the compiler is free to assume it doesn't happen.
If you assume two’s complement arithmetic, then this is always equivalent because you’re basically just calculating the answer in a modular ring.
I believe the term for this is scalar evolution.
Yep! That is it alright.
Here's a talk from an llvm conference with the details.
https://www.youtube.com/watch?v=AmjliNp0_00
We all have our skills and super powers and yours do not involve optimizing for time by switching from C to ASM. Fuck that! I bet you have super powers of some sort.
Compilers are not smarter than you - that's daft. The nutters that program the compilers and tweak and twiddle them, are better informed about how to deliver faster machine code for a given task.
One of your super powers is to know when to say: "fuck it, I'm trotting off and getting by with a five minutes runtime".
Sounds like problem 15 [1]?
[1]: https://projecteuler.net/problem=15
Yup! That was it!
If you want a few similar problems, in ascending order of difficulty, try #81 through #83:
https://projecteuler.net/problem=81
https://projecteuler.net/problem=82
https://projecteuler.net/problem=83
Couldn't all three of those be solved using the A* path-finding algorithm?
Sshhh...
Project Euler is a lot of fun if you like a dash of math in your programming. The problems generally won't apply to your job or even to interviews, so don't go in expecting that.
My favorite is https://projecteuler.net/problem=113, "Non-Bouncy Numbers." It takes some clever tricks to figure out but doesn't require any significant background knowledge, and the optimizations required to get it to run within 60 seconds (at least for my approach) all felt reasonable.
More than a dash. I would describe Project Euler as math problems that you need a computer to solve.
Well said, that is always how I saw it as well. The sort of math problem solving we did for fun in school but all the problems require programmatic thinking and usually eventually an algorithm. I learned so much from doing Eulers. Even basic stuff I thought I would know like the best way to get GCD
That’s mostly right, but some of them don’t require a computer. I’ve never solved one with just pencil and paper, but in some of the solution threads you will actually find pencil and paper solutions. I’m not sure if any of the later problems work like that, though.
Why problem 912 specifically?
On a side note, I remember when the website got hacked [1][2].
Many people, including myself, migrated to other platforms, but Project Euler problems always remained math-focused compared to the myriad of other websites like LeetCode and HackerRank, among others, listing programming-focused problems, which eventually popularized the use in modern tech interviews.
[1] https://news.ycombinator.com/item?id=9990221
[2] https://www.reddit.com/r/math/comments/28jp7x/x/
It's the most recent
The OG LeetCode. Highly recommend, helpful for becoming fluent in a programming language.
I wouldn't. The majority of problems there have mathematical optimizations which real problems never do. Worse if you start thinking in the way those problems encourage you to your programs will be completely invalidated even with a tiny change in the spec.
A good set of questions would be something between the advent of code - where the problems are hard because the spec is so bad on purpose - and project Euler - where the spec is so exact you don't really need a computer to solve the problems with enough thinking.
Something like 'plot the histogram of collatz sequence lengths of the first 100,000 numbers'.
The AoC spec isn't bad though?
The text always clearly states how your code has to behave, albeit it doesn't spell out every edge case you might overlook. Real world specs on the other hand is often contradictory and impossible to fullfil.
I'd argue you can usually find problems on project euler that are a little more obscure than what you'd typically get on leetcode.
Advent of Code is great for this.
Agreed about learning a new language starting with PE.
After that, I like to invent a big gnarly scenario that I don't have to completely solve, but one that takes me well out of the range of typical cookie-cutter tutorials. I want to find all the sharp edges on a new language/library/framework.
Chatgpt can write a solution for this problem if we go up to 10^10. 10^16 is too hard!
I don’t understand this problem (I didn’t tackle it myself) but wanted to see how quickly chatgpt could solve it and how far along it would go.
First it made a naive solution that would work until 10^6. Then we used that output to verify improved versions.
And we managed to improve it until 10^10 only. (Staying within a minute timeout.)
It did a DP approach (it suggested itself). I suggested to try numba and numpy. And with that it managed until 10^10. I think it’s still brute forcing it and one might leverage much better techniques in order to reach 10^16.
It hadn’t occurred to me that PE is a pretty good test for ChatGPT or SOTA LLM’s in general. Even if explicit solutions for earlier problems do appear somewhere on the web, it’s probably a fairly safe bet that the latest version of ChatGPT did not include the latest Project Euler problem in its training data.
For those who enjoy regular streams of programming puzzles similar to Project Euler, The Weekly Challenge is another fun resource: https://theweeklychallenge.org
I love Project Euler, it’s my go-to for learning the basics of new programming languages I want to learn.
I miss having the time and energy to do these. Maybe in retirement :)
I have spent a little time with Project Euler. Is it very popular with those here at HN?
I don't know that I'd gauge anything by popularity with the HN crowd. It's also a diverse group. I'm closing in on a hundred problems solved, in a not very completion-ist fashion (I've got a whole bunch of skips and random choices of puzzles).
Maths isn't my strongest suit, and I have no academic comp-sci background, so there's been a number of these I sort of brute force and then go read the answers in the thread; or I brute force the first few integers in the sequence and then try and wrap my head around what https://oeis.org/ is attempting to tell me about them.
It has challenged me a bit on some of my fundamentals with programming, really making me think about efficiency etc.
While I've done most of the problems in rust so far, I've been having to refresh my knowledge of Go recently, so I've started porting answers between the two languages, and it's definitely helping there a bunch.
https://oeis.org/ is an incredible website! Thanks for sharing.
It’s my game plan for keeping my brain active in retirement. Been heavily involved for the past 7 years. Been at the 100% solved level since summer 2023, though I’m back to one away the past couple weeks - PE910 is _hard_
Wow. The idea of getting to 100% on PE is almost incomprehensible to me. I've solved basically none outside the first couple pages.
What was your strategy like? How much math background do you have?
I've got a bachelor's in math, but that's 40+ years ago. I had intended to go on for a PhD in math, but fell into computers instead - programming was easier and way more lucrative, even in the early 80s. Once I was retired and found my way to Project Euler, it became an obsession, tickling that desire to go deeper into math that I had in my college days.
I attacked roughly the first 250 problems in order. The early problems build on each other to introduce new topics. I also got good at figuring out the right search term to find some random paper in number theory, combinatorics, probability, whatever.
Later problems introduced new, more niche areas, like chromatic polynomials and impartial & partisan game theory. But by then, I found it much easier to figure out what part of math a problem was based on and how to find relevant literature.
It helps to be really really stubborn, and to have the patience to let a problem stew in my brain, sometimes for weeks at a time. That seems to help lead to that Eureka moment.
One other thing - can't believe I forgot to mention this: once you solve a problem, read the solvers' thread in the forum! I learned so much by doing that, which fed into success on later problems. The link will be at the bottom of the problem page once you've solved it.
There are some much later problems where some obscure technique gets mentioned, even though the problem is doable without that technique. But then later on, there are other problems where that technique is practically required. I can think of multiple 100% difficulty problems which were actually much easier than that for me, because I had already seen and tried out the techniques that enable a fast solution.
And sorry, not going to mention any of those techniques. A lot of the fun I have in solving PE problems is that incremental increase in knowledge as time goes on.
I feel a lot better knowing that searching the literature is supposed to be a normal part of Project Euler.
Do you have a favorite?
As a class of problems, I'd say the combinatorial game theory ones are my favorites. There are a lot of impartial game theory problems - look for problems mentioning Nim or stone games. They build on each other nicely, from the mid 300s on. The site has been getting into partisan game theory problems in the past year, which finally got me to buy "Winning Ways For Your Mathematical Plays", vol 1, and "Lessons In Play". I find pretty much any problem with John Conway's influence fun to do.
As for a single problem, I'm fond of PE589, "Poohsticks Marathon". That was my 501st solution, two years after first attempting it (solved 5 years ago, yikes). I like it because it's a problem with a 95% difficulty rating, so very tough, but the development team slotted it in as an easy problem (problems normally get scheduled in batches of 6 with a cadence of medium/easy/medium/easy/medium/hard). Once I solved it, I agreed that it was relatively easy, in that it uses techniques introduced by early PE problems, but something about it makes using those techniques unexpectedly difficult.
I have been working on PE problems for most of 10 years. One thing I would sort of like to do is make a library (mostly I have been using python) for some of the more common functions. Do you have anything like that?
I do, but there's nothing too obscure in it. Efficient prime number sieve, prime factorization using trial division, generating list of divisors from the prime factorization, modular inverse via Euclidean algorithm, Chinese Remainder Theorem.
I also have some stand-alone modules, one to solve generalized Pell equations, another to find a polynomial given a sequence via the differences (e.g. 2, 5, 10, 17, first differences 3, 5, 7, second 2, 2 is enough to find n^2+1). There's another to find the closed form for a sequence as a linear recurrence.
Some solvers have much more extensive libraries, but I tend to grab bits of code from old solutions to reuse on the fly.
[flagged]
It’s been discussed quite a bit in the past. I love Project Euler, but haven’t been active for a few years. Even then I felt like the social aspect of it wasn’t designed for newcomers; a lot of activity on new problems happens as soon as it becomes available and then it mostly freezes. The problems get pretty hard on average once you’re past the first hundred, one reason I personally found it hard to keep going.
To be honest, I don't see people talking about it that much. But I'm certain that it would appeal to a good chunk of the people here.
But isn’t it mathematical computing? I feel like leetcode DSA is closer to your average HN user
There's certainly a strong element of computational mathematics. A lot of the problems deal with things like Mersenne Primes or Collatz sequences. On the other hand, there are problems like "Maximum Path Sum" which definitely make use of DSA-style tricks (https://projecteuler.net/problem=67).
Has here been any work done to find out about the average HN user? It seems like it would be a hard thing to do but it might be interesting to see. I do not even know what sort of metrics would be useful to measure.
I have great memories of competing with a college friend to solve these problems. Since I last seriously did the problems, probably 15 years ago, they have added a lot more.
Project Euler 241 I have never really understood...
Check if the sum of number's divisors divided by the number itself gives x.5
Oh - I understand the question. I just don't understand how to calculate the answer up to 10^18 in a reasonable amount of time!
This is cool! Are there solutions I can check my answers against?
Yes make an account and you can enter your solutions
ChatGPT o1 seems to understand the problem correctly but doesn't get far: https://chatgpt.com/share/670dbe0e-087c-8000-9856-996c3fbaa9...
o1 thought for 105 seconds, cycling through many relevant-sounding status messages like "looking for patterns," before writing a collection of thematic but flawed thoughts. The "Calculation Steps" approach is incorrect, but correctly implemented by the code.
It flubs a basic calculation that it correctly implements in python: "10^16 mod (10^9 + 7) = 49" (it's actually 930000007)
but succeeds in a seemingly harder calculation: "the modular inverse of 12 modulo 10^9 + 7 is 83333334"
Finally, o1 claims the code prints "0" when it actually prints "982790507" (both wrong answers).
Note: input was copied from the html-only Project Euler url since the formulas in the human-optimized url are not copyable: https://projecteuler.net/minimal=912