Everything web-facing, if it's not a static website delivered by a well-tested web server, happens in a high-risk environment. And doubly so, if, like in this case, stuff like custom cgi libraries are involved. One has to be either very confident in their skills to do that or very, very brave.
In any web application all data that comes from the outside world is potentially hostile. A decent web framework takes care of basic security measures, does input sanitation, provides referer checking and csrf for forms, etc. When you roll your own, your _are_ on your own to do that all properly yourself, if you even know all the potential pitfalls. And if you write your own cgi library like the op even more so. I'm not advocating for using WordPress either. I'm advocating for either having a static blog or using a decent, tested web framework or the the very least cgi module that provides tested implementations of common security features that in my experience are typically missed in self-made cgi scripts.
Nah. There are ways to mitigate the blast radius of experiments like this.
You can't utterly wrap developers up in cotton wool. Ultimately, people learn strong lessons from screwing up. You can at least make sure they're doing this learning within a sandbox in which the damage is contained. Like, maybe containerising it so it has limited access to anything that could do any real damage. If somebody builds a blogging app and the worst thing that happens is that they learn the value of sanitising your input, preventing SQL injection, that spam is an unfortunate fact of life, and that you should be very careful with how you manage cookies, I would consider that a _very_ big win for that novice developer.
I am not exactly sure about using containers (most likely you mean docker/podman) as sandboxes...
Please correct me wrong but for a better form of sandboxing, I would recommend something like microvm and the bottlefire thing which was recently shared if that might pique your interest as I found it to be interesting and then using it with something like https://github.com/Zouuup/landrun
There is also tinykvm and other projects too which can simply take a binary and run it and I think that maybe developers should also try out all the different sandboxes and different things just for understanding as well I suppose too. To me, I really like playing with different form of sandboxes or such technologies in general.
Also do note that I am not sure if bottlefire provides sandbox/isolation by default as they mention Sandbox with Landlock seperately so I am now a bit confused if they provide sandbox by default or not as I previously thought it might have.
I use perl for the same reasons. The few dynamic scripts that I need I want to write and forget them for years. Odds are with perl they'll still work 20 years from now.
I spend a lot of time blogging but all I use is a ~50 line Python file that converts my markdown pages to HTML, adds my template, and generates the ToC page. Then I push to GitHub Pages.
I can't imagine needing more than that. Why are these blog stacks so complex?
Speaking for myself at least, after you've been blogging for a quarter century or more there are some nice features you might want like pagination of your table-of-contents, RSS feeds (do it, everyone should do it), support for redirects so that ancient links mostly work across those decades (I've kept redirects from like three or four blogging systems now), tags pages for finding lost treasures and silly things. I been on both sides of "needing" comments tools over the decades, similar with things like WebMentions. With so much of blogging on social media WebMentions don't seem that big a deal this decade as it was in the one where every other person (in college) had at least one Blogger.com Blog or LiveJournal and a lot of discussions were cross-links between blogs.
Admittedly most of my blogging history has been something of a path towards simplification from hand-rolled PHP+MySQL, with custom "forum code" markup language, stuff before "blogging" was even an agreed upon term for it (and before Markdown was anywhere near as pervasive), to complex third-party beasts like Drupal, to homegrown Python (and reStructuredText), to very simple SSG tools (these days still Jekyll, but I don't like working in Ruby much, so I keep debating a switch to Lume but I don't think its Redirects plugin is yet compatible enough with GitHub Pages for my liking and I haven't tested its RSS support yet, both of which are personal hard requirements).
I had that thought too. My own blogging engine is ~100 lines of lua that accomplishes the same as you describe, plus RSS, with one additional library for markdown parsing. The author mentions Mustache templates and WebMentions, but ten dependencies still seems like a lot; I wonder what they are.
I've been thinking of how to make a blog simple recently, and I came across xslt. It looks really cool and seems pretty set in stone, so I thought I'd ask, what are the advantages/drawbacks of making your own tech stack versus xslt? At first glance, it seems perfectly able to handle rss and other simple linking patterns, and pretty much anything can easily be turned into an xml then xslt could be used to generate an html (server-side, or rather writer-side, not like the blog is gonna change) that you serve?
XSL is neat, and it is a functional language, but between XSL and XPath, it is quite verbose. Here's a small section of XSL I use to generate my website (not my blog):
And yes, there is other code I've omitted for brevity. This is used to generate the navigation links for the site. I initially write this ... prior to 2009 (that's when I moved it into git). There have been some minor fixes to the XSL over the years, but it's largely unchanged (for a reason that I hope is obvious). Yes, I still use it, because it still works, and it's for a static website.
I don't know man, every time I tried to learn Lua (to write nvim plugins and HammerSpoon spoons) I disliked the ergonomics of the language. I don't understand why people say it's an easy language—
Perhaps people's tastes vary? I find Lua an easy to use language. Easy to embed into C/C++ projects for scripting purposes. Also great when you can only spare 100 kB or so for an interpreter. (Arguably that was more important in the past.)
Agree, I find it a PITA. I think the good reviews are in contrast to C++ and vimscript. Fine for a small embedded script, but I hope WASM or some better language ecosystem starts to fill this niche.
There are languages that compile to Lua. Have you investigated those? For example, you can use a TypeScript syntax and pick up free typechecking along the way: https://typescripttolua.github.io/
The 1 indexes are only a difference from what you're used to. Lua was made by mathematicians, who of course wanted to address the first element as 1, the second element as 2, etc.
0-indexing makes sense in the context of C where the index operator is syntactic sugar for pointer arithmetic. In higher-level languages like C# and Python and others, it's pretty much just a leftover habit from C devs that we all got used to.
I don't have a background in web development and have a genuine question.
> Your blog is your place to experiment and program how you want it
I 100% agree with your statement and people don't need to justify their hobbies. I've done really pointless things simply for lolz and because I wanted.
My question arises because I was surprised in how ... architected and (dare I say) complex the tech stack in your blog is. In my blogging days I wrote my own HTML/CSS and published it on a Internet facing server. Later, I've used CSS templates and Markdown-to-HTML to generate the static content. What is the purpose of Lua and having a database and all the other complexity for what seems like a static blog? Again, "because I wanted to experiment" or "sharpen my skills" is a totally valid answer but seeing I don't have a background in web development I am inquiring to see if there is a technical reason for doing this. Would be curious to learn what, if any, technical problem warrants such a set up :)
Why not choose Hugo (https://gohugo.io) or Zola (https://www.getzola.org ?. Both are pretty well-supported by communities and have tonnes of blog themes. (Hugo has a truckload and is the top-3 widely deployed SSG's)
What are your thoughts on something like arturo which I know is quite recent but it has a lot of features for scripting and an argument might in fact be made that it is in fact it might have too many batteries but it was an absolute pleasure to learn and I had a lot of aha moments in their discord server and the community was really pleasant to follow through actually.
I know its definitely smaller but I just want your opinions on it and what you might think of the language and I may be a bit sorry if this comes across as a little off topic but your blog really reminded me of arturo and my attempts on creating something like hugo in arturo but the project was abandoned mid way but if I remember correctly it was just some 50 lines of code to convert from markdown to complete website or even less since arturo's battery include markdown syntax as well as well as a web server and its written in nim which I cherish too.
I am genuinely interested in your opinions about it!
I currently run a blog generator I wrote in fennel and have been considering switching to pollen haha. Slightly surprised you didn’t pick fennel over lua since you used racket before. Is there a reason you didn’t?
Why would you build your blog to fail if some article on it ever gets popular? The fact that the most hits you ever received was 50k in a week isn't relevant; a single important post could receive that in seconds.
It basically costs nothing to pre-render a static site, which then serves several orders of magnitude faster. I'm confused why anyone would do it this way in this day and age.
I liked this post, and I can totally understand where you’re coming from…
But couldn’t anything you say about Lua also be said about JS? You mentioned how Lua wasn’t batteries included, so you try to limit your libraries. Couldn’t you say the same for JS? JS itself doesn’t change much, it’s the ecosystem. Couldn’t you just pick out some small and stable libraries the same way you could with Lua?
For me, it isn't 100% language warts. It is the customer experience that matters the most for me. Can I compile and ship my products to my customers without having them to install a VMs, container runtime or a language runtime? That the question that is critical for me.
I agree, I think that you must really like golang and how easy it can make cross compilation and how fast it is to build.
Golang has one of the best developer experiences and there are only very very few minor nitpicks I might have of the language but the whole ecosystem on packaging software and what not is just so easy and I love golang.
LuaJIT bucks the trend of slow-warmup JITs. It is extremely quick to compile and load, and its interpreter is very fast -- faster than the JIT-compiled code from LuaJIT v1 IIRC, and certainly faster than the interpreter of Lua.
It wasn't until LuaJIT that I realized that JIT didn't inherently have to be these slow lumbering beasts that take hundreds of milliseconds just to wake from their slumber.
Yet I've witnessed Lua 5.1 launching faster than luajit for some of my use cases.
My point still stands though. Don't just use LuaJIT thinking it will magically make things faster in all cases. If you are embedding, LuaJIT is a no-brainer. If you are using a stand-alone interpreter, measure if you care about reality.
^^ the above combined with caddy git fs to have your md files cloned in memory and refresh every X interval is kind of magical. Git push a new md file and wait X minutes and your website updates.
I miss the days when "implement your own blogging engine" was one of the most popular learning projects for engineers.
We should bring that back! Its such a great way to play around with client- and server-side development options in an almost zero-risk environment.
Still a good project when one learns a new programming language.
Everything web-facing, if it's not a static website delivered by a well-tested web server, happens in a high-risk environment. And doubly so, if, like in this case, stuff like custom cgi libraries are involved. One has to be either very confident in their skills to do that or very, very brave.
What? Are you serious?
If you roll your own little cgi-bin perl script behind Apache you're far from vulnerable compared to, say, a WordPress website.
In any web application all data that comes from the outside world is potentially hostile. A decent web framework takes care of basic security measures, does input sanitation, provides referer checking and csrf for forms, etc. When you roll your own, your _are_ on your own to do that all properly yourself, if you even know all the potential pitfalls. And if you write your own cgi library like the op even more so. I'm not advocating for using WordPress either. I'm advocating for either having a static blog or using a decent, tested web framework or the the very least cgi module that provides tested implementations of common security features that in my experience are typically missed in self-made cgi scripts.
Nah. There are ways to mitigate the blast radius of experiments like this.
You can't utterly wrap developers up in cotton wool. Ultimately, people learn strong lessons from screwing up. You can at least make sure they're doing this learning within a sandbox in which the damage is contained. Like, maybe containerising it so it has limited access to anything that could do any real damage. If somebody builds a blogging app and the worst thing that happens is that they learn the value of sanitising your input, preventing SQL injection, that spam is an unfortunate fact of life, and that you should be very careful with how you manage cookies, I would consider that a _very_ big win for that novice developer.
Because we're talking about _developers_ here.
I am not exactly sure about using containers (most likely you mean docker/podman) as sandboxes...
Please correct me wrong but for a better form of sandboxing, I would recommend something like microvm and the bottlefire thing which was recently shared if that might pique your interest as I found it to be interesting and then using it with something like https://github.com/Zouuup/landrun
There is also tinykvm and other projects too which can simply take a binary and run it and I think that maybe developers should also try out all the different sandboxes and different things just for understanding as well I suppose too. To me, I really like playing with different form of sandboxes or such technologies in general.
Also do note that I am not sure if bottlefire provides sandbox/isolation by default as they mention Sandbox with Landlock seperately so I am now a bit confused if they provide sandbox by default or not as I previously thought it might have.
I use Lua for almost all my custom tools these days.
https://akkartik.name/freewheeling-apps
https://git.sr.ht/~akkartik/gen_site
Why not Fennel?
https://redbean.dev/ could be just perfect fit for you
I use perl for the same reasons. The few dynamic scripts that I need I want to write and forget them for years. Odds are with perl they'll still work 20 years from now.
This. I'm still sitting on some almost core code that's written on an over enthusiastic teenagers idea of a plan.
I spend a lot of time blogging but all I use is a ~50 line Python file that converts my markdown pages to HTML, adds my template, and generates the ToC page. Then I push to GitHub Pages.
I can't imagine needing more than that. Why are these blog stacks so complex?
Speaking for myself at least, after you've been blogging for a quarter century or more there are some nice features you might want like pagination of your table-of-contents, RSS feeds (do it, everyone should do it), support for redirects so that ancient links mostly work across those decades (I've kept redirects from like three or four blogging systems now), tags pages for finding lost treasures and silly things. I been on both sides of "needing" comments tools over the decades, similar with things like WebMentions. With so much of blogging on social media WebMentions don't seem that big a deal this decade as it was in the one where every other person (in college) had at least one Blogger.com Blog or LiveJournal and a lot of discussions were cross-links between blogs.
Admittedly most of my blogging history has been something of a path towards simplification from hand-rolled PHP+MySQL, with custom "forum code" markup language, stuff before "blogging" was even an agreed upon term for it (and before Markdown was anywhere near as pervasive), to complex third-party beasts like Drupal, to homegrown Python (and reStructuredText), to very simple SSG tools (these days still Jekyll, but I don't like working in Ruby much, so I keep debating a switch to Lume but I don't think its Redirects plugin is yet compatible enough with GitHub Pages for my liking and I haven't tested its RSS support yet, both of which are personal hard requirements).
I had that thought too. My own blogging engine is ~100 lines of lua that accomplishes the same as you describe, plus RSS, with one additional library for markdown parsing. The author mentions Mustache templates and WebMentions, but ten dependencies still seems like a lot; I wonder what they are.
Sometimes people just want to have fun
I've been thinking of how to make a blog simple recently, and I came across xslt. It looks really cool and seems pretty set in stone, so I thought I'd ask, what are the advantages/drawbacks of making your own tech stack versus xslt? At first glance, it seems perfectly able to handle rss and other simple linking patterns, and pretty much anything can easily be turned into an xml then xslt could be used to generate an html (server-side, or rather writer-side, not like the blog is gonna change) that you serve?
XSLT might be removed from the HTML spec soon, see discussion here: https://news.ycombinator.com/item?id=44952185
that would be client side if it happens, though, I'm talking about server side generation
xsltproc was preinstalled on my machine actually, the fact I could just run it without installing anything is pretty cool
XSL is neat, and it is a functional language, but between XSL and XPath, it is quite verbose. Here's a small section of XSL I use to generate my website (not my blog):
And yes, there is other code I've omitted for brevity. This is used to generate the navigation links for the site. I initially write this ... prior to 2009 (that's when I moved it into git). There have been some minor fixes to the XSL over the years, but it's largely unchanged (for a reason that I hope is obvious). Yes, I still use it, because it still works, and it's for a static website.I don't know man, every time I tried to learn Lua (to write nvim plugins and HammerSpoon spoons) I disliked the ergonomics of the language. I don't understand why people say it's an easy language—
Perhaps people's tastes vary? I find Lua an easy to use language. Easy to embed into C/C++ projects for scripting purposes. Also great when you can only spare 100 kB or so for an interpreter. (Arguably that was more important in the past.)
For me it is a much easier language than, say, Erlang.
but Erlang isn't a scripting language, it's a full-blown language with a rich ecosystem and framework (BEAM).
Erlang has multiple higher level barrier of entries than other languages. For starting, its syntax.
Agree, I find it a PITA. I think the good reviews are in contrast to C++ and vimscript. Fine for a small embedded script, but I hope WASM or some better language ecosystem starts to fill this niche.
There are languages that compile to Lua. Have you investigated those? For example, you can use a TypeScript syntax and pick up free typechecking along the way: https://typescripttolua.github.io/
There's also the venerable MoonScript: https://moonscript.org/
And YueScript, a personal fave: https://yuescript.org/doc/
A whole list: https://github.com/hengestone/lua-languages
Don't forget Fennel, if you are a Lisp lover!
I would suggest using it to add scripting functionality to your own C or C++ project. That's when it really clicked for me.
When you're doing stuff in nvim or HammerSpoon, you're dealing with someone else's interface and the decisions that they've made.
It really depends. I learned Lua in order to contribute to kulala plugin for neovim and found the language nice and easy to learn.
Of course it has its warts, but given the topic, almost everything is better than a vimscript imho.
it's easy, but the 1 indexes and global by default suck
The 1 indexes are only a difference from what you're used to. Lua was made by mathematicians, who of course wanted to address the first element as 1, the second element as 2, etc.
0-indexing makes sense in the context of C where the index operator is syntactic sugar for pointer arithmetic. In higher-level languages like C# and Python and others, it's pretty much just a leftover habit from C devs that we all got used to.
Global by default is a perpetual issue, agreed.
And a lot of the time it makes the syntax more compact than it would be with 0-indexing.
I don't feel that strongly for or against either way of indexing though, they both have their pros and cons.Author of the blog here in case anyone has questions.
I don't have a background in web development and have a genuine question.
> Your blog is your place to experiment and program how you want it
I 100% agree with your statement and people don't need to justify their hobbies. I've done really pointless things simply for lolz and because I wanted.
My question arises because I was surprised in how ... architected and (dare I say) complex the tech stack in your blog is. In my blogging days I wrote my own HTML/CSS and published it on a Internet facing server. Later, I've used CSS templates and Markdown-to-HTML to generate the static content. What is the purpose of Lua and having a database and all the other complexity for what seems like a static blog? Again, "because I wanted to experiment" or "sharpen my skills" is a totally valid answer but seeing I don't have a background in web development I am inquiring to see if there is a technical reason for doing this. Would be curious to learn what, if any, technical problem warrants such a set up :)
Why not choose Hugo (https://gohugo.io) or Zola (https://www.getzola.org ?. Both are pretty well-supported by communities and have tonnes of blog themes. (Hugo has a truckload and is the top-3 widely deployed SSG's)
What are your thoughts on something like arturo which I know is quite recent but it has a lot of features for scripting and an argument might in fact be made that it is in fact it might have too many batteries but it was an absolute pleasure to learn and I had a lot of aha moments in their discord server and the community was really pleasant to follow through actually.
I know its definitely smaller but I just want your opinions on it and what you might think of the language and I may be a bit sorry if this comes across as a little off topic but your blog really reminded me of arturo and my attempts on creating something like hugo in arturo but the project was abandoned mid way but if I remember correctly it was just some 50 lines of code to convert from markdown to complete website or even less since arturo's battery include markdown syntax as well as well as a web server and its written in nim which I cherish too.
I am genuinely interested in your opinions about it!
https://arturo-lang.io/
I currently run a blog generator I wrote in fennel and have been considering switching to pollen haha. Slightly surprised you didn’t pick fennel over lua since you used racket before. Is there a reason you didn’t?
Why would you build your blog to fail if some article on it ever gets popular? The fact that the most hits you ever received was 50k in a week isn't relevant; a single important post could receive that in seconds.
It basically costs nothing to pre-render a static site, which then serves several orders of magnitude faster. I'm confused why anyone would do it this way in this day and age.
I liked this post, and I can totally understand where you’re coming from…
But couldn’t anything you say about Lua also be said about JS? You mentioned how Lua wasn’t batteries included, so you try to limit your libraries. Couldn’t you say the same for JS? JS itself doesn’t change much, it’s the ecosystem. Couldn’t you just pick out some small and stable libraries the same way you could with Lua?
For me, it isn't 100% language warts. It is the customer experience that matters the most for me. Can I compile and ship my products to my customers without having them to install a VMs, container runtime or a language runtime? That the question that is critical for me.
I agree, I think that you must really like golang and how easy it can make cross compilation and how fast it is to build.
Golang has one of the best developer experiences and there are only very very few minor nitpicks I might have of the language but the whole ecosystem on packaging software and what not is just so easy and I love golang.
LuaJIT is awesome.
I've been converting my Bash scripts/functions to it, with much success thus far.
You may want to benchmark lua versus luajit if you are writing scripts or other short-lived programs.
JIT-compiled languages aren't generally faster in starting up, they generally are used to speed up long-lived programs that have hot sections.
LuaJIT bucks the trend of slow-warmup JITs. It is extremely quick to compile and load, and its interpreter is very fast -- faster than the JIT-compiled code from LuaJIT v1 IIRC, and certainly faster than the interpreter of Lua.
It wasn't until LuaJIT that I realized that JIT didn't inherently have to be these slow lumbering beasts that take hundreds of milliseconds just to wake from their slumber.
Yet I've witnessed Lua 5.1 launching faster than luajit for some of my use cases.
My point still stands though. Don't just use LuaJIT thinking it will magically make things faster in all cases. If you are embedding, LuaJIT is a no-brainer. If you are using a stand-alone interpreter, measure if you care about reality.
waldium can be good for blogs if you dont want to do a custom build. its geo-specific though
ha - I'm well behind you trying to get my blog from Mullenweg hell to pure Raku
glad to see cgi being loved again
Kinda curious why you did not go with OpenResty?
Why not go with redbean?
tl;dr: because no one uses Lua
You could also do something like this, have Caddy webserver parse your md files through a template
https://github.com/dbohdan/caddy-markdown-site/blob/master/C...
^^ the above combined with caddy git fs to have your md files cloned in memory and refresh every X interval is kind of magical. Git push a new md file and wait X minutes and your website updates.
https://github.com/mohammed90/caddy-git-fs
====
Or a one-file FastHTML (python web framework) solution:
https://gist.github.com/simonMoisselin/f63c52f087704c99b6a62...