Cool! Looks like is used dynamic loading. Which is certainly workable. One downside is that while dynamic loading is well-supported, dynamic unloading is generally not. Meaning if you are using dynamic loading and re-loading for hot updates the server process will start to accumulate memory. Not a huge issue of the server process is periodically restarted, but can potentially cause weird bugs.
You might be able to make something more robust by forking and loading the modules in a separate process. Then do something fancy with shared memory between the main process and the module processes. But I haven’t looked into this much
This hobby project is inspired by kernel modules and AWS Lambda. It lets you write raw C modules, upload them, and have the server compile and run only the module itself at runtime—no need to recompile the entire application or restart the server.
You can update routes and modules dynamically. Even WebSocket handlers can be updated without dropping existing connections.
Important question: Why would anyone develop a web application in C? Typically web applications lean heavily on garbage collection and memory safety, because their bottlenecks are very rarely CPU/memory issues. The ROI on manually managing memory just isn't there for a typical web application.
Speaking as someone who has done this back in the early wild days of the web:
* if what you're vending is the software instead of the service (not what people usually do now, but there was a time), then this approach does provide for some obfuscation of IP and various secrets.
* for some demand/resource profiles, CPU & memory issues are a lot easier to run into. The one I experienced with this project was targeting a serious e-commerce product to the context of 20-30 year old shared hosting environments (again, not what people would do now), but there may be different situational niches today.
* familiarity. Sometimes you use what you know. And in the late 90s today's most popular web languages were still years away from being the convenient platform they'd become. The other popular options were Perl, maybe Java, possibly ColdFusion/VB/PHP.
That said, you're correct: memory management was a pain, and by 2005 or so it was pretty clear that programmer cycles were as or more valuable than CPU and respectable frameworks were starting to coalesce in languages much better suited for string manipulation, so the ROI was not great. And of course, today you have other systems languages available like Go and Rust...
On the other hand memory management in web applications is quite easy.
Most of the stuff is only required for the lifetime of a request. Some stuff needs to be available the whole application life time.
You can do this with other languages (C# for example) as well. Memory is so cheap, though, that most companies should spend their money on increasing memory rather than on paying programmers to optimize memory usage.
Yes, for most use cases it doesn’t really matter which language is chosen.
Regarding just spending more money on memory - I agree that it’s definitely cheaper but it’s not only about wasting bytes of memory. If the garbage collector has a lot of work to do it may also impact response time/throughput.
And yes, C# did a pretty good job with implementing mechanisms for reducing allocations on a language level. This definitely helps to reduce garbage collection.
I’m all for using C for native development (and because I find it fun to work in occasionally), but I agree with your sentiment here. Not only do you have to manage memory manually, but you also have to do a lot more work for basic string manipulation which is the vast majority of web work. I would much rather work with a language with built in support for string manipulation and well known 3rd party libraries with good ergonomics for working with JSON, which a lot of your APIs will most likely be using.
Back then, C was one of a few viable choices. The original implementation of the 2ch BBS was written in C.[0] Later revisions used Perl. Between 1998 and 2001, the site was a widely-used BBS and written in C.
Cool! Looks like is used dynamic loading. Which is certainly workable. One downside is that while dynamic loading is well-supported, dynamic unloading is generally not. Meaning if you are using dynamic loading and re-loading for hot updates the server process will start to accumulate memory. Not a huge issue of the server process is periodically restarted, but can potentially cause weird bugs.
You might be able to make something more robust by forking and loading the modules in a separate process. Then do something fancy with shared memory between the main process and the module processes. But I haven’t looked into this much
Once you start forking wouldn't it make more sense to just exec? Woops, CGI.
Oh really interesting, will look into it! Was afraid it would leak memory.
This hobby project is inspired by kernel modules and AWS Lambda. It lets you write raw C modules, upload them, and have the server compile and run only the module itself at runtime—no need to recompile the entire application or restart the server.
You can update routes and modules dynamically. Even WebSocket handlers can be updated without dropping existing connections.
How ti integrate with extermal library ?
If you would want to link with external libraries, you would need to modify to server and make sure it has access to them when compiling the modules.
Important question: Why would anyone develop a web application in C? Typically web applications lean heavily on garbage collection and memory safety, because their bottlenecks are very rarely CPU/memory issues. The ROI on manually managing memory just isn't there for a typical web application.
> The ROI on manually managing memory just isn't there for a typical web application.
You can use a per-request memory arena built with a simple bump allocator and then free the entire block when the request has been handled.
Speaking as someone who has done this back in the early wild days of the web:
* if what you're vending is the software instead of the service (not what people usually do now, but there was a time), then this approach does provide for some obfuscation of IP and various secrets.
* for some demand/resource profiles, CPU & memory issues are a lot easier to run into. The one I experienced with this project was targeting a serious e-commerce product to the context of 20-30 year old shared hosting environments (again, not what people would do now), but there may be different situational niches today.
* familiarity. Sometimes you use what you know. And in the late 90s today's most popular web languages were still years away from being the convenient platform they'd become. The other popular options were Perl, maybe Java, possibly ColdFusion/VB/PHP.
That said, you're correct: memory management was a pain, and by 2005 or so it was pretty clear that programmer cycles were as or more valuable than CPU and respectable frameworks were starting to coalesce in languages much better suited for string manipulation, so the ROI was not great. And of course, today you have other systems languages available like Go and Rust...
> if what you're vending is the software instead of the service (not what people usually do now, but there was a time)
I'm very curious what this means. Can you give an example?
I could say speed, but the main reason for me is because it is fun. And I like to see what I can make C do. :D
On the other hand memory management in web applications is quite easy. Most of the stuff is only required for the lifetime of a request. Some stuff needs to be available the whole application life time.
You can do this with other languages (C# for example) as well. Memory is so cheap, though, that most companies should spend their money on increasing memory rather than on paying programmers to optimize memory usage.
Yes, for most use cases it doesn’t really matter which language is chosen.
Regarding just spending more money on memory - I agree that it’s definitely cheaper but it’s not only about wasting bytes of memory. If the garbage collector has a lot of work to do it may also impact response time/throughput.
And yes, C# did a pretty good job with implementing mechanisms for reducing allocations on a language level. This definitely helps to reduce garbage collection.
Cheap for whom? American programmers with FAANG salaries, or an under-funded third world NGO?
The title "Hobby Project" makes the point right from the beginning
I’m all for using C for native development (and because I find it fun to work in occasionally), but I agree with your sentiment here. Not only do you have to manage memory manually, but you also have to do a lot more work for basic string manipulation which is the vast majority of web work. I would much rather work with a language with built in support for string manipulation and well known 3rd party libraries with good ergonomics for working with JSON, which a lot of your APIs will most likely be using.
Back then, C was one of a few viable choices. The original implementation of the 2ch BBS was written in C.[0] Later revisions used Perl. Between 1998 and 2001, the site was a widely-used BBS and written in C.
[0] https://github.com/nekoruri/readcgi
Lightweight web frameworks are great for embedded applications.