Over engineering my homelab so I don't pay cloud providers

(ergaster.org)

78 points | by JNRowe 3 days ago ago

37 comments

  • 8fingerlouie 12 minutes ago

    While i get the whole homelab thing is exiting and a great learning experience, it's simply not worth the time and effort for the majority of people.

    You will end up paying much more for your services, along with spending a ton of time maintaining it (and if you don't, you will probably find yourself on the end of a 0-day hack sometime).

    In Northern/Western Europe, where power costs around €0.3/kWh on average, just the power consumption of a simple 4 bay NAS will cost you almost as much as buying Google Drive / OneDrive / iCloud / Dropbox / Jottacloud / Whatever.

    A simple Synology 4 bay NAS like a DS923+ with 4 x 4TB Seagate Ironwolf drives will use between 150 kWh and 300 kWh per year (100% idle vs 100% active, so somewhere in between), which will cost you between €45 and €90 per year, and that's power alone. Factoring in the cost of the hardware will probably double that (over a 5 year period).

    It's cheaper (and easier) to use public cloud, and then use something like Cryptomator (https://cryptomator.org/) to encrypt data before uploading it. That way you get the best of both worlds, privacy without any of the sysadm tasks.

  • BLKNSLVR 2 hours ago

    > But my server could be shut down because of a power outage or another reason. I might be at work or even on holidays when it happens, and even wireguard can’t solve this.

    A 'power outage' incident doesn't seem to have been mitigated. My homelab has had evolving mitigations: I cut a hole in the side of a small UPS so I could connect it to a larger (car) battery for longer uptime, which got replaced by a dedicated inverter/charger/transfer-switch attached to a big-ass AGM caravan battery (which on a couple of occasions powered through two-to-three hour power outages), and has now been replaced with these recent LiFePo4 battery power station thingies.

    Of course, it's only a homelab, there's nothing critically important that I'm hosting, but that's not the dang point, I want to beat most of "the things", and I don't like having to check that everything has rebooted properly after a minor power fluctuation (I have a few things that mount remote file stores and these mounts usually fail upon boot due to the speed at which certain devices boot up - and I've decided not to solve that yet).

    • qwertox an hour ago

      > I cut a hole in the side of a small UPS so I could connect it to a larger (car) battery for longer uptime

      Can you share more about this? I have a APC Back UPS PRO USV 1500VA (BR1500G-GR) and it would be nice to know if this is possible with that one as well.

      • BLKNSLVR an hour ago

        That UPS eventually died, and I'm not sure if it was because it was hooked up to a larger battery than it was designed for, but it's still only 12 volts so I don't think the electronics would notice. What they may notice is extended run-time in the event of a power failure.

        It was a crude mod. Take the cover off and remove the existing little security alarm battery, use tin snips to cut a hole in the side of the metal UPS cover (this was challenging, it was relatively thick metal, I'd recommend using an angle grinder in an appropriately safe environment far away from the internals of the UPS), and feed the battery cables out through the hole. I probably got some additional cables with appropriately sized terminations to effectively extend the short existing ones (since they were only designed to be used within the device). And then connect it up to a car battery.

        Cover any exposed metal on the connectors with that shrink rubber tubing or electrical tape. Be very careful with exposed metal around it anywhere, especially touching the RED POSITIVE pole of the battery. Get a battery box - I got one for the big-ass AGM battery.

        Test it out on a laptop that's had it's battery removed or disconnected that, just in case, you don't care too much about losing.

        Get a battery charger that can revive a flat battery, and do a full refresh/renew charge on the car battery once a year or after it's had to push through a power outage that may have used more than a few percent of its capacity.

        Personally, I think it's safer a less hassle to go for a LiFePo4 (LFP) Power Station style device that has UPS capabilities. LFP batteries have 3,000-ish cycle lifetimes, which could be nearly ten years with daily use.

      • throaway920181 41 minutes ago

        This really doesn't seem like something one would want to mess around with if they don't know what they're doing (fire hazards and all...)

    • preisschild an hour ago

      I just use a small UPS to make sure all data is written to the drives properly before the battery runs out.

      Do you have power outages often? Even if I have one, my services can come up automatically without doing anything, when the power is restored.

  • cyprien_g 13 minutes ago

    Building a homelab is an awesome way to learn a lot of things.

    I also used to over-engineer my homelab, but I recently took a more simplistic approach (https://www.cyprien.io/posts/homelab/), even though it’s probably still over-engineered for most people.

    I realized that I already do too much of this in my day job, so I don’t want to do it at home anymore.

  • arjie 26 minutes ago

    I have a setup that is perhaps not as robust, but where my primary aim was that I should be able to incrementally encapsulate the parts. https://wiki.roshangeorge.dev/w/One_Quick_Way_To_Host_A_WebA...

    As an example, I use cloudflare tunnel to point to an nginx that reverse proxies all the services, but I could just as well point DNS to that nginx and it would still work. I had to rebuild the entire thing on my home server when I found that the cheap VPS I was using was super over-provisioned ($2/mo for 2 Ryzen 7950 cores? Of course it was) and I had this thing at home anyway, and this served me well for that use-case.

    When I rebuilt it, I was able to get it running pretty quickly and each piece could be incrementally done: i.e. I could run without cloudflare tunnel and then add it to the mix, I could run without R2 and then switch file storage to R2 because I used FUSE s3fs to mount R2, so on and so forth.

  • MaKey 27 minutes ago

    Your VPS provider likely uses servers with ECC RAM, this home server doesn't. For most people it doesn't seem to matter but for me it does - a home server where I store my data needs to have ECC RAM.

    • ninjin 18 minutes ago

      Seconded, but hard to find for small boxes. I have seen in-band ECC on Asus Nucs, but that is as good as it gets from what I can tell.

    • mvanbaak 4 minutes ago

      totally agree. That's why my homelab storage server(s) are 2nd hand enterprise machines. They come with ECC.

  • treve 5 hours ago

    If anyone is looking to get started with a homelab at a good price, I can highly recommend checking ebay for a Dell Wyse 5070. They flooded the market for $50 and are likely powerful enough for many needs. They have a M.2 slot that support SATA. The 'extended' version also has space for a small pcie card and has a parallel and 2 serial ports for a blast to the past.

    • wraptile 2 hours ago

      Another tip: second hand gaming PCs! They can be incredibly cheap and powerful due to upgrade cycles just make sure to put a raid 1 on it as second hand gamer gear might be less reliable.

    • rwyinuse 2 hours ago

      For a bit more money, Optiplex Micro / Lenovo Tiny / HP Mini series with at least 8th gen i5 are a good option too. Can be found from Ebay for about 70 - 120 USD, much more powerful than Wyse 5070 while still quite power efficient (about 10W idle, as opposed to 5W of Wyse). Usually they come with one NVME, one SATA 2.5" slot, some premium models even with PCIE.

      • ninjin 2 hours ago

        All good options, just noting that based on some searches I made they all seem to lack serial ports compared to the Wyse if that is something you care about (I personally do). There could be variants out there though with serial ports and if would be happy to hear about them and even more happy if there are fanless variants/alternatives for those of us with very limited space at home and a need to avoid noise.

        • vladvasiliu an hour ago

          I don't know about fanless, especially with an i5 as opposed to n-something.

          But not all those minis are the same. G4 (intel 8th gen) and G5 (intel 9th gen) HPs are horrendous. The fan makes an extremely aggravating noise, and I haven't found a way to fix it. Bonus points for both the fan and heatsink having custom mounts, so even if you wanted to have an ugly but quiet machine by slapping a standard cooler, you couldn't.

          G6 versions (intel 10th gen) seem to have fixed this, and they're mostly inaudible on a desk, unless you're compiling something for half an hour.

    • finnjohnsen2 2 hours ago

      I would say raspberry pi 5. cheap, small and widely used so much of the stuff is already done many times

      • cookiengineer 2 hours ago

        Looking at Raspberry Pi prices inside EU, I can get 8 core laptops for a cheaper price, with display, dGPU et al.

        No idea what happened, but Raspberry Pis are super expensive for the last couple years, which is why I decided to just go with used Intel NUCs instead. They cost around 80-150EUR and they use more electricity but they are a quite good bang for the buck, and some variants also have 3x HDMI or Gbit/s ethernet or m2 slots you can use to have a SATA RAID in them.

      • cenamus 2 hours ago

        Is it better than a n100 setup from China? When you factor in the storage, power supply, case, (fan), and so on?

    • yamapikarya 3 hours ago

      i bought dell wyse 5070 for building talos cluster using proxmox. pretty great and you can upgrade ram to 32gb

    • pandemic_region 3 hours ago

      What if power consumption is taken into account? Are there any devices in that category that are ok to leave on 24/7 ?

      • vladvasiliu an hour ago

        I have a somewhat bigger machine that hosts my homelab, an HP 800 G2 SFF. It takes "normal" components, so can ben modified. The only custom thing is the PSU, but the standard one is good enough for my needs. Bonus points for not requiring an external power adaptor.

        It has an i5-6500, 32 GB RAM (16 + 2x8 DIMMs), 2 SATA SSDs and a 2x10Gb Connect-X3. It runs 24/7 hosting OpnSense and HomeAssistant on top of KVM (Arch Linux Hardened – didn't do anything specific to lower the power draw). Sometimes other stuff, but not right now.

        I haven't measured it with this specific nic, but before it had a 4x1Gb i350. With all ports up, all VMs running but not doing much, some power meter I got off Amazon said it pulled a little over 14W. The peak was around 40 when booting up.

        Electricity costs 0.22 €/kWh here. The machine itself cost me 0 (they were going to throw it out at work), 35 for the nic and maybe 50 for the RAM. It would take multiple years to break even by buying one of these small machines. My plan is to wait out until they start having 10 Gb nics and this machine won't be able to keep up anymore.

      • ninjin 3 hours ago

        Quick search online tells me ~5W for the Dell Wyse 5070, which does not sound unrealistic as I have similar boxes that draw ~10W. So, 32 to 62kWh per year and then we have ~USD 6.5 to 13 per year assuming 20 cents per kWh which another online search told me was reasonably realistic for the US.

        • fho 2 hours ago

          Tangent, but it's always crazy to hear what other countries pay per kWh compared to the 0.4€/kWh in Germany.

          • jurip 41 minutes ago

            Yeah, and Germany is expensive compared to the Nordics. 6.35 c/kWh right now in Finland, 2.54 c/kWh average over the last 30 days.

            (clarification: that's euro cent, so 0.0635€ etc)

          • bombela 2 hours ago

            Bay area, California: $0.61 base, $0.80 from 16:00 to 21:00.

          • MrDresden 2 hours ago

            And in Iceland the average is around 0.07€/kWh.

        • rwyinuse 2 hours ago

          Yea, I own Wyse 5070 extended, and measured around 5W from the wall when nothing attached to the PCIE slot.

    • senectus1 4 hours ago

      curious to know what you would use this for?

      • denkmoon 4 hours ago

        local dns, static site hosting, local apt cache, various other network services (unifi controller if you've got those APs for example), remote/headless dev machine (maybe not for kernel or bigcorp java development), or whatever else you want. mail if you want. Anything :)

        Those little thin clients aren't gonna be fast doing "big" things, but serving up a few dns packets or whatever to your local network is easy work and pretty useful.

      • treve 4 hours ago

        I use it for media hosting. Backups (connected USB disks), home assistant, syncthing

      • rwyinuse 2 hours ago

        Even these low-power CPU's are surprisingly capable. As an example of more fancy thing, one could slap in some external storage, install Jellyfin, and run their own local streaming service off such a machine. The CPU is modern enough for efficient hardware transcoding of a stream.

  • tietjens an hour ago

    This is such a great post. I have a small collection of posts for inspiration in creating my homelab and this is getting added to it. Current have a Pi 4 with PiHole and a Beelink. Going to add one or two more machines.

  • timc3 3 hours ago

    The encryption is interesting but I wouldn’t call this over engineered at all, in fact it’s rather basic compared to a lot of homelabs I see people building particularly where people are doing K8s or similar over multiple machines.

    Also Proxmox was called out as the only choice when that is very much not the case. It is a good choice for sure, but there are others.

  • czhu12 4 hours ago

    This was exactly a use case I had in mind when building https://canine.sh -- also uses k3s as a provider, and provides a Heroku-like devex.

    How to actually reliably expose a homelab to the broader internet is a little tricky, cloudflare tunnels mostly does the trick but can only expose one port at a time, so the set up is somewhat annoying

    • BLKNSLVR an hour ago

      I've got basically raw internet coming in to my OPNSense device, although I had to request certain ports to be removed from the ISP's by-default-blocked policy, since I host a mail server - but the ISP is fine with this, they have a form for it, super easy.

      Some family members are behind CGNAT, and I'm not sure if their ISP has the option to move out from behind that, but since they don't self-host it's probably slightly more secure from outside probes. We're still able to privately share communications via my VPN hub to which they connect, which allows me to remotely troubleshoot minor issues.

      I haven't looked into cloudflare tunnels, but haven't felt the need.

    • vladvasiliu an hour ago

      What do you mean by "one port at a time"?

      I run cloudflared on one machine, and it proxies one subdomain to one port, and another to a unix socket (could have been a second port, no pb).