Our 5 year ELA for VMware went from 1.5M USD to 12M USD. I work in Higher ed.
Our Hyper-V environment came online a few months ago. It was already included with our ELA with Microsoft so we were able to splash out a bit for some higher tier support.
Granted, we have a separate team working on "genAI stuff."
We started converting virtual machines about 3 weeks ago and we've gotten through ~500 of about 3500 or so.
Our grant based HPC environment is just moving back to bare metal. The VM conversion is just for ad-hoc HPC and then all of our general infrastructure. Some of our larger application servers (SAP Hana) are possibly staying on VMWare if SAP won't support them on Hyper-V.
This is a hot topic among some of my nerdier SME friends, and our conclusion is that the major players are HPE and Nutanix. At least from our perspective over here in Sweden.
HPE did a big brain move to support multiple hypervisor backends with their own frontend. The only way to go forward imho.
I'm using Proxmox at my current $dayjob, and we're quite happy with it. I come from a big VMware shop and I think most businesses could easily replace VMware with Proxmox.
I think Proxmox should just launch an Enterprise contract, regardless of the cost, just have one. Because right now I think the main obstacle halting adoption is their lack of any Enterprise SLA.
On a personal level I would love to see KubeVirt, or Openshift with KubeVirt, take over more. It just seems like a genius move to use the already established APIs of kubernetes with a hypervisor runtime.
Proxmox is about to miss their window of opportunity here. They are uniquely positioned to take on VMWare, but their outfit seems like a fairly tiny and conservative company with zero ambition to take on the world, so to speak.
If they aren't interested in that business, then it isn't really a window of opportunity for them. In fact I respect a company that chooses to not pursue business opportunities that don't fit their goals, and instead focus on being a good fit for the market they are in. Growth isn't the most important thing.
It helps that they’re not a publicly traded company [A]. If you’re beholden to stockholders, you’re beholden to a market demanding growth at all costs. Even if the leadership at the moment wants this stable strategy, all investor pressure tends toward aggressive moves to the contrary.
[A] probably? I couldn’t conclusively determine this, and I’m not an expert
I've been at multiple companies that wasted millions courting large enterprise contacts only to not make a single sale. It does make the sales update more exciting though—if we just get this one sale…
I can't blame any company for wanting to stay out of that market.
You can have a look at XCP-ng. They have the expertise and it's originally a fork of Citrix XenServer however they are completely on their own feet now delivering some interesting advancements.
Access to Enterprise repository
Complete feature-set
Support via Customer Portal
Unlimited support tickets
Response time: 2 hours* within a business day
Remote support (via SSH)
Offline subscription key activation
You asked for an Enterprise SLA. Not all Enterprise SLAs are 24/7. IM(Professional)E, most are not 24/7.
> What's a business day?
From the FAQ on the page linked to by guerby:
What are the business days/hours for support?
Ticket support provided by the Proxmox Enterprise support team is available on Austrian business days (CET/CEST timezone) for all Basic, Standard, or Premium subscribers, please see all details in the Subscription Agreement.
For different timezones, contact one of our qualified Proxmox resellers who will be able to offer you help with Proxmox solutions in your timezone and your local language.
Check out the actual FAQ entry to chase down the links embedded in those words that I'm too lazy to try to reproduce.
> ...definitely [a] 24/7 SLA is what Proxmox needs to break into the enterprise sector I have experience with.
Well, their FAQ says:
For different timezones, contact one of our qualified Proxmox resellers who will be able to offer you help with Proxmox solutions in your timezone and your local language.
Consulting the list of resellers that that page links to finds one that blatantly advertises 24x7 support, and it's likely that others will offer it if asked. See [0].
> You asked for an Enterprise SLA. Not all Enterprise SLAs are 24/7. IM(Professional)E, most are not 24/7.
Any serious enterprise software or hardware company absolutely has a 24/7 support option. They all have a base option that is not 24/7 for a significantly lower price.
There’s no way you’re replacing VMware in any company of any size without 24/7 support.
Microsoft seems perfectly capable of advertising 24/7 support whilst never managing to call back within 24 hours on business crippling sev1 tickets. Just look at how often someone on /r/sysadmin is shocked to find this is the norm.
I know thst youre right about the wording turning off orgs but I do wonder when the biggest enterprise organisation can barely offer it in practice what really is the show stopper for business.
The trap is you need Microsoft support training & strategy. If you buy unified and open a sev a, they just fuck around and assign an engineer from Antartica who works from 3AM-6:20AM Mongolian time, then reassign at 6:19AM to dude in Japan to reset their 2 hour SLA for the incident manager. In general, if you are big, you're better off buying Premier from a partner, and declaring a crit sit. Many issues are fixable by less dumb third party L2 techs, and you can leverage the partner's juice with Microsoft to get somebody. You have the ability to inflict real pain on the reseller, but all Microsoft will do for a strategic customer is send some VP of something to apologize profusely at great length and suggest the more staff meet with your TAM/CSM so they can get a dramatic reading of a powerpoint. Companies like this only understand pain, so you need leverage.
Microsoft is uniquely bad at this type of stuff. Anyone committing serious infrastructure where bad things are gonna happen when it goes down is insane for using HyperV. But you're also insane expecting a small reseller of some small company to pull your chestnuts out of the fire.
Anyone who is really committed to their infrastructure will not build it on top of highly proprietary stuff where you have 0 visibility into what's actually happening so you can only hope that somebody fixes it sustainably, in a reasonable time frame and permanently.
With open source, if you have the right people, you can find/ bisect down to the commit and function where the problem is exactly, which speeds up the remedy immensely. We have done such a thing with backup restores from the Proxmox Backup Server. The patches are now in Proxmox VE 9.0 because the low hanging fruit problem was actually with the client code not the Proxmox Backup Server.
It’s not about the support. It’s about the blame shifting. The CTO has a piece of paper which means he’s no longer accountable. Gartner says they are good, the occasional sales lunches are expensive, and the golf game can continue.
Formally, yes, they are 24/7. However, getting the expert you really need to solve the issue, that can be much harder on weekends. Sometimes it only amounts to handholding till Monday.
I can second technion's observations about Microsoft's "24/7" support SLA.
Anyway, as the FAQ answer that I quoted mentions, there are plenty of qualified Proxmox resellers who offer support for folks who are dissatisfied with what is offered by Proxmox Server Solutions GmbH. One reseller explicitly advertises 24x7 support [0]. I expect others would offer 24x7 support if you asked, but don't see the need to advertise it up-front.
Yes, I'd think Openshift with Kubevirt would be positioned to move in. Lots of Openshift in some of the sectors I've worked with so seems like a natural expansion.
I forgot about MSFT's ability to bundle Hyper-V though which seems to come up in this thread a lot.
I work for an MSP, mostly with small to medium companies. Licensing costs went up a ton when broadcom acquired vmware. They went up a ton more this year with minimum core counts, current licensing costs are roughly $20k a year minimum. They might hike the price again, even medium businesses that see some value in avoiding an expensive migration want to avoid this uncertainty. Basically they don't want to deal with small and medium sized businesses. I'm sure large businesses are facing price hikes too but I don't have experience with that.
If you are on a perpetual license you can put the management vlan on a network not connected to the internet if it wasn't already and realistically this buys a few years. You will not be able to patch, eventually auditors will not accept that. For the rest not on perpetual licensing, when the licensing expires you will not be able to power on machines, if they reboot they stay off.
About half of clients we are migrating to hyper-v. Most are already running windows servers. There are some differences but hyperv covers the important features and the licensing is basically already included. Beeam makes the virtual to virtual move a lot easier, this is what most of our customers use for backups
For a good chunk they are migrating to azure or another hosted environment. If you don't have a main office with a file server or some more demanding line of business apps this is a pretty easy move.
A few are going to nutanix. Or more of expanding nutanix.
Kind of sad seeing businesses getting screwed by closed source proprietary software, then making the same choices all over again.
Nutanix also seeing huge demand.
Not everyone is repeating their mistakes, with Proxmox and Xcp-ng seeing huge new level of business, as well, which is nice.
I'm part of the Apache CloudStack project and that too is seeing unparalleled levels of demand.
The KVM hypervisor has sort of become the de facto choice, thanks to virt-v2v tool which can help migrate VMware guests.
I used KVM for years, but it wasn't reliable. That changed a year ago or so, but even then I ripped all virtualization out, because it's just operationally annoying.
I basically don't trust any virtualization implementation to not have been written by idiots.
Curious, I ditched our last VMWare servers ~a decade ago for KVM (via ganeti) for 100-200VMs running our dev, stg, and production loads, and it's been super reliable.
Broadcom turned up the heat on our pot fast enough that we’re evacuating over to proxmox. I and several others in IT had run it at home for a while, so when Broadcom made the definite losses to continue on VMWare far higher than the likely losses from any migration outage, it became a no-brainer to migrate.
Migrating part of the farm and A/B testing shows good results and we’ll be able to complete it in-place before the next pizzo payment to Broadcom is due.
Thanks for the nudge, Broadcom! As far as I’m concerned, Broadcom and Oracle are tied for first on my “do not voluntarily do business with” list. Equaling Oracle in this way is a feat…
At PriorCo, I did a slide deck presentation of our options at the time (2023/2024) and pitched essentially three pathways: stay on VMware, move to Apache Cloudstack, or move to Nutanix. The deck was roundly ignored in favor of a lift-and-shift to AWS for remaining infrastructure.
If I were running the migration, my preferred pathway would’ve been to Apache Cloudstack. We had the expertise to pull it off, and it would’ve freed us from vendor partners. Nutanix was really only on the list purely from its technology portfolio; its lack of profitability and shifting towards SaaS for features like cost analysis meant that we’d be moving into a similarly bad situation as VMware at the time (wholly beholden to their business priorities instead of our own), which I didn’t care for.
There’s a lot of options out there, most of which are built atop either KVM/QEMU or OpenStack. Virtuozzo’s offerings impressed me, but the lack of a “comprehensive” product was a turnoff. Oxide was incredibly interesting from a simplicity and integration perspective, but the appetite wasn’t there to try a startup’s product. Microsoft and Oracle were both ruled out due to higher costs and more onerous licensing than VMware/Broadcom, while IBM/OpenShift were ruled out as our private cloud estate was 100% VMs with only ~20% of our internal products capable of containerization support.
The biggest advice I can give is to understand your workload today, and determine options accordingly. Everyone’s pitching K8s and containers, but if your estate is majority VMs, then a lot of those options just aren’t worthwhile.
1. Proxmox cannot even join a network using DHCP requiring manual IP configuration.
2. Disk encryption is a hell instead of checkbox in installer
3. Wi-Fi - no luck (rarely used for real servers, but frequently for r&d racks)
Of course, it is a Debian core underneath and a lot of things are possible given enough time and persistence, but other solutions have them out of the box.
I'm with a block storage vendor that works with a lot of companies migrating off VMware, and the diversity of KVM-based cloud management platforms we're seeing is fascinating. We have customers moving to OpenNebula, CloudStack, Proxmox, OpenStack, HP VME, Oracle Virtualization, and even some homegrown solutions. The common thread is that they're all looking for a storage backend that is not tied to a specific hypervisor and can deliver predictable high performance. The beauty of the KVM ecosystem is the freedom to choose the best tool for the job, and that extends to the storage layer. A good software-defined block storage solution should have the features (data migration, disaster recovery) and capabilities to make the transition away from VMware as smooth as possible.
Mostly bitching to corporate IT to make it possible to use alternative tools and workflows.
Not kidding, that’s the main blocker. We have the DevOps knowledge on our team to go to containers, prepackaged dev environments, etc. But corporate cyber tends to respond to our requests to discuss cyber policy and escalate via proper channels with “sorry that’s against policy”.
This is not my experience at one company but multiple good, name brand companies that generally do good engineering and software work.
Because the old “trusted” vendor is now absurdly expensive and switching to another one helps increase profit? E.g. why did organizations switch from bare metal to virtualization in the first place?
Non-Profit Liberal Arts Higher Ed here. Trying to move to the Azure Hybrid Cloud as fast as possible. Going to have to eat one more year of 170% price hike in our current VMware contract.
Heard from a peer school on the east coast that had to sign a 600% hike in their most recent contract. Absolutely evil.
I would not call it evil. Orgs are _voluntarily_ making the choice to lock in for expensive useless proprietary software by either being stupid or - actually evil - receiving kickbacks from those proprietary companies.
Every manager signing off the contract with vendor lock in should understand possible repercussions. Its not a rocket science. Yep they will screw you along the way with prices. Yea, that sales rep saying they won't is a lying bastard that can't care ess about your business the moment your pen raised from your signature in contract.
Seeing a lot of Nutanix especially for VDI/Citrix heavy workloads or typical 3-tier applications. HP VME is also becoming a thing as an almost drop-in and VERY cost effective alternative to VMWare. In telco Openstack is still king AFAIK.
Check out libvirtd based stacks, because that's what's supported by upstream Linux.
Some shops here migrate to proxmox as a UI because of certification requirements, but I migrated some of my customers to cockpit dashboard, and some to kubernetes. It's always a matter of scale and provisioning requirements.
Cockpit is my favorite so far because it's easy to setup, but its focus isn't cluster scale, which is what most larger companies need. You have to setup basically two cockpit variants: the webui and lots of cockpit server daemons (aka libvirtd on remote machines). The webui then uses SSH to login to other machines to manage them (e.g. via the known_hosts file on the webui server). [3]
Proxmox is pretty old and Perl, but it's doable. Usually storage clustering is a bit painful because you need something on a filesystem layer like ceph clusters.
There's also openshift but no idea if that is an IBM/RedHat lock-in as well, so the SMEs didn't want that risk.
In my sphere most companies are going to either Hyper-V or the cloud. Hyper-V kinda won by default as a lot of orgs already had Windows Server licenses.
We're driving as many apps as possible to containers, and replacing most of our virtual infrastructure with Talos Linux [1] which is a reasonably hardened OS dedicated to Kubernetes hosting. I strongly recommend using the terraform provider to help manage at scale. The docs seem a little sparse for beginners, but if you already know kubernetes concepts, it's pretty easy to pick up. If you know flux style gitups talhelper+sops is far better than naked talosctl. We're also trying to migrate off of IAAS provided kubernetes and migrate to talos within instances. It's an effort to reduce dependency on specific IAAS while also minimizing number of technologies we need to support.
We're driving anything that cannot be containerized to lift and shift to IAAS and forcing the app owners to pay for it out of their budget as motivation to modernize. They have to explain to the board why their spending increased and they are still on legacy.
I don’t really consider OpenShift in the same category. VMWare and its enabling software such as vSphere and vCenter are in another category than OpenShift to the point that there is a symbiotic relationship between VMWare and Dell in the corporate/enterprise setting
I'm seeing a bit of everything: renegotiating (which Broadcom doesn't really do), optimizing and consolidating hosts (to lower costs), public cloud migration (which is why I see the most given my line of work, but may not represent everything), forays into other hypervisors, etc.
Proxmox may come to many an HN visitor's mind (and I use it myself extensively, all my home services run on it), but it actually doesn't have a lot of enterprise features and isn't a drop-in replacement.
Most of the time I don't need a full VM and run inside a container via systemd-nspawn. This runs on the existing kernel instance but isolates everything nicely. Mainly use it for complex builds so they don't bork my system.
Ah, 5x? At $WORK, the low code tool vendor that is used to build the monolith (and that of our sister company) is bought by a private equity firm. Our sister company will face a 7x increase. Another fun thing is that the license is based on a percentage of licensing cost to their customers.
Their game is clearly to squeeze very hard for a few years, and then deprecate the product. I can't imagine that there are companies that are fine with such price hikes.
#1 reason people don't wholesale move off of VMW is 1 word, RISK. Too much risk to move legacy applications of enterprises without a lengthy project to move. This that do and start to, are choosing KVM variants, RH Open shift Virt, or Hyper-V. Promos would be good if it had enterprise support.
Nutanix is just as expensive and also a locked in option.
Hard to say much given we weren't given much info on how it'd be used.
E.g. Parallel's is only useful for people looking to run VMs locally on their Mac, but Hyper-V can be anything from that for a Windows PC to a full-blown headless hypervisor cluster with HA, shared volumes, replication, etc.
For several of the common categories, these are my takes:
- Traditional Enterprise: Nutanix [paid] if money is available, otherwise Hyper-V [paid] if a large Microsoft contract is already in place. If neither fit: fall through to acting like an SMB.
- SMB/Modern Mid-Sized Enterprise: Cloud [paid] only and/or Proxmox [free/paid]
- Tech Company: Doesn't matter, they'll do whatever sounds cool that year and make it work well enough
- Home Lab: Proxmox [free/paid]
- Windows PC: Hyper-V [free w/ Windows] (it's meh, but it's integrated - doubly so if you plan on using WSL on the side).
- Mac PC: Parallels [paid] if you need a GPU accelerated Windows guest, UTM [free] otherwise.
- Linux PC: QEMU+KVM [free], the choice of (optional) GUI client is up to preference.
Some extra notes by solution:
- Nutanix: Enterprises were staring to use this more and more even prior to the VMware sale. It's definitely the spiritual successor of traditional VMware usage in the data center. A bit less full of themselves, for now at least, than VMware ever managed to keep themselves (IMO).
- Proxmox: Has a bit of a habit of feeling like it always ends up a little broken by the time you've used an install/cluster for 6 months, but is by far the best option for the homelabber type use case (even ignoring that it's free as a reason). It's basically like someone configured KVM with what you want to be able to just (try to) use it without thinking about what's underneath, while still having access to the underneath to un-stick it in certain situations. Also does host-native containers! I never did have the guts to pitch my company try to run anything production on a cluster, but they do have reasonably priced support plans and advanced feature tiers for that.
- Parallels: Kind of sucks for the price, but there isn't anything else on macOS with the same GPU acceleration for Windows.
- Hyper-V: I think this is mostly still around because it helps Microsoft stay sticky at companies when the yearly renewal comes up. That said, it's alright - and it's also integrated into Windows in some pretty nifty ways for local use these days.
- UTM: Fantastic QEMU client for macOS, worth giving a few bucks for even though it's free.
I don't think Proxmox is anywhere near ready for that sort of shift. It's interesting what a big hole in the market VMWare is leaving and nothing quite fills it. OpenStack is the closest, but way more complicated than VMWare, and doesn't work at all for smaller deployments.
I’m not sure that’s true for larger scale installs but small scale VMware installs are probably less easily replaced by solutions that are also as well supported and have a path for expanding.
Doing a head-on VMware takeout path hasn’t been a good business strategy for companies that tried it.
For small SMBs using Proxmox is reasonably ok-ish. Running in production for 2+ years already our customers are quite happy. We also sent some patches to Proxmox for other much larger clients...
But, even if you restrict it to 'x86 virtualization', the alternative for the current crop of 'enterprise' OS environments is ...server sprawl. I'm a big fan of discrete hw for some things, but it can be a hard sell for everything.
The primary alternative to full system VMs is containers (or jails, zones... whatever your OS might call them). You don't need to go server sprawl or VMs as the only two options.
If you're so ignorant of the space you think virtualization is 20 years old, you're too ignorant to make proclamations about what anyone else should do.
I hate virtualization with a passion, because it's just more crap that doesn't work written by people that shouldn't touch computers.
In theory, it's great. In practice, if you need to get "support" from someone else, it's not so great anymore, as all these companies have been discovering.
I would use VM technology if whoever wrote it would provide me with a contract saying that if anyone were to find just one program that would crash their VM (while not crashing a real machine) or miscompute, that I would get a billion dollars.
To answer your question: I was smart enough to never use it in the first place.
The only kind of person who would say that they hate virtualization is a hardware vendor. Because of conflict of interests.
In this light, Broadcom, a hardware vendor, who buys a popular virtualization product does a "smart" move - it supposedly eliminates the very thing that eats away their profits. But it only looks smart to the vendor itself. For everyone else, the move looks unprofessional and incompetent.
Our 5 year ELA for VMware went from 1.5M USD to 12M USD. I work in Higher ed.
Our Hyper-V environment came online a few months ago. It was already included with our ELA with Microsoft so we were able to splash out a bit for some higher tier support.
Granted, we have a separate team working on "genAI stuff."
We started converting virtual machines about 3 weeks ago and we've gotten through ~500 of about 3500 or so.
Our grant based HPC environment is just moving back to bare metal. The VM conversion is just for ad-hoc HPC and then all of our general infrastructure. Some of our larger application servers (SAP Hana) are possibly staying on VMWare if SAP won't support them on Hyper-V.
This summer sucked big time but we'll make it.
This is a hot topic among some of my nerdier SME friends, and our conclusion is that the major players are HPE and Nutanix. At least from our perspective over here in Sweden.
HPE did a big brain move to support multiple hypervisor backends with their own frontend. The only way to go forward imho.
I'm using Proxmox at my current $dayjob, and we're quite happy with it. I come from a big VMware shop and I think most businesses could easily replace VMware with Proxmox.
I think Proxmox should just launch an Enterprise contract, regardless of the cost, just have one. Because right now I think the main obstacle halting adoption is their lack of any Enterprise SLA.
On a personal level I would love to see KubeVirt, or Openshift with KubeVirt, take over more. It just seems like a genius move to use the already established APIs of kubernetes with a hypervisor runtime.
Proxmox is about to miss their window of opportunity here. They are uniquely positioned to take on VMWare, but their outfit seems like a fairly tiny and conservative company with zero ambition to take on the world, so to speak.
If they aren't interested in that business, then it isn't really a window of opportunity for them. In fact I respect a company that chooses to not pursue business opportunities that don't fit their goals, and instead focus on being a good fit for the market they are in. Growth isn't the most important thing.
It helps that they’re not a publicly traded company [A]. If you’re beholden to stockholders, you’re beholden to a market demanding growth at all costs. Even if the leadership at the moment wants this stable strategy, all investor pressure tends toward aggressive moves to the contrary.
[A] probably? I couldn’t conclusively determine this, and I’m not an expert
I've been at multiple companies that wasted millions courting large enterprise contacts only to not make a single sale. It does make the sales update more exciting though—if we just get this one sale…
I can't blame any company for wanting to stay out of that market.
They're a european business. I don't think they're interested in the stress involved in selling to enterprise.
You can have a look at XCP-ng. They have the expertise and it's originally a fork of Citrix XenServer however they are completely on their own feet now delivering some interesting advancements.
Xcp-ng seems better positioned with a familiar vmwareish experience.
https://www.proxmox.com/en/products/proxmox-virtual-environm...
"Premium"
>Response time: 2 hours* within a business day
What's a business day? I wouldn't call that a 24/7 SLA.
> I wouldn't call that a 24/7 SLA.
You asked for an Enterprise SLA. Not all Enterprise SLAs are 24/7. IM(Professional)E, most are not 24/7.
> What's a business day?
From the FAQ on the page linked to by guerby:
Check out the actual FAQ entry to chase down the links embedded in those words that I'm too lazy to try to reproduce.I should have been more clear then, definitely 24/7 SLA is what Proxmox needs to break into the enterprise sector I have experience with.
It's kind of frustrating because it's such a tiny detail that could make them a real contender in this new power vacuum.
> ...definitely [a] 24/7 SLA is what Proxmox needs to break into the enterprise sector I have experience with.
Well, their FAQ says:
Consulting the list of resellers that that page links to finds one that blatantly advertises 24x7 support, and it's likely that others will offer it if asked. See [0].[0] <https://news.ycombinator.com/item?id=45637767>
> You asked for an Enterprise SLA. Not all Enterprise SLAs are 24/7. IM(Professional)E, most are not 24/7.
Any serious enterprise software or hardware company absolutely has a 24/7 support option. They all have a base option that is not 24/7 for a significantly lower price.
There’s no way you’re replacing VMware in any company of any size without 24/7 support.
Microsoft seems perfectly capable of advertising 24/7 support whilst never managing to call back within 24 hours on business crippling sev1 tickets. Just look at how often someone on /r/sysadmin is shocked to find this is the norm.
I know thst youre right about the wording turning off orgs but I do wonder when the biggest enterprise organisation can barely offer it in practice what really is the show stopper for business.
They suck balls.
The trap is you need Microsoft support training & strategy. If you buy unified and open a sev a, they just fuck around and assign an engineer from Antartica who works from 3AM-6:20AM Mongolian time, then reassign at 6:19AM to dude in Japan to reset their 2 hour SLA for the incident manager. In general, if you are big, you're better off buying Premier from a partner, and declaring a crit sit. Many issues are fixable by less dumb third party L2 techs, and you can leverage the partner's juice with Microsoft to get somebody. You have the ability to inflict real pain on the reseller, but all Microsoft will do for a strategic customer is send some VP of something to apologize profusely at great length and suggest the more staff meet with your TAM/CSM so they can get a dramatic reading of a powerpoint. Companies like this only understand pain, so you need leverage.
Microsoft is uniquely bad at this type of stuff. Anyone committing serious infrastructure where bad things are gonna happen when it goes down is insane for using HyperV. But you're also insane expecting a small reseller of some small company to pull your chestnuts out of the fire.
Anyone who is really committed to their infrastructure will not build it on top of highly proprietary stuff where you have 0 visibility into what's actually happening so you can only hope that somebody fixes it sustainably, in a reasonable time frame and permanently.
With open source, if you have the right people, you can find/ bisect down to the commit and function where the problem is exactly, which speeds up the remedy immensely. We have done such a thing with backup restores from the Proxmox Backup Server. The patches are now in Proxmox VE 9.0 because the low hanging fruit problem was actually with the client code not the Proxmox Backup Server.
It’s not about the support. It’s about the blame shifting. The CTO has a piece of paper which means he’s no longer accountable. Gartner says they are good, the occasional sales lunches are expensive, and the golf game can continue.
Formally, yes, they are 24/7. However, getting the expert you really need to solve the issue, that can be much harder on weekends. Sometimes it only amounts to handholding till Monday.
I can second technion's observations about Microsoft's "24/7" support SLA.
Anyway, as the FAQ answer that I quoted mentions, there are plenty of qualified Proxmox resellers who offer support for folks who are dissatisfied with what is offered by Proxmox Server Solutions GmbH. One reseller explicitly advertises 24x7 support [0]. I expect others would offer 24x7 support if you asked, but don't see the need to advertise it up-front.
[0] <https://www.proxmox.com/en/partners/find-partner/all/partner...>
Yes, I'd think Openshift with Kubevirt would be positioned to move in. Lots of Openshift in some of the sectors I've worked with so seems like a natural expansion.
I forgot about MSFT's ability to bundle Hyper-V though which seems to come up in this thread a lot.
Love the username.
I work for an MSP, mostly with small to medium companies. Licensing costs went up a ton when broadcom acquired vmware. They went up a ton more this year with minimum core counts, current licensing costs are roughly $20k a year minimum. They might hike the price again, even medium businesses that see some value in avoiding an expensive migration want to avoid this uncertainty. Basically they don't want to deal with small and medium sized businesses. I'm sure large businesses are facing price hikes too but I don't have experience with that.
If you are on a perpetual license you can put the management vlan on a network not connected to the internet if it wasn't already and realistically this buys a few years. You will not be able to patch, eventually auditors will not accept that. For the rest not on perpetual licensing, when the licensing expires you will not be able to power on machines, if they reboot they stay off.
About half of clients we are migrating to hyper-v. Most are already running windows servers. There are some differences but hyperv covers the important features and the licensing is basically already included. Beeam makes the virtual to virtual move a lot easier, this is what most of our customers use for backups
For a good chunk they are migrating to azure or another hosted environment. If you don't have a main office with a file server or some more demanding line of business apps this is a pretty easy move.
A few are going to nutanix. Or more of expanding nutanix.
Microsoft gaining the most I reckon.
Kind of sad seeing businesses getting screwed by closed source proprietary software, then making the same choices all over again.
Nutanix also seeing huge demand.
Not everyone is repeating their mistakes, with Proxmox and Xcp-ng seeing huge new level of business, as well, which is nice.
I'm part of the Apache CloudStack project and that too is seeing unparalleled levels of demand. The KVM hypervisor has sort of become the de facto choice, thanks to virt-v2v tool which can help migrate VMware guests.
I used KVM for years, but it wasn't reliable. That changed a year ago or so, but even then I ripped all virtualization out, because it's just operationally annoying.
I basically don't trust any virtualization implementation to not have been written by idiots.
Curious, I ditched our last VMWare servers ~a decade ago for KVM (via ganeti) for 100-200VMs running our dev, stg, and production loads, and it's been super reliable.
Broadcom turned up the heat on our pot fast enough that we’re evacuating over to proxmox. I and several others in IT had run it at home for a while, so when Broadcom made the definite losses to continue on VMWare far higher than the likely losses from any migration outage, it became a no-brainer to migrate.
Migrating part of the farm and A/B testing shows good results and we’ll be able to complete it in-place before the next pizzo payment to Broadcom is due.
Thanks for the nudge, Broadcom! As far as I’m concerned, Broadcom and Oracle are tied for first on my “do not voluntarily do business with” list. Equaling Oracle in this way is a feat…
I place Broadcom higher. At least Oracle seems to want customers that pay. I can't find evidence that of Broadcom wanting customers.
At PriorCo, I did a slide deck presentation of our options at the time (2023/2024) and pitched essentially three pathways: stay on VMware, move to Apache Cloudstack, or move to Nutanix. The deck was roundly ignored in favor of a lift-and-shift to AWS for remaining infrastructure.
If I were running the migration, my preferred pathway would’ve been to Apache Cloudstack. We had the expertise to pull it off, and it would’ve freed us from vendor partners. Nutanix was really only on the list purely from its technology portfolio; its lack of profitability and shifting towards SaaS for features like cost analysis meant that we’d be moving into a similarly bad situation as VMware at the time (wholly beholden to their business priorities instead of our own), which I didn’t care for.
There’s a lot of options out there, most of which are built atop either KVM/QEMU or OpenStack. Virtuozzo’s offerings impressed me, but the lack of a “comprehensive” product was a turnoff. Oxide was incredibly interesting from a simplicity and integration perspective, but the appetite wasn’t there to try a startup’s product. Microsoft and Oracle were both ruled out due to higher costs and more onerous licensing than VMware/Broadcom, while IBM/OpenShift were ruled out as our private cloud estate was 100% VMs with only ~20% of our internal products capable of containerization support.
The biggest advice I can give is to understand your workload today, and determine options accordingly. Everyone’s pitching K8s and containers, but if your estate is majority VMs, then a lot of those options just aren’t worthwhile.
What's the reason for not considering Proxmox?
My 2 cents: Proxmox is too rigid. For example:
1. Proxmox cannot even join a network using DHCP requiring manual IP configuration.
2. Disk encryption is a hell instead of checkbox in installer
3. Wi-Fi - no luck (rarely used for real servers, but frequently for r&d racks)
Of course, it is a Debian core underneath and a lot of things are possible given enough time and persistence, but other solutions have them out of the box.
They seriously need to invest in a well engineered multi node cluster filesystem. VMFS made VMware into the behemoth it is.
Without that your options for HA shared storage is Ceph (which proxmox makes decently easy to run), or NFS.
I'm with a block storage vendor that works with a lot of companies migrating off VMware, and the diversity of KVM-based cloud management platforms we're seeing is fascinating. We have customers moving to OpenNebula, CloudStack, Proxmox, OpenStack, HP VME, Oracle Virtualization, and even some homegrown solutions. The common thread is that they're all looking for a storage backend that is not tied to a specific hypervisor and can deliver predictable high performance. The beauty of the KVM ecosystem is the freedom to choose the best tool for the job, and that extends to the storage layer. A good software-defined block storage solution should have the features (data migration, disaster recovery) and capabilities to make the transition away from VMware as smooth as possible.
Openstack second wind was definitelly not on my 2020s bingo card. But I agree kvm solutions have a lot of momentum.
Mostly bitching to corporate IT to make it possible to use alternative tools and workflows.
Not kidding, that’s the main blocker. We have the DevOps knowledge on our team to go to containers, prepackaged dev environments, etc. But corporate cyber tends to respond to our requests to discuss cyber policy and escalate via proper channels with “sorry that’s against policy”.
This is not my experience at one company but multiple good, name brand companies that generally do good engineering and software work.
3rd party trust is not a joke. Why should they drop what they're doing to go and audit a new critical vendor?
Because the old “trusted” vendor is now absurdly expensive and switching to another one helps increase profit? E.g. why did organizations switch from bare metal to virtualization in the first place?
Because the old vendor started charging 10x the price
This. This always gets them. Just a matter of time.
"VMware's in court again. Customer relationships rarely go this wrong", 190 comments (2025), https://news.ycombinator.com/item?id=45167239
"Proxmox VE: Import Wizard for Migrating VMware ESXi VMs", 100 comments (2024), https://news.ycombinator.com/item?id=39841363
Non-Profit Liberal Arts Higher Ed here. Trying to move to the Azure Hybrid Cloud as fast as possible. Going to have to eat one more year of 170% price hike in our current VMware contract.
Heard from a peer school on the east coast that had to sign a 600% hike in their most recent contract. Absolutely evil.
I would not call it evil. Orgs are _voluntarily_ making the choice to lock in for expensive useless proprietary software by either being stupid or - actually evil - receiving kickbacks from those proprietary companies.
Every manager signing off the contract with vendor lock in should understand possible repercussions. Its not a rocket science. Yep they will screw you along the way with prices. Yea, that sales rep saying they won't is a lying bastard that can't care ess about your business the moment your pen raised from your signature in contract.
But people still believe in tales aren't they
Seeing a lot of Nutanix especially for VDI/Citrix heavy workloads or typical 3-tier applications. HP VME is also becoming a thing as an almost drop-in and VERY cost effective alternative to VMWare. In telco Openstack is still king AFAIK.
We use Proxmox.
NVidia are pushing hard in the direction of combined accelerators and ARM CPU (i.e. DGX, Thor, Jetson, etc).
Some of the upcoming hardware hits a sweet spot in terms of performance / $ / W. It's hard to ignore.
But Proxmox is ignoring ARM. Which is a big mistake IMO
Check out libvirtd based stacks, because that's what's supported by upstream Linux.
Some shops here migrate to proxmox as a UI because of certification requirements, but I migrated some of my customers to cockpit dashboard, and some to kubernetes. It's always a matter of scale and provisioning requirements.
Cockpit is my favorite so far because it's easy to setup, but its focus isn't cluster scale, which is what most larger companies need. You have to setup basically two cockpit variants: the webui and lots of cockpit server daemons (aka libvirtd on remote machines). The webui then uses SSH to login to other machines to manage them (e.g. via the known_hosts file on the webui server). [3]
Proxmox is pretty old and Perl, but it's doable. Usually storage clustering is a bit painful because you need something on a filesystem layer like ceph clusters.
There's also openshift but no idea if that is an IBM/RedHat lock-in as well, so the SMEs didn't want that risk.
[1] https://cockpit-project.org/
[2] https://www.proxmox.com/en/
[3] https://cockpit-project.org/guide/latest/feature-machines.ht...
In my sphere most companies are going to either Hyper-V or the cloud. Hyper-V kinda won by default as a lot of orgs already had Windows Server licenses.
Same here as well
We're driving as many apps as possible to containers, and replacing most of our virtual infrastructure with Talos Linux [1] which is a reasonably hardened OS dedicated to Kubernetes hosting. I strongly recommend using the terraform provider to help manage at scale. The docs seem a little sparse for beginners, but if you already know kubernetes concepts, it's pretty easy to pick up. If you know flux style gitups talhelper+sops is far better than naked talosctl. We're also trying to migrate off of IAAS provided kubernetes and migrate to talos within instances. It's an effort to reduce dependency on specific IAAS while also minimizing number of technologies we need to support.
We're driving anything that cannot be containerized to lift and shift to IAAS and forcing the app owners to pay for it out of their budget as motivation to modernize. They have to explain to the board why their spending increased and they are still on legacy.
- [1] - https://www.talos.dev/
Are you saying you have a board that isn't retarded? That would be a first.
That's not the word I'd use.
I might say when they have an opinion on something it can be very specific.
Red Hat is offering OpenShift virtualization, which is Kubernetes with Kubevirt. So some people might just use Kubernetes with Kubevirt.
There's also Harvester "open source hyperconverged infrastructure" https://harvesterhci.io/
Or some Xen spinoff like https://xcp-ng.org/
Smaller shops are migrating to Proxmox.
Red Hat has an implementation of OpenStack, which is more of a VMware replacement than OpenShift is.
I don’t really consider OpenShift in the same category. VMWare and its enabling software such as vSphere and vCenter are in another category than OpenShift to the point that there is a symbiotic relationship between VMWare and Dell in the corporate/enterprise setting
Do not ever use openshift
Can you explain why I shouldn't use OpenShift?
I know they pulled a mini-Broadcom on us and sharply raised all our prices after our first two years of having our OpenShift clusters.
They have a dominant percentage of banking workloads at this point.
This is damning with faint praise.
I find that regular libvirt/qemu with virt-manager or cockpit front-end on RHEL/Alma/Rocky is perfectly fine for plenty of situations.
Linux. kvm or lxc.
Hire a couple of sysadmins who know their ass from a hole in the ground.
I'm seeing a bit of everything: renegotiating (which Broadcom doesn't really do), optimizing and consolidating hosts (to lower costs), public cloud migration (which is why I see the most given my line of work, but may not represent everything), forays into other hypervisors, etc.
Proxmox may come to many an HN visitor's mind (and I use it myself extensively, all my home services run on it), but it actually doesn't have a lot of enterprise features and isn't a drop-in replacement.
Most of the time I don't need a full VM and run inside a container via systemd-nspawn. This runs on the existing kernel instance but isolates everything nicely. Mainly use it for complex builds so they don't bork my system.
croit.io, provides 24*7 enterprise support as a Proxmox Gold partner with a follow the sun support team.
Out of the loop: what's up with VMware?
Bought by Broadcom, now implementing classic strategy of leveraging vendor lock-in to milk customers.
The increase is massive ( I’ve heard x5 over existing contracts in some places )
Ah, 5x? At $WORK, the low code tool vendor that is used to build the monolith (and that of our sister company) is bought by a private equity firm. Our sister company will face a 7x increase. Another fun thing is that the license is based on a percentage of licensing cost to their customers.
Their game is clearly to squeeze very hard for a few years, and then deprecate the product. I can't imagine that there are companies that are fine with such price hikes.
Not only that, add dip downs in quality. For instance, VMware was famous for stuter-less graphics, now it's a 15 FPS show.
Milking customers is already a thin ice but in combination with declining quality it's a death sentence.
Fork?
Are you under the impression that VMware is free open source software?
Bought out by Broadcom, who realized if you increase prices by 10x and lose 75% of customers, you end with more revenue and less support costs
Hock Tan's only goal is to increase stock value. Period. At the expense of anything and everything else, stock value.
That's how he gets paid - increasing short term stock price.
"Show me the incentives and I'll show you the results" - Charlie Munger
Broadcom competing with Oracle for most hated company prize:
https://www.reddit.com/r/sysadmin/comments/1jfumvw/broadcom_...
https://www.theregister.com/2025/05/22/euro_cloud_body_ecco_...
Maybe Oracle can buy Broadcom and settle it
Broadcom bought VMware and changed the pricing.
Imagine seeing a 300% or more cost increase for no particular reason.
the reason is greed, old as time
I work for a vendor covering subset of functionality from VMware.
I am seeing Nutanix the most, then Proxmox, Openshift.
For some sub products, Avi is often going to HAproxy, Aria to a combination of Terraform, Datadog (and others)
I use VMWare Workstation a lot for testing and it's a very good workhorse for that. I hope they won't mess that up.
#1 reason people don't wholesale move off of VMW is 1 word, RISK. Too much risk to move legacy applications of enterprises without a lengthy project to move. This that do and start to, are choosing KVM variants, RH Open shift Virt, or Hyper-V. Promos would be good if it had enterprise support.
Nutanix is just as expensive and also a locked in option.
A major european bank is about to move everything they got on VMware to Hyper-V
Hyper-V and Windows Server 2025. It ticks all the boxes, except being BigScaryCorp for some people.
What’s next best alternative (regardless of cost)?
Anything else? Which is best?Did you just propose Virtualbox as a replacement for ESX?
That's the $XXM question.
Hard to say much given we weren't given much info on how it'd be used.
E.g. Parallel's is only useful for people looking to run VMs locally on their Mac, but Hyper-V can be anything from that for a Windows PC to a full-blown headless hypervisor cluster with HA, shared volumes, replication, etc.
For several of the common categories, these are my takes:
- Traditional Enterprise: Nutanix [paid] if money is available, otherwise Hyper-V [paid] if a large Microsoft contract is already in place. If neither fit: fall through to acting like an SMB.
- SMB/Modern Mid-Sized Enterprise: Cloud [paid] only and/or Proxmox [free/paid]
- Tech Company: Doesn't matter, they'll do whatever sounds cool that year and make it work well enough
- Home Lab: Proxmox [free/paid]
- Windows PC: Hyper-V [free w/ Windows] (it's meh, but it's integrated - doubly so if you plan on using WSL on the side).
- Mac PC: Parallels [paid] if you need a GPU accelerated Windows guest, UTM [free] otherwise.
- Linux PC: QEMU+KVM [free], the choice of (optional) GUI client is up to preference.
Some extra notes by solution:
- Nutanix: Enterprises were staring to use this more and more even prior to the VMware sale. It's definitely the spiritual successor of traditional VMware usage in the data center. A bit less full of themselves, for now at least, than VMware ever managed to keep themselves (IMO).
- Proxmox: Has a bit of a habit of feeling like it always ends up a little broken by the time you've used an install/cluster for 6 months, but is by far the best option for the homelabber type use case (even ignoring that it's free as a reason). It's basically like someone configured KVM with what you want to be able to just (try to) use it without thinking about what's underneath, while still having access to the underneath to un-stick it in certain situations. Also does host-native containers! I never did have the guts to pitch my company try to run anything production on a cluster, but they do have reasonably priced support plans and advanced feature tiers for that.
- Parallels: Kind of sucks for the price, but there isn't anything else on macOS with the same GPU acceleration for Windows.
- Hyper-V: I think this is mostly still around because it helps Microsoft stay sticky at companies when the yearly renewal comes up. That said, it's alright - and it's also integrated into Windows in some pretty nifty ways for local use these days.
- UTM: Fantastic QEMU client for macOS, worth giving a few bucks for even though it's free.
Jumping into bed with another single vendor.
You dont think enterprise IT does sensible things like have multiple vendors to avoid single points of failure.
A mix of Proxmox and Hyper-V. Also, Apache CloudStack using KVM.
Will be interesting to see if large organizations move to Proxmox
I don't think Proxmox is anywhere near ready for that sort of shift. It's interesting what a big hole in the market VMWare is leaving and nothing quite fills it. OpenStack is the closest, but way more complicated than VMWare, and doesn't work at all for smaller deployments.
I’m not sure that’s true for larger scale installs but small scale VMware installs are probably less easily replaced by solutions that are also as well supported and have a path for expanding.
Doing a head-on VMware takeout path hasn’t been a good business strategy for companies that tried it.
For small SMBs using Proxmox is reasonably ok-ish. Running in production for 2+ years already our customers are quite happy. We also sent some patches to Proxmox for other much larger clients...
Production: Hyper V Dev: Proxmox
This was question at a very very very slow moving org and industry I was at until about a year ago.
They went to Nutanix right before the broadcom acquisition and never looked back.
They were much happier, and HCI was very nice for k8s nodes.
docker is the way!
Virtualization is a 20 years old tech. Quit it.
Rather a lot older than that.
But, even if you restrict it to 'x86 virtualization', the alternative for the current crop of 'enterprise' OS environments is ...server sprawl. I'm a big fan of discrete hw for some things, but it can be a hard sell for everything.
The primary alternative to full system VMs is containers (or jails, zones... whatever your OS might call them). You don't need to go server sprawl or VMs as the only two options.
If you're so ignorant of the space you think virtualization is 20 years old, you're too ignorant to make proclamations about what anyone else should do.
Why?
Docker or podman your stuff? There's an image of every OS.
I hate virtualization with a passion, because it's just more crap that doesn't work written by people that shouldn't touch computers.
In theory, it's great. In practice, if you need to get "support" from someone else, it's not so great anymore, as all these companies have been discovering.
I would use VM technology if whoever wrote it would provide me with a contract saying that if anyone were to find just one program that would crash their VM (while not crashing a real machine) or miscompute, that I would get a billion dollars.
To answer your question: I was smart enough to never use it in the first place.
The only kind of person who would say that they hate virtualization is a hardware vendor. Because of conflict of interests.
In this light, Broadcom, a hardware vendor, who buys a popular virtualization product does a "smart" move - it supposedly eliminates the very thing that eats away their profits. But it only looks smart to the vendor itself. For everyone else, the move looks unprofessional and incompetent.