I recently moved to a Dutch municipality that runs its own non-profit ISP. They installed a symmetric 1 Gbps fiber connection with a static IP at my house for 40 euros per month.
The service is solid, there’s no upselling or throttling, and hosting things from home just works. I bring this up because when we talk about “open”, “fair” and “monopolies” the model of a local, non-profit ISP backed by the municipality could offer a real alternative. It doesn’t directly solve the peering issues, but it shifts the balance of power (and cost) somewhat.
i've wondered for a long time why this isn't a more common solution to these services that are almost inevitably monopolous. power, water, and internet kind of things.
While this is great when it works it does raise some interesting challenges, what happens if the ISP loses money, should taxes be used to cover the cost since this is a public service? Is it reasonable for this ISP to undercut commercial offerings? Internet is in a weird grey zone where it's almost a utility but not on the same level as water or sewage.
I'm glad this non-profit ISP exists but on a national level I would prefer (strong) net neutrality laws. Probably not an issue in NL but in less developed countries neutrality isn't guaranteed.
Usually these ISPs are part of or under the municipal utility provider. So if they lose money, it first gets offset by profits from other utilities and eventually yes, the taxpayer will step in, directly or indirectly. No big deal. No one is complaining about subsidies for water or power lines in rural areas, neither should they when it comes to internet. Remember: These ISPs were founded because the market was already failing to provide decent offers or any at all.
In Austria the Internet, like the postal service, is a Universal Service ("Universaldienst"). As such, any completely unserviced citizen can petition the current Finance Minister to decree an ISP of his choice - usually A1, which is the privatized form of our former public office for postal services and telegraphs - to service their area. The costs of facilitating the servicing of this area are covered by all active ISPs of a certain size operating in Austria.
Telecommunications law in Europe is a very interesting thing.
After a famously bad wind storm in 2008, my house (along with thousands of others) was without power for about two continuous weeks.
The internet connection, which was FTTN VDSL, never skipped a beat. It was completely solid.
This was accomplished by using batteries and generators.
The ISP was The Phone Company, so their Cold War-era central office had very good backup power.
The VRAD nodes scattered all over town had enough battery backup that (at least in my neighborhood) things stayed up until they brought out generators for those nodes.
And at my house, the VDSL box had its own UPS. And I also had a rather overkill UPS, and a portable generator
We ran the generator intermittently, mostly to charge batteries and chill down refrigerators.
It wasn't an awesome time. It was hot as hell. It was a pain in the ass to keep the generator fueled. We didn't even try to run the desktop PC rigs.
But, yeah: The internet was working fine.
(We charged batteries for neighbors, too. One or two neighbors also dragged over extension cords to run their own fridge. And I opened up the WiFi completely so everyone nearby could use it.
So if you were my neighbour in that 2008 power outage, I'd have just taken care of that internet problem for you. The range at 2.4GHz was amazingly good in that abnormally-quiet RF environment.)
When we loose power here internet usually works just fine. All you need is a generator and you're back up and running. Usually the POP still has power so this works for a long time. Sometimes they are or run out of (backup) power too if its widespread and prolonged. Cell service including LTE is usually still up / up for longer, so again as long as you have a generator, you're good.
One doesn't have to rely on others for power. One can run their own generator, or set up a solar power system, or if you live out in the sticks, run a mini hydro system or use wind power. All of these can provide power to the home.
There's voting with your wallet and voting with your, well, vote.
In some sense a democracy is also a market and can lead to efficient allocation of resources, particularly common resources for common good.
This is why public utilities tend to work so well in practice. People, especially in the US, don't seem to realise that such services are also subject to strong market forces, just a different kind of market.
Voters care a lot about good public services, and they also care a lot about not getting taxed much. This can lead to very efficient outcomes in well functioning democracies, often more efficient than those that come out of private enterprise, when it comes to services that most of the population needs.
If it loses money, do some combination of raising prices and cutting costs. Aim for an average of zero profit/loss over the long term. Undercutting commercial offerings is perfectly fine if it works out that way.
This isn't unique to Vodafone. Google has also been slowly withdrawing from IXes globally in favor of PNIs and "VPPs" (verified peering providers). This only makes it harder for smaller networks to establish presence on the internet and feels pretty anti-competitive.
On the flip side, IXes are becoming harder and less desirable to participate in: port fees are going up, useful networks are withdrawing, low quality network participants are joining and widening blast radius. I'm not sure what the answer to this is, but this has not been a great year for the "open" internet.
I thought Google was _always_ like this. At least going back to 2015 when I left the ISP game, peering with them was notoriously difficult if you didn't have the traffic volumes required. Our network suffered from asynchronous routing to Google and Netflix for years because they refused to allow our routes despite checking all the boxes they require. Customers eventually left because other (larger) ISPs didn't have this issue.
I get why the enshittification of IXPs is occurring. Over the years many small and careless ISPs have caused issues for IXPs (and peers) based on what I've seen on mailing lists. It's hard work managing many hundreds or thousands of peers, let alone the equipment cost with multi-100Gbit ports becoming the norm for larger providers.
Google publishes a peering policy. It's reasonable to expect that they will peer with you if you hit all the requirements in the policy.
Afaik, their requirements have never been judgement based: just bandwidth minimums, port types and locations. I would expect that they prioritize new connections in some way, so if you barely hit the criteria and are somewhere well served by transit, you'll be low priority, and the requirements might change before your connection gets setup and if so, you might not get connected because you don't meet the new requirements, but otherwise, seems like if you meet the requirements, send in the application, and have some patience, the peering connection should turn up eventually.
It's not like they have a mostly balanced flows requirement like Tier 1 ISPs usually do. Also, even in their current peering policy, they don't require presence in multiple metros; just substantial traffic (10gbps), fast ports (100G), two pops in the same metro.
> IXes are becoming harder and less desirable to participate in
Could this be due to the rise of services like Equinix Fabric and Inter.link? Google doesn't need to peer directly with most anymore because there is always a middleman somewhere who can handle it, and for many businesses the convenience of a point and click web gui outways whatever it costs?
Bell Canada also has had a long-standing policy of refusing to peer with internet exchanges. They'll only truly peer with other direct backbone providers and a handful of one-off peer with other large networks (google, cloud flare, etc), but their historical position as Canada's base backbone (not so much anymore, but it was definitely a thing pre-2005) has meant their policy is most people should pay them to peer. I'm not sure if it's still the case, but IIRC for awhile they also refused to peer with any other domestic backbone providers.
The result has been some funny routes sometimes. I live in Toronto and have seen trace routes bounce over to Chicago to connect to stuff colocated here in Toronto.
It's frustrating as their fibre is my only real high speed option; also their lack of IPv6 on anything but their mobile network is annoying.
FWIW, this has being going on for decades. Bell literally doesn't care if there's a chance they can suck up some peering fees.
There have been periodic times where it became an acute problem, like early in the YouTube and Netflix years there was a lot of congestion in their upstream peers and they held out hoping those orgs would pay for the peering. They were also over provisioned in early DSL days where their upstreams became saturated and there were few alternative paths.
Maybe that's a way to give them bad PR and convince then to change policies? Unfortunately most people probably don't understand this well enough and they have a pretty well oiled PR machine with all their control over tv and radio stations, etc
> also their lack of IPv6 on anything but their mobile network is annoying.
This gives me even less confidence after BCE took over ZiplyFiber, US PNW provider. There's a long running joke about IPv6 just one more lab test away from deployment.
It's annoying how they're the only big ISP offering fiber everywhere when they're also the ones that don't support ipv6 and have the shitty peering policy. I've heard you can use another isp though (teksavvy maybe?) that uses Bell's fibre and supports ipv6
The solution - as always - is regulation. ISPs typically already have very generous business models with widespread monopolies on customers, overwhelming barriers to entry for new companies, and a lack of rate controls allowing then to price arbitrarily - all of which supports immensely profitable businesses without the need for additional extraction of capital from other parties. Regulators and consumers alike should be screaming in rage at the idea that their ISPs are now multi-dipping for revenue, but we’ve done a piss-poor job of explaining how this works to the common man and thus can’t count on them to support the Open Internet as we’d like to see it.
That being said, the threat to the open internet is also more than just ISPs being gigantic assholes: it’s centralization in general. A majority of web traffic passes into or through one of three main cloud compute providers; Cloudflare has such an outsized impact that regional IP blocks can disrupt global traffic; and ISPs have been permitted to consolidate through mergers and acquisitions into expansive monopolies. The internet is fiercely centralized and largely closed already, which is why these ploys by shitty ISPs are likely to work absent Government intervention.
You want to protect the open internet? Regulate the shit out of its major players again. Force them to keep it open, especially when it hinders expanding profit margins.
The current trend in govt regulation is actually going in the opposite direction, with telecom lobbies in Europe pushing for "fair share" (pretty much an implementation through law of what Deutsche Telekom and Vodafone Germany are doing right now) through the Digital Networks Act.
South Korea pioneered "fair share" govt regulations in 2016 (which caused Twitch to exit the market in 2024 due the exorbitant "fair share" fees).
Because western governments (and those whose governments were modeled from western regimes - like post-war South Korea and Japan) have become victim to regulatory capture and corruption. It’s why the FCC has repeatedly killed, blocked, or reversed reforms like net neutrality or “nutrition labels” on ISPs, and why South Korea gave in to “fair share” regulations that deterred further investment. Tech money is hugely influential, and the industry is almost exclusively made up of rent-seeking slumlords at this point, particularly at the top (Oracle, Microsoft, Google, Adobe, Apple, etc). It’s why DMCA reform is blocked, why pirates get jail time while AI grifts get a hand-wave, and why right-to-repair or data privacy remains a fractured and piecemeal reform instead of a national agenda item.
The problem isn’t regulation, but regulatory capture ensuring companies get the regulations they desire and benefit from.
> The problem isn’t regulation, but regulatory capture ensuring companies get the regulations they desire and benefit from.
Aka regulation..... Nearly all regulation is for regulatory capture and if you think of something that isn't it probably just outlived who it was designed to capture for.
Regulation isn't good enough. The government needs to make their own competing ISP.
Hell, at least in the US, there's precedent for this: government builds and maintain all the roads; they run most transit and intercity rail operations; and they run physical mail delivery. At one point they even owned most of the railroads[0]. Communications and travel infrastructure are things government is moderately good at.
For some reason, we just decided not to have a government-sponsored telecom company, even when Ma Bell made it patently obvious that having all the country's telecom infrastructure be privately owned by one company was a bad idea. It's obvious that a government-run ISP is about as crucial to life in 2025 as a government-run postal carrier was in the early 1800s.
[0] In the 1970s, all of America's railroads went bankrupt. First, they discharged their passenger rail mandates into Amtrak, then they went bankrupt anyway, and then they got nationalized.
As I’ve stated in other comments, the reason western governments don’t do this more often boils down largely to regulatory capture. Every single time there’s been a large mobilization of efforts to regulate some aspect of tech - municipal broadband expansion, cable box standardization and openness, right to repair, DMCA reform, privacy reforms, mandatory binding arbitration clauses, EULA’s, provider monopolies, etc - tech money floods into regulators and political races to counter the will of the mobilizers and their supporters. Then those same ghouls repeat mantras like “disrupt” and “deregulate” to convince people that actually it’s a good thing you only have three cellular networks, one cable provider, one telephone provider, two operating systems, and four media conglomerates to choose from. At one point these slimeballs claimed anyone who used anything else (like Linux, or GrapheneOS, or FOSS) was obviously a criminal who wanted something for nothing, such was their fear of an open ecosystem.
Regulations get a bad rap because for decades the only ones to really get passed have only entrenched existing players and (rent-seeking) business models while blocking new entrants or competitors. I’m 100% in agreement with you that every single state and country should have an internet network that’s open access and governed solely by that country’s constitutional law - a sort of digital state, if you will, with which they can court business and interest groups alike to represent their interests globally. Instead, we’re presently stuck with a “whoever donates the most money to politicians wins” model, and that means the open internet exists in spite of the interests of Capital, not because of their good graces.
What you are describing sounds mostly like a US-problem, not sure it's a western gov problem in general?
In my city, the municipality owns much of the fibre. The country I live in owns a bank where you can get a mortgage pretty cheap. The good parts of GDPR or CRA are very good and was not disrupted by large corporations?
> Hell, at least in the US, there's precedent for this: government builds and maintain all the roads; they run most transit and intercity rail operations; and they run physical mail delivery.
Correct me if I'm wrong but it seems to me the US government is doing a terrible job at all of these.
Awesome. The only internet connection we get IN THE MIDDLE OF BERLIN is Vodafone Cable. Deutsche Telekom wants to build fiber here, but our landlord refuses to open the door to the cellar because he wants to kick everybody out and raise the rent for new tenants.
> our landlord refuses to open the door to the cellar
The simple solution would be to make this illegal, i.e. require landlords to allow at least two competing wired ISPs to connect each household.
No need to make them pay for it; I suspect it would be more than enough to end their very lucrative arrangement of somehow rewarding exclusivity. (I don't have any evidence that landlords are getting paid for it by Vodafone directly, but I highly doubt that there's any above board reason for the status quo.)
Yes. It's the so called Berlin issue. If you're lucky enough to find an apartment, you have a good chance of getting a slumlord who does everything to make your life miserable.
I'd buy my own place, if there would be anything available. Probably need to move to another city or country.
How one can still buy into Berlin's hype in 2025?! Fuck their rental market, their landlords, three months deposit for completely empty apartment without even lightbulbs, copyright trolls, rundfunkbeitrag, rude customer service, schufa. Turn around and walk away.
You forgot the government who spends billions to 3km of motorway a few people uses. To an airport that is one of the worst in EU and almost a decade late. And of course now cut from arts because of all this.
And every single construction project takes forever. And costs a fortune. And it is impossible to build housing fast enough.
The reason to be in Berlin has always been its great art scene. Now they are actively destroying it. What's left is a few Rossmanns and an Edeka.
Deutsche Telekom is just as bad as Vodafone. IIRC the government stepped in and said they had to offer peering to everyone, so now they offer peering to everyone in some random hamlet in the middle of nowhere for 5000€ per month per gigabit, while peering at other locations and prices is still at their sole discretion.
Is there anything that Vodafone customers can do legally to punish Vodafone or not delivering on their broadband contracts?
If you're paying for a 1Gbps connection and Netflix is only able to stream to you at 0.93 Mbps because Vodafone or Inter.link are choking off the supply, surely that's breach of contract on Vodafone's part?
I'm sure Cory Doctorow has a word for what's happening here.
You can file a complaint with the "Federal ministry for digital transformation" (formed this year). It does actually work, but it's a lengthy process.
I did force my cell phone carrier to grant me proper 4G speeds last year, after spending many hours with their help line and ultimately complaining to the (then) ministry of transportation and digital infrastructure.
My plan advertises "up to 50 Mbit/s" on a 4G connection. I was getting less than 1 Mbit/s a lot of the time. Websites and videos would not load properly.
I downloaded the app of the german ministry that allows you to take speed tests and file a complaint. After multiple weeks of measuring connection speeds on the cellular network, I was able to file a complaint.
What did the provider do? Did they put your IMEI onto some list of other customers that complained, where all of you get better network prioritization?
That's why you are paying for a "up to" 1Gbps connection. (I think it was already a struggle that they had to put the "up to" in the big advertisement)
Surely there's a reasonable expectation that Netflix would work at decent speeds, especially given that Netflix's infrastructure, nor the network load as a whole are to blame, but rather the specific ISP bureaucracy? Getting 1/1000 the listed speed does not strike me as something even a 75 year old computer neophyte of a judge would take kindly too, unless it were for very good reasons.
I don't think there really is much that can be done. Even under ideal conditions, an ISP could only possibly guarantee the advertised link speed between you and their routers, not between you and any particular node on the Internet. Is it possible an ISP might be doing things that harm the QoS? Yeah, sure. But the angle to approach that problem is not by complaining about instances of limited bandwidth.
But the true link speed's not even what's being asked for. 4K Netflix never goes above 20 Mbps as far as I know, so getting just 1/50 the advertised speed to one of the most well-known internet services in existence, hardly seems like a big ask, especially when the only reason that it can't reach that speed or higher is because of the ISP, given that swapping to one that aren't being knobheads about it fixes the problem. It should be the responsibility of the ISP to keep links to other parts of the internet as open as possible. If real-world constraints prevent the speed from being all that high, because it's a shitty server in Australia, then that's understandable. This however, isn't that.
All I'm getting from this is that it's a good idea to label ISPs utilities and bring the hammer down if they're being knobheads about it.
It is mostly the middle, and not either of the endpoints, that is the real problem. You have a 1Gbps link, the Netflix DC you're reaching probably has multiple links with aggregated bandwidth measured in Tbps, but at some point in between the two there's a 10Gbps link being shared between 5000 subscribers at peak times and now the bottleneck is 2Mbps per subscriber. This link may or may not be under your ISP's (or Netflix's ISP's) control, and it may or may not be the only relevant bottleneck.
The solution that was developed in the Netflix-Comcast fight over a decade ago is content distribution. Instead of trying to build out extra capacity in every possible link, you shorten the path and thus reduce the number of contended links involved in each interaction. This scales much better, but it has two major problems: the first is rightsholders and their obnoxious anti-piracy restrictions, and the second is good old jurisdictional friction and economic misalignment. Somebody has to own the physical servers in all the myriad locations that keep the content closer to the consumer. If the ISP owns them, then they naturally want to exploit them. If Netflix owns them, they naturally don't want to serve their competitors. If a third party owns them, you address those two problems (potentially) but add new ones around liability, non-disclosure, competitiveness, etc.
If regulation is going to be useful here, it needs to focus on opening up opportunities to serve the unsexy middle of the infrastructure puzzle and not just the most visible parts that consumers/voters usually interact with. Also, "Netflix" needs to be understood as just a stand-in for any high-bandwidth Internet service, as the landscape is constantly changing.
at some point in between the two there's a 10Gbps link being shared between 5000 subscribers at peak times and now the bottleneck is 2Mbps per subscriber. This link may or may not be under your ISP's (or Netflix's ISP's) control, and it may or may not be the only relevant bottleneck.
No, that link is absolutely under Vodafone's control. They're deliberately not upgrading it so that they can extort money from Netflix.
The solution ... is content distribution.
CDNs have been worldwide, including Germany, for a long time. That's not the problem here.
If the CDN is so poorly interconnected with Vodafone that there's one bottlenecked link, then it's not really accomplishing its job, at least as far as "inside of Germany" is concerned. It might have reduced pressure on another bottleneck, like links between the US and the EU, but it still needs to spread out more. If Vodafone is blocking that, then pressure should be applied to force them to open up more connections. I'm assuming this CDN serves more than just Netflix, mind you.
Secondly, the question of responsibility cannot be answered the same way today that it was answered in the Internet of universities. Netflix and Vodafone are not peers. The bandwidth ratio between them is incredibly lopsided. This will never change, there is no foreseeable scenario under which Vodafone has a reason to send anywhere near the same amount of data to Netflix as it gets back. This asymmetrical relationship inherently implies a different kind of business arrangement than traditional peering.
What Vodafone (any ISP) provides to Netflix (any content provider) is access to consumers. This is a service, and services are not free. The natural monopoly ISPs enjoy implies some degree of regulatory restraint must be applied on them, but it does not mean they bear all the costs of all the infrastructure either.
However, my bigger point is that this cannot constantly be reduced to these two-party analyses. Netflix is waning, others are rising, this problem needs to be solved in a scalable way.
In the US the FCC named and shamed broadband ISPs for their low speeds and "magically" those speeds increased over the following years. Overcome with greed, some ISPs eventually found ways to cheat on the benchmarks though. https://www.fcc.gov/general/measuring-broadband-america
I think it's actually a quite complicated question and it only works because people are playing somewhat nice with each other. Like imagine if Netflix refused to peer with one particular ISP unless they paid an extortionate amount of money. Should the ISP be legally required to pay any price they name? I don't think that would be fair.
One solution could be to have geographically distributed test points. Any connection to be able to claim a certain speed has to be able to get that speed to those test points. And the test points are legally required to connect to anyone that can bring fiber to their doorstep. If someone plays hardball with peering there will then always be the backup option of routing traffic through one of the test points.
At least here in the Czech Republic, ISPs have to also list a "guaranteed speed", and it can't be less than some fraction of the advertised maximum. I don't know what part of the Internet that speed is supposed to be measured against, though.
There's always an implicit reasonableness requirement on these things. "Up to" isn't a wildcard that lets you do anything you want. It will protect you if you struggle to maintain top speeds during peak hours, or if a technical fault cuts speeds in half for a few days. But if you provide 1/1000th of what you claim to offer, "up to" won't stop a judge from observing that you're not really providing what you said you would.
There's definitely not anything in the contract that promises performance to a 3rd party, especially not in a residential contract. The legal options are switch to a different ISP and/or start a new one. Not always easy or practical, but there you go.
Who is to say where the performance problem is? Certainly not your contract.
Maybe if the last mile is cronically congested, or between the local aggregation switch and their regional exchange points, you might have a legal case. But if the issue is insufficient connectivity between their network and other networks, I would be very surprised if the contract terms covered that at all.
There's a bunch of networks throughout the world where their policies mean you can get more economically acheive better connectivity to their customers by hosting outside the geographic boundaries of the network rather than inside it. Doesn't make sense from a theoretical point of view, but when German ISPs won't interconnect within Germany, serve their customers from Poland or France and the connectivity picture may change significantly. Worst case, serve them from the US (but the latency may be too high)
Depends on the region. Often there are smaller regional companies providing fiber internet. Prices for these fiber connections a still somewhat higher than the cheapest vodafone tier, but you also get better service for your money.
Afaik almost a monopoly: there is Deutsche Telekom which does the same thing and Vodafone. I think apart from some local providers almost everybody else is just a reseller of one of the two.
There are resellers that do not just rebrand a whitebox product, but have their own IP addresses, network and peering polices. Their customers are not necessarily impacted by the IP peering policies of the company that owns the access network.
This is a bit messy but if there are completing options at a given location install multiple ISPs and run them concurrently and log the details—download speeds, etc.
There's nothing as good as hard verifiable data—even if regulators play hardball and favor ISPs then you've the evidence to whip up political action (claim biased decisions, etc.).
Unfortunately, having a landline capable of DSL is no longer the default in Germany.
Some apartment buildings exlusively offer DOCSIS via a single provider (as there's never been any unbundling of the DOCSIS "local loop"; presumably under the assumption that a landline will always be available anyway?).
If that one provider is oversubscribed, you're pretty much out of luck.
High speed internet is a market not just internet access. Email might not care that your on a DSL connection but a streamer can’t generally use DSL as a substitute.
Yes and no. There are other providers in Germany. However, with the EU's neoliberal privatization policy the governments privatized many existing infrastructure. Vodafone bought the previous government company that owned all of the the cable TV infrastructure of Germany. So they are a monopoly of a particular type of internet connection. Depending on the place the alternatives could be too slow since Germany also has an aging population that do not {care about, demand} higher internet speeds and didn't upgrade its copper infrastructure due to corruption.
Some new apartments also simply lack phone lines. No idea how that's legal (since there is no competition at all on DOCSIS, unlike on DSL, in Germany), but it's a thing these days.
In Germany in specific building there is only one provider available in your telephone socket, and one in your cable socket in your apartment. Frequently there is no cable socket.
That's not relevant. Over a Deutsche Telekom phone line you can choose an ISP. The ISP sometimes has a layer two connection to you and therefore has their own infrastructure or they have a layer 3 connection in which case you suffer from the Telekom policies.
Layer 2 = their infrastructure connects you to the internet
Layer 3 = theyre literally just a reseller, DTAG is providing your internet connection, the ISP just billing etc.
> Instead, all traffic will flow through a single company called Inter.link, which charges content providers based on how much data they send to Vodafone customers.
As a past customer, I'd like to challenge the implication that it's possible to send any data over Vodafone's network. (My DOCSIS connection with them peaked at fractions of an Mbps for many months during the pandemic, with latency measured in multiple seconds.)
I'm surprised to read an obviously AI written article ("This isn't about efficiency—it's about extraction") from a tech/news site. Does anyone else find this weird? It make me question the editors note about how much background research was actually done.
Also, am I going crazy or do the "comment" and "share" buttons under the header just tick up but don't allow you to actually do anything? This feels like a vibe-coded website but it could be Firefox being weird.
HeyHey. The website uses Ghost right now, and a lit based web components catalogue. Some features are not yet entirely carved out. The commenting system being one of them. Components are ready, but some integration work has to be done. We also want to enable highlighting stuff directly, so people can comment on specific referenced stuff...
The commenting APIs in ghost are a little obscure.
The easiest way to notice it is its excessive use of what Wikipedia calls "negative parallel construction". [1] It's a super common genai tic
> This isn't about efficiency—it's about extraction
> The problem isn't your connection to Vodafone—it's Vodafone's restrictive connections to the rest of the internet.
> Vodafone's exit from public peering isn't an isolated technical decision—it's part of a broader pattern of large telecoms trying to reshape internet economics in their favor
The more obnoxious signs though is the excessive length, loose structure, repetition, and lack of serious editing. Writing ~3000 words used to take quite a bit of effort, so you'd need to be at least a strong enough writer to organize and structure your thoughts to make it that far. Now it's so easy anyone can put out tons of generated content on whatever topic they want.
The big difference here is that you're almost never forced to use Deutsche Telekom for wired internet (there are many DSL resellers, and many of them actually provide routing themselves), but in some buildings, there is literally only Vodafone, with no choice of any alternative service provider on top.
Until I switched, it would only peer with other Tier 1 providers 2000 mi away from my location, even though there is a large IX 5 mi from home co-located with a large regional ISP with several other networks and appliances connected to it.
I filed a complaint but it is impossible to escape the event horizon of the customer service black hole, and customer protection regulation agents fail to appreciate how clownish it is to have 100 ms ping to my university 5 mi away.
So I switched and recommended everyone within earshot to do so as well.
To this day I fail to understand the logic behind not peering locally.
> There's a reason your internet feels like magic. When you click a YouTube video in Berlin, that data doesn't travel some convoluted path through half of Europe to reach you. It flows through something called an "internet exchange point"—a giant room full of routers where hundreds of networks connect directly, swapping traffic efficiently and, crucially, for free.
When you open a Youtube video page, the video is probably loaded from Google's caching servers located in your ISPs network.
> When Deutsche Telekom customers want to watch YouTube, that traffic flows directly from Google's network to Deutsche Telekom's network at a Frankfurt exchange point—maybe four or five router hops, minimal latency, no intermediaries. It's elegant. It's efficient. And it's exactly what Vodafone is abandoning.
The Telekom story mentioned in this article is 100% as bad as they make it out to be, most of the users we support with issues reaching our services are with Telekom Germany. Or in an authoritarian nation that blocks access to western services.
Yeah, the blog post took a bit of a weird turn there + the AI slop headers are also quite off putting. I know this may be conspiracy territory, but it does make me wonder whether this is guerilla marketing to ensure big E meets the sale goals for his $1T payment.
I think the open internet like it existed in the late 90s and early 2000s is never coming back. Luckily tech got cheap enough that I believe we eventually will have continent wide grass roots mesh networks and uncensorable free communication again.
Mesh networks don't really scale to continent size and there really isn't a lot of free/unlicensed spectrum to work with, it's already really crowded. WISPs do a lot with what they have to work with but it's still not going to supplant the internet without some major technical breakthroughs.
Not really, as US ISPs have been repeatedly trying to game the system into becoming landlords for decades. The difference is, ironically, their own self-imposed monopolies: Comcast may be a T1 ISP, but they’re largely a monopoly in the markets they serve. Same goes for Verizon, Spectrum, Cox, TDS, etc. The end result is a sort of “forced cooperation” with each other, though occasionally one will try to extort the others for cash (I seem to recall L3 and Cogent both engaging in this bullshit around the time streaming video got big).
Companies are extractive by nature, and they will always try to find new ways of squeezing blood from a stone absent regulations saying otherwise (and suitable punishments ensuring anyone caught violating them is crippled in the marketplace, if not outright destroyed). This has been going on for decades and will continue absent regulatory intervention. Just look at how the US Electrical grid bills to see how this could end up (higher prices, bullshit fees, redundant billing).
Exactly. Regulations without suitable punishments and investigatory powers are essentially only barriers to new entrants, not deterrents of bad behavior.
Germany has had a weird approach when it comes to the Internet for several decades, but sure, it must somehow be related to recent nuclear power policies.
It doesn’t make sense to conflate a decision made by the government with that of a private company, just because they happen to be in the same country.
Is doing business with Inter.link really structurally different from getting connectivity to an exchange like DE-CIX and doing business there? I know that in theory, you get settlement-free peering at exchanges, but only for those networks that participate.
And who funds Inter.link? Their publicly available balance sheet shows significant, growing debts to a linked company, but it doesn't mention its name.
Thank you for writing this article. It is a rare take on what is happening, and not everyone has the expertise to write it. It seems easy enough to share to consumers who may be affected by this. And it does have some action steps too.
Ironic that I switched to HN from chess.com because Deutsche Telekom‘s peering with Cloudflare (or more its lack of) made the site even more unusable than usual. 5 minutes ago I thought „Maybe it’s time to switch to Vodafone“
The disclaimer at the top of the article is really mind blowing:
> We may have failed in some areas to grasp the issue entirely. The reader is advised that not everything might be correct and you should follow the sources and conduct your own research to get an adequate understanding of the subject at hand.
it sounded to me like an AI generated article covering it's ass, or at least written by someone who did naught but AI generated research as their own 'research'.
A banner by itself is very ignorable, but in this case the website is going to be annoyingly slow, and people are gonna notice that and be thinking a little about why. In that scenario, they're much more likely to notice and pay attention to something like a banner.
I worked with Vodafone years ago to do an integration with ticketing systems. It seems like no one actually worked for Vodafone it was all contractors or contractors of contractors of contractors.
Outsourcing peering to a 3rd party seems like their playbook.
At such organizations, you can usually communicate with any contractor, but you must go through a project manager. These managers, who are often contractors themselves, act as a support center for the other contractors.
I used service of Vodafone Germany once. In paperwork fever I only scanned the contract and saw somewhere 6000 but signed off and moved along with remaining paperwork. I thought "it has to be at least 60Mbit/s, right?". Nope. 6Mbit/s DSL, two years contract, cancellation by letter. Fuck you, Vodafone.
It's very simple. I host my stuff on a network with an open peering policy. If you as an ISP somehow have peering issues with that, then that's a you problem. I will not pay a ransom to some shady middleman that you decide to use because your network admins are too lazy. I will (rightfully) blame you and tell your customers to switch ISPs if they have issues.
Play stupid games, win stupid prices. Just wait until Vodafone Germany customers get slow speeds and an automated warning banner on every other website they visit. "Too big to fail" until it isn't.
This is a very reasonable approach until somehow your competitors service loads twice as fast on the Vodafone Germany network.
As a business, at that point, you're basically extorted to pay the ransom or deal with a loss of revenue. Since the ransom is most likely lower it won't take long for your other competitors to start paying it as well leaving you with an objectively worse product, irrespective of your warning banner (which lefty Linda or Gradma Garry isn't going to understand).
Fucking around with peering is the specialty of German ISPs. Telekom (our biggest provider) is sometimes unusable for YouTube/Netflix/Cloudflare/Steam in big cities because of similar shenanigans.
I was shocked just how slow and poor the mobile networks were in Germany. When I last visited (circa 2014) I literally switched my prepaid sim from T-Mobile to Vodaphone because the experience was so bad - only to have the same bad experience. I had barely usable LTE and connections dropped to EDGE on the train between Hamburg and Berlin. Google Maps barely loaded in the cities let alone the fact I was playing Ingress at the time and it was pretty much unusable
This was surprising to my Canadian sensibilities. Our mobile networks are expensive, but I generally get solid 4G and now 5G coverage between Toronto and Montreal and had full 4G (at the time) coverage on a road trip between Saskatoon and Calgary.
When can we start doing meshnets again? I was excited about it in '05 long before I could afford networking equipment. 20 years later and we've only regressed.
Interesting take, but generally large, incumbent eyeball networks have refrained from open peering at IX's for decades at this point. They maintained presences, but usually just to grab a few specific peers they wanted, not to peer broadly with everyone across the exchange, and the bulk of traffic from large providers into eyeball networks comes across PNIs or on-net CDN nodes, not IX.
If anything, this move to centralized PNIaaS platforms makes interconnecting with the eyeball networks even easier for smaller providers. The portals allow for straightforward visibility on what they want to charge for paid peering, and instant automated EVCs and turnup, shortcutting the long and windy process of negotiating terms and establishing individual XCs in DCs that you agree to peer in.
I recently moved to a Dutch municipality that runs its own non-profit ISP. They installed a symmetric 1 Gbps fiber connection with a static IP at my house for 40 euros per month.
The service is solid, there’s no upselling or throttling, and hosting things from home just works. I bring this up because when we talk about “open”, “fair” and “monopolies” the model of a local, non-profit ISP backed by the municipality could offer a real alternative. It doesn’t directly solve the peering issues, but it shifts the balance of power (and cost) somewhat.
i've wondered for a long time why this isn't a more common solution to these services that are almost inevitably monopolous. power, water, and internet kind of things.
In the US the governments have actively killed them as a favor to the large corporate Internet providers.
While this is great when it works it does raise some interesting challenges, what happens if the ISP loses money, should taxes be used to cover the cost since this is a public service? Is it reasonable for this ISP to undercut commercial offerings? Internet is in a weird grey zone where it's almost a utility but not on the same level as water or sewage.
I'm glad this non-profit ISP exists but on a national level I would prefer (strong) net neutrality laws. Probably not an issue in NL but in less developed countries neutrality isn't guaranteed.
Usually these ISPs are part of or under the municipal utility provider. So if they lose money, it first gets offset by profits from other utilities and eventually yes, the taxpayer will step in, directly or indirectly. No big deal. No one is complaining about subsidies for water or power lines in rural areas, neither should they when it comes to internet. Remember: These ISPs were founded because the market was already failing to provide decent offers or any at all.
In Austria the Internet, like the postal service, is a Universal Service ("Universaldienst"). As such, any completely unserviced citizen can petition the current Finance Minister to decree an ISP of his choice - usually A1, which is the privatized form of our former public office for postal services and telegraphs - to service their area. The costs of facilitating the servicing of this area are covered by all active ISPs of a certain size operating in Austria.
Telecommunications law in Europe is a very interesting thing.
Why make the assumption that municipalities must treat their internet services as a second-rate utility? Many local governments are competently run.
If the internet is out, it's going to be just as visible and probably will yield as many complaints as losing power, sewer, and water.
> If the internet is out, it's going to be just as visible and probably will yield as many complaints as losing power, sewer, and water.
More. Far more visible. Much easier to go without municipal power than without internet.
I don't find that it is usually even possible to have internet without power. How would that work?
After a famously bad wind storm in 2008, my house (along with thousands of others) was without power for about two continuous weeks.
The internet connection, which was FTTN VDSL, never skipped a beat. It was completely solid.
This was accomplished by using batteries and generators.
The ISP was The Phone Company, so their Cold War-era central office had very good backup power.
The VRAD nodes scattered all over town had enough battery backup that (at least in my neighborhood) things stayed up until they brought out generators for those nodes.
And at my house, the VDSL box had its own UPS. And I also had a rather overkill UPS, and a portable generator
We ran the generator intermittently, mostly to charge batteries and chill down refrigerators.
It wasn't an awesome time. It was hot as hell. It was a pain in the ass to keep the generator fueled. We didn't even try to run the desktop PC rigs.
But, yeah: The internet was working fine.
(We charged batteries for neighbors, too. One or two neighbors also dragged over extension cords to run their own fridge. And I opened up the WiFi completely so everyone nearby could use it.
So if you were my neighbour in that 2008 power outage, I'd have just taken care of that internet problem for you. The range at 2.4GHz was amazingly good in that abnormally-quiet RF environment.)
When we loose power here internet usually works just fine. All you need is a generator and you're back up and running. Usually the POP still has power so this works for a long time. Sometimes they are or run out of (backup) power too if its widespread and prolonged. Cell service including LTE is usually still up / up for longer, so again as long as you have a generator, you're good.
Seems like a question of whether solar power with battery backups or satellite internet is easier, no?
One doesn't have to rely on others for power. One can run their own generator, or set up a solar power system, or if you live out in the sticks, run a mini hydro system or use wind power. All of these can provide power to the home.
There's voting with your wallet and voting with your, well, vote.
In some sense a democracy is also a market and can lead to efficient allocation of resources, particularly common resources for common good.
This is why public utilities tend to work so well in practice. People, especially in the US, don't seem to realise that such services are also subject to strong market forces, just a different kind of market.
Voters care a lot about good public services, and they also care a lot about not getting taxed much. This can lead to very efficient outcomes in well functioning democracies, often more efficient than those that come out of private enterprise, when it comes to services that most of the population needs.
If it loses money, do some combination of raising prices and cutting costs. Aim for an average of zero profit/loss over the long term. Undercutting commercial offerings is perfectly fine if it works out that way.
This isn't unique to Vodafone. Google has also been slowly withdrawing from IXes globally in favor of PNIs and "VPPs" (verified peering providers). This only makes it harder for smaller networks to establish presence on the internet and feels pretty anti-competitive.
On the flip side, IXes are becoming harder and less desirable to participate in: port fees are going up, useful networks are withdrawing, low quality network participants are joining and widening blast radius. I'm not sure what the answer to this is, but this has not been a great year for the "open" internet.
Google gave a presentation on this that I think is helpful context for "why": https://nanog.org/events/nanog-94/content/5452/
Direct YouTube link: https://youtu.be/Yg-qV6Fktjw
> low quality network participants are joining
(Genuinely curious because I truly don't know in this context) What is a low quality network participant? One of the "bulletproof" hosts?
I thought Google was _always_ like this. At least going back to 2015 when I left the ISP game, peering with them was notoriously difficult if you didn't have the traffic volumes required. Our network suffered from asynchronous routing to Google and Netflix for years because they refused to allow our routes despite checking all the boxes they require. Customers eventually left because other (larger) ISPs didn't have this issue.
I get why the enshittification of IXPs is occurring. Over the years many small and careless ISPs have caused issues for IXPs (and peers) based on what I've seen on mailing lists. It's hard work managing many hundreds or thousands of peers, let alone the equipment cost with multi-100Gbit ports becoming the norm for larger providers.
Why did your company expect Google to readily accept peering?
If there was such a large difference in volume they would be choosing to intentionally make it more difficult for themselves.
Google publishes a peering policy. It's reasonable to expect that they will peer with you if you hit all the requirements in the policy.
Afaik, their requirements have never been judgement based: just bandwidth minimums, port types and locations. I would expect that they prioritize new connections in some way, so if you barely hit the criteria and are somewhere well served by transit, you'll be low priority, and the requirements might change before your connection gets setup and if so, you might not get connected because you don't meet the new requirements, but otherwise, seems like if you meet the requirements, send in the application, and have some patience, the peering connection should turn up eventually.
It's not like they have a mostly balanced flows requirement like Tier 1 ISPs usually do. Also, even in their current peering policy, they don't require presence in multiple metros; just substantial traffic (10gbps), fast ports (100G), two pops in the same metro.
> IXes are becoming harder and less desirable to participate in
Could this be due to the rise of services like Equinix Fabric and Inter.link? Google doesn't need to peer directly with most anymore because there is always a middleman somewhere who can handle it, and for many businesses the convenience of a point and click web gui outways whatever it costs?
What sort of verification are they doing? Is this trend being pushed by the lack of proper security on BGP?
Bell Canada also has had a long-standing policy of refusing to peer with internet exchanges. They'll only truly peer with other direct backbone providers and a handful of one-off peer with other large networks (google, cloud flare, etc), but their historical position as Canada's base backbone (not so much anymore, but it was definitely a thing pre-2005) has meant their policy is most people should pay them to peer. I'm not sure if it's still the case, but IIRC for awhile they also refused to peer with any other domestic backbone providers.
The result has been some funny routes sometimes. I live in Toronto and have seen trace routes bounce over to Chicago to connect to stuff colocated here in Toronto.
It's frustrating as their fibre is my only real high speed option; also their lack of IPv6 on anything but their mobile network is annoying.
it’s also insane because probably no Canadian wants their traffic going through the US unnecessarily, since all the 51st state takeover crap.
FWIW, this has being going on for decades. Bell literally doesn't care if there's a chance they can suck up some peering fees.
There have been periodic times where it became an acute problem, like early in the YouTube and Netflix years there was a lot of congestion in their upstream peers and they held out hoping those orgs would pay for the peering. They were also over provisioned in early DSL days where their upstreams became saturated and there were few alternative paths.
Maybe that's a way to give them bad PR and convince then to change policies? Unfortunately most people probably don't understand this well enough and they have a pretty well oiled PR machine with all their control over tv and radio stations, etc
That's insane given that Front St. 151 is probably the best spot on North America when it comes to connectivity.
> also their lack of IPv6 on anything but their mobile network is annoying.
This gives me even less confidence after BCE took over ZiplyFiber, US PNW provider. There's a long running joke about IPv6 just one more lab test away from deployment.
It's annoying how they're the only big ISP offering fiber everywhere when they're also the ones that don't support ipv6 and have the shitty peering policy. I've heard you can use another isp though (teksavvy maybe?) that uses Bell's fibre and supports ipv6
The solution - as always - is regulation. ISPs typically already have very generous business models with widespread monopolies on customers, overwhelming barriers to entry for new companies, and a lack of rate controls allowing then to price arbitrarily - all of which supports immensely profitable businesses without the need for additional extraction of capital from other parties. Regulators and consumers alike should be screaming in rage at the idea that their ISPs are now multi-dipping for revenue, but we’ve done a piss-poor job of explaining how this works to the common man and thus can’t count on them to support the Open Internet as we’d like to see it.
That being said, the threat to the open internet is also more than just ISPs being gigantic assholes: it’s centralization in general. A majority of web traffic passes into or through one of three main cloud compute providers; Cloudflare has such an outsized impact that regional IP blocks can disrupt global traffic; and ISPs have been permitted to consolidate through mergers and acquisitions into expansive monopolies. The internet is fiercely centralized and largely closed already, which is why these ploys by shitty ISPs are likely to work absent Government intervention.
You want to protect the open internet? Regulate the shit out of its major players again. Force them to keep it open, especially when it hinders expanding profit margins.
The current trend in govt regulation is actually going in the opposite direction, with telecom lobbies in Europe pushing for "fair share" (pretty much an implementation through law of what Deutsche Telekom and Vodafone Germany are doing right now) through the Digital Networks Act.
South Korea pioneered "fair share" govt regulations in 2016 (which caused Twitch to exit the market in 2024 due the exorbitant "fair share" fees).
Because western governments (and those whose governments were modeled from western regimes - like post-war South Korea and Japan) have become victim to regulatory capture and corruption. It’s why the FCC has repeatedly killed, blocked, or reversed reforms like net neutrality or “nutrition labels” on ISPs, and why South Korea gave in to “fair share” regulations that deterred further investment. Tech money is hugely influential, and the industry is almost exclusively made up of rent-seeking slumlords at this point, particularly at the top (Oracle, Microsoft, Google, Adobe, Apple, etc). It’s why DMCA reform is blocked, why pirates get jail time while AI grifts get a hand-wave, and why right-to-repair or data privacy remains a fractured and piecemeal reform instead of a national agenda item.
The problem isn’t regulation, but regulatory capture ensuring companies get the regulations they desire and benefit from.
> The problem isn’t regulation, but regulatory capture ensuring companies get the regulations they desire and benefit from.
Aka regulation..... Nearly all regulation is for regulatory capture and if you think of something that isn't it probably just outlived who it was designed to capture for.
Unfortunately you're going to get regulatory capture and extortion when the "bad guys" are local but the "good guys" are foreign.
Regulation isn't good enough. The government needs to make their own competing ISP.
Hell, at least in the US, there's precedent for this: government builds and maintain all the roads; they run most transit and intercity rail operations; and they run physical mail delivery. At one point they even owned most of the railroads[0]. Communications and travel infrastructure are things government is moderately good at.
For some reason, we just decided not to have a government-sponsored telecom company, even when Ma Bell made it patently obvious that having all the country's telecom infrastructure be privately owned by one company was a bad idea. It's obvious that a government-run ISP is about as crucial to life in 2025 as a government-run postal carrier was in the early 1800s.
[0] In the 1970s, all of America's railroads went bankrupt. First, they discharged their passenger rail mandates into Amtrak, then they went bankrupt anyway, and then they got nationalized.
As I’ve stated in other comments, the reason western governments don’t do this more often boils down largely to regulatory capture. Every single time there’s been a large mobilization of efforts to regulate some aspect of tech - municipal broadband expansion, cable box standardization and openness, right to repair, DMCA reform, privacy reforms, mandatory binding arbitration clauses, EULA’s, provider monopolies, etc - tech money floods into regulators and political races to counter the will of the mobilizers and their supporters. Then those same ghouls repeat mantras like “disrupt” and “deregulate” to convince people that actually it’s a good thing you only have three cellular networks, one cable provider, one telephone provider, two operating systems, and four media conglomerates to choose from. At one point these slimeballs claimed anyone who used anything else (like Linux, or GrapheneOS, or FOSS) was obviously a criminal who wanted something for nothing, such was their fear of an open ecosystem.
Regulations get a bad rap because for decades the only ones to really get passed have only entrenched existing players and (rent-seeking) business models while blocking new entrants or competitors. I’m 100% in agreement with you that every single state and country should have an internet network that’s open access and governed solely by that country’s constitutional law - a sort of digital state, if you will, with which they can court business and interest groups alike to represent their interests globally. Instead, we’re presently stuck with a “whoever donates the most money to politicians wins” model, and that means the open internet exists in spite of the interests of Capital, not because of their good graces.
What you are describing sounds mostly like a US-problem, not sure it's a western gov problem in general?
In my city, the municipality owns much of the fibre. The country I live in owns a bank where you can get a mortgage pretty cheap. The good parts of GDPR or CRA are very good and was not disrupted by large corporations?
> Hell, at least in the US, there's precedent for this: government builds and maintain all the roads; they run most transit and intercity rail operations; and they run physical mail delivery.
Correct me if I'm wrong but it seems to me the US government is doing a terrible job at all of these.
Awesome. The only internet connection we get IN THE MIDDLE OF BERLIN is Vodafone Cable. Deutsche Telekom wants to build fiber here, but our landlord refuses to open the door to the cellar because he wants to kick everybody out and raise the rent for new tenants.
What a time to be alive.
> our landlord refuses to open the door to the cellar
The simple solution would be to make this illegal, i.e. require landlords to allow at least two competing wired ISPs to connect each household.
No need to make them pay for it; I suspect it would be more than enough to end their very lucrative arrangement of somehow rewarding exclusivity. (I don't have any evidence that landlords are getting paid for it by Vodafone directly, but I highly doubt that there's any above board reason for the status quo.)
The issue here seems to be the landlord.
Yes. It's the so called Berlin issue. If you're lucky enough to find an apartment, you have a good chance of getting a slumlord who does everything to make your life miserable.
I'd buy my own place, if there would be anything available. Probably need to move to another city or country.
How one can still buy into Berlin's hype in 2025?! Fuck their rental market, their landlords, three months deposit for completely empty apartment without even lightbulbs, copyright trolls, rundfunkbeitrag, rude customer service, schufa. Turn around and walk away.
You forgot the government who spends billions to 3km of motorway a few people uses. To an airport that is one of the worst in EU and almost a decade late. And of course now cut from arts because of all this.
And every single construction project takes forever. And costs a fortune. And it is impossible to build housing fast enough.
The reason to be in Berlin has always been its great art scene. Now they are actively destroying it. What's left is a few Rossmanns and an Edeka.
Part of the issue is that the landlord gets any say whatsoever about a public utility.
Deutsche Telekom is just as bad as Vodafone. IIRC the government stepped in and said they had to offer peering to everyone, so now they offer peering to everyone in some random hamlet in the middle of nowhere for 5000€ per month per gigabit, while peering at other locations and prices is still at their sole discretion.
Is there anything that Vodafone customers can do legally to punish Vodafone or not delivering on their broadband contracts?
If you're paying for a 1Gbps connection and Netflix is only able to stream to you at 0.93 Mbps because Vodafone or Inter.link are choking off the supply, surely that's breach of contract on Vodafone's part?
I'm sure Cory Doctorow has a word for what's happening here.
You can file a complaint with the "Federal ministry for digital transformation" (formed this year). It does actually work, but it's a lengthy process.
I did force my cell phone carrier to grant me proper 4G speeds last year, after spending many hours with their help line and ultimately complaining to the (then) ministry of transportation and digital infrastructure.
Can you elaborate on what was wrong with your cellular connection?
My plan advertises "up to 50 Mbit/s" on a 4G connection. I was getting less than 1 Mbit/s a lot of the time. Websites and videos would not load properly.
I downloaded the app of the german ministry that allows you to take speed tests and file a complaint. After multiple weeks of measuring connection speeds on the cellular network, I was able to file a complaint.
What did the provider do? Did they put your IMEI onto some list of other customers that complained, where all of you get better network prioritization?
I'm genuinely curious.
"If you're paying for a 1Gbps connection"
That's why you are paying for a "up to" 1Gbps connection. (I think it was already a struggle that they had to put the "up to" in the big advertisement)
Surely there's a reasonable expectation that Netflix would work at decent speeds, especially given that Netflix's infrastructure, nor the network load as a whole are to blame, but rather the specific ISP bureaucracy? Getting 1/1000 the listed speed does not strike me as something even a 75 year old computer neophyte of a judge would take kindly too, unless it were for very good reasons.
I don't think there really is much that can be done. Even under ideal conditions, an ISP could only possibly guarantee the advertised link speed between you and their routers, not between you and any particular node on the Internet. Is it possible an ISP might be doing things that harm the QoS? Yeah, sure. But the angle to approach that problem is not by complaining about instances of limited bandwidth.
But the true link speed's not even what's being asked for. 4K Netflix never goes above 20 Mbps as far as I know, so getting just 1/50 the advertised speed to one of the most well-known internet services in existence, hardly seems like a big ask, especially when the only reason that it can't reach that speed or higher is because of the ISP, given that swapping to one that aren't being knobheads about it fixes the problem. It should be the responsibility of the ISP to keep links to other parts of the internet as open as possible. If real-world constraints prevent the speed from being all that high, because it's a shitty server in Australia, then that's understandable. This however, isn't that.
All I'm getting from this is that it's a good idea to label ISPs utilities and bring the hammer down if they're being knobheads about it.
It is mostly the middle, and not either of the endpoints, that is the real problem. You have a 1Gbps link, the Netflix DC you're reaching probably has multiple links with aggregated bandwidth measured in Tbps, but at some point in between the two there's a 10Gbps link being shared between 5000 subscribers at peak times and now the bottleneck is 2Mbps per subscriber. This link may or may not be under your ISP's (or Netflix's ISP's) control, and it may or may not be the only relevant bottleneck.
The solution that was developed in the Netflix-Comcast fight over a decade ago is content distribution. Instead of trying to build out extra capacity in every possible link, you shorten the path and thus reduce the number of contended links involved in each interaction. This scales much better, but it has two major problems: the first is rightsholders and their obnoxious anti-piracy restrictions, and the second is good old jurisdictional friction and economic misalignment. Somebody has to own the physical servers in all the myriad locations that keep the content closer to the consumer. If the ISP owns them, then they naturally want to exploit them. If Netflix owns them, they naturally don't want to serve their competitors. If a third party owns them, you address those two problems (potentially) but add new ones around liability, non-disclosure, competitiveness, etc.
If regulation is going to be useful here, it needs to focus on opening up opportunities to serve the unsexy middle of the infrastructure puzzle and not just the most visible parts that consumers/voters usually interact with. Also, "Netflix" needs to be understood as just a stand-in for any high-bandwidth Internet service, as the landscape is constantly changing.
at some point in between the two there's a 10Gbps link being shared between 5000 subscribers at peak times and now the bottleneck is 2Mbps per subscriber. This link may or may not be under your ISP's (or Netflix's ISP's) control, and it may or may not be the only relevant bottleneck.
No, that link is absolutely under Vodafone's control. They're deliberately not upgrading it so that they can extort money from Netflix.
The solution ... is content distribution.
CDNs have been worldwide, including Germany, for a long time. That's not the problem here.
There are two issues here.
If the CDN is so poorly interconnected with Vodafone that there's one bottlenecked link, then it's not really accomplishing its job, at least as far as "inside of Germany" is concerned. It might have reduced pressure on another bottleneck, like links between the US and the EU, but it still needs to spread out more. If Vodafone is blocking that, then pressure should be applied to force them to open up more connections. I'm assuming this CDN serves more than just Netflix, mind you.
Secondly, the question of responsibility cannot be answered the same way today that it was answered in the Internet of universities. Netflix and Vodafone are not peers. The bandwidth ratio between them is incredibly lopsided. This will never change, there is no foreseeable scenario under which Vodafone has a reason to send anywhere near the same amount of data to Netflix as it gets back. This asymmetrical relationship inherently implies a different kind of business arrangement than traditional peering.
What Vodafone (any ISP) provides to Netflix (any content provider) is access to consumers. This is a service, and services are not free. The natural monopoly ISPs enjoy implies some degree of regulatory restraint must be applied on them, but it does not mean they bear all the costs of all the infrastructure either.
However, my bigger point is that this cannot constantly be reduced to these two-party analyses. Netflix is waning, others are rising, this problem needs to be solved in a scalable way.
Nah, you're just an apologist for rent-seeking ISPs and you're trying to cloud the argument with unnecessary details.
"Comments should get more thoughtful and substantive, not less, as a topic gets more divisive."
https://news.ycombinator.com/newsguidelines.html
In the US the FCC named and shamed broadband ISPs for their low speeds and "magically" those speeds increased over the following years. Overcome with greed, some ISPs eventually found ways to cheat on the benchmarks though. https://www.fcc.gov/general/measuring-broadband-america
The question was whether a customer could do something.
I suppose in aggregate the customers could use their elected government to fix the problem. In theory.
I think it's actually a quite complicated question and it only works because people are playing somewhat nice with each other. Like imagine if Netflix refused to peer with one particular ISP unless they paid an extortionate amount of money. Should the ISP be legally required to pay any price they name? I don't think that would be fair.
One solution could be to have geographically distributed test points. Any connection to be able to claim a certain speed has to be able to get that speed to those test points. And the test points are legally required to connect to anyone that can bring fiber to their doorstep. If someone plays hardball with peering there will then always be the backup option of routing traffic through one of the test points.
Idk, just throwing out ideas here.
At least here in the Czech Republic, ISPs have to also list a "guaranteed speed", and it can't be less than some fraction of the advertised maximum. I don't know what part of the Internet that speed is supposed to be measured against, though.
There's always an implicit reasonableness requirement on these things. "Up to" isn't a wildcard that lets you do anything you want. It will protect you if you struggle to maintain top speeds during peak hours, or if a technical fault cuts speeds in half for a few days. But if you provide 1/1000th of what you claim to offer, "up to" won't stop a judge from observing that you're not really providing what you said you would.
There's definitely not anything in the contract that promises performance to a 3rd party, especially not in a residential contract. The legal options are switch to a different ISP and/or start a new one. Not always easy or practical, but there you go.
Who is to say where the performance problem is? Certainly not your contract.
Maybe if the last mile is cronically congested, or between the local aggregation switch and their regional exchange points, you might have a legal case. But if the issue is insufficient connectivity between their network and other networks, I would be very surprised if the contract terms covered that at all.
There's a bunch of networks throughout the world where their policies mean you can get more economically acheive better connectivity to their customers by hosting outside the geographic boundaries of the network rather than inside it. Doesn't make sense from a theoretical point of view, but when German ISPs won't interconnect within Germany, serve their customers from Poland or France and the connectivity picture may change significantly. Worst case, serve them from the US (but the latency may be too high)
Are there competing options, or are they a monopoly?
Depends on the region. Often there are smaller regional companies providing fiber internet. Prices for these fiber connections a still somewhat higher than the cheapest vodafone tier, but you also get better service for your money.
Afaik almost a monopoly: there is Deutsche Telekom which does the same thing and Vodafone. I think apart from some local providers almost everybody else is just a reseller of one of the two.
There are resellers that do not just rebrand a whitebox product, but have their own IP addresses, network and peering polices. Their customers are not necessarily impacted by the IP peering policies of the company that owns the access network.
This is a bit messy but if there are completing options at a given location install multiple ISPs and run them concurrently and log the details—download speeds, etc.
There's nothing as good as hard verifiable data—even if regulators play hardball and favor ISPs then you've the evidence to whip up political action (claim biased decisions, etc.).
Sounds like their largest competitor (DT) is already doing this.
No monopoly. Only for cable internet, which may be a possible argument. For landline internet (DSL), there's plenty of alternatives.
Unfortunately, having a landline capable of DSL is no longer the default in Germany.
Some apartment buildings exlusively offer DOCSIS via a single provider (as there's never been any unbundling of the DOCSIS "local loop"; presumably under the assumption that a landline will always be available anyway?).
If that one provider is oversubscribed, you're pretty much out of luck.
High speed internet is a market not just internet access. Email might not care that your on a DSL connection but a streamer can’t generally use DSL as a substitute.
They mean VDSL; that's 100~200 Mbit down and 10~24~50 Mbit up.
VDSL2 can hit those speeds in optimal conditions, but at the end of last year ~14% of Germans have internet under 10Mbps and ~17% where 10-30Mbps.
Yes and no. There are other providers in Germany. However, with the EU's neoliberal privatization policy the governments privatized many existing infrastructure. Vodafone bought the previous government company that owned all of the the cable TV infrastructure of Germany. So they are a monopoly of a particular type of internet connection. Depending on the place the alternatives could be too slow since Germany also has an aging population that do not {care about, demand} higher internet speeds and didn't upgrade its copper infrastructure due to corruption.
Some new apartments also simply lack phone lines. No idea how that's legal (since there is no competition at all on DOCSIS, unlike on DSL, in Germany), but it's a thing these days.
In Germany in specific building there is only one provider available in your telephone socket, and one in your cable socket in your apartment. Frequently there is no cable socket.
That's not relevant. Over a Deutsche Telekom phone line you can choose an ISP. The ISP sometimes has a layer two connection to you and therefore has their own infrastructure or they have a layer 3 connection in which case you suffer from the Telekom policies.
Layer 2 = their infrastructure connects you to the internet
Layer 3 = theyre literally just a reseller, DTAG is providing your internet connection, the ISP just billing etc.
That's too many words for simply saying "The fastest available DSL is 16Mbit/s and the customer service will be rude and useless".
> Instead, all traffic will flow through a single company called Inter.link, which charges content providers based on how much data they send to Vodafone customers.
As a past customer, I'd like to challenge the implication that it's possible to send any data over Vodafone's network. (My DOCSIS connection with them peaked at fractions of an Mbps for many months during the pandemic, with latency measured in multiple seconds.)
I'm surprised to read an obviously AI written article ("This isn't about efficiency—it's about extraction") from a tech/news site. Does anyone else find this weird? It make me question the editors note about how much background research was actually done.
Also, am I going crazy or do the "comment" and "share" buttons under the header just tick up but don't allow you to actually do anything? This feels like a vibe-coded website but it could be Firefox being weird.
HeyHey. The website uses Ghost right now, and a lit based web components catalogue. Some features are not yet entirely carved out. The commenting system being one of them. Components are ready, but some integration work has to be done. We also want to enable highlighting stuff directly, so people can comment on specific referenced stuff...
The commenting APIs in ghost are a little obscure.
Hate it, but doesn't surprise me one bit.
This article is 2700 words of repetitive slop. It seems that people are adapting to this new world.
It didn't seem AI to me.
The easiest way to notice it is its excessive use of what Wikipedia calls "negative parallel construction". [1] It's a super common genai tic
> This isn't about efficiency—it's about extraction
> The problem isn't your connection to Vodafone—it's Vodafone's restrictive connections to the rest of the internet.
> Vodafone's exit from public peering isn't an isolated technical decision—it's part of a broader pattern of large telecoms trying to reshape internet economics in their favor
The more obnoxious signs though is the excessive length, loose structure, repetition, and lack of serious editing. Writing ~3000 words used to take quite a bit of effort, so you'd need to be at least a strong enough writer to organize and structure your thoughts to make it that far. Now it's so easy anyone can put out tons of generated content on whatever topic they want.
[1] https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
It looks like the website was founded not so long ago, and I guess AI-generated slop is the only way to make a profitable web media nowadays.
Seems pretty much a private equivalent of "fair share" govt regulation currently being pushed by ISPs in Europe through lobbying with the Digital Networks Act (https://www.namex.it/toward-the-digital-networks-act-the-fut..., https://stopdna.eu/).
South Korea pioneered fair share govt regulations in 2016 (which caused Twitch to exit the market in 2024 due the exorbitant "fair share" fees).
It's important to keep in mind that Deutsche Telekom is basically doing the same, and has been… forever?
I disagree with this move, but it is not without precedent.
They mention it extensively in the article.
I suppose OP is hinting at the fact that Germans are probably already used to having one of the shittiest internet services in the Western world.
They certainly do; however, we all know the game: Headlines only. Reading TFA? Meh
The big difference here is that you're almost never forced to use Deutsche Telekom for wired internet (there are many DSL resellers, and many of them actually provide routing themselves), but in some buildings, there is literally only Vodafone, with no choice of any alternative service provider on top.
Still Telekom has a big market share there.
Telefonica does this.
Until I switched, it would only peer with other Tier 1 providers 2000 mi away from my location, even though there is a large IX 5 mi from home co-located with a large regional ISP with several other networks and appliances connected to it.
I filed a complaint but it is impossible to escape the event horizon of the customer service black hole, and customer protection regulation agents fail to appreciate how clownish it is to have 100 ms ping to my university 5 mi away.
So I switched and recommended everyone within earshot to do so as well.
To this day I fail to understand the logic behind not peering locally.
Even the first paragraph is wrong:
> There's a reason your internet feels like magic. When you click a YouTube video in Berlin, that data doesn't travel some convoluted path through half of Europe to reach you. It flows through something called an "internet exchange point"—a giant room full of routers where hundreds of networks connect directly, swapping traffic efficiently and, crucially, for free.
When you open a Youtube video page, the video is probably loaded from Google's caching servers located in your ISPs network.
> When Deutsche Telekom customers want to watch YouTube, that traffic flows directly from Google's network to Deutsche Telekom's network at a Frankfurt exchange point—maybe four or five router hops, minimal latency, no intermediaries. It's elegant. It's efficient. And it's exactly what Vodafone is abandoning.
Tab closed
Why?
AI writing cadence I presume.
Edit: https://news.ycombinator.com/item?id=45850683
Thanks. I thought it was something factually wrong.
It is also likely factually wrong as most major ISPs use GGC for popular traffic
Refusing to peer directly is not “killing the free internet”. This level of hyperbole doesn’t help your argument.
The Telekom story mentioned in this article is 100% as bad as they make it out to be, most of the users we support with issues reaching our services are with Telekom Germany. Or in an authoritarian nation that blocks access to western services.
Can confirm, recently moved to o2, it was insufferable.
For anyone wondering: netzbremse.de/
EN https://netzbremse.de/en/
Would you recommend o2?
Using Starlink as your Internet provider does not protect you in any way when it comes to changes in pricing, speed or connectivity.
Yeah, the blog post took a bit of a weird turn there + the AI slop headers are also quite off putting. I know this may be conspiracy territory, but it does make me wonder whether this is guerilla marketing to ensure big E meets the sale goals for his $1T payment.
Is anyone in Germany able to recommend ISPs which aren't Vodafone or Telekom?
I think the open internet like it existed in the late 90s and early 2000s is never coming back. Luckily tech got cheap enough that I believe we eventually will have continent wide grass roots mesh networks and uncensorable free communication again.
Mesh networks don't really scale to continent size and there really isn't a lot of free/unlicensed spectrum to work with, it's already really crowded. WISPs do a lot with what they have to work with but it's still not going to supplant the internet without some major technical breakthroughs.
Rare comparative W for American ISPs?
Not really, as US ISPs have been repeatedly trying to game the system into becoming landlords for decades. The difference is, ironically, their own self-imposed monopolies: Comcast may be a T1 ISP, but they’re largely a monopoly in the markets they serve. Same goes for Verizon, Spectrum, Cox, TDS, etc. The end result is a sort of “forced cooperation” with each other, though occasionally one will try to extort the others for cash (I seem to recall L3 and Cogent both engaging in this bullshit around the time streaming video got big).
Companies are extractive by nature, and they will always try to find new ways of squeezing blood from a stone absent regulations saying otherwise (and suitable punishments ensuring anyone caught violating them is crippled in the marketplace, if not outright destroyed). This has been going on for decades and will continue absent regulatory intervention. Just look at how the US Electrical grid bills to see how this could end up (higher prices, bullshit fees, redundant billing).
California's net neutrality law bans these kinds of paid interconnections, but they likely exist as all these deals are wrapped in 15 layers of NDAs
Exactly. Regulations without suitable punishments and investigatory powers are essentially only barriers to new entrants, not deterrents of bad behavior.
I think Comcast charges for peering as well (but not through an intermediary).
Germany is on a weird path; first the nuclear power withdrawal and now this. It's all a bit topsy turvy.
Germany has had a weird approach when it comes to the Internet for several decades, but sure, it must somehow be related to recent nuclear power policies.
It doesn’t make sense to conflate a decision made by the government with that of a private company, just because they happen to be in the same country.
Is doing business with Inter.link really structurally different from getting connectivity to an exchange like DE-CIX and doing business there? I know that in theory, you get settlement-free peering at exchanges, but only for those networks that participate.
And who funds Inter.link? Their publicly available balance sheet shows significant, growing debts to a linked company, but it doesn't mention its name.
Thank you for writing this article. It is a rare take on what is happening, and not everyone has the expertise to write it. It seems easy enough to share to consumers who may be affected by this. And it does have some action steps too.
Inter.link is openly peering on IX's tho. So I'm unclear why the angst. https://inter.link/knowledgebase/peering-policy/
Ironic that I switched to HN from chess.com because Deutsche Telekom‘s peering with Cloudflare (or more its lack of) made the site even more unusable than usual. 5 minutes ago I thought „Maybe it’s time to switch to Vodafone“
Do not switch to Vodafone. Even without their peering it's still a horrible ISP
I hate this line of thinking.. Netflix isn't just sending data to random users, it's data Vodafone users request and want to receive.
The disclaimer at the top of the article is really mind blowing:
> We may have failed in some areas to grasp the issue entirely. The reader is advised that not everything might be correct and you should follow the sources and conduct your own research to get an adequate understanding of the subject at hand.
it sounded to me like an AI generated article covering it's ass, or at least written by someone who did naught but AI generated research as their own 'research'.
The play by major content providers is "not to pay" and "block inter.link"
Sure, you lose Vodafone germany. Then you explain clearly why to every major media.
This coukd be stopped fairly quickly.
Blocking seems overkill. Just put up a banner explaining why the service is slow, warning customers about their ISPs.
Because a banner is ignorable. A block will actually spur action.
A banner by itself is very ignorable, but in this case the website is going to be annoyingly slow, and people are gonna notice that and be thinking a little about why. In that scenario, they're much more likely to notice and pay attention to something like a banner.
"The play by major content providers" like Google? Where in order to become a "Verified Peering Provider" you are not permitted to use a IX?!
https://peering.google.com/#/options/verified-peering-provid...
Doesn’t seem to work in regards to Deutsche Telekom so far.
I worked with Vodafone years ago to do an integration with ticketing systems. It seems like no one actually worked for Vodafone it was all contractors or contractors of contractors of contractors.
Outsourcing peering to a 3rd party seems like their playbook.
At such organizations, you can usually communicate with any contractor, but you must go through a project manager. These managers, who are often contractors themselves, act as a support center for the other contractors.
I used service of Vodafone Germany once. In paperwork fever I only scanned the contract and saw somewhere 6000 but signed off and moved along with remaining paperwork. I thought "it has to be at least 60Mbit/s, right?". Nope. 6Mbit/s DSL, two years contract, cancellation by letter. Fuck you, Vodafone.
It's very simple. I host my stuff on a network with an open peering policy. If you as an ISP somehow have peering issues with that, then that's a you problem. I will not pay a ransom to some shady middleman that you decide to use because your network admins are too lazy. I will (rightfully) blame you and tell your customers to switch ISPs if they have issues.
Play stupid games, win stupid prices. Just wait until Vodafone Germany customers get slow speeds and an automated warning banner on every other website they visit. "Too big to fail" until it isn't.
This is a very reasonable approach until somehow your competitors service loads twice as fast on the Vodafone Germany network.
As a business, at that point, you're basically extorted to pay the ransom or deal with a loss of revenue. Since the ransom is most likely lower it won't take long for your other competitors to start paying it as well leaving you with an objectively worse product, irrespective of your warning banner (which lefty Linda or Gradma Garry isn't going to understand).
Fucking around with peering is the specialty of German ISPs. Telekom (our biggest provider) is sometimes unusable for YouTube/Netflix/Cloudflare/Steam in big cities because of similar shenanigans.
I was shocked just how slow and poor the mobile networks were in Germany. When I last visited (circa 2014) I literally switched my prepaid sim from T-Mobile to Vodaphone because the experience was so bad - only to have the same bad experience. I had barely usable LTE and connections dropped to EDGE on the train between Hamburg and Berlin. Google Maps barely loaded in the cities let alone the fact I was playing Ingress at the time and it was pretty much unusable
This was surprising to my Canadian sensibilities. Our mobile networks are expensive, but I generally get solid 4G and now 5G coverage between Toronto and Montreal and had full 4G (at the time) coverage on a road trip between Saskatoon and Calgary.
When can we start doing meshnets again? I was excited about it in '05 long before I could afford networking equipment. 20 years later and we've only regressed.
Not to say that Mr. Musk seems popular in Germany, nor that orbital dynamics are friendly to high user densities, but ...
https://inter.link "Trusted By: SpaceX"
Wow, thanks I hate it.
New selling point for all the VPN sponsored segments... "if you're on Vodafone Germany, make your connection speeds faster with YetAnotherVPN!"
I'm not sure that will not speed anything up, since you face the same peering issue with connecting to XYZ vpn's server
The VPN will be fast if they pay the extortion. It's basically a paid fast lane.
A VPN won't make your route shorter
If the shortest route is congested, a longer route can be advantageous if it avoids congested hops.
Interesting take, but generally large, incumbent eyeball networks have refrained from open peering at IX's for decades at this point. They maintained presences, but usually just to grab a few specific peers they wanted, not to peer broadly with everyone across the exchange, and the bulk of traffic from large providers into eyeball networks comes across PNIs or on-net CDN nodes, not IX.
If anything, this move to centralized PNIaaS platforms makes interconnecting with the eyeball networks even easier for smaller providers. The portals allow for straightforward visibility on what they want to charge for paid peering, and instant automated EVCs and turnup, shortcutting the long and windy process of negotiating terms and establishing individual XCs in DCs that you agree to peer in.