Trusting a server to store an application's state is a different thing from trusting it to author changes or to read the data.
Servers should become dumber, and clients should become smarter.
When I use an app, I want the app to load E2E encrypted state from storage (possibly on another machine, possibly not owned by me) make whatever changes and produce new encrypted data to send back to the server.
The server should just be trusted for durability, and to prevent unauthorized access, but not to tell the truth about doing either of those things.
Blobcache provides an API to facilitate transactions on E2EE state between a dumb storage server and any smart client.
Blobcache can be installed on old hardware along with a VPN like Tailscale and then loaded up with data from other devices.
Configuration is like SSH, drop a key in a configuration file to grant access.
It removes most of the friction associated with consuming and producing storage as a resource.
I'm using it to build E2EE version control like Git, but for your whole home directory.
I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio).
Looking through the docs on Peergos, it looks like it's built on top of IPFS.
I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here.
Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it.
A significant difference is that Blobcache Volumes expose a transaction API with serializable isolation semantics.
IPFS provides distributed, available-but-inconsistent, cryptographically signed cells.
IPFS chooses availability, and Blobcache chooses consistency.
A Blobcache Volume corresponds to a specific entity maintained and controlled by a specific Node.
An IPFS name exists as a distributed entity on the network.
Most applications need some sort of consistent transactional cell (even if they don't realize it), but in order to be useful, inconsistent-but-available cells have to be used carefully in an application specific way.
I blame this required application-specific care for the lack of adoption of CRDTs.
There's a long tail of other differences too.
IPFS was pretty badly behaved the last time I used it, trying to configure my router, and creating lots of connections to other nodes.
Blobcache is more like a web browser; it creates transient connections in immediate response to user actions.
That whole ecosystem is filled with complicated abstractions. Just as an example, the Multihash format is pervasive.
It amounts to a tag for the algorithm used to create a hash, and then the hash output.
I'd rather not have that indirection.
All the hashes in Blobcache are 256 bits, and you set the algorithm per Volume.
In Go that means the hashes can just be `[32]byte` instead of a slice and a tag and a table of algorithms.
I haven't used IPFS in a while, but I became pretty familiar with it awhile ago. Had I been able to build any of the stuff I was interested in on top of it, I probably wouldn't have written Blobcache.
The good news is Peergos also has serializable transactional modifications. This comes from us storing signed roots in a db on your home server (not ipns). We also have our own minimal ipfs implementation that uses 1000x fewer resources than kubo, aka go-ipfs.
Both of these proposals (as far as I've read them, YMMV) fail the evolutionary test. At the scale we're talking about, ideas must proceed as evolution does: not with a far-away goal in mind, but with incremental changes, each of which individually must be an improvement over the status quo.
We are at (near) a significant local maximum, and (again, as far as I've read, which is not all of it for sure) the people pitching this form of information control have given no set of steps from here to there without significant cost/effort.
Of course they don't have to have the whole path in mind. By definition they just need the first step or two. But they must be steps up.
You don't get wings by wanting to fly; first you need feathers to keep warm (I am not an evolutionary biologist, I don't know if that's a valid theory).
> each of which individually must be an improvement over the status quo
I agree. And looking at the average web user specifically, is "owning your own data" enough of a UX improvement? Maybe paired with less ads and products that optimize for the end-user rather than advertisers? I think... maybe. I hope so. It's going to take a lot of work done for little money, which is concerning, but I'm optimistic.
99.9% of BlueSky users use only Bluesky services. But BlueSky has a Personal Data Service for each. That means:
Those users have credible exit to take their data off BlueSky's hosting to someplace else (and as of a week or two ago to move back to BlueSky if they want).
Those users can put whatever kind of data they want in their PDS. They can host their git data via https://tangled.org . They can store their music listening scrobbles with https://teal.fm . They can blog on https://leaflet.pub .
And there's been rapidly advancing host it yourself options. Plenty of folk individually or collectively host PDS. There are alternate relays that collect &n syndicate out everyone's PDS data as that changes. Hosting the aggregation layer is significantly harder especially if you are trying to fully connect the network but there are a couple & progress is good.
it feels like a huge improvement over the status quo, and there's extremely visible developer energy building forward & rolling with the concepts. The breakdown on architecture allows for wins and work in various areas. The base seems solid, the core seems coherent & well built, built to scale not as one big thing but coherent layers. I think it's doing what you are asking for, and the signs of advancement & uptake warm my heart to see.
Among the first page and 2nd page (top 60) there is always atleast 1 post about how we're gonnna "take back the web" or make it back into some form of our 90s millenial nostalgia memories, self hosting, federated this or that, etc etc.
Meanwhile - Nothing changes, everything generally gets worse and younger generations come into the world with no memories of the 90s internet or the world before mobile devices or surveillence everywhere.
Applying for a job or apartment or anything today means creating endless pointless copies of your pesonal information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc
I dont know the way out if there is one, I guess we can keep fantasizing and thinking about it. It just feels like it would be easier to get the earth to start spinning the other way sometimes.
> “Applying for a job or apartment or anything today means creating endless pointless copies of your pesonal information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc”
This problem is practically fixed in the EU (to the extent that legislation can fix it). Data protection laws have enough teeth that real companies can’t afford to keep or sell customer information illegally.
But people only see the tip of the iceberg and think EU data protection is something to do with annoying cookie banners. We need to do a better job of celebrating Europe’s real achievements in making the digital world better for its citizens. Instant zero-fee bank transfers are another example.
Yes just make user data hoarding and targeted advertising a nonviable business model, and watch the horrible secondary effects start to dissipate. it requires a lot of political will that currently isn’t there but we have become too resigned in the US that things can’t change. I still hate cookie banners though :).
It doesn’t happen because when a company replaces advertising with a subscription, people balk and then switch to a competitor that doesn’t charge anything by using advertising.
The problem is that a lot of these services are just worthless. As in their market price is precisely zero dollars and zero cents. The reason you won't get me to subscribe to your random recipe or news website isn't the competition - the site simply provides no value. If it also costs nothing, then I might be indifferent to browsing it when it appears as a search result. If it costs anything, I definitely won't. I also feel the same about your competitors, so I'm not replacing you with them - I'm just browsing this type of content less. And that's a good thing for me and for society overall.
We need to (once again) define “free” pricing models as predatory and broadly outlaw them. They distort the idea of a free and fair marketplace by poisoning consumer expectations of what things should cost.
That will never happen as long as people are terrified with anxiety from continuous media exaggeration and "Security and Defense" are hidden behind thick veils and dark budgets.
>This problem is practically fixed in the EU (to the extent that legislation can fix it). Data protection laws have enough teeth that real companies can’t afford to keep or sell customer information illegally
Not even close to the case for any big player. It just exists as a moat for smaller companies.
https://www.enforcementtracker.com/ and sort by amount, these are not small companies and amounts aren't exactly trivial either, with a mechanism to get bigger if ignored.
Meta appear 4 times in the top 10 with a total of about 2.25bn in fines. That sounds like a lot but it's only 1.6% of their revenue. As a cost of doing business that's probably acceptable to the Meta board. It'd cost them more to do things properly, so there's little incentive to do so.
Like with most laws, smaller companies have smaller chance to get caught and smaller likely penalties.
But I've noticed there are two kinds of people when it comes to entrepreneurship and regulations. There are people who go all gung-ho and do what they want and ignore the law as much as they can get away with. And there are people who are so scared of things like laws that they never become entrepreneurs. I don't see much of a middle ground in practice.
I've worked with many large enterprises, including US megacorps, who have completely changed how they handle EU data post-GDPR. It's not perfect, but it's certainly not just a toll to be paid to continue old practices.
Was this posted from a Brussels IP? This certainly seems to reflect how the EU regulators see themselves, but I haven't met many real Europeans who have themselves realized any actual value coming from their laughable, vague attempts at legislating the problems away. The best they've managed is making some Europeans smug, but their data still exists in all the same places. Worst case a few fines get levied, for megacorps that can easily afford them, while small businesses grapple with confusing and vague language that threaten to punish them even absent any actual harms or even ill intentions.
So, if Europeans think these rules improved the situation, they are smug and dont count.
Frankly, in here EU did a good job, certainly better then USA does. It would be neat if USA made similar laws too.
Megacorps do get bigger fines then small companies, actually. Megacorps existence is also literally result of winner takes all and rich are untouchable legal system cranked to 11 Americans are proud of.
People in the EU are still using Instagram/Facebook/WhatsApp. Zuckerberg did a "ok, if you don't want us to track you, you can pay 12€/month" and everyone just smashed the "I consent to get my data mined forever" button.
Not to mention that we *still* have lobbying for chat control.
Every measure from the EU is, as always, meant to look like our beloved bureaucrats are doing something but absolute ineffective at changing the status quo.
If even the people who experience a different time gives up because "nothing changes" then it's truly over.
We need to do what we preach: sure, things are worse in certain things but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever.
Also I think people forget to realise that the type of people who were online in the 90s are still online, many still does exactly the same things. The Internet just got so much easier to use for the rest of the people who doesn't really see the magic of it all. And that's ok.
People always complaining how bad things currently are, they are doing a disservice to all the services and communities still around. They are not sexy or cool but they exist.
Self hosting is so much easier than before, though. Tools like docker and Tailscale make operating servers and using VPNs pretty painless.
Routing to your home address could be hard, but it’s also pretty easy and cheap to set up a reverse proxy from a server you can rent. Routing through a public CDN is also easy and cheap and solves a lot of problems like DDoS.
>but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever.
Understand your enthusiasm but to relate the discussion back to Tim Berners-Lee idea for SOLID data storage protocol... Running self-hosted things like email, NextCloud, Plex, sandstorm.io, etc -- are not relevant to the gp's "nothing changes" complaint.
Without dissecting the SOLID protocol, the basic idea is that transactional data is stored on a separate user-specified "storage pod". It's not just simplistic sharing of "name/address" profile data. Imagining some idealized scenarios might help:
- Spotify music : instead of "playlists, listening history" being stored on Spotify's servers, it is stored on the user's storage pod. Spotify makes API calls to constantly save that data to the user-controlled data location. If the user then cancels Spotify and switches to Apple Music service, Apple can just read the "music playlists data storage pod" and all the recommendations work as expected. No import/export.
- Amazon shopping: instead of order history being in a data silo on Amazon servers. It could be stored in user's "ecommerce orders storage pod". The user can then give permission to Walmart.com to read it to provide product recommendations.
The user "doesn't own their own data" continues with the current AI chat tools. The users' ChatGPT "prompts history" is stored at OpenAI instead of a user-controlled "storage pod".
The walled-garden and data silos don't just restrict consumers. Businesses have the same issue. They use SAP accounting software package or a SaaS tool and their data is locked up in those services. Exports are sometimes possible but cumbersome.
Therefore, self-hosting Plex on local server for a personal music library instead of using Spotify cloud doesn't affect the "nothing changes" narrative. TBL still wants people to have the flexibility/convenience of using cloud services but somehow still keep "ownership of their data".
On the other hand, if you were self-hosting a SOLID Storage Pod at home, and a company like Spotify wrote listening data to it, that's when the narrative changes.
It should be obvious that companies are not incentivized to write transactional data to users' storage pods which explains why the SOLID protocol doesn't seem to gain much traction for the last 9 years.
> It should be obvious that companies are not incentivized to write transactional data to users' storage pods which explains why the SOLID protocol doesn't seem to gain much traction for the last 9 years.
Not simply "not incentived" but actually decentivized. It's not just that companies lose the ability to have a better algorithm to recommend products, but the data itself is worth a fortune. Google, Facebook, etc are worth as much as they are because of the give amount of personal data they've gathered. And, the reason it's worth so much (well, one reason, and probably the least-scary one) is advertising.
Online advertising is the keystone keeping this pile of shit upright and I can't wait until that bubble finally pops. That is when the narrative will change. None of the ideas in this article will come to pass until all of the data that Google hoards is suddenly useless.
> Online advertising is the keystone keeping this pile of shit upright and I can't wait until that bubble finally pops. That is when the narrative will change.
This can't happen until there's another viable revenue stream. Which requires smoothing out everything about microtransactions, creating a culture where people now expect to pay, and building trust that it won't get stuffed with ads anyway.
thats why this is a legal battle as much it a technological one
it comes down to the rights to own the data you produce, and have it easily accesible. Solid is just a way of giving people option to excercise this right
> but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever.
Sometimes HN makes me feel like I'm the literal last remaining person on the planet who just... uses a desktop computer, and stores data on SSDs and HDDs, all physically connected to the machine, and never worries about how to access this data from another device because there are no other devices from which it should be accessed.
I mean, okay, fine, I do things like publishing to GitHub. But I still have a local copy, and I'm in control.
edit: I have no idea what people think they're talking about when they're like "people should just" and "you should just." The cage is not all in your mind, dude; it's an actual cage, guarded by people with guns.
Not OP, but I am self-hosting a bunch of things, like my blog. I am trying to move away from Google, my primary email for important things is under my domain (not purely self-hosted, but still). I am also creating backups so that I can recover if a service is gone for any reason.
Exactly, I've stopped worrying so much about what "everyone" is doing, and just continue to do my own thing. I've self-hosted E-mail and web for 15+ years at this point. I keep my music and movies on spinning metal in my garage with an NFS server running on it. Photos stored locally too, and everything backed up on my own storage. I don't care how locked-in Spotify keeps you, because I don't need Spotify. I don't care how much data Netflix collects, because I don't use it.
It's always fun to read articles about how urgently we need to go back to local-this and self-hosted that, knowing I never left!
Not simple website hosting, but if you want to do something like running social media, there are a bunch of regulations in the way that used to not exist, and regulations are enforced by people with guns (who are called police officers).
> creating endless pointless copies of your pesonal (sic) information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc… I dont know the way out if there is
The data needs to be viewed by the holder of that data as a dangerous liability, not an asset. If there were headlines about “Megabank Files Bankruptcy Over Data Breach, Executives Jailed” instead of the general sentiment of “LOL another data breach, here’s a free trial of LifeLock,” there would be changing attitudes about storing arbitrary user data.
I think it's advantageous for data to be viewed as an asset, but an asset owned by the source of the data. If Megabank was like; 'Oops, we left our vault unlocked and someone walk off with your savings' people would be up in arms.
You’d think that if anybody could pull off reshaping how data is stored and shared on the Internet, it would be him. And the technology is, well, solid.
Unfortunately, it doesn’t have as much traction as I would hope. Probably because it requires a new way of thinking about many parts of the tech stack. It’s not as simple as swapping out one library for another one. The existing web has so much momentum, and so many of today’s tools and frameworks have assumptions built into them that aren’t necessarily convenient for building a web where users have true data ownership.
Still, I’m rooting for Solid and the team behind it. They clearly understand these issues. They’ve been building libraries and scaffolding tools to make it easier to adopt Solid, For new projects, it’s pretty easy these days.
In general, I think these types of sticky behaviors only change when there's an application that people gravitate towards with the changing behavior embedded.
One such candidate is cryptocurrency and personal finances. The cryptocurrency wallet will necessarily need to be cryptographically secure, so this at least provides an opening for privacy. Tying it to finances means that there's an immediate application, payment processing, that people might want to use and put up with clunky behavior, at least initially.
All this lacks specificity and finances, cryptocurrency or no, bring their own drawbacks, but it does seem like it's possible to me.
The Internet's attention can be fickle and it's easy to forget that sometimes. IBM used to be a titan before Microsoft supplanted it. Proprietary server operating system, including web servers and databases used to deeply embedded until they were supplanted by FOSS alternatives. Digg, Friendster, Myspace, Yahoo, etc. used to fixtures of the Internet until they weren't.
This is demonstrably not fantasy as the example case is a fully productionized network (Bluesky and the rest of AT-net) that’s having real-world impact to the point where it’s under threat from several authoritarian states.
Don't get me wrong, I'm in the tech industry and generally more online then likely 95% of the population, but ime ... Nobody even knows what bluesky is?
(They also don't know what X is, though they DO know what Twitter is)
And even more niche products like mostodon, the fediverse altogether etc are entirely unknown to most of the tech industry too.
Tech is downstream of culture. Seems that smart people keep getting duped by this idea.
For example Twitter and Facebook didn’t result in a bunch of Democracies springing up after the Arab Spring, it resulted in the complete opposite. Tech simply amplifies the culture that was already there.
Honestly, that’s not been my experience. Granted the UK is less authoritarian than most. But the general attitude is people who care don’t even use Bluesky and those that don’t continue to use Meta services because why wouldn’t they if they don’t care.
I know the topic of mental health and social media is different from the topic of independence vs the monolithic web. But that doesn’t mean that there isn’t significant overlap in terms of those who are willing to boycott Meta for privacy reasons are also the kinds of people who likely dislike social media for other societal reasons too.
Bluesky is not decentralized. Building a centralized system on top of a protocol that can also theoretically support decentralized systems does not make it decentralized. https://arewedecentralizedyet.online/
> the point where it’s under threat from several authoritarian states.
This is a victim fantasy, and if being under intense attack from the state meant you were rebelling against the authoritarian system, then you would be capping for Parler, Gab, X and Tiktok. Bluesky, however, is only under attack from its own users, who are authoritarian trolls. At least the management seem to be getting sick of them, because it is actively inhibiting their growth* that they've been used as a base for the angriest, most entitled, least interesting people on the planet. It must be hell trying to manage a site filled with people demanding to speak to the manager.
It is also just a centralized twitter clone backed by VC looking for a return; not a revolution.
[*] Of course, it was their strategy to cater to that group because of all the free advertising they'd get from the media. But it had and has nothing to do with Dorsey's hopeful redemption arc, which was only about decentralization (i.e. not having speech under the control of people like him) and resilience. Bluesky was supposed to be bittorrent.
Wasn’t BlueSky kinda ruined by the whole leftist Twitter exodus while simultaneously being fawned over and settled by Reddity political types? Maybe I’m missing something but I’ve tried to use it a few times and it just feels like another internet echo chamber silo (even if that’s due to user self-isolation and not the underlying tech).
Well, TFA, and sibling posts to mine, point out some ways in which federated networks are leading the change in this direction. I would add that alongside SOLID and the AT Protocol, ActivityPub also encourages people taking ownership of their own data.
So probably you need to focus your attention to where the change happens instead of waiting for large, ad filled, for profit networks to act on it. Because indeed they have no incentive.
<< instead of waiting for large, ad filled, for profit networks to act on it.
I think I agree. I know I started re-evaluating my internet presence as a whole. I accept that a lot can't or won't do much, but the same was true, when firefox was new and no one wanted to jump ship, but the people, who liked privacy focus and extensions. Those that can, will move. The herd will follow if they see it can work.
> I guess we can keep fantasizing and thinking about it.
Strong regulations is the answer. To think that big corporations are going to do anything for us out of their good heart is naive and dangerous.
If a society wants nice things then they need to fight for it. Get elected officials that care to fix things, that fights against big corporations, and that help to split their monopolies.
The USA thinks that they can get a better Internet by doing nothing, like by magic. The reality is that government and civil society are going to need to put a lot of effort to reign in the big tech monopolies.
Nothing changes because the ask is silly and disconnected from the reality of normal people's lives. So what happens if Google has all your data? To the best of my observations over the past 20 years: best in class services, cheap, paired with excellent security and data availability.
Exactly. "It's good for you and takes some effort" is a bad growth strategy. For this movement to win, something will have to replace social media and walled gardens with a better dopamine hit, that just happens to keep data private.
I genuinely disagree. At this point, the only real way to make sure something like this stays worthwhile is when it is not 'super easy and convenient'. In other words, it has to take effort ( and obviously right now it does take effort and that effort ranks close to 'impossible' --- that should be pared down a bit ).
I think we're still missing an "open social" closed social network. Something like old-Facebook where you can post to an intimate audience of friends and family, and your feed isn't stuffed full of ads and influencers. Just a little private windows into your friends' lives.
That feels like something that could displace other social media in a way that's difficult for for-profit businesses to replicate since it goes against every product manager's instinct to leave engagement on the table, and would stand in stark contrast to the current social media landscape.
That looks really promising. It checks a lot of the boxes I already had in mind for such a system, like being able to continue a thread without exposing the whole thing to untrusted parties
> So what happens if Google has all your data? To the best of my observations over the past 20 years: best in class services, cheap, paired with excellent security and data availability.
And hope you never have your identity stolen, or an account hijacked, since that was the only proof of who you are.
The weird thing is that there are still IRC federators - big servers with channels much like discord, but presumably running on some dude's computer in a basement, and there are tons of people (usually niche interest groups) are still using those.
Yep, it’s all totally pointless so why bother thinking and dreaming of a way out, right?
Even if the ideas in this post are a little unrealistic in the face of modern convenience, it’s productive to talk about it.
Is there something else we should be doing instead?
The way out is mostly antitrust and regulation of the private data market. But too many portfolios depend on the status quo; the way will be opened once the AI bubble pops. The Chrome lawsuit was the jab before an AdX haymaker is thrown just as the arena lights go out.
Everyone wants "free ad-free no tracking no payment" Internet. Nobody wants to compensate anyone for it, and therefore nobody wants to host it.
Then the people who have not viewed an ad or paid a subscription in 20 years complain that the internet sucks and we need to go back to IRC and chan boards. As if ideologically non-paying customers have a voice worth listening to.
This isn't even close to true. The people who are serious about privacy and the open Web, and in the technologies posited to bring that about (such as self-sovereign identity and federation), tend to spend much more money.
They buy servers to self-host services, extra hardware to store data locally and domain names to let others find them. Those who cannot afford it sometimes join niche communities like the Tildeverse as an outlet for the interest.
In my experience it's largely the 'just not interested' camp who always go for the free webmail and whatever free messaging service comes with their phone.
> "As if ideologically non-paying customers have a voice worth listening to."
Do people who ideologically refuse to spend money on meat-foods have nothing worth listening to about animal welfare? Who don't spend money on airline flights have nothing worth listening to about climate change? Who avoid companies which use slave labour in their supply chains have nothing worth listening to about human rights?
'Money talks' but that doesn't automatically mean money has anything worth listening to; markets are manipulated by money as well as using it for signalling, and as a goal-seeking mechanism they are prone to local maxima like other things are.
The only reason why I ever use these services is because they killed off any alternatives through anti competitive practices. And I hate it every time because they are awful and disrespect me every single millimeter of the way.
You are arguing on the premise that ads would somehow be a fair exchange. That is simply the opposite of the truth. Ads are parasitic. Services with ads are almost always worse than services without, not just by having ads but also in every other way. Ads do not incentive quality, they incentive treating your users as prey and feeding them SEO slop.
I want to compensate people for actual beneficial work they do. But with most for profit internet services that is simply not possible. If you give them a finger they will take your whole arm. For exampme I want to buy good movies. But I simply cannot. All I can "buy" is a pinky promise from them to let me watch a movie under their conditions which they can change at any time under their sole discretion and they can just revoke that possibility for me completely at any time. Would I pay for Netflix they would only give me 720p no matter how much money I give them, because I have to much control over my own hardware for them.
There are exceptions to this that I happily pay for, but those are all niche services that cater to the small group of people like me.
The web is bloated. Costs have exploded because what used to be done in a few megabytes now takes hundreds. You COULD host much of the modern web for much, much less, but you'd actually have to get your webdev house in order.
IRC has pretty much always been free without ads. You make it sound unworkable when it's become so much easier to run over time. And tons of forums are in the same category.
Also there isn't a way for people to pay their share of server cost for services like that. For your average non-video communication service your options are paying 0x or paying 50x.
IRC doesn't offer multi device, high availability log archives. IRC doesn't offer a lot things, actually. Fairly sure the standards don't offer persistent identity.
Adding that doesn't take many resources though. It's because IRC is old and somewhat neglected, not because it would be burdensome to provide for free.
And some networks provide bouncers so they basically do have that. And maybe some IRCv3 networks, I haven't looked into that much lately.
All the things you describe are achieved via 'bouncers' or dedicated clients living in a server that an impermanent consumption device like a mobile phone might be able to connect to.
No, they're not native to the protocol, nor are they required. However it's an open protocol. You are free to pick from a number of solutions that compose that goal.
I don't want to compose anything and neither does 99% of the world. It's a non solution and we're having the Dropbox announcement discussion 15 years later.
But such consistent "nagging" is what gets attention to the problem. In the EU, you have GDPR exactly because of this kind of nagging. Privacy has nothing to do with nostalgia.
I love the idea of personal data storage and I want it to be the default, but I think there are some possibly insurmountable technical problems. This article doesn't mention schema once, and schemas make seamless data portability virtually impossible. I've spent a week making sure a simple CRUD app could change a string field to a UUID field without causing any outage or bugs.
You can export your data from Google or Facebook today, but then you need to write a copy of the source UI that faithfully replicates the way all those data fields are supposed to display. And tomorrow the source makes a change so what used to be one field is now two fields, oh and they also removed another field entirely so that data is just gone. Well, in future dumps anyway. Are you going to use the old schema or the new schema for your display? Is it possible to do both?
When everything is in data silos, you can freely and safely change data format, which is something that needs to happen a lot as applications evolve. Even in a data silo, doing this is pretty tricky and bugs and data loss are significant risks. If you're trying to sync between an unbounded number of data repositories where each repository has potentially conflicting relationships with the data schema, data loss is practically assured.
Another big problem is schema permissions and identity. I might have some piece of data that says "person A is allowed to see this set of fields" and another piece that says "person A is blocked from seeing this other set of fields." This gets synced to 3 different servers, one of those servers has no idea that userA is in fact person A. So you fail closed, but then the data on that server practically does not exist if the goal of this data repository is sharing some data with person A. You really can't do any sort of fine-grained access controls in a system where trust/identity/auditing is decentralized.
Vertically integrated apps are much cheaper to run - Instagram stores only a small fraction of your photos and makes a lot of money from them. It is somewhat harder to explain why we pay for things like iCloud, which mostly has no web API, only APIs for Apple devices. (Plenty of value there because it keeps you from having to buy a bigger iPhone.) But there are lots of these "almost general purpose" solutions, paying to upload files and store them, but where you cannot use them as you like.
Why not dozens of apps running over the "web filesystem" like happens on the desktop? Two reasons:
1. Amazon pricing for transit/bandwidth is way higher than storage, and so it makes accessing your own data quite expensive if it is not in the same datacenter.
2. And there is a huge security and usability gap between "pick one photo" vs "give me [scoped] access to your Dropbox"
Often the general-purpose mode does not work that well, is quite slow, or just costs a lot in bandwidth, a thing nobody wants to pay extra for when they're already paying for storage.
It was an idea that never went away. Many people have wanted to self host everything. Sadly companies have found it easier to centralise, and then as a bonus can monetise that data.
It wasn’t the companies but the users that found it easier. There’s a reason why everyone’s on Facebook, instagram, and gmail instead of running their own hosts—because it’s vastly easier for the majority of people to do so, and because everyone else is there.
We have not solved decentralisation in an accessible and useful way yet, and the incentives won’t change until we do. If ever.
But those who actually want to do this should be allowed by law to practice their ownership over their data.
I, and many like me, would pay for centralised service or any other service if it meant that we own our data and can tune the algorithms to our own preferences. I wont pay for doom scrolling, but would gladly pay for algorithm to serve me content that would better my human experience.
Governments have given corporation to much power, people need to rise up agains that, if it remains the same in AI age, we humans, and our collective mind would erode to the point of no return.
Users have the most power, by far. Corporations are the garden plants and users hold the hose. The graveyard of companies who didn't follow consumer trends is huge.
Unequivocally, users water plants that deliver in demand fruit while being most convenient and cheapest.
95% of Americans had shitty upload bandwidth until very recently, since coaxial broadband is all they have at home. It still probably sucks for most.
There was no choice but to use someone else’s computers for moving around large files. Plus CGNAT and whatnot making people have to use dynamic DNS. If a turnkey solution could have existed 20 years ago, maybe a market for it would have developed before the big companies locked it down.
Does the performance of individual data ownership hosted at home actually change very much when people have gigabit upload speeds? Since applications can already make multiple asynchronous requests, if we’re imagining that applications would need to request user data from each user’s house, the upload speeds would primarily affect latency and not necessarily throughput. If this does affect throughput, and it certainly might, then I’d guess that everyone having gigabit upload speeds doesn’t fix the problem. If we’re talking about something like Reddit and Facebook needing to make external requests for every comment in a long thread, I’d wager that it wouldn’t matter if every single request could upload at 100GB/s, it would still be hundreds of times slower than what we have today.
Even if I’m wildly in favor of user control over data, I’d venture to say that there still is no choice but to use someone else’s computers, and not just for performance reasons. If applications have to gather every individual user’s data that gets shown to another user from somewhere outside their servers every time, won’t reliability and consistency and UX likely become nonexistent, in addition to the unusable performance?
I don't know why you're imagining such ridiculously bad infrastructure that it has to access every person's house every page load.
Decentralized does not need to be slow like that. And very limited upload does get to be a problem if you want more than a couple people/servers to be able to access your media posts at the same time.
I replied to a comment that was talking about user upload speed. They replied to a comment about other people’s computers. Did I misunderstand? How do you get good infrastructure without using other people’s computers?
If you think such a system would need to load every comment from a different computer when you visit a page and be hundreds of times slower because of that, then yes you did misunderstand something.
The person you replied to is assuming a reasonable distributed system.
Please elaborate. If true, and they were imagining some unstated infrastructure, then what is it and what does home upload speed have to do with anything? What exactly did I misunderstand?
The self-hosting machines are plenty to avoid the problem you described, where there's massive slowdowns getting anything at all, including tiny little text comments. I hope you don't need me to walk through every detail of how a distributed system can do comments in a reasonable way?
But self-hosting machines are susceptible to the "I can only upload pictures and videos at 5-10mbps" problem. That requires more difficult peer-to-peer systems.
The first problem only requires getting small bits of data onto the same machine. The second problem requires getting large amounts of data onto many machines. Or reasonably symmetrical upload speeds.
It is very easy to sign up to Facebook, Instagram, Gmail and everything else. No manual is needed for doom-scrolling and on-boarding is instant. Personally I would prefer to have my own full-on LAMP stack at home, with Postfix for email and everything accessible via my own subdomain.
So, why can't I have that?
During my standard install of my favourite distro, I would only need to enter my name, subdomain and email password for everything to be magically installed, so I have a standard web site, some file sharing and email out of the box.
However, it would take me a fortnight to get this setup and I wouldn't have a clue how the email actually worked, if it worked. This wouldn't be my first rodeo either, so I wouldn't be starting entirely from scratch. I am also sure that there are some that have setup umpteen virtual linux machines that they could get everything done by tea-time.
Whether two hours or two weeks, it is still not that much work in the bigger scheme of things, which makes me wonder, why haven't I got some all-singing and all-dancing bash script that automates the whole process? But why has nobody else done it either, to make it fully open source and as easy to obtain as it can be?
Also, why can't I buy a glorified router box that does all of this? It could take the mainboard and power circuitry from any laptop, and, out the box, provide a decent web server, mail server and whatever else.
There is a suspicious absence of products in this space.
> why can't I buy a glorified router box that does all of this?
Step 0 is to secure that box, as routers are obvious targets, even before they have self-hosted data. There are some products based on RPi, NAS and router form factors.
Not that I disagree with you, but that’s generally not how society works. If only everyone had some consideration, self-control, and curiosity, we wouldn’t have an environmental crisis, churches, corruption, or wars. Yet all of these things do exist and won’t go away no matter how I wish them to.
So the next best thing is trying to operate in the constraints that apply, such as most people being unwilling to learn new things and going down the path of least resistance.
Slightly offtopic, but the sheer scale of the phenomenon you allude to - of screen-addled zombification - is really turbo-charging my own misanthropy. People staggering around, necks hunched, eyes down, all but glued to their miserable little toys. Everywhere, everyone, all the time. It's just pathetic. I guess I had hoped humans would have more self-control than this.
Stop viewing them in isolation and view them as a product of their environment. They weren't born with a phone in hand, someone gave it to them and someone created Tik Tok for them.
That's a fair argument. It's also unfalsifiable and based on an underlying personal worldview. Specifically (I would venture) an "us and them" view of things where history is determined by groups and power - a left-wing outlook, basically! I'm a bit of a liberal individualist by nature, I see personal responsibility and autonomy as a thing. I'm not sure how I'd go about deprogramming myself of this even if I wanted to. But it would help with the misanthropy, for sure.
I’ve always had this like 70% formed idea about Plex and how it’s indicative of how people want to self host more than we realize, but I’ve never quite been able to articulate what I’m thinking here and what the larger implications are.
Plex is obviously not true self hosting, but it’s a lot closer to it than a Netflix subscription, and the number of people who I do not consider very tech savvy who have not only been joining other people servers but trying to set up their own is staggering lately. And they’re not simply doing it because they want free movies or something. A lot of them have done it for the same reason I initially started: their kids.
I am concerned about the media that is put in front of my kids. I care about what shows they are watching. Kids are going to get their hands on screens there almost is no getting around it, so I would rather not trust YouTube et al with deciding what my kids do and don’t see. I can’t realistically be there to catch literally everything they watch, but if they’re using my server I know they only have access to a certain Library at all times so I can rest a lot easier. In a lot of ways I imagine this is how our parents felt when we were kids. On cable television growing up there were only so many “weird” or troubling things that could pop up, definitely nothing as extreme as we see today, and you could be reasonably aware of what most of those things were and know what channels to forbid/what times your kids should not have free access to the TV.
I found a lot of other parents feel the same way here. They’re just tired of feeling like the Internet is such an incredibly hostile place and want to find ways to take a little power back into their own hands.
I don’t know hopefully something useful popped up in that rant above. I have a lot of disjointed thoughts about this I really haven’t been able to bring together.
Yup that’s why I started self-hosting, when my wife got pregnant and we started to think about what technology access for our future kids would look like.
I started with CasaOS and Jellyfin. Quickly outgrew Casa and moved to learning Docker and setting up my own container stack, moving from media self-hosting to adding new containers of stuff like budgeting apps. I’m still working on building out my server but every container I add, the goal is basically to self-host a version of something I’m doing on a centralized service on the web and ultimately take my data and privacy back.
I will say some peoples’ elitist attitudes about stuff can be annoying and discouraging; it’s the same general spillover attitude from the Linux supremacy crowd. When I started with Casa I had someone basically tell me I was wasting my time and if I wasn’t running everything in VMs why bother. Which is entirely the opposite attitude to get “normies” and low technical literacy people on board, they need easy one-click install solutions like CasaOS. And if they decide to move onto something more complex, well I’m sure they can figure out how to reimage and rebuild their server in ProxMox or Docker as part of that.
Ha we basically had the same journey though you are certainly further along than I am.
Definitely agree about the elitist attitude problem. The amount of people who dunk on people for using Plex when I think it’s a fantastic jumping off point for true self hosting…it’s just so unnecessary and becomes a missed opportunity.
Glad to see a mention to Opera Unite. I found it to be a really revolutionary idea, anyone could have a simple static website running in their browser with zero tech knowledge needed. I think the world would have been better if that idea succeeded as a way for people to share their content, rather than the highly monetized and manipulative social networks.
> Rather than being in countless separate places on the internet in the hands of whomever it had been resold to, your data is in one place, controlled by you.
I don't see how this follows. The moment you create/share data with a site, what's to prevent them from reselling it?
The only thing this seems to attempt to solve is portability/interop (and moving control of and responsibility for blocking/moderation/spam to users rather than sites).
I don't see how it helps at all with privacy or you "controlling" who gets your data. If you give it to site A but not data collector B, what's preventing A from selling it to B? As far as I can tell, the situation will remain identical to how it is today.
Your data will never be in one place unless you never share it. The moment you use it with other sites or services, it is stored there too, out of your control.
Nothing is preventing it, but 3rd party operates on a copy. You are still owner of the data and it is on one place which makes it easier for you to access it, share it, backup it, analyze it. So, this doesn't prevent reselling in general but prevents data locking. From there, I guess its not that hard to demonstrate which 3rd party sold your data and sue them. It also mandates nonproprietary data formats.
All that is much, much better than what we have now.
> From there, I guess its not that hard to demonstrate which 3rd party sold your data and sue them.
But it doesn't? Obviously every site's TOS will say that by providing them with your data they can use it for all sorts of purposes. If you sued, you'd lose.
And you're generally going to want to make your data available to the various services requesting it, because otherwise most people won't see your posts and comments on their preferred platform.
Your site, the source of the data, could also include a TOS. Plenty of working examples in the commercial world where licensees are allowed to use data but not compile or resell it.
So I actually started researching this, and it turns out that, by the principles in Field v. Google Inc. (2006), neither side would have an enforceable TOS, but that a user making their social data available on their host and not attempting to block any particular crawlers with robots.txt would create an implied license that would allow social media sites to store and display the posts. Which is what allows Google to display information, snippets, and images from sites it crawls.
Facebook couldn't enforce a TOS because the hosting user had never gone to facebook.com and created an account, so the user never agreed to a contract. But a user couldn't enforce a TOS either because the crawling was automated, so Facebook wouldn't be agreeing to a contract either. But Facebook would be allowed to use the data because that's what a user is inviting by making it publicly available to crawlers and not doing anything to restrict access to Facebook.
> The moment you create/share data with a site, what's to prevent them from reselling it?
If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it.
EU legislation shows that we can actually have the right to demand that a company forgets about us. Asserting such rights become easier the more accurately we define what data is ours.
> If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it.
Can you? A site's TOS will say that by sharing your data, you grant them the right to display, reuse and redistribute it, the same as you do now. And that would take precedence because your host provided the data. They requested and you provided.
The only thing that would change that is actual legislation. But then the legislation is orthogonal to personal data storage. If you want legislation for that, pursue legislation for that. Personal data storage is completely separate, and the two shouldn't be confused with each other.
The right granted by the TOS elapses when you cancel the respective service, or when you revoke your consent (in which case the service provider may possibly cancel the service). (Some TOS are also simply illegal to begin with.) That’s what the GP is referring to.
No they don't. I don't know where you've gotten that information, but none of it is correct.
I mean, a TOS could be written that way. But they're generally not, because companies don't want to self-impose limits like that.
The TOS usually has something like "grant the platform a perpetual, worldwide, royalty-free, non-exclusive license to host, display, distribute, modify, and otherwise use that content in connection with the service".
current data points are much more valuable than historical data points, so storing old data doesn't have much incentives
also by having ability to enable/disable access to your data, you have the power of who gets what and for which purpose
also reselling of your data should become illegal to start with, would you be OKAY if your lawyer sells your data? or your colorectal surgeon? off course not, we have laws in place for that, and same laws should be applied to whoever handles your personal data
> current data points are much more valuable than historical data points, so storing old data doesn't have much incentives
Not true -- advertising profiles are vastly more valuable when based on a lifetime of data.
> also by having ability to enable/disable access to your data, you have the power of who gets what
But realistically, when are you ever going to disable access? If you want people to be able to read your replies no matter what social network they're using, you're going to make those replies available to every social network.
> and for which purpose
also reselling of your data should become illegal to start with
This is my point. The solution here is legal, not technological. Personal data storage doesn't change anything legally, and changing the law would prevent reselling even if you didn't have personal data storage.
It seems important not to confuse the two, in order not to give people false hopes.
I agree that this is not just a technological problem to be solved. Technology by it self can't fix the problems, but it can help nudge the human experience in good or bad way. Right now, we gave our data to large corporations and we got the lovely attention economy thats being feed on human rage, envy and greed.
Solid idea is more in line with revolution and demand for our representatives to give their people internet that can push the humanity forward, and not just let us waist countless hours on doom scrolling.
When I was a kid, a 4GB pendrive was a huge thing for me. I used to think my 40GB HDD would never fill up, but then Internet started to grow. Today it doesn’t even matter how muc storage you have it’ll always fill up.
I have started to self host quite a lot of stuff but eve then every storage solution has a life of 5-6 years in which atleast one of the components would fail. We click enormous amounts of photos but they do not have any impact like printed photo albums.
With ever growing storage costs (both cloud based and self hosted) I’m thinking of going back to keep only important stuff that too in print format.
I run a NAS, in various forms, for almost 20 years. The lifetime is quite longer, I still have ~ 10 year old drives in the backup NAS built on a Ryzen 1600 (8 years) and the average power supply works for me 10-12 years. The primary NAS is still on hardware that is more than 5 years old, except the drives that I just replaced with higher capacity.
As I find the size of current drives bigger than my yearly additions (personal pictures and movies), I am quite happy with a 10 year lifetime at low usage. I would love some reliable and affordable long term offline storage, but backup tapes and a reader are not affordable and not in common use for end users. Otherwise I would build a tiered storage system with more reliability and even performance (nvme hot tier? maybe).
In terms of NAS, I have long wonder if there is a market for a combination of both online and offline. We will need at least 2 HDD for redundancy and to prevent bit riot. And the NAS will be sold as a whole package and subscription, with an encrypted backup services included for first 2 years and requires the backup subscription to work there after. The profit margin is first on the hardware and then on long tail backup which is charged like iCloud and Google storage per tier. Where your 1.5TB storage will be charged at 2TB storage.
Before 2014 I would have thought Apple to potentially take this route for Time Capsule. Instead they doubled down on iCloud. Google will never take this route. Microsoft is not interested. Amazon should have done this and bundled with cold storage back up but their track record are not good enough. I doubt people trust Meta enough even if the solution was perfect.
In pre 2012 you could at least bet on Apple to be somewhat customer centric.
May be UniFi will do it. They just announced their 2 Bay UNAS and I only just discovered, they are a 40B market cap company. ( I thought they were much smaller )
You both wrote "bit riot" but meant "bit rot", right?
I've been running a RAIDZ2 NAS (with ECC RAM) for like 5 years with no data loss/corruption issues. Are you saying if it was just regular RAIDZ there would be data integrity issues?
>with an encrypted backup services included for first 2 years and requires the backup subscription to work there after.
Its confusing if you mean the NAS will stop working if you stop paying for the subscription or not. If you can no longer access your data on the NAS without a subscription, then the NAS just becomes the cloud with an extra up front cost plus the cost of your own electricity.
Personally I have started moving as much of my data out of the cloud as possible. I've got a Synology and a few single board computers running various services with a Synology in my parent's home for their photos. Their photos back up to my NAS and my data to their Synology.
Its a shame Synology decided to enshitify this year for all products going forward, but UGreen looks like a suitable replacement when I outgrow my current NAS.
Data will need Backup to be safe. You could tell everyday customer to get NAS and they wouldn't know what is Bit Riot until they saw their Image and Video with errors or broken. They also wouldn't do off site backup. Company wants long subscription model.
Right now everyone is only talking about options that are extreme in both ends.
Systems Twin Intelligence, where a Pod represents the full space-time information for part of the world, using Solid Protocol: https://graphmetrix.com/trinpod-server
I like the convenience of the cloud. but don't know whether its due to declining literacy rates / awareness etc. the cloud is nice and e.g google storage, iCloud but now with fast microsd's you can buy 1TB for $100. have a few copies then boom, you own your own data. but now phones don't allow you to have microsd's so here we are.
likewise things like email etc instead of all of us being on gmail we could have community email servers etc.
I use Dropbox, but with an encryption overlay that also integrates into the iOS Files app for ease of use on mobile. So it’s possible to use cloud storage and still keep your data private.
The fact that the AT Protocol relies on everyone having a domain name, which is a centralized system over which few people have control, and about whose workings most people have no clue about, is problematic. Also impractical, once we consider that - as far as I can understand - 8 billion people should have their own domain name.
What's impractical about everyone having a domain name? It surely isn't due to lack of domain names, because foo.bar.baz.bim.bim.bap.com is a valid domain name.
It is true that full data sovereignty isn't something most people are interested in, but this is more about a cooperative model for data ownership and access. Having your data identifier be JackDaniels@yahoo.com isn't particularly different from it being jackdaniels.is.technically.bourbon.com. In both cases another organization owns some of the path to your identifier and could potentially lock you out of it. In both cases, verizon is near the top of that list (.com).
As far as the domain name system being centralized, I'm not sure I agree. DNS is like a feudal system with hundreds of kings (top level domains) who all work together with one pope (ICANN), and various lords and ladies occupying positions under those kings. If ICANN goes completely bonkers the kings can get a new pope, some of them are literally sovereign because they are nation states. Just for fun, some of those states are ruled by literal kings, too. There are experiments to run a TLD by Decentralized Autonomous Organization (DAO), but I think for the most part nobody really cares because the current system happens to work pretty OK. If you have an idea for a more decentralized way to organize a namespace that doesn't involve your grandmother typing in a massive UUID or onion address, and doesn't result in someone being able to domain squat literally everything; I would love to hear about it.
Ownership is probably the wrong word since the legal grant is term limited contract for exclusive use under terms of service. Selling subdomain usage grants (also under contract and TOS) feels quite similar.
Top level domains can change pricing, terms, or cease operation. Freenom is a great case study, as they previously operated TLDs. At the edges, a well-operated subdomain service could offer stronger ownership-like behavior than a top level domain.
> The fact that the AT Protocol relies on everyone having a domain name
Well, either that or someone else hosting their identity (see did:plc), which seems to be the part you say should exist?
Probably DNS is the most decentralized centralized system we have available today that most people can actually use, unless I'm missing some obviously better way of doing the same thing?
The thing your missing is ICANN is headquartered in the US. The US political situation is dire and I think this could be a real danger for the internet at large. We might end up with disagreeing DNS worldwide at some point. E.g. if you hold a domain and have a non-authorized viewpoint so your DNS entry gets snuffed.
But from a practical point of view a decentralised system should not rely on domain name ownership. Any computer can generate a private/public key pair, which is all you need for identify.
> Any computer can generate a private/public key pair, which is all you need for identify.
Right, but once you've generated those, then what? You need a global registry of sorts so people can lookup each others keys for example, which is why DNS kind of is the best we have available today.
I don't think there is any perfect solution here, but it's hard to come up with something that has better trade-offs than DNS. Sure, ICANN might be based in the US, but so far DNS been relatively safe to rely on, and if ends up not reliable in the future, I'm not sure social media profiles is the biggest worry at that point.
If there was no way of moving away from it, probably yeah. But since you can migrate from a did:plc to did:web, I don't feel like they're very similar situations at all.
It doesn’t really rely absolutely on domain names; at the very root there’s just a DID. DNS happens to be the best we’ve got right now as a human-readable username and address in-one goes.
There are around 10^99 different possible domain name labels (the part between the dots), so I don’t quite see the impracticality. Even going the route of Reddit’s autogenerated usernames like Eloquent-Salad9443.net would be viable.
> 8 billion people should have their own domain name
That is something that could be feathered in gradually -- your country, region, city, neighbourhood, etc could have their own domains, and you could be anon237@milan.italy or whatever, until you find it necessary or inspiring to obtain your own domain.
But what is the alternative. Systems that bind identity to the phone number give even less control. Systems that use a self generated cryptographic key (like Scuttlebutt) are even less practical.
DNS is not perfect but I think the best we have for now.
I love this idea, and I imagine with years of successful lobbying efforts we could potentially get some laws passed to provide rights and clarity around our own data that could move us into this direction. But until then, while BlueSky is solid, I'll wait and see.
For those of us who've been around for some time and still value privacy, this sort of paradigm is obvious.
The trouble isn't a lack of the right technologies - I'd argue it's a problem in the go-to-market strategy of those building these products/technologies.
Ideas flow along lines carved out by power/influence. Facebook's early strategy was to start with restricting its usage to people at Harvard University - arguably a highly influential institution - and then expand outwards to other highly influential institutions. Only once the "who's who" from those institutions were already onboard did they let down the walls to allow us plebs in, and we all rushed in head-first.
X's current strategy leverages Musk's visibility and influence (for better or worse).
Get the most prominent influencers onboard with your decentralized social network, and others will follow (dramatically easier said than done, of course). But without a significant contingent of influencers/powerful people, your network's DoA.
> prominent influencers onboard with your decentralized social network
That's sort of a contradiction, no? Or at least it assumes transplanting the same mechanisms into a new milieu -- which I argue is something to leave behind, because it's those very mechanisms that have ruined the current internet.
I think instead of tapping into the same addictive attention economy schemes, the distributed / decentralized socials could onboard people en-masse by providing what's missing there, and filling a real need.
As in self hosting? I love self hosting idea for myself out of principles.
But unforunately it will never take off in a huge way because convenience is king. Average Joe and Jane want to install things with as little efforts as possible.
You can self host, but in order to be reachable you need to be discoverable. If the discovery is based on a mechanism that is controlled by someone else that can become an evil party, self-hosting in isolation is not too useful.
Any utopian future that requires a party to put in a lot of effort to change something in a way that would be a net negative for them, is just not going to happen.
People do not spend money to change the world in a way that would be worse for them but better for other people.
I don't think the average citizen cares enough or even understands the benefits
But lets say you get them on board and pass some law. Unless its a huge market like the EU or USA, probably what immediately happens is everyone pulls out of that market. Not out of malice but because they suddenly have to rewrite their app and that's probably quite expensive.
you store ur photos on fb same way you store your money at the bank and your code on github, its delegation of concerns, you can make same argument for literally anything....not using your own silicon, growing your own food, financing your own venture, owning your own land, etc etc.... maybe its more "secure" vs "less efficient" or some other tradeoff. and you have to get the right balance or take risks for optimal efficiency / profit/whatever your values are
When I read the title I couldn't help but think "did everyone forgot about hard disks?"
I'm sure Tim Berners-Lee is much smarter than me, but I kind of feel there are some parallels between the idea of "owning" posts you made in a platform and the ludicrous idea of "owning" game items as NFTs in a blockchain. The latter promises interoperability that games would never deliver. I wonder about the former.
At least I feel the major dealbreaker with this technology is just that it's not worth it for both parties involved.
Right now, Facebook hosts all the posts and monetizes them with ads. So long as they are making money with ads, they have no reason to delete the posts they're hosting, as the posts are their money maker.
But what happens if Facebook no longer "owns" the posts?
So now your posts are in your "personal cloud", which means that unless they are encrypted any website or local app can display them, even without any ads. This means Facebook is no longer making money off the posts. Why would they accept this?
On the flip side, who is paying for the hosting? Facebook? It's no longer their servers hosting the content, so I don't think so? Is Facebook supposed to pay the cloud service for metered API access? Can a cloud service offer different rates to different companies? Is the user supposed to pay for their cloud storage? So you're going to make users pay money to use facebook?
What happens if a post violates the ToS? Can facebook delete my post in my cloud storage against my will? What happens if content that is legal where facebook operates is illegal where the cloud servers operate?
Can I manually edit the data in my cloud storage like I'd be able with a file and then facebook has to treat every post as if it were untrusted input?
What happens if my cloud storage closes my account? I just lose everything? Will I be able to back up my cloud to my hard disk and reupload it to another cloud so facebook can access it? How is facebook going to handle a single user with 2 clouds that have different content?
I feel like this is a very complex thing and there are infinite questions that we can have about how this would be implemented in practice, while it's presented as simply "you own your data."
If this takes off I fear big tech very quickly finding friends among those pushing for things like chat control, while potentially reevaluating some of its more consumer friendly "views" towards privacy. Very easy to undermine something when you start speaking of its potential to facilitate CSAM.
that is exactly what is going to happen, as more people become aware.
that's why we all need to exercise our rights and freedoms. I'm scared that if we fail to do this in next few years. And let the AI be used in similar ways like it has been used to create social media algorithms. Then we are all fucked!
Whoever owns your AI owns you, so it better be you who owns it!
The problem isnt technical feasibility it is market incentives.
Most companies have no incentive to let you hold your data when they can just hold it for you.
If they do this they can mine it for data to improve their product as well as sell or otherwise indirectly profit from it. And, it's easier.
Also, while the market for privacy focused products isnt nothing, the number of people willing to pay a lot extra to compensate for the missed opportunities companies get by collecting your data is, i think, smaller than many people imagine. Which is sad.
I think the only way it will grow to an appreciable size is by seeing up close and personal what a really vicious stasi-like secret police does with dragnet surveillance and come out the other side, with scars. I believe we've only seen a small taste of this.
> The problem isnt technical feasibility it is market incentives.
This is understating it honestly.
The software industry has become completely reliant on renting data access back to users to maintain subscription revenue. One effect of this is it has devalued the actual software in the eyes of users to such a degree that virtually no one will pay for alternatives, certainly not enough to compensate the development cost.
Most people have no incentive of owning their data. Otherwise the companies which don't give you that would die out because people wouldn't use them if they cared.
Same fallacy as believing smartphones are giant and with non-user swappable batteries because somehow smartphone making companies are forcing this on the market, instead of the real reason which is that it's what consumers want.
I don't think it's so black-and-white. There are multiple forces at play simultaneously.
I agree that people don't care enough about owning their data for it to matter more than what the companies want to push, which is of course monetizing the data and maximizing user lock-in.
Similarly, I think it's in the companies' interests to use non-swappable batteries: simpler and cheaper to manufacture (I think this is the main reason) and the device is made obsolete earlier which is an added bonus. Maybe small improvements in size etc., but that's a very small difference. Modern phones are already larger even with non-swappable batteries so I'm not sure it mattered. But again, having a non-swappable battery has to be weighed against other features, and availability of alternatives. In the end, people just care more about the other features, even though swappable battery would be a good thing.
Just to conclude: I don't believe markets work to fully cater to what customers actually want. It's more like customers (and other parties) get a compromise between what different parties in the market want.
> the real reason which is that it's what consumers want
Consumers want what they're told to want by a constant barrage of commercial propaganda.
Devices are large and non-serviceable because this way they can be sold with a higher profit margin. Side effect being that the larger screens make the embedded commercial propaganda more effective and easy to deliver.
> Whether these providers are strictly cooperatives in the formal sense isn't what's most important here though;
I think the context of “encouraging people to switch” to a pds/solid/data coop, how they operate IS important. For two reasons:
- data coop and controlling data opens the door to a new market if we’re going to join data coops, then we may as well try to share the profits from said coop fairly. Otherwise Facebook can step in as a “data-coop” and keep-on-keeping-on
- a secondary effect is that now there is an incentive to move off facebook. If I can join my local Nowheresville.USA.town data coop and benefit directly to my community by storing data together then I am encouraged to switch to this new paradigm
That is the major undiscussed shift to me. I believe the only way out of the Big Tech dystopia is to incentivize the switch. Even if the reward is pennies. Invest in the community oil well.
Ideas like the Solid protocol have a limited timeframe to make it or go away. Not sure why anyone is still talking about it. TBL is rightfully a legend but this is now just a windmill.
This comment has inspired me to target SOLID and “things I can do to help” on my Sunday afternoon research block. This type of commentary is rife in this article thread and is now just a windmill.
If Schneier can’t get more than 13 comments on a solid protocol crypto wallet, I personally don’t think that anyone will ever care about a solid protocol app of any kind. And I’m all for it, just calling it as I personally see it.
Some things are fire, some things are warm, and some things are DOA.
A Solid protocol cryptowallet. Arcane on top of arcane.
I think it’s entirely unfair to dismiss technology because it hasn’t demanded immediate adoption by society. Solid is attempting to help define a better data future. We have working mechanisms in place but everyone is at a disadvantage except the people loyal to these giant corps. Attempting to give people the power to organize their data as they wish and to be used as they wish is worth it. Even if it doesn’t bring a renaissance.
Crypto wallets are not nearly as arcane as Solid. How many people have Binance accounts?
Market share matters, critical mass matters, adoption matters. I'm suggesting that mindshare goes negative over time if these things aren't achieved, and when you have long-tail blog posts trying to pump life into it, it's pivot time.
Righteousness alone doesn't win any of those things. It's been a very long time since Solid was released and it's like a whisper in the wind.
I do not have a Binance account and think cryptocurrency is a terrible starting point for a Solid application. Not even people who buy cryptocurrency care about where their cryptocurrency is held.
Arguably it hasn’t taken off because no one has incentivized using it.
I wasn't happy the state of blogging (tracking, bloat, ads, paywalls...), so I built https://LMNO.lol. It's offline first and you can browse blogs from anywhere (even terminal). Your blog is a single Markdown file. Drag and drop it to the browser and your entire blog is generated.
Custom domains are welcome. My blog is running off LMNO.lol that https://xenodium.com
I don't see what advantage any company gets from choosing to build products that enable personal data ownership. I say this as someone working on a venture with these sorts of design aims, it feels like pushing a boulder uphill often.
The business model of cloud service providers makes a lot of sense- we have a system which stores and operates on your data, you pay some rental fee for us to store it and operate on it, easy peasy. The cost is related to both the utility of the operations the operator performs (to both the operator and the user) and the amount of data the user stores.
Fundamentally this is how everything from Dropbox to Facebook is governed- Dropbox does not devise much utility per GB and users store a lot, so you rent per GB, but at Facebook, they don't store lots of your stuff, and on the data side maybe you don't get much value from it as it's a cesspit, but the data is valuable to Facebook to sell ads, etc, so they can provide the service for free.
Importantly, you don't need to improve the product to continue extracting this rent, because the product you are selling is not Dropbox v4, Facebook v2.3, rather you are selling ongoing access to the rental.
As soon as you introduce even simply a federated system where a few corporate operators are involved, it becomes very hard to justify extracting rent there as the network designer, as the operators are taking on the cost of actually storing the data. You have to really be iterating on the core product to use a SaaS business model here. Some things simply don't need a v4, does Dropbox really need that much iteration?
Meanwhile as the system designer, life has become a lot more complex for you. Suddenly you cannot push unilateral sweeping changes to APIs, you need to version things in a way that is compatible between, say, one university updating their system but not the other. Since your users are a few large operators rather than millions of individuals, you lose the network effect advantage of being able to screw over a few users for the "greater good", since if you irritate one corporate client, you lose a lot of your install base. Why would you voluntarily choose this harder path as a company?
Things get even worse as you increase the level of decentralization. The reality is users expect the polished experience that the rental companies can give you; they want their data always accessible so that their friend can see the pic they shared without needing to keep their own computers running, they want the "like counter" to go up without their personal node subscribing to messages from other nodes, etc. The only users that will accept a worse experience are people who have are motivated by their philosophy re: personal data ownership, and this crowd will want a FOSS solution, so you can say goodbye to charging them for Dropbox v4, they are simply not interested if you're not giving them the source code for free. (I suspect this is where the author sits, but fundamentally I don't think it will get mass appeal, most people simply do not care about data ownership above something that "just works".)
So now you are dealing with problems like dynamic generation of redundant data and fault- and Byzantine-tolerant consensus algorithms so that your system can maintain function even when the user turns their computer off, and you have to deal with wrapped-key cryptography so that the redundant data can be split across all these user nodes without you worrying that an unauthorized user can read it, and then you have issues like how do you deal with nodes that are too slow to process updates (perhaps some user data needs to be stored in this conflict-free replicated datatype you devise), and eventually you go through all of this to... create a system that is less monetizable than the rental model, because you can't extract that rent for ongoing data storage, and we know users are not interested in actually paying for software.
This article seems pretty far detached from the problems that people experience using technology. It’s the kind of thing that only deeply technical people consider.
When someone uses a service like Dropbox or iCloud Drive or Google Drive, they really aren’t experiencing any kind of problem where their data “isn’t theirs” or is “trapped.” It’s not that hard to migrate to something else and the services themselves are reasonably low-friction.
In terms of social data, users don’t really have a major issue with the status quo, and those who do have already developed relatively popular solutions like Mastodon and BlueSky.
Even “proprietary” photos applications like Apple Photos and Google Photos have very easy migration paths to other services.
So what exactly is the problem we’re trying to solve here? Giving me an @Bob handle? Did I want that or need that?
> In terms of social data, users don’t really have a major issue with the status quo
That's exactly it. And with social media (unlike files and photo storage) migration isn't really something people care about, because it's about the present not the past.
If you move from Twitter to Bluesky, does anyone care about moving their tweet history? They just want their list of followers to migrate over as much as possible, which happens relatively organically anyways.
I find the ideas of data coops to be very appealing. I don't want to depend on faceless mega-corps like Google to host stuff like my email, but I also don't find the idea self-hosting to be realistic. I wouldn't mind paying for the security since losing access to certain accounts would be a disaster, but I'm already locked in, and the benefits of existing services would be marginal compared to the cost of moving.
ideally you should be able in a simple way to host your stuff, in this case in a POD. That service should be provided by a utility company, same way we have internet providers now. They will be well regulated and it would be in their interest to safely hold your data because if not, they would face legal and financial consequences.
All other services would read/write from your Pod.
Who has an incentive to provide a Solid server? Not big social media companies, who want the personal information that Solid attempts to withhold. I don't think anyone is prepared to offer a convenient, high quality Solid-based social media experience to everyone for free, because that costs a lot of money. And if you know anything about human nature, it will have to be convenient and completely free in order to have a chance of capturing any mindshare outside of weird tech nerd circles.
> the platforms should be asking us what kinds of data they may copy from our servers, and only with strictly temporary allowances.
Until practical homomorphic encryption arrives, I don't see how this temporariness can be enforced. If we rely on promises or regulation instead of the technical ability to enforce this, how is that any better than today's social media companies promising not to do anything bad with the data they have on us?
price of intelligence is dropping day by day like it or not, sooner or later price incentives for someone to host such social media experience could become financially viable
How do I post a message on Discord/Twitter/Instagram from my personal data storage? If this is not supported, this idea is born-dead. Very few will use it, for the regular person the conversation goes like this:
- Who can see my personal data storage posts? Can someone with Twitter see them?
- No, but you'll own your data
- Bye
So maybe start with something which backs-up what you post on Twitter/Instagram/Discord to your personal data storage through APIs/data export.... This has no downside if it's easy to "activate"
At this point distributed protocols are getting good enough that for a large class of social applications, network effects are the only thing keeping the incumbents in place.
The irony of ad supported free services is that if you just let the advertisers pay you directly for eyeball time then paid for your services, it'd be better for you financially while keeping the web pure outside of the "paid to consume ads" app.
You just wait. The closed services will close down or become hostile enough that people will migrate. Not everyone will, but over a longer period - enough.
People getting into Solid and ATproto today are like people using own XMPP servers decades ago, or Mastodon years ago, or Matrix. Some projects like that will succeed, others will fade. But one day, you won't be able to post to Discord due to some policy changes and you'll have to reevaluate options.
Also, you can't backup from Twitter anymore. Or Discord. Or google photos. Or many others - they cut off that option once they're big enough.
> You just wait. The closed services will close down or become hostile enough that people will migrate.
I've been waiting a long time. Over that time, the closed services have only gotten more popular and no regular person is ever complaining that they are "hostile".
Regular people don't like ads, but they dislike paying even more, so they're pretty OK with the status quo. They certainly don't want to be paying for a domain name and paying for hosting.
I’m continuing to explore ideas like this in my DN project (short for DownloadNet or Discernet). The core concept: a browser controller / instrumentation harness that, by default, saves everything you browse to disk, and makes it available via full-text search or a browsable alphabetical index.
The browser controller actually runs its own local server that handles indexing and archiving on your disk, while the front end lives inside your browser as a dashboard or control pane. So it’s both a locally hosted app and a browser extension of sorts.
This is still a work in progress, but one direction I want to push further is allowing users to publish curated collections or search indexes of their browsing history.
More likely, though, you’d create a separate archive centered on a topic you care about, and as you browse you selectively add pages to that topic. Over time, you end up with a niche search engine tied to your expertise.
If that archive is good, others might find it valuable—and you might choose to publish it from your own machine. With tunneling tech (Cloudflare, Tor, etc.), you can expose your local box to the public internet. The vision is: user-sovereign data, but still shareable.
You could even federate groups of topic-based archives into a shared search ecosystem, useful for domains like biotech or other specialized fields.
Another crucial point: DownloadNet archives your browsing in real time. It doesn’t crawl externally; it captures exactly what you see, including sites you access via institutional credentials (e.g. research journals behind paywalls). Then you can optionally share those archives with a trusted group.
I’m also exploring a web-document bundle format: package an interactive set of web pages (not just one) into a self-contained snapshot you can send (e.g. via email). The recipient can browse that snapshot locally, with all internal links intact, as of a particular moment in time.
It’s a simple but powerful idea, and I think it has real growth potential in the data-sovereignty space. I started this as a passion project, and I believe many others care deeply about these ideas too. If you’re interested or want to get involved, head to the repository.
One way my vision differs from something like Solid is the philosophy of adoption: rather than launching with a full-blown protocol, you start with a simple tool that users adopt, extend, and share. Over time, emergent use cases and community practices shape the system. It’s bottom-up rather than top-down.
I’m not dissing Solid — I understand its aims and don’t see this as strictly competitive or exclusive. But I feel the incremental, user-led route is likelier to produce something sustainable. You grow it in the wild, learn what users actually need, and adapt. Instead of trying to design for all cases in advance, you let real-world use teach you what matters.
Anyway, that’s the gist of my vision—and how it diverges from other approaches like the one in the article you referenced. While it may seem as a condemnation of other ideas, it's not. So please don't take it that way.
If this is something you could get into, I encourage you come on over to the repo and share your contribution. I also riff more on Solid, this article and the approach of DN if you're interested, here: https://github.com/DO-SAY-GO/dn/wiki/What-is-DiskerNet-and-h...
There was a neat idea a bit back to allow Service Workers to work across origin: foreign fetch. It wasn't on the internet, was only in the scope of your browser, but I thought it was such a neat advancement. Would have done so much to allow the offline web to weave itself. Alas, deprecated. https://developer.chrome.com/blog/foreign-fetch
I work on a FOSS project in this space, Blobcache.
https://github.com/blobcache/blobcache
Trusting a server to store an application's state is a different thing from trusting it to author changes or to read the data. Servers should become dumber, and clients should become smarter. When I use an app, I want the app to load E2E encrypted state from storage (possibly on another machine, possibly not owned by me) make whatever changes and produce new encrypted data to send back to the server. The server should just be trusted for durability, and to prevent unauthorized access, but not to tell the truth about doing either of those things. Blobcache provides an API to facilitate transactions on E2EE state between a dumb storage server and any smart client.
Blobcache can be installed on old hardware along with a VPN like Tailscale and then loaded up with data from other devices. Configuration is like SSH, drop a key in a configuration file to grant access. It removes most of the friction associated with consuming and producing storage as a resource.
I'm using it to build E2EE version control like Git, but for your whole home directory.
https://github.com/gotvc/got
We should talk. This very similar to how apps use E2EE data in Peergos. Maybe we can join forces. https://peergos.org/posts/a-better-web
I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio).
Looking through the docs on Peergos, it looks like it's built on top of IPFS. I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here.
Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it. A significant difference is that Blobcache Volumes expose a transaction API with serializable isolation semantics. IPFS provides distributed, available-but-inconsistent, cryptographically signed cells. IPFS chooses availability, and Blobcache chooses consistency. A Blobcache Volume corresponds to a specific entity maintained and controlled by a specific Node. An IPFS name exists as a distributed entity on the network.
Most applications need some sort of consistent transactional cell (even if they don't realize it), but in order to be useful, inconsistent-but-available cells have to be used carefully in an application specific way. I blame this required application-specific care for the lack of adoption of CRDTs.
There's a long tail of other differences too. IPFS was pretty badly behaved the last time I used it, trying to configure my router, and creating lots of connections to other nodes. Blobcache is more like a web browser; it creates transient connections in immediate response to user actions.
That whole ecosystem is filled with complicated abstractions. Just as an example, the Multihash format is pervasive. It amounts to a tag for the algorithm used to create a hash, and then the hash output. I'd rather not have that indirection. All the hashes in Blobcache are 256 bits, and you set the algorithm per Volume. In Go that means the hashes can just be `[32]byte` instead of a slice and a tag and a table of algorithms.
I haven't used IPFS in a while, but I became pretty familiar with it awhile ago. Had I been able to build any of the stuff I was interested in on top of it, I probably wouldn't have written Blobcache.
Thanks! I'll send you an email.
The good news is Peergos also has serializable transactional modifications. This comes from us storing signed roots in a db on your home server (not ipns). We also have our own minimal ipfs implementation that uses 1000x fewer resources than kubo, aka go-ipfs.
There is also https://remotestorage.io/ for per-user storage.
Both of these proposals (as far as I've read them, YMMV) fail the evolutionary test. At the scale we're talking about, ideas must proceed as evolution does: not with a far-away goal in mind, but with incremental changes, each of which individually must be an improvement over the status quo.
We are at (near) a significant local maximum, and (again, as far as I've read, which is not all of it for sure) the people pitching this form of information control have given no set of steps from here to there without significant cost/effort.
Of course they don't have to have the whole path in mind. By definition they just need the first step or two. But they must be steps up.
You don't get wings by wanting to fly; first you need feathers to keep warm (I am not an evolutionary biologist, I don't know if that's a valid theory).
> each of which individually must be an improvement over the status quo
I agree. And looking at the average web user specifically, is "owning your own data" enough of a UX improvement? Maybe paired with less ads and products that optimize for the end-user rather than advertisers? I think... maybe. I hope so. It's going to take a lot of work done for little money, which is concerning, but I'm optimistic.
99.9% of BlueSky users use only Bluesky services. But BlueSky has a Personal Data Service for each. That means:
Those users have credible exit to take their data off BlueSky's hosting to someplace else (and as of a week or two ago to move back to BlueSky if they want).
Those users can put whatever kind of data they want in their PDS. They can host their git data via https://tangled.org . They can store their music listening scrobbles with https://teal.fm . They can blog on https://leaflet.pub .
And there's been rapidly advancing host it yourself options. Plenty of folk individually or collectively host PDS. There are alternate relays that collect &n syndicate out everyone's PDS data as that changes. Hosting the aggregation layer is significantly harder especially if you are trying to fully connect the network but there are a couple & progress is good.
it feels like a huge improvement over the status quo, and there's extremely visible developer energy building forward & rolling with the concepts. The breakdown on architecture allows for wins and work in various areas. The base seems solid, the core seems coherent & well built, built to scale not as one big thing but coherent layers. I think it's doing what you are asking for, and the signs of advancement & uptake warm my heart to see.
99.9% of BlueSky users use only Bluesky services.
I highly, highly doubt this, even in the narrowest sense of how many BlueSky users still actively post on X.
Among the first page and 2nd page (top 60) there is always atleast 1 post about how we're gonnna "take back the web" or make it back into some form of our 90s millenial nostalgia memories, self hosting, federated this or that, etc etc.
Meanwhile - Nothing changes, everything generally gets worse and younger generations come into the world with no memories of the 90s internet or the world before mobile devices or surveillence everywhere.
Applying for a job or apartment or anything today means creating endless pointless copies of your pesonal information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc
I dont know the way out if there is one, I guess we can keep fantasizing and thinking about it. It just feels like it would be easier to get the earth to start spinning the other way sometimes.
> “Applying for a job or apartment or anything today means creating endless pointless copies of your pesonal information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc”
This problem is practically fixed in the EU (to the extent that legislation can fix it). Data protection laws have enough teeth that real companies can’t afford to keep or sell customer information illegally.
But people only see the tip of the iceberg and think EU data protection is something to do with annoying cookie banners. We need to do a better job of celebrating Europe’s real achievements in making the digital world better for its citizens. Instant zero-fee bank transfers are another example.
Yes just make user data hoarding and targeted advertising a nonviable business model, and watch the horrible secondary effects start to dissipate. it requires a lot of political will that currently isn’t there but we have become too resigned in the US that things can’t change. I still hate cookie banners though :).
It doesn’t happen because when a company replaces advertising with a subscription, people balk and then switch to a competitor that doesn’t charge anything by using advertising.
Converting a service to a subscription is hard. Customers get used to "free" and will always be resentful.
Starting as a subscription service at least doesn't feel like a broken promise.
The problem is that a lot of these services are just worthless. As in their market price is precisely zero dollars and zero cents. The reason you won't get me to subscribe to your random recipe or news website isn't the competition - the site simply provides no value. If it also costs nothing, then I might be indifferent to browsing it when it appears as a search result. If it costs anything, I definitely won't. I also feel the same about your competitors, so I'm not replacing you with them - I'm just browsing this type of content less. And that's a good thing for me and for society overall.
We need to (once again) define “free” pricing models as predatory and broadly outlaw them. They distort the idea of a free and fair marketplace by poisoning consumer expectations of what things should cost.
> We need to (once again) define “free” pricing models as predatory and broadly outlaw them
Free services funded by ads have been a boon for the poor.
That rips off the advertisers and/or leaves the poor poorer.
For any given ad supported service, one of two things must be true:
(1) the ad spend was more than or equal to the cost of the service for those users
(2) the ad spend was less than the cost of the service for those users
From fork (2), it follows that the service isn't sustainable anyway.
From fork (1), it follows that the buyers of the ad slots in turn only make a profit if those ads led to sales higher than the ad spend.
But for any given poor person, that necessarily means spending more than they would have on a non-ad-supported version of the same thing.
That will never happen as long as people are terrified with anxiety from continuous media exaggeration and "Security and Defense" are hidden behind thick veils and dark budgets.
Idk if it's the thought that the US can't change things, but these concerns are mostly hypothetical for almost all people.
How are real people's lives being effected by these problems?
centralisation of power leads to fascism and historically people didn't really like that ie 2. WW
>This problem is practically fixed in the EU (to the extent that legislation can fix it). Data protection laws have enough teeth that real companies can’t afford to keep or sell customer information illegally
Not even close to the case for any big player. It just exists as a moat for smaller companies.
https://www.enforcementtracker.com/ and sort by amount, these are not small companies and amounts aren't exactly trivial either, with a mechanism to get bigger if ignored.
Meta appear 4 times in the top 10 with a total of about 2.25bn in fines. That sounds like a lot but it's only 1.6% of their revenue. As a cost of doing business that's probably acceptable to the Meta board. It'd cost them more to do things properly, so there's little incentive to do so.
The fines will increase if they continue breaking the rules, so there is incentive.
Besides fines being able to grow that's global revenue, probably a bigger part of EU revenue. And their margins aren't 100%.
Like with most laws, smaller companies have smaller chance to get caught and smaller likely penalties.
But I've noticed there are two kinds of people when it comes to entrepreneurship and regulations. There are people who go all gung-ho and do what they want and ignore the law as much as they can get away with. And there are people who are so scared of things like laws that they never become entrepreneurs. I don't see much of a middle ground in practice.
I've worked with many large enterprises, including US megacorps, who have completely changed how they handle EU data post-GDPR. It's not perfect, but it's certainly not just a toll to be paid to continue old practices.
Was this posted from a Brussels IP? This certainly seems to reflect how the EU regulators see themselves, but I haven't met many real Europeans who have themselves realized any actual value coming from their laughable, vague attempts at legislating the problems away. The best they've managed is making some Europeans smug, but their data still exists in all the same places. Worst case a few fines get levied, for megacorps that can easily afford them, while small businesses grapple with confusing and vague language that threaten to punish them even absent any actual harms or even ill intentions.
So, if Europeans think these rules improved the situation, they are smug and dont count.
Frankly, in here EU did a good job, certainly better then USA does. It would be neat if USA made similar laws too.
Megacorps do get bigger fines then small companies, actually. Megacorps existence is also literally result of winner takes all and rich are untouchable legal system cranked to 11 Americans are proud of.
> Frankly, in here EU did a good job
People in the EU are still using Instagram/Facebook/WhatsApp. Zuckerberg did a "ok, if you don't want us to track you, you can pay 12€/month" and everyone just smashed the "I consent to get my data mined forever" button.
Not to mention that we *still* have lobbying for chat control.
Every measure from the EU is, as always, meant to look like our beloved bureaucrats are doing something but absolute ineffective at changing the status quo.
If even the people who experience a different time gives up because "nothing changes" then it's truly over.
We need to do what we preach: sure, things are worse in certain things but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever.
Also I think people forget to realise that the type of people who were online in the 90s are still online, many still does exactly the same things. The Internet just got so much easier to use for the rest of the people who doesn't really see the magic of it all. And that's ok.
People always complaining how bad things currently are, they are doing a disservice to all the services and communities still around. They are not sexy or cool but they exist.
~the internet~ got easier to consume but self hosting in many ways became harder because of how hostile the internet has become.
Self hosting is so much easier than before, though. Tools like docker and Tailscale make operating servers and using VPNs pretty painless.
Routing to your home address could be hard, but it’s also pretty easy and cheap to set up a reverse proxy from a server you can rent. Routing through a public CDN is also easy and cheap and solves a lot of problems like DDoS.
Not really that much harder, if it’s only for personal use.
Not really. But sure didn’t get easier. Entropy and all that.
>but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever.
Understand your enthusiasm but to relate the discussion back to Tim Berners-Lee idea for SOLID data storage protocol... Running self-hosted things like email, NextCloud, Plex, sandstorm.io, etc -- are not relevant to the gp's "nothing changes" complaint.
Without dissecting the SOLID protocol, the basic idea is that transactional data is stored on a separate user-specified "storage pod". It's not just simplistic sharing of "name/address" profile data. Imagining some idealized scenarios might help:
- Spotify music : instead of "playlists, listening history" being stored on Spotify's servers, it is stored on the user's storage pod. Spotify makes API calls to constantly save that data to the user-controlled data location. If the user then cancels Spotify and switches to Apple Music service, Apple can just read the "music playlists data storage pod" and all the recommendations work as expected. No import/export.
- Amazon shopping: instead of order history being in a data silo on Amazon servers. It could be stored in user's "ecommerce orders storage pod". The user can then give permission to Walmart.com to read it to provide product recommendations.
The user "doesn't own their own data" continues with the current AI chat tools. The users' ChatGPT "prompts history" is stored at OpenAI instead of a user-controlled "storage pod".
The walled-garden and data silos don't just restrict consumers. Businesses have the same issue. They use SAP accounting software package or a SaaS tool and their data is locked up in those services. Exports are sometimes possible but cumbersome.
Therefore, self-hosting Plex on local server for a personal music library instead of using Spotify cloud doesn't affect the "nothing changes" narrative. TBL still wants people to have the flexibility/convenience of using cloud services but somehow still keep "ownership of their data".
On the other hand, if you were self-hosting a SOLID Storage Pod at home, and a company like Spotify wrote listening data to it, that's when the narrative changes.
It should be obvious that companies are not incentivized to write transactional data to users' storage pods which explains why the SOLID protocol doesn't seem to gain much traction for the last 9 years.
> It should be obvious that companies are not incentivized to write transactional data to users' storage pods which explains why the SOLID protocol doesn't seem to gain much traction for the last 9 years.
Not simply "not incentived" but actually decentivized. It's not just that companies lose the ability to have a better algorithm to recommend products, but the data itself is worth a fortune. Google, Facebook, etc are worth as much as they are because of the give amount of personal data they've gathered. And, the reason it's worth so much (well, one reason, and probably the least-scary one) is advertising.
Online advertising is the keystone keeping this pile of shit upright and I can't wait until that bubble finally pops. That is when the narrative will change. None of the ideas in this article will come to pass until all of the data that Google hoards is suddenly useless.
> Online advertising is the keystone keeping this pile of shit upright and I can't wait until that bubble finally pops. That is when the narrative will change.
This can't happen until there's another viable revenue stream. Which requires smoothing out everything about microtransactions, creating a culture where people now expect to pay, and building trust that it won't get stuffed with ads anyway.
thats why this is a legal battle as much it a technological one
it comes down to the rights to own the data you produce, and have it easily accesible. Solid is just a way of giving people option to excercise this right
Well its a double whammy -companies are disincitivized, but also the average consumer does not understand or care what this means.
Most comsumers just want websites to work. Something like SOLID would add friction. People who care about privacy are a vocal minority.
> but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever.
Sometimes HN makes me feel like I'm the literal last remaining person on the planet who just... uses a desktop computer, and stores data on SSDs and HDDs, all physically connected to the machine, and never worries about how to access this data from another device because there are no other devices from which it should be accessed.
I mean, okay, fine, I do things like publishing to GitHub. But I still have a local copy, and I'm in control.
> We need to do what we preach
You start.
edit: I have no idea what people think they're talking about when they're like "people should just" and "you should just." The cage is not all in your mind, dude; it's an actual cage, guarded by people with guns.
Not OP, but I am self-hosting a bunch of things, like my blog. I am trying to move away from Google, my primary email for important things is under my domain (not purely self-hosted, but still). I am also creating backups so that I can recover if a service is gone for any reason.
So yea, some of us are practicing what we preach.
Exactly, I've stopped worrying so much about what "everyone" is doing, and just continue to do my own thing. I've self-hosted E-mail and web for 15+ years at this point. I keep my music and movies on spinning metal in my garage with an NFS server running on it. Photos stored locally too, and everything backed up on my own storage. I don't care how locked-in Spotify keeps you, because I don't need Spotify. I don't care how much data Netflix collects, because I don't use it.
It's always fun to read articles about how urgently we need to go back to local-this and self-hosted that, knowing I never left!
Ok, done. You next.
Sorry, what? There are people with guns preventing us from self hosting websites? That’s certainly news to me.
Not simple website hosting, but if you want to do something like running social media, there are a bunch of regulations in the way that used to not exist, and regulations are enforced by people with guns (who are called police officers).
Metaphorical guns, but yes. And if needs be, actual ones.
> creating endless pointless copies of your pesonal (sic) information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc… I dont know the way out if there is
The data needs to be viewed by the holder of that data as a dangerous liability, not an asset. If there were headlines about “Megabank Files Bankruptcy Over Data Breach, Executives Jailed” instead of the general sentiment of “LOL another data breach, here’s a free trial of LifeLock,” there would be changing attitudes about storing arbitrary user data.
I think it's advantageous for data to be viewed as an asset, but an asset owned by the source of the data. If Megabank was like; 'Oops, we left our vault unlocked and someone walk off with your savings' people would be up in arms.
The most compelling and plausible solution to this that I have seen is a set of standards called Solid, made by Tim Berners Lee, who invented the web.
https://en.wikipedia.org/wiki/Solid_(web_decentralization_pr...
You’d think that if anybody could pull off reshaping how data is stored and shared on the Internet, it would be him. And the technology is, well, solid.
Unfortunately, it doesn’t have as much traction as I would hope. Probably because it requires a new way of thinking about many parts of the tech stack. It’s not as simple as swapping out one library for another one. The existing web has so much momentum, and so many of today’s tools and frameworks have assumptions built into them that aren’t necessarily convenient for building a web where users have true data ownership.
Still, I’m rooting for Solid and the team behind it. They clearly understand these issues. They’ve been building libraries and scaffolding tools to make it easier to adopt Solid, For new projects, it’s pretty easy these days.
Yeah, that’s... that’s what the whole post was about...
In general, I think these types of sticky behaviors only change when there's an application that people gravitate towards with the changing behavior embedded.
One such candidate is cryptocurrency and personal finances. The cryptocurrency wallet will necessarily need to be cryptographically secure, so this at least provides an opening for privacy. Tying it to finances means that there's an immediate application, payment processing, that people might want to use and put up with clunky behavior, at least initially.
All this lacks specificity and finances, cryptocurrency or no, bring their own drawbacks, but it does seem like it's possible to me.
The Internet's attention can be fickle and it's easy to forget that sometimes. IBM used to be a titan before Microsoft supplanted it. Proprietary server operating system, including web servers and databases used to deeply embedded until they were supplanted by FOSS alternatives. Digg, Friendster, Myspace, Yahoo, etc. used to fixtures of the Internet until they weren't.
This is demonstrably not fantasy as the example case is a fully productionized network (Bluesky and the rest of AT-net) that’s having real-world impact to the point where it’s under threat from several authoritarian states.
It has?
Don't get me wrong, I'm in the tech industry and generally more online then likely 95% of the population, but ime ... Nobody even knows what bluesky is?
(They also don't know what X is, though they DO know what Twitter is)
And even more niche products like mostodon, the fediverse altogether etc are entirely unknown to most of the tech industry too.
Sounds like a feature. I like some self-selection bias, it might have character. Maybe a little less global competition for my attention.
You must live in a different tech industry than I do. They might not be using it, but most know about it.
Sometimes tech leads the world, however unwillingly, to better outcomes.
Tech is downstream of culture. Seems that smart people keep getting duped by this idea.
For example Twitter and Facebook didn’t result in a bunch of Democracies springing up after the Arab Spring, it resulted in the complete opposite. Tech simply amplifies the culture that was already there.
Honestly, that’s not been my experience. Granted the UK is less authoritarian than most. But the general attitude is people who care don’t even use Bluesky and those that don’t continue to use Meta services because why wouldn’t they if they don’t care.
I know the topic of mental health and social media is different from the topic of independence vs the monolithic web. But that doesn’t mean that there isn’t significant overlap in terms of those who are willing to boycott Meta for privacy reasons are also the kinds of people who likely dislike social media for other societal reasons too.
Bluesky is not decentralized. Building a centralized system on top of a protocol that can also theoretically support decentralized systems does not make it decentralized. https://arewedecentralizedyet.online/
> the point where it’s under threat from several authoritarian states.
This is a victim fantasy, and if being under intense attack from the state meant you were rebelling against the authoritarian system, then you would be capping for Parler, Gab, X and Tiktok. Bluesky, however, is only under attack from its own users, who are authoritarian trolls. At least the management seem to be getting sick of them, because it is actively inhibiting their growth* that they've been used as a base for the angriest, most entitled, least interesting people on the planet. It must be hell trying to manage a site filled with people demanding to speak to the manager.
It is also just a centralized twitter clone backed by VC looking for a return; not a revolution.
[*] Of course, it was their strategy to cater to that group because of all the free advertising they'd get from the media. But it had and has nothing to do with Dorsey's hopeful redemption arc, which was only about decentralization (i.e. not having speech under the control of people like him) and resilience. Bluesky was supposed to be bittorrent.
Wasn’t BlueSky kinda ruined by the whole leftist Twitter exodus while simultaneously being fawned over and settled by Reddity political types? Maybe I’m missing something but I’ve tried to use it a few times and it just feels like another internet echo chamber silo (even if that’s due to user self-isolation and not the underlying tech).
> Meanwhile - Nothing changes
Well, TFA, and sibling posts to mine, point out some ways in which federated networks are leading the change in this direction. I would add that alongside SOLID and the AT Protocol, ActivityPub also encourages people taking ownership of their own data.
So probably you need to focus your attention to where the change happens instead of waiting for large, ad filled, for profit networks to act on it. Because indeed they have no incentive.
<< instead of waiting for large, ad filled, for profit networks to act on it.
I think I agree. I know I started re-evaluating my internet presence as a whole. I accept that a lot can't or won't do much, but the same was true, when firefox was new and no one wanted to jump ship, but the people, who liked privacy focus and extensions. Those that can, will move. The herd will follow if they see it can work.
> I guess we can keep fantasizing and thinking about it.
Strong regulations is the answer. To think that big corporations are going to do anything for us out of their good heart is naive and dangerous.
If a society wants nice things then they need to fight for it. Get elected officials that care to fix things, that fights against big corporations, and that help to split their monopolies.
The USA thinks that they can get a better Internet by doing nothing, like by magic. The reality is that government and civil society are going to need to put a lot of effort to reign in the big tech monopolies.
Nothing changes because the ask is silly and disconnected from the reality of normal people's lives. So what happens if Google has all your data? To the best of my observations over the past 20 years: best in class services, cheap, paired with excellent security and data availability.
Exactly. "It's good for you and takes some effort" is a bad growth strategy. For this movement to win, something will have to replace social media and walled gardens with a better dopamine hit, that just happens to keep data private.
I genuinely disagree. At this point, the only real way to make sure something like this stays worthwhile is when it is not 'super easy and convenient'. In other words, it has to take effort ( and obviously right now it does take effort and that effort ranks close to 'impossible' --- that should be pared down a bit ).
I think we're still missing an "open social" closed social network. Something like old-Facebook where you can post to an intimate audience of friends and family, and your feed isn't stuffed full of ads and influencers. Just a little private windows into your friends' lives.
That feels like something that could displace other social media in a way that's difficult for for-profit businesses to replicate since it goes against every product manager's instinct to leave engagement on the table, and would stand in stark contrast to the current social media landscape.
You may like Peergos (creator here) https://peergos.org/posts/decentralized-social-media
That looks really promising. It checks a lot of the boxes I already had in mind for such a system, like being able to continue a thread without exposing the whole thing to untrusted parties
Thanks! You can play around with it on https://peergos-demo.net
I wish I understood why people will pay for streaming tv subscriptions but not for social subscriptions.
I suppose social subscriptions have to overcome network effects and a plethora of “free” alternatives - ranging from iMessage to facebook.
I think at least one take on this is that people see it as paying for the content of streaming subscriptions, not the streaming infrastructure itself.
So the idea of paying for the infrastructure needed to see the content produced by your social network doesn't feel like a good deal.
google has all data > google creates AI from data > google embeds their values into AI > you use the AI > you become what ever the google AI wants
"over the past 20 years" is not the same as next 20 years
unless you travel to the 25% of the world they antagonize politically.
or unless you don’t comply quickly enough when they say “jump” and they unilaterally take away “your” gvoice number.
Look at QC Safe sometime. Same idea applies. Incentives are not consistent over time.
Giving all your data for better services is easily hijackable.
> So what happens if Google has all your data? To the best of my observations over the past 20 years: best in class services, cheap, paired with excellent security and data availability.
And hope you never have your identity stolen, or an account hijacked, since that was the only proof of who you are.
Most of those 20 years have coincided with low interest rates and the internet growing constantly (and hardware and software maturing).
What happens when the rising tide stops but the boats still have to rise?
My bet is that we will hate Google, Facebook, Amazon, modern Microsoft a lot more than people in the 80s and 90s hated IBM and old Microsoft.
...while selling you crap you don't need because they follow you everywhere.
I think it's about showing that different models are possible for people who do care and are willing to reflect and change the way they operate.
The big majority goes with the comfort of the mainstream, almost by definition.
The weird thing is that there are still IRC federators - big servers with channels much like discord, but presumably running on some dude's computer in a basement, and there are tons of people (usually niche interest groups) are still using those.
Yep, it’s all totally pointless so why bother thinking and dreaming of a way out, right? Even if the ideas in this post are a little unrealistic in the face of modern convenience, it’s productive to talk about it. Is there something else we should be doing instead?
The way out is mostly antitrust and regulation of the private data market. But too many portfolios depend on the status quo; the way will be opened once the AI bubble pops. The Chrome lawsuit was the jab before an AdX haymaker is thrown just as the arena lights go out.
Everyone wants "free ad-free no tracking no payment" Internet. Nobody wants to compensate anyone for it, and therefore nobody wants to host it.
Then the people who have not viewed an ad or paid a subscription in 20 years complain that the internet sucks and we need to go back to IRC and chan boards. As if ideologically non-paying customers have a voice worth listening to.
This isn't even close to true. The people who are serious about privacy and the open Web, and in the technologies posited to bring that about (such as self-sovereign identity and federation), tend to spend much more money.
They buy servers to self-host services, extra hardware to store data locally and domain names to let others find them. Those who cannot afford it sometimes join niche communities like the Tildeverse as an outlet for the interest.
In my experience it's largely the 'just not interested' camp who always go for the free webmail and whatever free messaging service comes with their phone.
> "As if ideologically non-paying customers have a voice worth listening to."
Do people who ideologically refuse to spend money on meat-foods have nothing worth listening to about animal welfare? Who don't spend money on airline flights have nothing worth listening to about climate change? Who avoid companies which use slave labour in their supply chains have nothing worth listening to about human rights?
'Money talks' but that doesn't automatically mean money has anything worth listening to; markets are manipulated by money as well as using it for signalling, and as a goal-seeking mechanism they are prone to local maxima like other things are.
The thing is that they still use the services/products. It's just ad-blocking and piracy.
So to follow your analogy, they eat meat by stealing it, and feel like they are sending a message about animal welfare.
The only reason why I ever use these services is because they killed off any alternatives through anti competitive practices. And I hate it every time because they are awful and disrespect me every single millimeter of the way.
You are arguing on the premise that ads would somehow be a fair exchange. That is simply the opposite of the truth. Ads are parasitic. Services with ads are almost always worse than services without, not just by having ads but also in every other way. Ads do not incentive quality, they incentive treating your users as prey and feeding them SEO slop.
I want to compensate people for actual beneficial work they do. But with most for profit internet services that is simply not possible. If you give them a finger they will take your whole arm. For exampme I want to buy good movies. But I simply cannot. All I can "buy" is a pinky promise from them to let me watch a movie under their conditions which they can change at any time under their sole discretion and they can just revoke that possibility for me completely at any time. Would I pay for Netflix they would only give me 720p no matter how much money I give them, because I have to much control over my own hardware for them.
There are exceptions to this that I happily pay for, but those are all niche services that cater to the small group of people like me.
The web is bloated. Costs have exploded because what used to be done in a few megabytes now takes hundreds. You COULD host much of the modern web for much, much less, but you'd actually have to get your webdev house in order.
IRC has pretty much always been free without ads. You make it sound unworkable when it's become so much easier to run over time. And tons of forums are in the same category.
Also there isn't a way for people to pay their share of server cost for services like that. For your average non-video communication service your options are paying 0x or paying 50x.
IRC doesn't offer multi device, high availability log archives. IRC doesn't offer a lot things, actually. Fairly sure the standards don't offer persistent identity.
Adding that doesn't take many resources though. It's because IRC is old and somewhat neglected, not because it would be burdensome to provide for free.
And some networks provide bouncers so they basically do have that. And maybe some IRCv3 networks, I haven't looked into that much lately.
All the things you describe are achieved via 'bouncers' or dedicated clients living in a server that an impermanent consumption device like a mobile phone might be able to connect to.
No, they're not native to the protocol, nor are they required. However it's an open protocol. You are free to pick from a number of solutions that compose that goal.
I don't want to compose anything and neither does 99% of the world. It's a non solution and we're having the Dropbox announcement discussion 15 years later.
But such consistent "nagging" is what gets attention to the problem. In the EU, you have GDPR exactly because of this kind of nagging. Privacy has nothing to do with nostalgia.
I love the idea of personal data storage and I want it to be the default, but I think there are some possibly insurmountable technical problems. This article doesn't mention schema once, and schemas make seamless data portability virtually impossible. I've spent a week making sure a simple CRUD app could change a string field to a UUID field without causing any outage or bugs.
You can export your data from Google or Facebook today, but then you need to write a copy of the source UI that faithfully replicates the way all those data fields are supposed to display. And tomorrow the source makes a change so what used to be one field is now two fields, oh and they also removed another field entirely so that data is just gone. Well, in future dumps anyway. Are you going to use the old schema or the new schema for your display? Is it possible to do both?
When everything is in data silos, you can freely and safely change data format, which is something that needs to happen a lot as applications evolve. Even in a data silo, doing this is pretty tricky and bugs and data loss are significant risks. If you're trying to sync between an unbounded number of data repositories where each repository has potentially conflicting relationships with the data schema, data loss is practically assured.
Another big problem is schema permissions and identity. I might have some piece of data that says "person A is allowed to see this set of fields" and another piece that says "person A is blocked from seeing this other set of fields." This gets synced to 3 different servers, one of those servers has no idea that userA is in fact person A. So you fail closed, but then the data on that server practically does not exist if the goal of this data repository is sharing some data with person A. You really can't do any sort of fine-grained access controls in a system where trust/identity/auditing is decentralized.
Vertically integrated apps are much cheaper to run - Instagram stores only a small fraction of your photos and makes a lot of money from them. It is somewhat harder to explain why we pay for things like iCloud, which mostly has no web API, only APIs for Apple devices. (Plenty of value there because it keeps you from having to buy a bigger iPhone.) But there are lots of these "almost general purpose" solutions, paying to upload files and store them, but where you cannot use them as you like.
Why not dozens of apps running over the "web filesystem" like happens on the desktop? Two reasons: 1. Amazon pricing for transit/bandwidth is way higher than storage, and so it makes accessing your own data quite expensive if it is not in the same datacenter. 2. And there is a huge security and usability gap between "pick one photo" vs "give me [scoped] access to your Dropbox" Often the general-purpose mode does not work that well, is quite slow, or just costs a lot in bandwidth, a thing nobody wants to pay extra for when they're already paying for storage.
It was an idea that never went away. Many people have wanted to self host everything. Sadly companies have found it easier to centralise, and then as a bonus can monetise that data.
It wasn’t the companies but the users that found it easier. There’s a reason why everyone’s on Facebook, instagram, and gmail instead of running their own hosts—because it’s vastly easier for the majority of people to do so, and because everyone else is there.
We have not solved decentralisation in an accessible and useful way yet, and the incentives won’t change until we do. If ever.
But those who actually want to do this should be allowed by law to practice their ownership over their data.
I, and many like me, would pay for centralised service or any other service if it meant that we own our data and can tune the algorithms to our own preferences. I wont pay for doom scrolling, but would gladly pay for algorithm to serve me content that would better my human experience.
Governments have given corporation to much power, people need to rise up agains that, if it remains the same in AI age, we humans, and our collective mind would erode to the point of no return.
Users have the most power, by far. Corporations are the garden plants and users hold the hose. The graveyard of companies who didn't follow consumer trends is huge.
Unequivocally, users water plants that deliver in demand fruit while being most convenient and cheapest.
95% of Americans had shitty upload bandwidth until very recently, since coaxial broadband is all they have at home. It still probably sucks for most.
There was no choice but to use someone else’s computers for moving around large files. Plus CGNAT and whatnot making people have to use dynamic DNS. If a turnkey solution could have existed 20 years ago, maybe a market for it would have developed before the big companies locked it down.
Does the performance of individual data ownership hosted at home actually change very much when people have gigabit upload speeds? Since applications can already make multiple asynchronous requests, if we’re imagining that applications would need to request user data from each user’s house, the upload speeds would primarily affect latency and not necessarily throughput. If this does affect throughput, and it certainly might, then I’d guess that everyone having gigabit upload speeds doesn’t fix the problem. If we’re talking about something like Reddit and Facebook needing to make external requests for every comment in a long thread, I’d wager that it wouldn’t matter if every single request could upload at 100GB/s, it would still be hundreds of times slower than what we have today.
Even if I’m wildly in favor of user control over data, I’d venture to say that there still is no choice but to use someone else’s computers, and not just for performance reasons. If applications have to gather every individual user’s data that gets shown to another user from somewhere outside their servers every time, won’t reliability and consistency and UX likely become nonexistent, in addition to the unusable performance?
I don't know why you're imagining such ridiculously bad infrastructure that it has to access every person's house every page load.
Decentralized does not need to be slow like that. And very limited upload does get to be a problem if you want more than a couple people/servers to be able to access your media posts at the same time.
I replied to a comment that was talking about user upload speed. They replied to a comment about other people’s computers. Did I misunderstand? How do you get good infrastructure without using other people’s computers?
If you think such a system would need to load every comment from a different computer when you visit a page and be hundreds of times slower because of that, then yes you did misunderstand something.
The person you replied to is assuming a reasonable distributed system.
Please elaborate. If true, and they were imagining some unstated infrastructure, then what is it and what does home upload speed have to do with anything? What exactly did I misunderstand?
The self-hosting machines are plenty to avoid the problem you described, where there's massive slowdowns getting anything at all, including tiny little text comments. I hope you don't need me to walk through every detail of how a distributed system can do comments in a reasonable way?
But self-hosting machines are susceptible to the "I can only upload pictures and videos at 5-10mbps" problem. That requires more difficult peer-to-peer systems.
The first problem only requires getting small bits of data onto the same machine. The second problem requires getting large amounts of data onto many machines. Or reasonably symmetrical upload speeds.
It is very easy to sign up to Facebook, Instagram, Gmail and everything else. No manual is needed for doom-scrolling and on-boarding is instant. Personally I would prefer to have my own full-on LAMP stack at home, with Postfix for email and everything accessible via my own subdomain.
So, why can't I have that?
During my standard install of my favourite distro, I would only need to enter my name, subdomain and email password for everything to be magically installed, so I have a standard web site, some file sharing and email out of the box.
However, it would take me a fortnight to get this setup and I wouldn't have a clue how the email actually worked, if it worked. This wouldn't be my first rodeo either, so I wouldn't be starting entirely from scratch. I am also sure that there are some that have setup umpteen virtual linux machines that they could get everything done by tea-time.
Whether two hours or two weeks, it is still not that much work in the bigger scheme of things, which makes me wonder, why haven't I got some all-singing and all-dancing bash script that automates the whole process? But why has nobody else done it either, to make it fully open source and as easy to obtain as it can be?
Also, why can't I buy a glorified router box that does all of this? It could take the mainboard and power circuitry from any laptop, and, out the box, provide a decent web server, mail server and whatever else.
There is a suspicious absence of products in this space.
> why can't I buy a glorified router box that does all of this?
Step 0 is to secure that box, as routers are obvious targets, even before they have self-hosted data. There are some products based on RPi, NAS and router form factors.
> suspicious absence of products in this space
Earlier efforts:
Active OSS projects include Proxmox (https://community-scripts.github.io/ProxmoxVE/), Paperless-NGX (docs), Immich (photos), NextCloud and others, https://github.com/awesome-selfhosted/awesome-selfhostedGod forbid that people actually have to learn and do something instead of sitting around being a doomscrolling tiktok zombie... /s
There's all sorts of things I have no interest in learning because they seem unspeakably dull.
That some people don't want to spend time learning the thing that you happen to find interesting doesn't mean they're wasting their lives.
Not that I disagree with you, but that’s generally not how society works. If only everyone had some consideration, self-control, and curiosity, we wouldn’t have an environmental crisis, churches, corruption, or wars. Yet all of these things do exist and won’t go away no matter how I wish them to.
So the next best thing is trying to operate in the constraints that apply, such as most people being unwilling to learn new things and going down the path of least resistance.
Slightly offtopic, but the sheer scale of the phenomenon you allude to - of screen-addled zombification - is really turbo-charging my own misanthropy. People staggering around, necks hunched, eyes down, all but glued to their miserable little toys. Everywhere, everyone, all the time. It's just pathetic. I guess I had hoped humans would have more self-control than this.
Stop viewing them in isolation and view them as a product of their environment. They weren't born with a phone in hand, someone gave it to them and someone created Tik Tok for them.
That's a fair argument. It's also unfalsifiable and based on an underlying personal worldview. Specifically (I would venture) an "us and them" view of things where history is determined by groups and power - a left-wing outlook, basically! I'm a bit of a liberal individualist by nature, I see personal responsibility and autonomy as a thing. I'm not sure how I'd go about deprogramming myself of this even if I wanted to. But it would help with the misanthropy, for sure.
Ticktoks and Phones do not exist without a creator. Buck stops with the software dev and exec.
I got screwed, I had to pay quite a few hundred dollars with a 2 year contract with ATT and I waited in line at 6AM for my first smart phone.
Even today, I doubt I could get anyone to just give me a smartphone.
I’ve always had this like 70% formed idea about Plex and how it’s indicative of how people want to self host more than we realize, but I’ve never quite been able to articulate what I’m thinking here and what the larger implications are.
Plex is obviously not true self hosting, but it’s a lot closer to it than a Netflix subscription, and the number of people who I do not consider very tech savvy who have not only been joining other people servers but trying to set up their own is staggering lately. And they’re not simply doing it because they want free movies or something. A lot of them have done it for the same reason I initially started: their kids.
I am concerned about the media that is put in front of my kids. I care about what shows they are watching. Kids are going to get their hands on screens there almost is no getting around it, so I would rather not trust YouTube et al with deciding what my kids do and don’t see. I can’t realistically be there to catch literally everything they watch, but if they’re using my server I know they only have access to a certain Library at all times so I can rest a lot easier. In a lot of ways I imagine this is how our parents felt when we were kids. On cable television growing up there were only so many “weird” or troubling things that could pop up, definitely nothing as extreme as we see today, and you could be reasonably aware of what most of those things were and know what channels to forbid/what times your kids should not have free access to the TV.
I found a lot of other parents feel the same way here. They’re just tired of feeling like the Internet is such an incredibly hostile place and want to find ways to take a little power back into their own hands.
I don’t know hopefully something useful popped up in that rant above. I have a lot of disjointed thoughts about this I really haven’t been able to bring together.
Yup that’s why I started self-hosting, when my wife got pregnant and we started to think about what technology access for our future kids would look like.
I started with CasaOS and Jellyfin. Quickly outgrew Casa and moved to learning Docker and setting up my own container stack, moving from media self-hosting to adding new containers of stuff like budgeting apps. I’m still working on building out my server but every container I add, the goal is basically to self-host a version of something I’m doing on a centralized service on the web and ultimately take my data and privacy back.
I will say some peoples’ elitist attitudes about stuff can be annoying and discouraging; it’s the same general spillover attitude from the Linux supremacy crowd. When I started with Casa I had someone basically tell me I was wasting my time and if I wasn’t running everything in VMs why bother. Which is entirely the opposite attitude to get “normies” and low technical literacy people on board, they need easy one-click install solutions like CasaOS. And if they decide to move onto something more complex, well I’m sure they can figure out how to reimage and rebuild their server in ProxMox or Docker as part of that.
Ha we basically had the same journey though you are certainly further along than I am.
Definitely agree about the elitist attitude problem. The amount of people who dunk on people for using Plex when I think it’s a fantastic jumping off point for true self hosting…it’s just so unnecessary and becomes a missed opportunity.
Glad to see a mention to Opera Unite. I found it to be a really revolutionary idea, anyone could have a simple static website running in their browser with zero tech knowledge needed. I think the world would have been better if that idea succeeded as a way for people to share their content, rather than the highly monetized and manipulative social networks.
> Rather than being in countless separate places on the internet in the hands of whomever it had been resold to, your data is in one place, controlled by you.
I don't see how this follows. The moment you create/share data with a site, what's to prevent them from reselling it?
The only thing this seems to attempt to solve is portability/interop (and moving control of and responsibility for blocking/moderation/spam to users rather than sites).
I don't see how it helps at all with privacy or you "controlling" who gets your data. If you give it to site A but not data collector B, what's preventing A from selling it to B? As far as I can tell, the situation will remain identical to how it is today.
Your data will never be in one place unless you never share it. The moment you use it with other sites or services, it is stored there too, out of your control.
Nothing is preventing it, but 3rd party operates on a copy. You are still owner of the data and it is on one place which makes it easier for you to access it, share it, backup it, analyze it. So, this doesn't prevent reselling in general but prevents data locking. From there, I guess its not that hard to demonstrate which 3rd party sold your data and sue them. It also mandates nonproprietary data formats.
All that is much, much better than what we have now.
> From there, I guess its not that hard to demonstrate which 3rd party sold your data and sue them.
But it doesn't? Obviously every site's TOS will say that by providing them with your data they can use it for all sorts of purposes. If you sued, you'd lose.
And you're generally going to want to make your data available to the various services requesting it, because otherwise most people won't see your posts and comments on their preferred platform.
Your site, the source of the data, could also include a TOS. Plenty of working examples in the commercial world where licensees are allowed to use data but not compile or resell it.
So I actually started researching this, and it turns out that, by the principles in Field v. Google Inc. (2006), neither side would have an enforceable TOS, but that a user making their social data available on their host and not attempting to block any particular crawlers with robots.txt would create an implied license that would allow social media sites to store and display the posts. Which is what allows Google to display information, snippets, and images from sites it crawls.
Facebook couldn't enforce a TOS because the hosting user had never gone to facebook.com and created an account, so the user never agreed to a contract. But a user couldn't enforce a TOS either because the crawling was automated, so Facebook wouldn't be agreeing to a contract either. But Facebook would be allowed to use the data because that's what a user is inviting by making it publicly available to crawlers and not doing anything to restrict access to Facebook.
> The moment you create/share data with a site, what's to prevent them from reselling it?
If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it.
EU legislation shows that we can actually have the right to demand that a company forgets about us. Asserting such rights become easier the more accurately we define what data is ours.
> If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it.
Can you? A site's TOS will say that by sharing your data, you grant them the right to display, reuse and redistribute it, the same as you do now. And that would take precedence because your host provided the data. They requested and you provided.
The only thing that would change that is actual legislation. But then the legislation is orthogonal to personal data storage. If you want legislation for that, pursue legislation for that. Personal data storage is completely separate, and the two shouldn't be confused with each other.
The right granted by the TOS elapses when you cancel the respective service, or when you revoke your consent (in which case the service provider may possibly cancel the service). (Some TOS are also simply illegal to begin with.) That’s what the GP is referring to.
No they don't. I don't know where you've gotten that information, but none of it is correct.
I mean, a TOS could be written that way. But they're generally not, because companies don't want to self-impose limits like that.
The TOS usually has something like "grant the platform a perpetual, worldwide, royalty-free, non-exclusive license to host, display, distribute, modify, and otherwise use that content in connection with the service".
See the word "perpetual"? That's standard.
A TOS cannot override https://gdpr-info.eu/art-17-gdpr/.
And the GDPR doesn't apply outside the EU.
It sounded to me like you were making a general statement about TOS's.
current data points are much more valuable than historical data points, so storing old data doesn't have much incentives
also by having ability to enable/disable access to your data, you have the power of who gets what and for which purpose
also reselling of your data should become illegal to start with, would you be OKAY if your lawyer sells your data? or your colorectal surgeon? off course not, we have laws in place for that, and same laws should be applied to whoever handles your personal data
> current data points are much more valuable than historical data points, so storing old data doesn't have much incentives
Not true -- advertising profiles are vastly more valuable when based on a lifetime of data.
> also by having ability to enable/disable access to your data, you have the power of who gets what
But realistically, when are you ever going to disable access? If you want people to be able to read your replies no matter what social network they're using, you're going to make those replies available to every social network.
> and for which purpose also reselling of your data should become illegal to start with
This is my point. The solution here is legal, not technological. Personal data storage doesn't change anything legally, and changing the law would prevent reselling even if you didn't have personal data storage.
It seems important not to confuse the two, in order not to give people false hopes.
I agree that this is not just a technological problem to be solved. Technology by it self can't fix the problems, but it can help nudge the human experience in good or bad way. Right now, we gave our data to large corporations and we got the lovely attention economy thats being feed on human rage, envy and greed.
Solid idea is more in line with revolution and demand for our representatives to give their people internet that can push the humanity forward, and not just let us waist countless hours on doom scrolling.
When I was a kid, a 4GB pendrive was a huge thing for me. I used to think my 40GB HDD would never fill up, but then Internet started to grow. Today it doesn’t even matter how muc storage you have it’ll always fill up.
I have started to self host quite a lot of stuff but eve then every storage solution has a life of 5-6 years in which atleast one of the components would fail. We click enormous amounts of photos but they do not have any impact like printed photo albums. With ever growing storage costs (both cloud based and self hosted) I’m thinking of going back to keep only important stuff that too in print format.
> ever growing storage costs (both cloud based and self hosted)
That’s not my experience at all.
We still print photo albums. I can strongly recommend this!
In the age of abundance, smart prioritization is needed.
I run a NAS, in various forms, for almost 20 years. The lifetime is quite longer, I still have ~ 10 year old drives in the backup NAS built on a Ryzen 1600 (8 years) and the average power supply works for me 10-12 years. The primary NAS is still on hardware that is more than 5 years old, except the drives that I just replaced with higher capacity.
As I find the size of current drives bigger than my yearly additions (personal pictures and movies), I am quite happy with a 10 year lifetime at low usage. I would love some reliable and affordable long term offline storage, but backup tapes and a reader are not affordable and not in common use for end users. Otherwise I would build a tiered storage system with more reliability and even performance (nvme hot tier? maybe).
In terms of NAS, I have long wonder if there is a market for a combination of both online and offline. We will need at least 2 HDD for redundancy and to prevent bit riot. And the NAS will be sold as a whole package and subscription, with an encrypted backup services included for first 2 years and requires the backup subscription to work there after. The profit margin is first on the hardware and then on long tail backup which is charged like iCloud and Google storage per tier. Where your 1.5TB storage will be charged at 2TB storage.
Before 2014 I would have thought Apple to potentially take this route for Time Capsule. Instead they doubled down on iCloud. Google will never take this route. Microsoft is not interested. Amazon should have done this and bundled with cold storage back up but their track record are not good enough. I doubt people trust Meta enough even if the solution was perfect.
In pre 2012 you could at least bet on Apple to be somewhat customer centric.
May be UniFi will do it. They just announced their 2 Bay UNAS and I only just discovered, they are a 40B market cap company. ( I thought they were much smaller )
> for redundancy and to prevent bit riot
What are you doing to your hard drives that the bits are rioting?
BTRFS / ZFS.
You both wrote "bit riot" but meant "bit rot", right?
I've been running a RAIDZ2 NAS (with ECC RAM) for like 5 years with no data loss/corruption issues. Are you saying if it was just regular RAIDZ there would be data integrity issues?
Synology sells cloud backup services for their NASes. And a bunch of other brands at least can easily connect to other services.
>with an encrypted backup services included for first 2 years and requires the backup subscription to work there after.
Its confusing if you mean the NAS will stop working if you stop paying for the subscription or not. If you can no longer access your data on the NAS without a subscription, then the NAS just becomes the cloud with an extra up front cost plus the cost of your own electricity.
Personally I have started moving as much of my data out of the cloud as possible. I've got a Synology and a few single board computers running various services with a Synology in my parent's home for their photos. Their photos back up to my NAS and my data to their Synology.
Its a shame Synology decided to enshitify this year for all products going forward, but UGreen looks like a suitable replacement when I outgrow my current NAS.
>> And the NAS will be sold as a whole package and subscription...
Misses the point entirely.
Data will need Backup to be safe. You could tell everyday customer to get NAS and they wouldn't know what is Bit Riot until they saw their Image and Video with errors or broken. They also wouldn't do off site backup. Company wants long subscription model.
Right now everyone is only talking about options that are extreme in both ends.
Systems Twin Intelligence, where a Pod represents the full space-time information for part of the world, using Solid Protocol: https://graphmetrix.com/trinpod-server
The W3C Linked Web Storage (LWS) working group is transforming Solid into a web standard: https://www.w3.org/groups/wg/lws/
Bluesky’s PDS is currently fairly limited due to the lack of support for private data and inadequate permissions [1]. Hopefully they’ll fix that soon.
[1] https://bsky.app/profile/byarielm.fyi/post/3lz4vzzhybk2b
I like the convenience of the cloud. but don't know whether its due to declining literacy rates / awareness etc. the cloud is nice and e.g google storage, iCloud but now with fast microsd's you can buy 1TB for $100. have a few copies then boom, you own your own data. but now phones don't allow you to have microsd's so here we are.
likewise things like email etc instead of all of us being on gmail we could have community email servers etc.
I use Dropbox, but with an encryption overlay that also integrates into the iOS Files app for ease of use on mobile. So it’s possible to use cloud storage and still keep your data private.
Sony phones continue to have MicroSD slots, headphone jacks, AND remain water resistant. They have been that way for at least a decade.
The fact that the AT Protocol relies on everyone having a domain name, which is a centralized system over which few people have control, and about whose workings most people have no clue about, is problematic. Also impractical, once we consider that - as far as I can understand - 8 billion people should have their own domain name.
What's impractical about everyone having a domain name? It surely isn't due to lack of domain names, because foo.bar.baz.bim.bim.bap.com is a valid domain name.
It is true that full data sovereignty isn't something most people are interested in, but this is more about a cooperative model for data ownership and access. Having your data identifier be JackDaniels@yahoo.com isn't particularly different from it being jackdaniels.is.technically.bourbon.com. In both cases another organization owns some of the path to your identifier and could potentially lock you out of it. In both cases, verizon is near the top of that list (.com).
As far as the domain name system being centralized, I'm not sure I agree. DNS is like a feudal system with hundreds of kings (top level domains) who all work together with one pope (ICANN), and various lords and ladies occupying positions under those kings. If ICANN goes completely bonkers the kings can get a new pope, some of them are literally sovereign because they are nation states. Just for fun, some of those states are ruled by literal kings, too. There are experiments to run a TLD by Decentralized Autonomous Organization (DAO), but I think for the most part nobody really cares because the current system happens to work pretty OK. If you have an idea for a more decentralized way to organize a namespace that doesn't involve your grandmother typing in a massive UUID or onion address, and doesn't result in someone being able to domain squat literally everything; I would love to hear about it.
Small point but
> foo.bar.baz.bim.bim.bap.com
is owned by the owner of bap.com, under the current system.
Ownership is probably the wrong word since the legal grant is term limited contract for exclusive use under terms of service. Selling subdomain usage grants (also under contract and TOS) feels quite similar.
Top level domains can change pricing, terms, or cease operation. Freenom is a great case study, as they previously operated TLDs. At the edges, a well-operated subdomain service could offer stronger ownership-like behavior than a top level domain.
> The fact that the AT Protocol relies on everyone having a domain name
Well, either that or someone else hosting their identity (see did:plc), which seems to be the part you say should exist?
Probably DNS is the most decentralized centralized system we have available today that most people can actually use, unless I'm missing some obviously better way of doing the same thing?
The thing your missing is ICANN is headquartered in the US. The US political situation is dire and I think this could be a real danger for the internet at large. We might end up with disagreeing DNS worldwide at some point. E.g. if you hold a domain and have a non-authorized viewpoint so your DNS entry gets snuffed.
But from a practical point of view a decentralised system should not rely on domain name ownership. Any computer can generate a private/public key pair, which is all you need for identify.
> Any computer can generate a private/public key pair, which is all you need for identify.
Right, but once you've generated those, then what? You need a global registry of sorts so people can lookup each others keys for example, which is why DNS kind of is the best we have available today.
I don't think there is any perfect solution here, but it's hard to come up with something that has better trade-offs than DNS. Sure, ICANN might be based in the US, but so far DNS been relatively safe to rely on, and if ends up not reliable in the future, I'm not sure social media profiles is the biggest worry at that point.
> Well, either that or someone else hosting their identity (see did:plc)
Wouldn't that turn into did:plc:facebook all over again?
If there was no way of moving away from it, probably yeah. But since you can migrate from a did:plc to did:web, I don't feel like they're very similar situations at all.
It doesn’t really rely absolutely on domain names; at the very root there’s just a DID. DNS happens to be the best we’ve got right now as a human-readable username and address in-one goes.
We can work to make DNS /ICANN et.al. more democratically operated and people-owned while at the same time devising wholly alternate paradigms like Handshake and similar: https://blog.webb.page/2025-08-21-dap-the-handshake-successo...
There are around 10^99 different possible domain name labels (the part between the dots), so I don’t quite see the impracticality. Even going the route of Reddit’s autogenerated usernames like Eloquent-Salad9443.net would be viable.
> 8 billion people should have their own domain name
That is something that could be feathered in gradually -- your country, region, city, neighbourhood, etc could have their own domains, and you could be anon237@milan.italy or whatever, until you find it necessary or inspiring to obtain your own domain.
With did:plc, you don’t have to have your own domain, if you are willing to delegate some responsibility.
But what is the alternative. Systems that bind identity to the phone number give even less control. Systems that use a self generated cryptographic key (like Scuttlebutt) are even less practical.
DNS is not perfect but I think the best we have for now.
I love this idea, and I imagine with years of successful lobbying efforts we could potentially get some laws passed to provide rights and clarity around our own data that could move us into this direction. But until then, while BlueSky is solid, I'll wait and see.
For those of us who've been around for some time and still value privacy, this sort of paradigm is obvious.
The trouble isn't a lack of the right technologies - I'd argue it's a problem in the go-to-market strategy of those building these products/technologies.
Ideas flow along lines carved out by power/influence. Facebook's early strategy was to start with restricting its usage to people at Harvard University - arguably a highly influential institution - and then expand outwards to other highly influential institutions. Only once the "who's who" from those institutions were already onboard did they let down the walls to allow us plebs in, and we all rushed in head-first.
X's current strategy leverages Musk's visibility and influence (for better or worse).
Get the most prominent influencers onboard with your decentralized social network, and others will follow (dramatically easier said than done, of course). But without a significant contingent of influencers/powerful people, your network's DoA.
> prominent influencers onboard with your decentralized social network
That's sort of a contradiction, no? Or at least it assumes transplanting the same mechanisms into a new milieu -- which I argue is something to leave behind, because it's those very mechanisms that have ruined the current internet.
I think instead of tapping into the same addictive attention economy schemes, the distributed / decentralized socials could onboard people en-masse by providing what's missing there, and filling a real need.
As in self hosting? I love self hosting idea for myself out of principles.
But unforunately it will never take off in a huge way because convenience is king. Average Joe and Jane want to install things with as little efforts as possible.
You can self host, but in order to be reachable you need to be discoverable. If the discovery is based on a mechanism that is controlled by someone else that can become an evil party, self-hosting in isolation is not too useful.
This is never going to happen.
The incentives do not make sense.
Any utopian future that requires a party to put in a lot of effort to change something in a way that would be a net negative for them, is just not going to happen.
People do not spend money to change the world in a way that would be worse for them but better for other people.
> The incentives do not make sense
Commercial incentives, no. If this preference exists, it would need to be pursued civically.
I don't think the average citizen cares enough or even understands the benefits
But lets say you get them on board and pass some law. Unless its a huge market like the EU or USA, probably what immediately happens is everyone pulls out of that market. Not out of malice but because they suddenly have to rewrite their app and that's probably quite expensive.
you store ur photos on fb same way you store your money at the bank and your code on github, its delegation of concerns, you can make same argument for literally anything....not using your own silicon, growing your own food, financing your own venture, owning your own land, etc etc.... maybe its more "secure" vs "less efficient" or some other tradeoff. and you have to get the right balance or take risks for optimal efficiency / profit/whatever your values are
How do I, as a complete noob, use the powers of atproto and the fact I own a domain?
Isn't this what web3 was about? Was it the wrong approach?
When I read the title I couldn't help but think "did everyone forgot about hard disks?"
I'm sure Tim Berners-Lee is much smarter than me, but I kind of feel there are some parallels between the idea of "owning" posts you made in a platform and the ludicrous idea of "owning" game items as NFTs in a blockchain. The latter promises interoperability that games would never deliver. I wonder about the former.
At least I feel the major dealbreaker with this technology is just that it's not worth it for both parties involved.
Right now, Facebook hosts all the posts and monetizes them with ads. So long as they are making money with ads, they have no reason to delete the posts they're hosting, as the posts are their money maker.
But what happens if Facebook no longer "owns" the posts?
So now your posts are in your "personal cloud", which means that unless they are encrypted any website or local app can display them, even without any ads. This means Facebook is no longer making money off the posts. Why would they accept this?
On the flip side, who is paying for the hosting? Facebook? It's no longer their servers hosting the content, so I don't think so? Is Facebook supposed to pay the cloud service for metered API access? Can a cloud service offer different rates to different companies? Is the user supposed to pay for their cloud storage? So you're going to make users pay money to use facebook?
What happens if a post violates the ToS? Can facebook delete my post in my cloud storage against my will? What happens if content that is legal where facebook operates is illegal where the cloud servers operate?
Can I manually edit the data in my cloud storage like I'd be able with a file and then facebook has to treat every post as if it were untrusted input?
What happens if my cloud storage closes my account? I just lose everything? Will I be able to back up my cloud to my hard disk and reupload it to another cloud so facebook can access it? How is facebook going to handle a single user with 2 clouds that have different content?
I feel like this is a very complex thing and there are infinite questions that we can have about how this would be implemented in practice, while it's presented as simply "you own your data."
If this takes off I fear big tech very quickly finding friends among those pushing for things like chat control, while potentially reevaluating some of its more consumer friendly "views" towards privacy. Very easy to undermine something when you start speaking of its potential to facilitate CSAM.
that is exactly what is going to happen, as more people become aware.
that's why we all need to exercise our rights and freedoms. I'm scared that if we fail to do this in next few years. And let the AI be used in similar ways like it has been used to create social media algorithms. Then we are all fucked!
Whoever owns your AI owns you, so it better be you who owns it!
This guy has eyes and eyes can be used to visualize CSAM! What if...
The problem isnt technical feasibility it is market incentives.
Most companies have no incentive to let you hold your data when they can just hold it for you.
If they do this they can mine it for data to improve their product as well as sell or otherwise indirectly profit from it. And, it's easier.
Also, while the market for privacy focused products isnt nothing, the number of people willing to pay a lot extra to compensate for the missed opportunities companies get by collecting your data is, i think, smaller than many people imagine. Which is sad.
I think the only way it will grow to an appreciable size is by seeing up close and personal what a really vicious stasi-like secret police does with dragnet surveillance and come out the other side, with scars. I believe we've only seen a small taste of this.
> The problem isnt technical feasibility it is market incentives.
This is understating it honestly.
The software industry has become completely reliant on renting data access back to users to maintain subscription revenue. One effect of this is it has devalued the actual software in the eyes of users to such a degree that virtually no one will pay for alternatives, certainly not enough to compensate the development cost.
You got the market incentives wrong.
Most people have no incentive of owning their data. Otherwise the companies which don't give you that would die out because people wouldn't use them if they cared.
Same fallacy as believing smartphones are giant and with non-user swappable batteries because somehow smartphone making companies are forcing this on the market, instead of the real reason which is that it's what consumers want.
I don't think it's so black-and-white. There are multiple forces at play simultaneously.
I agree that people don't care enough about owning their data for it to matter more than what the companies want to push, which is of course monetizing the data and maximizing user lock-in.
Similarly, I think it's in the companies' interests to use non-swappable batteries: simpler and cheaper to manufacture (I think this is the main reason) and the device is made obsolete earlier which is an added bonus. Maybe small improvements in size etc., but that's a very small difference. Modern phones are already larger even with non-swappable batteries so I'm not sure it mattered. But again, having a non-swappable battery has to be weighed against other features, and availability of alternatives. In the end, people just care more about the other features, even though swappable battery would be a good thing.
Just to conclude: I don't believe markets work to fully cater to what customers actually want. It's more like customers (and other parties) get a compromise between what different parties in the market want.
> the real reason which is that it's what consumers want
Consumers want what they're told to want by a constant barrage of commercial propaganda.
Devices are large and non-serviceable because this way they can be sold with a higher profit margin. Side effect being that the larger screens make the embedded commercial propaganda more effective and easy to deliver.
I get what you're saying.
People want vendor lock in...otherwise they wouldnt pay for it.
People want bait and switch sales tactics...otherwise they wouldnt work.
People are perfectly fine with high rents...if they didnt, they would not pay them.
People want their smartphones to be deliberately slowed down when they get old...otherwise theyd vote against it with their wallet.
> Whether these providers are strictly cooperatives in the formal sense isn't what's most important here though;
I think the context of “encouraging people to switch” to a pds/solid/data coop, how they operate IS important. For two reasons:
- data coop and controlling data opens the door to a new market if we’re going to join data coops, then we may as well try to share the profits from said coop fairly. Otherwise Facebook can step in as a “data-coop” and keep-on-keeping-on
- a secondary effect is that now there is an incentive to move off facebook. If I can join my local Nowheresville.USA.town data coop and benefit directly to my community by storing data together then I am encouraged to switch to this new paradigm
That is the major undiscussed shift to me. I believe the only way out of the Big Tech dystopia is to incentivize the switch. Even if the reward is pennies. Invest in the community oil well.
Ideas like the Solid protocol have a limited timeframe to make it or go away. Not sure why anyone is still talking about it. TBL is rightfully a legend but this is now just a windmill.
Next, please.
This comment has inspired me to target SOLID and “things I can do to help” on my Sunday afternoon research block. This type of commentary is rife in this article thread and is now just a windmill.
Next, please.
If Schneier can’t get more than 13 comments on a solid protocol crypto wallet, I personally don’t think that anyone will ever care about a solid protocol app of any kind. And I’m all for it, just calling it as I personally see it.
Some things are fire, some things are warm, and some things are DOA.
And I’m typing this on my Linux desktop (f’real).
https://www.schneier.com/blog/archives/2024/07/data-wallets-...
A Solid protocol cryptowallet. Arcane on top of arcane.
I think it’s entirely unfair to dismiss technology because it hasn’t demanded immediate adoption by society. Solid is attempting to help define a better data future. We have working mechanisms in place but everyone is at a disadvantage except the people loyal to these giant corps. Attempting to give people the power to organize their data as they wish and to be used as they wish is worth it. Even if it doesn’t bring a renaissance.
Crypto wallets are not nearly as arcane as Solid. How many people have Binance accounts?
Market share matters, critical mass matters, adoption matters. I'm suggesting that mindshare goes negative over time if these things aren't achieved, and when you have long-tail blog posts trying to pump life into it, it's pivot time.
Righteousness alone doesn't win any of those things. It's been a very long time since Solid was released and it's like a whisper in the wind.
I do not have a Binance account and think cryptocurrency is a terrible starting point for a Solid application. Not even people who buy cryptocurrency care about where their cryptocurrency is held.
Arguably it hasn’t taken off because no one has incentivized using it.
Here is the Solid website list of apps:
https://solidproject.org/apps
> Meanwhile - Nothing changes, everything generally gets worse
https://LMNO.lol is my grain of sand.
I wasn't happy the state of blogging (tracking, bloat, ads, paywalls...), so I built https://LMNO.lol. It's offline first and you can browse blogs from anywhere (even terminal). Your blog is a single Markdown file. Drag and drop it to the browser and your entire blog is generated.
Custom domains are welcome. My blog is running off LMNO.lol that https://xenodium.com
I don't see what advantage any company gets from choosing to build products that enable personal data ownership. I say this as someone working on a venture with these sorts of design aims, it feels like pushing a boulder uphill often.
The business model of cloud service providers makes a lot of sense- we have a system which stores and operates on your data, you pay some rental fee for us to store it and operate on it, easy peasy. The cost is related to both the utility of the operations the operator performs (to both the operator and the user) and the amount of data the user stores.
Fundamentally this is how everything from Dropbox to Facebook is governed- Dropbox does not devise much utility per GB and users store a lot, so you rent per GB, but at Facebook, they don't store lots of your stuff, and on the data side maybe you don't get much value from it as it's a cesspit, but the data is valuable to Facebook to sell ads, etc, so they can provide the service for free.
Importantly, you don't need to improve the product to continue extracting this rent, because the product you are selling is not Dropbox v4, Facebook v2.3, rather you are selling ongoing access to the rental.
As soon as you introduce even simply a federated system where a few corporate operators are involved, it becomes very hard to justify extracting rent there as the network designer, as the operators are taking on the cost of actually storing the data. You have to really be iterating on the core product to use a SaaS business model here. Some things simply don't need a v4, does Dropbox really need that much iteration?
Meanwhile as the system designer, life has become a lot more complex for you. Suddenly you cannot push unilateral sweeping changes to APIs, you need to version things in a way that is compatible between, say, one university updating their system but not the other. Since your users are a few large operators rather than millions of individuals, you lose the network effect advantage of being able to screw over a few users for the "greater good", since if you irritate one corporate client, you lose a lot of your install base. Why would you voluntarily choose this harder path as a company?
Things get even worse as you increase the level of decentralization. The reality is users expect the polished experience that the rental companies can give you; they want their data always accessible so that their friend can see the pic they shared without needing to keep their own computers running, they want the "like counter" to go up without their personal node subscribing to messages from other nodes, etc. The only users that will accept a worse experience are people who have are motivated by their philosophy re: personal data ownership, and this crowd will want a FOSS solution, so you can say goodbye to charging them for Dropbox v4, they are simply not interested if you're not giving them the source code for free. (I suspect this is where the author sits, but fundamentally I don't think it will get mass appeal, most people simply do not care about data ownership above something that "just works".)
So now you are dealing with problems like dynamic generation of redundant data and fault- and Byzantine-tolerant consensus algorithms so that your system can maintain function even when the user turns their computer off, and you have to deal with wrapped-key cryptography so that the redundant data can be split across all these user nodes without you worrying that an unauthorized user can read it, and then you have issues like how do you deal with nodes that are too slow to process updates (perhaps some user data needs to be stored in this conflict-free replicated datatype you devise), and eventually you go through all of this to... create a system that is less monetizable than the rental model, because you can't extract that rent for ongoing data storage, and we know users are not interested in actually paying for software.
This article seems pretty far detached from the problems that people experience using technology. It’s the kind of thing that only deeply technical people consider.
When someone uses a service like Dropbox or iCloud Drive or Google Drive, they really aren’t experiencing any kind of problem where their data “isn’t theirs” or is “trapped.” It’s not that hard to migrate to something else and the services themselves are reasonably low-friction.
In terms of social data, users don’t really have a major issue with the status quo, and those who do have already developed relatively popular solutions like Mastodon and BlueSky.
Even “proprietary” photos applications like Apple Photos and Google Photos have very easy migration paths to other services.
So what exactly is the problem we’re trying to solve here? Giving me an @Bob handle? Did I want that or need that?
> In terms of social data, users don’t really have a major issue with the status quo
That's exactly it. And with social media (unlike files and photo storage) migration isn't really something people care about, because it's about the present not the past.
If you move from Twitter to Bluesky, does anyone care about moving their tweet history? They just want their list of followers to migrate over as much as possible, which happens relatively organically anyways.
I find the ideas of data coops to be very appealing. I don't want to depend on faceless mega-corps like Google to host stuff like my email, but I also don't find the idea self-hosting to be realistic. I wouldn't mind paying for the security since losing access to certain accounts would be a disaster, but I'm already locked in, and the benefits of existing services would be marginal compared to the cost of moving.
ideally you should be able in a simple way to host your stuff, in this case in a POD. That service should be provided by a utility company, same way we have internet providers now. They will be well regulated and it would be in their interest to safely hold your data because if not, they would face legal and financial consequences.
All other services would read/write from your Pod.
Who has an incentive to provide a Solid server? Not big social media companies, who want the personal information that Solid attempts to withhold. I don't think anyone is prepared to offer a convenient, high quality Solid-based social media experience to everyone for free, because that costs a lot of money. And if you know anything about human nature, it will have to be convenient and completely free in order to have a chance of capturing any mindshare outside of weird tech nerd circles.
> the platforms should be asking us what kinds of data they may copy from our servers, and only with strictly temporary allowances.
Until practical homomorphic encryption arrives, I don't see how this temporariness can be enforced. If we rely on promises or regulation instead of the technical ability to enforce this, how is that any better than today's social media companies promising not to do anything bad with the data they have on us?
See this response: https://news.ycombinator.com/item?id=45480884
Aka: I agree it can’t be dine with technology; it has to be done with regulation, and the EU example already models a lot of it.
'that costs a lot of money'
price of intelligence is dropping day by day like it or not, sooner or later price incentives for someone to host such social media experience could become financially viable
How do I post a message on Discord/Twitter/Instagram from my personal data storage? If this is not supported, this idea is born-dead. Very few will use it, for the regular person the conversation goes like this:
- Who can see my personal data storage posts? Can someone with Twitter see them?
- No, but you'll own your data
- Bye
So maybe start with something which backs-up what you post on Twitter/Instagram/Discord to your personal data storage through APIs/data export.... This has no downside if it's easy to "activate"
At this point distributed protocols are getting good enough that for a large class of social applications, network effects are the only thing keeping the incumbents in place.
The irony of ad supported free services is that if you just let the advertisers pay you directly for eyeball time then paid for your services, it'd be better for you financially while keeping the web pure outside of the "paid to consume ads" app.
You just wait. The closed services will close down or become hostile enough that people will migrate. Not everyone will, but over a longer period - enough.
People getting into Solid and ATproto today are like people using own XMPP servers decades ago, or Mastodon years ago, or Matrix. Some projects like that will succeed, others will fade. But one day, you won't be able to post to Discord due to some policy changes and you'll have to reevaluate options.
Also, you can't backup from Twitter anymore. Or Discord. Or google photos. Or many others - they cut off that option once they're big enough.
> You just wait. The closed services will close down or become hostile enough that people will migrate.
I've been waiting a long time. Over that time, the closed services have only gotten more popular and no regular person is ever complaining that they are "hostile".
Regular people don't like ads, but they dislike paying even more, so they're pretty OK with the status quo. They certainly don't want to be paying for a domain name and paying for hosting.
The creator/consumer divide is still 90/10. Your example just doesn't matter.
I think you got the ratio backwards, but assuming that then your argument serves to bolster GP's position.
If I don't create anything, and just consume creators, what do I need a personal data store for?
Just your existence itself already create a lot of data ;)
you just created a comment here.
also your government, your service providers and many other entities are creating data on your behalf
I’m continuing to explore ideas like this in my DN project (short for DownloadNet or Discernet). The core concept: a browser controller / instrumentation harness that, by default, saves everything you browse to disk, and makes it available via full-text search or a browsable alphabetical index.
The browser controller actually runs its own local server that handles indexing and archiving on your disk, while the front end lives inside your browser as a dashboard or control pane. So it’s both a locally hosted app and a browser extension of sorts.
This is still a work in progress, but one direction I want to push further is allowing users to publish curated collections or search indexes of their browsing history.
More likely, though, you’d create a separate archive centered on a topic you care about, and as you browse you selectively add pages to that topic. Over time, you end up with a niche search engine tied to your expertise.
If that archive is good, others might find it valuable—and you might choose to publish it from your own machine. With tunneling tech (Cloudflare, Tor, etc.), you can expose your local box to the public internet. The vision is: user-sovereign data, but still shareable.
You could even federate groups of topic-based archives into a shared search ecosystem, useful for domains like biotech or other specialized fields.
Another crucial point: DownloadNet archives your browsing in real time. It doesn’t crawl externally; it captures exactly what you see, including sites you access via institutional credentials (e.g. research journals behind paywalls). Then you can optionally share those archives with a trusted group.
I’m also exploring a web-document bundle format: package an interactive set of web pages (not just one) into a self-contained snapshot you can send (e.g. via email). The recipient can browse that snapshot locally, with all internal links intact, as of a particular moment in time. It’s a simple but powerful idea, and I think it has real growth potential in the data-sovereignty space. I started this as a passion project, and I believe many others care deeply about these ideas too. If you’re interested or want to get involved, head to the repository.
One way my vision differs from something like Solid is the philosophy of adoption: rather than launching with a full-blown protocol, you start with a simple tool that users adopt, extend, and share. Over time, emergent use cases and community practices shape the system. It’s bottom-up rather than top-down.
I’m not dissing Solid — I understand its aims and don’t see this as strictly competitive or exclusive. But I feel the incremental, user-led route is likelier to produce something sustainable. You grow it in the wild, learn what users actually need, and adapt. Instead of trying to design for all cases in advance, you let real-world use teach you what matters.
Anyway, that’s the gist of my vision—and how it diverges from other approaches like the one in the article you referenced. While it may seem as a condemnation of other ideas, it's not. So please don't take it that way.
If this is something you could get into, I encourage you come on over to the repo and share your contribution. I also riff more on Solid, this article and the approach of DN if you're interested, here: https://github.com/DO-SAY-GO/dn/wiki/What-is-DiskerNet-and-h...
> Another spiritually similar idea being championed at the time came from the Opera browser folks who wanted to put "a web server in your browser".
Opera Unite was such an awesome idea. https://arstechnica.com/information-technology/2009/06/opera...
There was a neat idea a bit back to allow Service Workers to work across origin: foreign fetch. It wasn't on the internet, was only in the scope of your browser, but I thought it was such a neat advancement. Would have done so much to allow the offline web to weave itself. Alas, deprecated. https://developer.chrome.com/blog/foreign-fetch
Aka, more dunking on "the cloud". Now it's cool to be able to do so.
How about we go back 20yr and train a generation of unix sysadmins and self host at companies and at home.
IPFS and Filecoin exist to solve this problem.
https://ipfs.tech https://filecoin.io
thought the same.