Consuming the Bluesky firehose for less than $2.50/mo

(bsky.bad-example.com)

187 points | by colinprince 5 hours ago ago

44 comments

  • bradgessler 3 hours ago

    Two reasons to like BlueSky:

    1. Your username can be your website. I'm @bradgessler.com.

    2. Copy and pasting images into a message works on iOS (this has been broken on X/Twitter for a long time).

    There's more reasons to like it, but these two feel pretty great.

    I put a Ruby hacker starter pack together if you're looking for a way in:

    https://go.bsky.app/HXB2cPh

    There's tons of other ones and communities as well.

    • LeoPanthera 3 hours ago

      > Your username can be your website. I'm @bradgessler.com.

      But more importantly, you have to verify that you own the domain. I can't sign up as google.com

    • throwawaycities an hour ago

      This is the reason I initially got on the waitlist to join Bluesky.

      11 years ago I made a social media aggregator where users had to have a verified domain to join - think using your domain as username and using you domain to authenticate your social media profiles. My show HN: https://news.ycombinator.com/item?id=6529523

      This probably sounds silly but in the end I didn’t actually use Bluesky because they don’t support emoji domains.

      • extraduder_ire an hour ago

        Did you try using the emoji domain as-is, or did you use the punycode version? e.g. <US flag>.com vs xn--w77hd.com

        Because I can imagine them not implementing support to limit lookalike codepoints being used in handles, or they just hadn't gotten around to supporting punycode in their domain verifier.

        • throwawaycities 36 minutes ago

          I attempted both the emoji character and punycode.

          Emojis are tricky, most domain registrars can’t get them right, most browsers can’t get at least some right (usually the ones with ZWJ characters.

          So by no means a knock on Bluesky, but I was waiting for an invite code for so long just to be able to use an emoji as my identity.

          Not directed at you, but not sure why my initial comment was downvoted, it’s pretty damn hard for new social media platforms to get users from other platforms - and as domain identity/usernames is a main competitive feature and I was super excited then didn’t end up using it, sharing my experience might be worth more than downvoting

    • Vaslo 3 hours ago

      It’s going to go the way of Threads and Mastadon. Lukewarm results at best, but glad you are enjoying it.

      • nmfisher an hour ago

        I jumped on Bluesky, Mastodon and Threads during earlier exodi. They all waned for various reasons (Mastodon was slow and unreliable, Threads was basically an Instagram feed of nothingness, and Bluesky was dead at the time).

        This wave does feel very different though. People joining in large numbers (I gained more followers in a day than in a year on Mastodon). The Bluesky app itself is actually very nice. Twitter is becoming more and more unbearable with the force-feeding of ads and totally broken search (that's without even mentioning the post-election environment).

        You might be right, but I do feel like this time is different. Let's check back in 12 months and see.

        @nickfisherau.bsky.social if anyone wants to say hello.

      • bradgessler 3 hours ago

        Perhaps, but I haven’t seen this level of momentum and consensus behind a text-based social network since early Twitter.

        Time will tell—ultimately it’s a self-fulfilling prophecy. If people “believe” and sign up, it will happen.

  • monksy 4 hours ago

    Ooo we're back in the old good days of twitter again! The time where you could build against it and do things.

    I wrote a contest winning bot back at one tie:

    https://theexceptioncatcher.com/blog/2015/11/how-i-got-my-bo...

  • evenw 4 hours ago

    The bsky firehose is really fun and feels like a misty-eyed throwback to 2005.

    I used it last weekend to throw together this very silly little thing: https://bigmood.blue/

    • cyberlimerence 3 hours ago

      I like the concept, but it is flashing way too fast. Maybe do a random selection every 1 sec, rather than a straight fire hose ?

    • zamadatix 3 hours ago

      Very nifty, I like it quite a bit.

      I wonder if instead of black+repeating images (assumption here is that's the display and not an artifact of the firehose, the same unique image often repeats a lot in the same short instance) it'd be cleaner to have some sort of exponential backoff queue (the closer it is to empty the slower it empties like 0.5s/numInQueue, last element in queue will wait no less than 0.5s but also wait for the next element to queue). This would make the flow a bit more consistent, never blank, and never repeating.

      Also if images kept their aspect ratio.

    • esperent 3 hours ago

      Is there any way to make it flash less? I'm not epileptic, and I appreciate that you put a warning first, but it's still highly unpleasant after around 2 seconds. If I could look at it for a bit long it would be a cool experiment.

      • h0l0cube 3 hours ago

        +1

        Would be good if it just tiled the images, and then wrapped around after the screen is filled

    • lelandfe 4 hours ago

      I've got the perfect soundtrack for it (flashing lights warning) https://www.youtube.com/watch?v=wNtxgxYY7sI&t=66s

  • ks2048 4 hours ago

    This is cool and a big reason to root for BlueSky over X, all politics aside.

    I wonder if they (or some third party) offers zip files of dumps of various kinds? It seems that would go easier on thier servers.

  • 101008 13 minutes ago

    I have an important account on Twitter where I publish manually posts, thoughts, links, announcement, etc. I tried to look for a way to crosspost to Threads and Bluesky automatically, but I couldn't find anything. Any suggestions?

  • kylehotchkiss 4 hours ago

    It was fun playing with Twitters firehose way back when that was accessible to the common developer. Excited to see the idea make a comeback!

  • caesil 4 hours ago

    Wow this is neat!

    Stark contrast to the stupid locked-down new API rules twitter rolled out post-Elon.

  • smeeger 18 minutes ago

    decentralized social media is the achilles heel of democrats. they just keep throwing themselves at it and it never works and never will work. while democrats work feverishly on this concept, X continues to grow daily. i would put money on this if polymarket hadnt just been the victim of a politically motivated FBI raid…

  • tonetegeatinst 4 hours ago

    I sense this becoming training data for some AI/LLM in the near future.

    • hieu 3 hours ago

      Only two outcomes: either everyone can access it or only few big companies can. While not ideal, the former is much better.

      • llm_trw an hour ago

        It really isn't when you realise everyone includes people who sells home burglary as a service as scale.

    • paxys 2 hours ago

      Better that everyone have access to the data rather than just a couple large corporations.

    • BobbyJo 4 hours ago

      TBF, a walk down the street where cctv cameras are present is likely already training data for some AI.

    • mewse-hn 3 hours ago

      I think researchers want vintage data thats not contaminated with llm vomit

      • NewJazz 3 hours ago

        You mean uncurated 100 character blurbs of plausible sounding words isn't quick diddle making sense to the time of the hour?

  • brrrrrm 4 hours ago

    your single word firehose demo is awesome. I can see a lot of creative people doing really cool things with this

  • donatj 4 hours ago

    I'm still irate Twitter broke all my automaton and then had the gal to try to charge me $100 a month!? Looks like it's up to $200 now.

    That's as much as my car loan... No normal individual can justify that.

  • foundart 3 hours ago

    Great opening line!

    > It's fun to play with data[citation needed]

  • Onavo 3 hours ago

    What language are you using for the backend server? 256MB is pretty tight unless you are using something like Go.

    • Retr0id 3 hours ago

      I was consuming the full authenticated firehose (not jetstream) with tens of megabytes of RAM with python+sqlite (unfortunately I ran out of bandwidth on my cheapo VPS, I've yet to refactor to use jetstream instead)

  • hobs 4 hours ago

    I dont understand why I would build on bluesky when it seems like it would have the same medium term issues that the twitter api had anyway, but enjoy it while it lasts I guess.

    • marviel 3 hours ago

      say more? I thought the protocol was designed to be able to handle decentralization

      • Zamiel_Snawley 3 hours ago

        There is still major centralization. Essentially, the relays crawl the network and provide an aggregated data stream, their firehose.

        Independent relays can exist, which is better than the fully centralized options.

        There is no financial incentive to bear the cost of running a public relay.

        • marviel 2 hours ago

          Thank you for the explanation!

          Are there efforts to build a similar type of network which rewards running a public relay? I'm not sure how, for instance, Mastodon solves this problem

          • Zamiel_Snawley 2 hours ago

            Mastodon has a completely different architecture.

            In Activity Pub each node is essentially the full stack of what ATProto breaks up into individual entities. This means that each node in the fediverse is incented to connect to other nodes in the network, to provide good coverage to their users.

      • koolala 3 hours ago

        So a firehose isn't different than re-hosting Bluesky and it encourages re-hosting?

        • marviel 3 hours ago

          If you're critiquing, could you spell out your point more plainly?

  • jeffbee 39 minutes ago

    I mean, the punch line of this post is that the "firehose" of even a successful social network is trivial to handle. Even Twitter at its pinnacle of popularity barely breached ~10000 posts/second intraday peaks. If you only want to observe and transiently store such traffic you barely even need a computer.