Yggdrasil Network

(yggdrasil-network.github.io)

213 points | by BSDobelix 13 hours ago ago

77 comments

  • foundry27 8 hours ago

    The first thing I tried to find on their website and their GitHub was a protocol specification, to be able to implement it independently from the reference implementation. I thought this would be straightforward since it’s advertised as a scheme/protocol, but such a spec isn’t referenced anywhere! Digging on my own I eventually found [1] on a side-branch of one of their other GitHub projects.

    Kudos to the author: I think it actually covers a lot of what you’d need to know: crypto identities, message formats, wire protocols, peering and stream semantics, spanning tree updates and root selection, the DHT, forwarding logic, sessions, etc. A couple things are TODOs like how to verify and sign root updates, and there’s some ambiguity in the tiebreaker algorithm for next-hop selection.

    It seems to be very tightly coupled to TCP as the transport layer though, since all packets need to be delivered reliably and in the order they were sent, and need to be capable of being fragmented into smaller packets for varying MTU sizes.

    [1] https://github.com/yggdrasil-network/yggdrasil-specs/blob/ys...

    • neilalexander 7 hours ago

      We did spend a little bit of time documenting the earlier v0.3 protocol, as you have linked, but the protocol has changed significantly in design twice since then. v0.4 changed the DHT quite a bit and v0.5 removed the DHT altogether. As a research project it likely will continue to change until we settle on a design we are happier with, at which point we will definitely spend more time documenting it.

      The need for ordered/reliable links is mostly for convenience of development at this stage, but that can be fixed for sure.

      • Rhapso 6 hours ago

        Look at https://arxiv.org/pdf/1502.06461 if you want to try a chord dht again.

        Kademlia is a lot less intuitive, but by not ever assuming it's tables are correct, it handles and corrects inconsistency (and malicious nodes) better.

        Chapter 6 of this pile of (my) crap https://scholarworks.gsu.edu/cs_diss/106/ talks about doing latency optimization on dht routing. Basically just embedding then network graph into a metric space.

      • godelski 39 minutes ago

        Some documentation can help with those issues though. I find it helps more because you’re writing to yourself why you’re making certain decisions and it helps when you decide to make others. It just so happens that it’s also a great way to onboard people.

    • colordrops 7 hours ago

      Is coupling with TCP a problem? Does it do anything that goes against their goal of full decentralization?

      • macawfish 7 hours ago

        Makes it hard to do hole punching I think? At any rate, direct connections currently cannot be established between multi-hop peers, traffic gets routed through peers instead. I think this has something to do with the TCP choice.

        • foundry27 4 hours ago

          Yeaaah. TCP hole punching is goofy and unreliable, last I checked. You have to do some arcane ritual of having both peers start a three-way handshake to each others’s public endpoints simultaneously, relying on NATs to accept inbound SYN packets if they match the outgoing SYN. And nobody’s NAT devices implement simultaneous-open the same way, so all your connections just fail.

          Naturally this leads to slapping even more arcane fixes on top of that, like NAT port assignment oracles to adversarial interoperate with different port allocation strategies (random, sequential, single, etc.) by analyzing patterns in previous port assignments. Networking sucks.

        • AyyEye 2 hours ago

          > At any rate, direct connections currently cannot be established between multi-hop peers, traffic gets routed through peers instead. I think this has something to do with the TCP choice.

          Yggdrasil is designed for physical links and multi-hop routing first and foremost. Internet peering is just a way to test/use/join the network until then.

        • Karrot_Kream 4 hours ago

          I think this is a pragmatic choice. NAT Hole Punching can be hit or miss no matter the method but doing peer routing guarantees even a client that can only initiate outbound connections can route packets. It can be slow though.

          I also know there's support for other transports like QUIC but TCP is the main default.

  • hahajk 9 hours ago

    Ok, so as I understand it, yggdrasil and cjdns are virtual P2P networks that offer the normal layer 3 routing services, but built on top of the existing internet. So they still require ISPs and internet backbones, etc.

    Are there any projects attempting to build a worldwide P2P network that can replace the IP layer? Like a mesh network that can operate without verizon, cisco routers, etc? I know of some mesh network technologies aimed at small disconnected networks but nothing consumer-facing and supporting anything more than a few thousand nodes.

    • progval 7 hours ago

      That was the original goal of cjdns, which is why it automatically peers with other nodes reachable over Ethernet (no IP needed), including WiFi (see the first paragraph of https://github.com/cjdelisle/cjdns/blob/master/doc/Whitepape...). Unfortunately, its approach to routing turned out to not scale in practice. Yggdrasil uses a different routing algorithm so it might.

      • neilalexander 7 hours ago

        Yggdrasil was actually inspired in part by cjdns but the design is very different. We care deeply about scale and we want an Yggdrasil network to be able to grow organically without requiring a strict hierarchy, huge amounts of state etc. We’re still working on it of course but for now the public test network is somewhere around 5000-6000 nodes and continues to work pretty well as it grows.

    • YesThatTom2 8 hours ago

      IP was originally an overlay network on top of the telco network.

      That has many benefits most importantly it makes adoption easy.

      Now we run telco networks over IP for legacy apps. If this Yggdrasil stuff is successful, I presume eventually we’ll run IP over it for legacy systems.

    • u8080 8 hours ago

      There was cjdroute project with own OpenWRT-based Yggdrasil firmware for routers. But it seems failed to gain traction and died - https://habr.com/ru/companies/cjdns/articles/198428/ [in Russian]

    • bityard 7 hours ago

      People have been dreaming of mesh networks forever. Unfortunately they scale very poorly (among other issues) and this is a fundamental limitation of their design. The Internet (ARPAnet) started out as a mesh network and the concept of trunks, backbones, and routing came about to solve those scaling issues.

    • rapnie 7 hours ago

      Maybe Irdest [0] mesh network.

      > Irdest is a networking research project that explores different technologies and ideas on how to build more sustainable, user-controlled communication networks.

      [0] https://irde.st/

    • Communitivity 8 hours ago

      There was the Locker project by Jeremie Miller (XMPP), but it failed to gain traction and I think he pivoted into a more small scale commercial effort with it IIRC. The telehash protocol of Locker was extremely interesting.

    • ajsnigrutin 8 hours ago

      Why would you want to remove the IP layer?

      Or are you thinking about IP layer, just not on the "internet", but on a separate network? If this, then how do you suggest connecting people together? Mesh becomes innefficient due to mesh routing at larger sizes and sooner or later you just reinvent "your own internet", but not worldwide, because you don't have the resources to actually connect the whole world together.

      • hahajk 7 hours ago

        In order to access the internet you are required to enter into a contract with a corporate entity. That's not because the internet is "theirs" (like Facebook's servers and systems are Meta's), but because the network layer was design with the assumption that companies would do the work of setting up ISPs, core routers, peering agreements, etc.

        I'd like to see a P2P protocol that doesn't assume this but instead is designed to be completely decentralized, and anyone running the protocol can join. This protocol would provide addressing routing like our current IP protocol, and TCP/UDP etc can run on top of it. Would this be a separate "internet" or could it have gateways to the proper internet? Preferably the latter. There are obvious technical challenges with routing, addressing, mobility, all in a decentralized way but are they truly impossible?

        • toast0 4 hours ago

          What you've described sounds a lot like running an AS with BGP. Yes, there's centralized allocation of ASNs and IP ranges. Otherwise, the whole thing is pretty decentralized, but you've got to figure out how to connect to peers and transit providers.

          Many peers will connect without a real contract, especially if you're both present on a peering fabric, but transit usually needs a contract because transit isn't mutually beneficial.

          For some sort of overlay/alternative network, reliable transit seems highly likely to have a cost too. Probably not a contracted cost while it's experimental; and maybe optimistically, much lower than today's costs for IP transit, but still there would be a cost. Actually, IP transit costs are much lower today than years ago, but last mile transport costs are more important to your bill and running wires requires skilled labor and specific capital equipment, so it remains expensive; bandwidth capacity of wiring increases over time, but you still need one connection per home for best service; although wireless seems poised to reduce costs for good enough service in favorable conditions.

        • DanAtC 5 hours ago

          Yggdrasil can do this but you have to bring the physical layer.

        • ajsnigrutin 6 hours ago

          It's not nearly like that, it's designed that you can build your own, separate networks, before the concept of internet providers even existed (and a few colleges and DoD were the only users). It is completely decentralized (with the exception of DNS, which can be decentralized easily, but isn't).

          Setting up networks is easy and cheap. the expensive part is pulling cables and connecting people, that's why many countries have the local governments do that. If you live in an apartment building, you can easily create a separate network for all the apartments. If you want to connect to the next building, you'll need a lot more cables and someone to actually dig in the cable or erect the poles and use those to carry the cable... but who will pay for that? What if you want to connect to the next city over... who will pay for the cabling, digging etc? And of course, the paperwork? Underseas cable? Good luck with that.

          It's not a protocol problem, it's a cost problem.

    • bythreads 8 hours ago

      6lowpan was also a pretty nice attempt at overcoming some of the deficiencies - i think that operated on both lvl 2 and 3

    • dartos 8 hours ago

      There’s meshtastic, but it’s not a full internet stack replacement iirc

      • prurigro 4 hours ago

        It can do tcp/ip, but it's extremely slow. Like 5+ seconds for a character to appear over ssh with a direct connection.

    • prussia 8 hours ago

      reticulum.network perhaps? It certainly fits the "replace the IP layer" requirement, and I believe in theory it can be very large scale, though unsure how it would do in reality.

    • rolph 7 hours ago

      mesh over a starlink like system, but i think he would want a goodly sum for it.

    • fragmede 8 hours ago

      It's a very romantic notion, but there's a lot of resources (time/money/hardware/effort) that go into the existing IP layer that's totally invisible. Without a plan on how to supplant those resources, any replacement network will struggle.

      • lambdaone 8 hours ago

        I think the idea here is that somebody else runs an underlying IP layer, and this rides on top as an overlay network.

        You could, of course, run a local wireless IP layer and use this to route, but peer-to-peer wireless has well-known scaling problems.

        Still, it looks like a very interesting and reasonably well thoughout out idea.

  • coppsilgold an hour ago

    If you want an actual mesh p2p IP-network that can punch through firewalls/NATs you can use Tailscale/Headscale

    If you want a crypto-key addressable p2p connection-network there is a somewhat recent project which does this rather well:

    https://www.iroh.computer

    It punches through firewalls/NATs and establishes QUIC connections.

    They have two already useful PoC's:

    https://github.com/n0-computer/sendme

    https://github.com/n0-computer/dumbpipe

  • dang 4 hours ago

    Related:

    Yggdrasil Network - https://news.ycombinator.com/item?id=41669625 - Sept 2024 (3 comments)

    Yggdrasil P2P mesh E2EE IPv6 network - https://news.ycombinator.com/item?id=30156551 - Jan 2022 (77 comments)

    Yggdrasil – Early-stage implementation of an end-to-end encrypted IPv6 network - https://news.ycombinator.com/item?id=27577201 - June 2021 (102 comments)

    Show HN: Yggdrasil Network – compact mesh routing experiment for mesh networks - https://news.ycombinator.com/item?id=18863554 - Jan 2019 (15 comments)

    Announcing Yggdrasil Network v0.3 - https://news.ycombinator.com/item?id=18751991 - Dec 2018 (3 comments)

    Yggdrasil: End-To-end Encrypted IPv6 Networking - https://news.ycombinator.com/item?id=18666245 - Dec 2018 (1 comment)

  • linsomniac 9 hours ago

    I was pretty excited about it 3-4 years ago, but it seems like it's kind of an abandoned project at this point. Anyone making use of it and have any impressions?

    • neilalexander 7 hours ago

      Definitely not abandoned, but it’s a free-time project for myself and another developer. At the end of last year we released version 0.5 with a new protocol design, and roughly a month ago released 0.5.9 with link cost changes to dramatically improve network latency.

      • linsomniac 7 hours ago

        Thanks for that update, you might want to post a quick blog update because that was where I was looking to see what the activity was. I get it about free-time projects, I have some of those myself. Thanks for your work on this, it is definitely very neat!

        • neilalexander 7 hours ago

          One or two others have also asked for a project update on the blog so I’ll be sure to draft something up soon! :-) Thanks for your interest!

        • PhilippGille 5 hours ago

          Why would you not look at the code repo for checking activity? There are so many active projects without regular blog posts.

    • DanAtC 7 hours ago

      There have been a few updates recently including a revival of the iOS app which had languished for some time.

      I use it as a VPN to connect my phone to my home network which are both peered privately to a VPS.

      It's a bit convoluted vs directly connecting to home, but it was easier to set up than worrying about dynamic IPs, port forwarding, and exchanging Wireguard keys.

      Multicast peering is neat in that I can access my home server directly using the same Ygg IP when I'm home. Problem is, I have to use an IP; the iOS app doesn't support configuring a custom DNS server for the Ygg VPN connection.

      Headscale is really a better solution for this use-case, but it's kind-of neat to know there's an alternative Internet available with just an additional peering.

      • mrbluecoat 4 hours ago

        Agreed. If the Yggdrasil Android and iOS apps supported zero-touch MDM configuration like Tailscale, I'd try it out but my guess is the performance still wouldn't match WireGuard.

        Update: 83% comparitive speed using a US QUIC peer, not bad actually...

      • sunshine-o 6 hours ago

        Using Yggdrasil as a mesh VPN for your devices could be a great use case.

        From a quick search it seems you do not even need a static IP address [0]

        I am not familiar with Yggdrasil and can't wrap my hear around how this is possible !

        - [0] https://lemmy.sdf.org/comment/472679

        • scottyeager 2 hours ago

          To join the larger public Yggdrasil network, you need to peer with at least one publicly reachable node. Most likely that machine has a static IPv4 address. There are a number of such nodes operated by volunteers, and they enable the magic that allows any device to join the network and immediately start receiving inbound traffic from the rest of the network. By opening an outbound connection to the public node, a channel is created for traffic to flow back to the non public node.

    • prurigro 3 hours ago

      I use it all the time to connect to my boxes at home when I'm out and about, and I chat with friends on an IRC server running on there.

      Development is pretty active, and the latest release just improved the routing algorithm by having it favour hops with the lowest latency which had a noticeable improvement.

      If you're looking for a big community hub within the network you might be disappointed (you could always try to set one up!), but there are a lot of people using it for their own purposes and the protect is far from abandoned.

    • evbogue 4 hours ago

      Yggdrasil just works, so there is less of a need for developers to be in the chatroom discussing how to fix the problems with it.

      I use yggdrasil right now on all of my devices so I can ssh between them even if they are behind NAT.

      Using termux on android and the yggdrasil android app I can access files located on my home computer while I'm on the go without storing them in a cloud somewhere.

  • PhilipRoman 9 hours ago

    I really like the idea of address being derived from public key, but there is a problem with this approach - since Yggdrasil currently uses IPv6 addresses, the length is very limited and you can find collisions (there is a workaround which involves brute forcing a key with more leading bits). As I understand, the long term plan is adding a custom protocol which has no limits for address length.

    • neilalexander 7 hours ago

      Truncating the public key to fit in an IPv6 address isn’t totally ideal, agreed, but for now it means that just about any existing IPv6-capable application will work over Yggdrasil without modification, which is a nice property for a testnet.

      • Retr0id 5 hours ago

        What about truncating a hash of the whole public key? (what's what I'd assumed was happening already)

    • Retr0id 8 hours ago

      My napkin math says it'd be plausible to generate a pair of colliding addresses (birthday paradox etc.), but still implausible to collide with the set of existing in-use addresses. How much would the former actually matter, in the context of Yggdrasil?

    • Its_Padar 8 hours ago

      Why not use the entire public key and let entropy do the rest, like the Reticulum Network?

  • Its_Padar 8 hours ago

    Something else in this space includes the Reticulum Network Stack https://reticulum.network/

  • poincaredisk 9 hours ago

    >Yggdrasil is a new experimental compact routing scheme

    Not that new anymore, right? It's at least 6 years old.

    • 1oooqooq 9 hours ago

      is anything using something similar?

      • jeanlucas 9 hours ago

        by the original comment logic, everything that is older than 1 moment is not new anymore

        • poincaredisk 6 hours ago

          I wrote my comment, because I had to recheck if this is the same Yggdrassil I've read about 5 years ago. When I read about a new thing I also wonder will it be more popular in the future, and knowing it's already many years old reduces the chance of explosive growth in the future. At some point things just... stop being new.

          • jeanlucas 2 hours ago

            I didn't mean to be mean to you, but in terms of network protocols something can be a new approach for a while, especially if others are still trying out. What is it compared to? TCP/IP is past 50 years old.

            Then again, I agree with you: it wasn't created yesterday.

  • myspeed 6 hours ago

    Sounds like Teredo tunnels which was part of Windows 7. It builds ipv6 tunnel over ipv4 and assigns a global IPv6 address to Windows machines. But these tunnels were later removed from Windows 10.

  • block_dagger 5 hours ago

    Came in thinking this was an extension for the game Valheim. Different yggdrasil apparently.

    • jeroenhd an hour ago

      Yggdrasill is a name from Norse mythology. It's the tree along which the nine worlds were believed to exist.

  • jcmontx 6 hours ago

    I don't know a lot about networking. Where does this stand in the networking layers? Transport? Network?

  • pjmlp 7 hours ago

    And me thinking it was a Linux distribution.

    https://en.m.wikipedia.org/wiki/Yggdrasil_Linux/GNU/X

  • omani 7 hours ago

    something else in this space includes (New Kind of Network) NKN. (https://nkn.org)

  • varunnrao 6 hours ago

    This is not a technical point but does anyone know which font was used to typeset the logo? It looks really nice and clean.

  • A4ET8a8uTh0 8 hours ago

    FAQ >> Is Yggdrasil anonymous? No, it is not a goal of the Yggdrasil project to provide anonymity.

    I understand that the problem is hard, has its own set of issues to solve beyond just technical, but this honestly makes it a non-starter for me. Anything that would be an actual internet evolution would need to include actual anonymity. Apart from this, I simply do not see what problem it actually solves for the existing internet that is not already solved with the current setup.

    • neilalexander 7 hours ago

      Anonymity isn’t a goal for Yggdrasil anymore than it is a goal for for BGP, OSPF, BATMAN etc. Anonymous networks also generally have very high costs/overheads as they often engineer long and indirect paths for obscurity. See the generally poor performance/reliability of Tor circuits for an example of why we probably wouldn’t want the entire Internet to work this way.

  • epapsiou 5 hours ago

    50 comments and no one mentioned Treeship or Hyperion!!

    • gautamcgoel 3 hours ago

      Yeah, this is what I was thinking! The Templars would be so disappointed...

  • fred_is_fred 9 hours ago

    I get why the name was used but if you start a project that you want to be heavily adopted, please pick a simpler name. The complexity of spelling or pronouncing this for most people creates an actual barrier to adoption. MP3 was easy to say and tell your friends about, Ogg Vorbis was not.

    • opan 9 hours ago

      Ogg Vorbis seems very pronounceable to me, and without an obvious wrong way to say it, using an english language perspective.

      Yggdrasil is a wild one, though, agreed. Better a unique name than another thing called Gemini or Atom or something, though.

      • NemoNobody 7 hours ago

        No, Fred is right - it would better if it was atom or gemini, that's literally what he is saying.

        Yggdrasil - I just had to type the entire word out and even then autocorrect didn't tell me I had a word. I think the Mp3 vs Ogg Vorbis is perfect analogy.

        Tbh, I wouldn't use a Scandinavian language word for a global application as it will automatically frustrate any English as a second language users - the words defy practically all rules of English, they frustrate me even as no matter if I can read them, I often have no idea how to pronounce them unless I've already heard them said.

        This is one of those words I encountered many times before I first heard it said and actually knew how to say it.

        Fred is right 100

        • anotherhue 6 hours ago

          To give a contrary opinion I think it's a beautiful world and an excellent gateway to one of our greatest mythologies. "The World Tree" is an aspect of human literary history.

          I'm not an ESL so I can only imagine the difficulties but I do not think we should be robbing the world of beauty, history and nuance for the sake of business English. Few English speakers can spell or pronounce it correctly so it even becomes a shared difficulty.

          Fun fact: several names of days of the week come from Norse mythology. Look up the names of the months if you want something more modern.

          • F3nd0 4 hours ago

            I have English as my second language and can't think of a single reason why foreign words should frustrate me. On the contrary, I feel like I have an advantage; since English spelling/pronunciation is very messy, coming from a language with more regularity (and just being multilingual in general) probably just makes non-English words feel more natural to me.

            One anecdotal example is the name of ‘GNU’. Somewhat often, I see English speakers on the internet mock the name for being difficult or odd to pronounce, and they usually end up explaining it by writing ‘guh-noo’, which somehow clarifies the matter. To me, ‘GNU’ reads naturally, I find the official explanation ‘like “grew” but with an “n”’ very clear, and I can’t fathom how ‘guh-noo’ can feel more clear or comfortable to anyone, because to me it just looks utterly ridiculous. So for deviating from English, I have a hard time seeing a background in other languages as anything but an advantage.

        • NemoNobody 7 hours ago

          Haha, I just realized I actually have used a Scandinavian word in an app I intended for global use - I just respelled the word so that it made sense in English.

      • majoe 7 hours ago

        The English language has the habit of taking perfectly fine Latin words and pronounce them in the most unintuitive way.

        Gemini is actually a good example, I rather take Yggdrasil.

    • prmoustache 7 hours ago

      I am pretty sure most of the world can figure out how to ponounce yggdrasil much easier than how to pronounce infrastructure or litterature in english.

      https://youtu.be/RpCTu2ymqiM?feature=shared

      • NemoNobody 7 hours ago

        Not once they have familiarity with the language at all.

        One of your examples has a word within a word, so it's like half pronounced if you can say "structure" which I think difficult to mispronounce.

        • poincaredisk 6 hours ago

          Every kid learning English in my country will pronounce "structure" incorrectly at first, because it's similar to a word (with the same meaning) in my native language, and the correct English pronunciation of -ture doesn't make sense. I've looked up Yggdrassil pronunciation and... it's not surprising and I guessed the pronunciation correctly already?

    • askvictor 4 hours ago

      I thought it was an knock-off brand sold on amazon

    • neilalexander 7 hours ago

      The name wouldn’t necessarily stay if we succeed in our goals and formally specify a protocol, but for now it hasn’t really been much of a barrier in terms of interest or experimental deployments.

    • dizhn 7 hours ago

      mp3 is easy to say.. in English. "ogg" is much easier. i.e that was not the reason.