They really should be big-endian, because that’s network byte order.
IMHO it makes sense to use decimal-encoded ASCII digits instead and keep the protocol readable. Nothing like ‘telnet host.example 80’ followed by ‘GET / HTTP/1.0.’
> (1 bit) request_type: integer representing type of request
With two types already defined. No room for future extensions, then. Is the idea to just use another protocol altogether if change is necessary?
I love how the term "endianness" was picked straight from Gulliver's Travels. It's a very good fit - old wise computer wizards must've known how it's gonna be.
I'm not sure. I was confused for years thinking that "big-endian" meant "the big number is at the end". It was only after reading Gulliver's Travels that it clicked.
I'm all for silly names, but I think this one went a little too much into obscure references and metaphors.
That's not a good reason. It's the byte order used for some network protocols, but definitely not all. And given that protocols aren't interchangeable there's no advantage from having all protocols use the same endianness.
Little endian makes way more sense today because all modern computers are little endian. The network itself doesn't care what you use. It's just a stream of bytes to the network.
I know that's the standard but for all private projects I always write little endian so I can sleep soundly at night knowing I've saved precious picoseconds not flipping to cpu order.
Most CPUs suitable to implement such a protocol can do endian swapping for free e.g. a load+bswap instruction, a microarchitecture that fuses load+bswap. Don't worry about byte swapping fixed sized int across the network. Just keep in mind that for some applications it maters e.g. most bignum operations favour little endian, but databases profit from big endian (memcmp sorts in numerical order).
I set out to create a "simpler HTTP" once. Ended up concluding that by the time you've added the basic features, you need most of what a minimal HTTP request gives you. There might be some marginal gains to be made, but it's simply not worth it. HTTP is too good.
Commenting on this proposal directly, I don't see how a stateful protocol could ever be simpler than a subset of HTTP/1.1 with range requests.
Refining the ideas behind pasta, noodles, lasagna to create a simple dish for reliably ending hunger across the world. I came up with the unique name "french fries" all by myself!!1
(this document is a first draft, and is not intended to be implemented in its current form. But I am posting my trivial 5 minutes write-up with a click-bait title to Hackernews, anyway.)
I'd expect the "one" protocol to be able to sync files, especially if they are "large", as advertised. In other words, instead of a case "transfer" implement a universal "sync" to "rule them all".
It's almost as if it makes sense to have more than one protocol, given that there is more than one possible task one might perform on data/objects in computer networks.
Having worked with RTP, I was honestly very confused why anyone would pick this as the one protocol to rule them all. (Especially as most applications need RTCP too, so at best it would be two protocols to rule them all…)
It’s odd that someone would propose a new protocol but not check the IANA port number registry for whether the name is already in use. Surely you’ll eventually want a port for your new protocol when it’s massively successful.
> RTP is commonly used as an abbreviation for the Real-time Transport Protocol, and is not descriptive enough. This is a protocol for reliably downloading large files. it is not designed to be a drop in replacement for http or BitTorrent.
> I am currently drafting a successor proposal that addresses these issues.
I think you could save one round-trip if you included the resource size in OPEN response. It could be optional if you want to support streaming responses.
> good single-source performance (unlike BitTorrent)
Is that true? I've never had use for it, but I've heard of people copying files from one machine to another with it and they seemed to think it worked well.
Well... Sure? Unless I've missed a trick, I don't think it's physically possible to make a protocol that's not bottlenecked on the cumulative upload speed, which in the single-uploader case is the upload speed of that uploader. I would argue that the question is purely a matter of if bittorrent can or cannot saturate a given uplink if there is a single uploader, and I thought the answer is either yes or that it can get really close.
* For the variable length data (uri in open request, error payload, etc) you'll want to specify a length in the message. It allows for more efficient parsing (knowing a buffer size up front) and prevents some security(ish) issues - e.g. i could just send an open followed by uri that follows the scheme but is essentially gibberish until the connection is closed by the server (or the server chokes on it somehow).
You may also want to consider a lenth field for READ responses - in the specified range where the server has all of the resource this is redundant, but if the server doesn't have all the resource, it allows the client to request from another at an adjusted offset even while still receiving the values from the first READ.
* Tokens: If i'm understanding the draft (incl erata) correctly you're using tokens as handles for message pairs, additionally one of the tokens is used to associate the open request and read request, and they are all chosen by the client.
Like this:
Client Server
Open ---------new_token(val1)-->
<---------reuest_token(val1)-- OpenResponse
Read --new_token(val2), open_token(val1) -->
< --- request_token(val2) --- ReadResponse
I'd suggest making the payload of the OPEN request be a server chosen open_token. Having the client manage all of the token values forces the server to track tokens and sender IP or other, similar unique identifying information. It also opens the door for various token collision and/or exhaustion attacks.
* Specify various edge cases well (client closes connection early, connection breaks, etc), because they will have interop consequences, and affect server design (e.g. how tokens are handled in the program) too.
Even if the three letter acronym space is a bit thin, perhaps best to avoid overloading the name of any one of the ten most used protocols out there?
> all integers are little-endian
They really should be big-endian, because that’s network byte order.
IMHO it makes sense to use decimal-encoded ASCII digits instead and keep the protocol readable. Nothing like ‘telnet host.example 80’ followed by ‘GET / HTTP/1.0.’
> (1 bit) request_type: integer representing type of request
With two types already defined. No room for future extensions, then. Is the idea to just use another protocol altogether if change is necessary?
I love how the term "endianness" was picked straight from Gulliver's Travels. It's a very good fit - old wise computer wizards must've known how it's gonna be.
I'm not sure. I was confused for years thinking that "big-endian" meant "the big number is at the end". It was only after reading Gulliver's Travels that it clicked.
I'm all for silly names, but I think this one went a little too much into obscure references and metaphors.
The RFC makes the reference explicit. https://www.rfc-editor.org/ien/ien137.txt
> Nothing like ‘telnet host.example 80’ followed by ‘GET / HTTP/1.0.’
The 80 in port 80 is not ASCII encoded on the wire. That's an UI feature of telnet and/or your OS. (The 1.0 in HTTP is, though.)
> because that's network byte order.
That's not a good reason. It's the byte order used for some network protocols, but definitely not all. And given that protocols aren't interchangeable there's no advantage from having all protocols use the same endianness.
Little endian makes way more sense today because all modern computers are little endian. The network itself doesn't care what you use. It's just a stream of bytes to the network.
I know that's the standard but for all private projects I always write little endian so I can sleep soundly at night knowing I've saved precious picoseconds not flipping to cpu order.
Most CPUs suitable to implement such a protocol can do endian swapping for free e.g. a load+bswap instruction, a microarchitecture that fuses load+bswap. Don't worry about byte swapping fixed sized int across the network. Just keep in mind that for some applications it maters e.g. most bignum operations favour little endian, but databases profit from big endian (memcmp sorts in numerical order).
I set out to create a "simpler HTTP" once. Ended up concluding that by the time you've added the basic features, you need most of what a minimal HTTP request gives you. There might be some marginal gains to be made, but it's simply not worth it. HTTP is too good.
Commenting on this proposal directly, I don't see how a stateful protocol could ever be simpler than a subset of HTTP/1.1 with range requests.
This, HTTP/1.1 already is a simple protocol with the “things you will forget to add in the first version” added.
This is not trying to replace HTTP, it's an attempt at an alternative to "HTTP for big blob downloads". Likewise for the other protocols mentions.
Does it add anything that HTTP can't do with range requests?
I remember hearing that range requests are clunky to implement for HTTP (reverse) proxies, but otherwise they seem to do their job just fine?
I'm guessing range requests are not problematic for proxies, just for caches (which are usually proxies also).
A pure proxy (reverse or not) should have no problem with a range request.
French Fries - one dish to rule them all?
Refining the ideas behind pasta, noodles, lasagna to create a simple dish for reliably ending hunger across the world. I came up with the unique name "french fries" all by myself!!1
(this document is a first draft, and is not intended to be implemented in its current form. But I am posting my trivial 5 minutes write-up with a click-bait title to Hackernews, anyway.)
Open and read should be coalesced. When would you ever open and not read?
And, why not call it RDLF (reliably download large files protocol) instead of RTP? And what does RTP stand for in this case anyway?
> One protocol to rule them all?
I'd expect the "one" protocol to be able to sync files, especially if they are "large", as advertised. In other words, instead of a case "transfer" implement a universal "sync" to "rule them all".
It's almost as if it makes sense to have more than one protocol, given that there is more than one possible task one might perform on data/objects in computer networks.
But there's already something called RTP -- protocol for transferring video and audio...
Having worked with RTP, I was honestly very confused why anyone would pick this as the one protocol to rule them all. (Especially as most applications need RTCP too, so at best it would be two protocols to rule them all…)
It’s odd that someone would propose a new protocol but not check the IANA port number registry for whether the name is already in use. Surely you’ll eventually want a port for your new protocol when it’s massively successful.
From the errata in the article:
> RTP is commonly used as an abbreviation for the Real-time Transport Protocol, and is not descriptive enough. This is a protocol for reliably downloading large files. it is not designed to be a drop in replacement for http or BitTorrent.
> I am currently drafting a successor proposal that addresses these issues.
I think you could save one round-trip if you included the resource size in OPEN response. It could be optional if you want to support streaming responses.
> good single-source performance (unlike BitTorrent)
Is that true? I've never had use for it, but I've heard of people copying files from one machine to another with it and they seemed to think it worked well.
Single source performance is good if the source uploads with the desired download speed for a single leech. This is not always true.
Well... Sure? Unless I've missed a trick, I don't think it's physically possible to make a protocol that's not bottlenecked on the cumulative upload speed, which in the single-uploader case is the upload speed of that uploader. I would argue that the question is purely a matter of if bittorrent can or cannot saturate a given uplink if there is a single uploader, and I thought the answer is either yes or that it can get really close.
To be clear RTP stands for "Reliable Transfer Protocol"?
Some interesting stuff here. I would love to see a numeric table of RTP compared to the other 4 you mention.
Here's my user test: https://news.pub/?try=https://www.youtube.com/embed/0-snH3SN...
That's an awesome user test! Upvoted.
Neat! A couple of suggestions:
* For the variable length data (uri in open request, error payload, etc) you'll want to specify a length in the message. It allows for more efficient parsing (knowing a buffer size up front) and prevents some security(ish) issues - e.g. i could just send an open followed by uri that follows the scheme but is essentially gibberish until the connection is closed by the server (or the server chokes on it somehow).
You may also want to consider a lenth field for READ responses - in the specified range where the server has all of the resource this is redundant, but if the server doesn't have all the resource, it allows the client to request from another at an adjusted offset even while still receiving the values from the first READ.
* Tokens: If i'm understanding the draft (incl erata) correctly you're using tokens as handles for message pairs, additionally one of the tokens is used to associate the open request and read request, and they are all chosen by the client.
Like this:
I'd suggest making the payload of the OPEN request be a server chosen open_token. Having the client manage all of the token values forces the server to track tokens and sender IP or other, similar unique identifying information. It also opens the door for various token collision and/or exhaustion attacks.* Specify various edge cases well (client closes connection early, connection breaks, etc), because they will have interop consequences, and affect server design (e.g. how tokens are handled in the program) too.
xkcd 927 anyone?