A key problem is that compilation operates on an implicit (compiler writers have this in the back of their heads) notion of correctness which is very roughly “preservation of observable behaviors” where “observable” is sequences of system calls and then return value. That is, the final output of a compiler should never add new sequences of observable behaviors.
Security properties on the other hand are very often about the relationship between these sequences. For example we like to say that an external observer/attacker can’t distinguish internal state by the external observations (confidentiality) which requires that two observable traces given different hidden values don’t have different observations from the same starting observable state.
If you’re an LTS nerd you know this difference as trace properties vs hyper properties. Compilers try to preserve the former but not the latter.
Separately there is the problem of “what is observable?” For example, if you include timing in your observable behaviors suddenly the kinds of compiler passes that are able to preserve observations tends to zero rather quickly.
The compiler is allowed to transform your code if it can prove that the result would interact with the outside world in exactly the same way as your original code would, right? You can optimize on "I already checked the value of X" if you can prove that nothing could have changed X.
Well, it sounds like a lot of compilers are making unjustified assumptions about what the outside world is allowed to affect or observe. Maybe with the encouragement of specs, maybe not.
I don't think that's the problem, and I think your "if it can prove..." isn't really accurate.
The compile can transform your code if the result of the computations it makes is the same (plus any ordering guarantees you've encoded with the correct primitives, etc.). "Interact with the outside world in exactly the same way" is way too strong a guarantee.
The canonical example is the constant-time comparison. You want to compare a provided password hash with the one in your database in such a way that every comparison, regardless of success or failure, completes in exactly the same amount of time. The compiler does not care about this desire of yours, though, and can and will try to optimize things so it will stop the comparison as soon as it knows they don't match, which can take different amounts of time depending on the input. This is perfectly valid and reasonable, but breaks security sometimes.
Another example is to allocate some memory, write to it, and free it, without reading it. The compiler is free to optimize the entire thing away. We have `volatile` in C because sometimes merely writing to a memory address has side-effects that aren't visible to the compiler, but if you don't use it, the compiler can do what it wants.
> The compile can transform your code if the result of the computations it makes is the same (plus any ordering guarantees you've encoded with the correct primitives, etc.). "Interact with the outside world in exactly the same way" is way too strong a guarantee.
The word "result" is doing a lot of work there. `printf ("%d\n", 2+2);` isn't interesting because 4 appears in a memory cell; it's interesting because 4 appears on stdout. Which one is the "result"?
If you're going to make assumptions about what's "inside" the program and what's "outside", you have to make them explicit. And they have to be reasonable assumptions. An assumption that memory is "inside" has to be justified in the presence of shared memory, virtual memory, debuggers, or whatever. You have to actually explain what you mean in a lot more detail than I think the average spec has a chance of doing.
If I create an unlinked temp file, and the compiler can observe that I'm holding the only FD open on that file, should that file be seen as "inside the computation", or as "a collection of results and inputs"?
Without reading the specs, I can be 95 percent sure that they don't nail down all the issues... and 100 percent sure that if they do nail down all the issues, or even all the possibly important issues, the corner cases are unknown to almost all actual programmers. Which means either that it's not appropriate for the compiler to rely on just any rule regardless of what the spec says, or that it's not reasonable to write code in the language.
If you want to write code for a VAX, then you can use the K&R C compiler where it had defined outcomes for everything. If you want to write portable C code for modern CPUs then it's fair to ask what the C language standard is supposed to define for each of those CPUs and OSes and ABIs.
And a bunch of people were nice enough to do that for you and I, but because they are not deities, there are things that they had to leave out to make the language useful, so they did.
Not at all the same. C and C++ are full of hazards and I get the impression it’s genuinely difficult to avoid entirely in normal code bases, and typically impossible to avoid statically. Whereas in Rust it’s all gated behind the unsafe keyword, and if you don’t use it (and most code bases never need to use it), you cannot encounter UB; and that scoping makes it far easier to control and handle correctly.
Sure Rust is better on it, my point is that it is there, despite that community making it a Big Freakin' Deal to have all the memory safe they could design into the compiler and language.
If even they had to add escape hatches despite the presences of powerful language primitives like types, traits, borrow-checking, maybe the people charged with making it all work with 80s compiler technology weren't the literal Antichrist for also having UB as Rust does.
For what it's worth, C and C++ are much different in terms of hazard, so when you bucket them together it makes me wonder how familiar you are with the actual risk of UB in practice.
Nowadays it generally stems from doing weird things.... but no one is making you do weird things, any more than people are making you use Rust's unsafe keyword.
I think this is a bit exaggerated. I mostly find it not difficult to avoid UB in C. There are mainly five areas where you can have problems: type safety issues, signed integer overflow, out-of-bounds accesses, use-after-free, and race conditions. Type safety is generally not a problem if you avoid unsafe casts (and casts are easy to screen for just like "unsafe"), signed overflow one can protect against via sanitizers or one can rule it out statically, and out-of-bounds accesses you can avoid by using safe buffer and string abstractions and never doing open-coded pointer arithmetic.
Use-after-free and race conditions are the areas where Rust has a clear advantage. Here one needs to have a clear strategy and enforce it manually (or using tools, but we lack good open-source tools for this). Valgrind and similar tools also help.
As someone who coded C++ (15+ years) and later Rust (about 4 years now) for my dayjob: there are more than those you listed I have seen commonly. Unaligned accesses is a perrenial favourite, as is ODR violations and reliance on the undefined order of static constructors between translation units. In C++ I didn't see much of unsafe casts, except related to enums (always specify an underlying type to mitigate this).
Rust protects against all of these, but if you think Rust is only about memory safety, I don't believe you have seriously tried it. It does a lot of things in std API design as well to steer you away from bugs. Some examples:
- The pervasive use of Result and Option makes it impossible to forget to handle (or forward) the error case.
- Because of usage of RAII (C++ has this too, but not as pervasively, C doesn't except using some very new GCC extension) it is very hard to forget to free resources such as files, sockets, database connections, mutexes, etc.
- Enums can carry payload in their variants (C devs: think tagged unions, but safe, C++ devs: think std::variant but with match/case rather than bulky visitor pattern), which means you can make API designs that cannot represent invalid states.
- The typestate pattern is a bit hard to explain briefly, but it allows a state machine with types at compile time, to make sure you dont misuse an API. For example it can be used to prevent forgetting setting required fields in a builder before building. Or in embedded microcontrollers to make sure you can't hand out the same GPIO pin to different parts of the code base.
I often find that my code in Rust works first try, while that almost never happen in C++ for non-trivial code. It is what all those Haskell devs were talking about all these years, but in a systems language (no GC is critical to my day job in hard realtime control systems) and without the incomprehensible abstract math lingo.
I was only speaking for C not C++ (I fled C++ a long time ago). My code usually works first try in C, but my experience also working with students is that you need to learn to use safe patterns and strategies first. I can imagine that Rust enforces those.
You can do a lot more in C too: You can design safe interfaces based around incomplete structure types. This also should allows what you call typestate pattern (if I understand it correctly). You can build a decent option type / result type. You can have a bounds safe vector type. You can have safe string types. One can have type-safe dynamic casts. One can annotate return values so that they can't be ignored. One can use many different tools for safety. People coming from C++ often think that one can not do this in C because "it lacks abstractions", but this is not really true.
It has been a long time since I coded C. Last I did I remember hating C strings and the standard functions for dealing with them. So easy to get buffer overflows. I'd rather have the language design be so that I dont constantly have to think about not tripping over various things, instead I want to focus on the hard and interesting domain specific problems.
EDIT: Also, errno is an awful design. Forgetting to check for errors, or screw up which error you report is so easy in C. Exceptions in C++ are also bad, it is very easy to have no idea what exceptions are possible 5 layers down and end up with unhandled exceptions.
True, if you do pointer arithmetic on C strings or similar low-level buffer operations without introducing safe abstractions, there is basically no way to do this safely.
And I think that illustrates my point well. Yes there is MISRA C++ and CERT for C when you write safety critical code. But that is a lot of extra rules to follow and remember (and have linting tools check where possible). It is basically a entirely separate dialect of the parent languages. And if you aren't doing safety critical you won't be dealing with these but have to come up with your own (company specific or individual) rules. (You dont want to write MISRA C++ unless you have to, large parts of it are quite miserable).
In Rust I get good defaults, and a language that guides me in the right direction. The rules for safety critical rules are somewhat still under development but so far they look a lot shorter (you still need the "don't allocate in hard realtime tasks except at startup" and similar rules for example). And if you aren't doing safety critical you can safely use all of the language as long as you stay away from unsafe.
And for most code you dont need unsafe, and even when you do someone else has likely done the hard work for you already, providing safe abstractions on top. (The exception is FFI to other languages, it is impossible to avoid unsafe when calling code in another language that the compiler can't reason about, you should build a safe Rust API on top of the raw bindings. For popular libraries this often already exists.)
And my point is that "do not do low-level pointer arithmetic" is not really much harder to follow in practice than do not use "unsafe". In safety critical systems you may also care about panics, memory leaks, etc. I am not sure this is so simply in Rust as well.
That you do not get safe libraries out-of-the-box in C is a major problem. But I also see the supply chain situation in the Rust world as highly problematic.
Two counterpoints: Searching for "unsafe" is a lot easier when you want to audit the code (there is even a lint you can enable to forbid all unsafe in a crate (library)). And there is a pervasive culture to avoid unsafe where possible and document all the unsafe you do have explaining why it is in fact ok.
There are also experiments in formalizing the safety comments with attributes. To me the current prototypes look halfway towards formal verification, with unsafe functions specifying named requirements that must be upheld when calling them and the callsites needing to "discharge" them by name. Time will tell if this is a good idea for general code and it it catches on.
Almost but not quite. You are promised that safe Rust doesn't have Undefined Behaviour, but that's cultural, not technological. The technology is just enabling Rust's culture to deliver what they promised, but it would be very easy to purposefully (indeed there are known bugs where it does happen, look for "Rust soundness bug" if you want examples) inject UB which happens in your safe Rust.
The extent to which this is about culture should not be underestimated, to me that's the most hilarious part of Bjarne Stroustrup's big rant on memory safety a few years ago. The C word appears exactly once in his slides, in a quote from somebody else about what needs fixing. But Bjarne never addresses this once, even though it's the actual problem.
> I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you.
They did. Most other languages, the vast majority of which are also memory safe languages, go out of their way to do the opposite, and give meaning even to erroneous programs. Some things slip through the cracks, and generally language designers and implementers work hard to get rid of UB.
C/C++ is the only ecosystem that has fully embraced UB as a way of life. They are the only compilers that make full use of "UB is bad and cannot ever happen" as a core tenet in how optimizations are designed. Rust UB is at least a little different. Rust UB can only be the result of unsafe code and is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster.
C/C++ have a surprisingly large set of UB, too. Thankfully, the rest of the software world is rising up and the committees are starting to make things like gasp signed arithmetic overflow into defined behavior.
For, C we have already removed at lot of UB from the working draft for C2y. My hope is that only the UB is left that is difficult to remove without requiring extensive changes to code or compilers, and that this can then be addressed by a technical specification that defines a safe subset of C.
But note that UB also does not necessarily mean your program has no meaning. A good compiler can do something reasonable by defining the UB. There is no mandate in the specification that a compiler has to break things. It is also the user's responsibility to put pressure on compiler developers to do something reasonable.
> Rust UB […] is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster.
This isn't true. Rust UB is meant to only be possible to trigger via `unsafe` code, but if you do trigger it, the compiler is free to do whatever it wants, and in practice it will make full use of that freedom. Rustc uses LLVM, it shares most of its optimizations with Clang!
If you don't want UB to mean "Absolutely anything might happen" which necessarily has to include "... forever" then you need Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead.
To some extent in C and even more in C++ there's a much worse problem, IFNDR [Ill-formed No Diagnostic Required]. Programs which the language specification insists mean nothing at all, but your tools won't (in many cases can't) notice so the result might do anything. It's not Undefined Behaviour, your program never had any defined behaviour at all.
> Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead
It’s worth being clear here that this is not what Fil-C does, it still has UB, and can still explode in many of the same ways as C and all (after all, it’s a clang fork). Fil-C takes one particular class of allocation related bugs and UB off the table, but leaves many of them behind.
Whatever happens for the other UB remains bounded memory safely in Fil-C. (according to a definition of memory safety that excludes protection of subobject bounds, but Rust also redefines memory safety to what the Rust compiler can do, e.g. excludes memory leaks).
This is important. Undefined Behaviour is a behaviour, and so if we can ensure the behaviour doesn't happen, our problem is averted. For example if there's UB when a zero size file is loaded by our software, we can instruct operators to check the file has a non-zero size and we're preventing whatever horrible UB would arise.
We can even rope off whole parts of the software. If the Postscript printing code has UB, simply instruction operators only to use the HP inkjet printers for which we know Postscript is not used can prevent this UB from happening.
I don't think this is true. You're going to have to point to the exact place in the spec that says this, because optimizations in the presence of UB in most compilers make absolutely no assumptions.
undefined behavior: "behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this document imposes no requirements"
The C++ spec at some point changed this to explicitly allow changing anything in the program not just the specific behavior implied by the "for which". The C spec never did this.
Because people were confused about this, in C23 we added the following note. "Note 3 to entry: Any other behavior during execution of a program is only affected as a direct consequence of the concrete behavior that occurs when encountering the erroneous or non-portable program construct or data. In particular, all observable behavior (5.1.2.4) appears as specified in this document when it happens before an operation with undefined behavior in the execution of the program."
Compilers mostly follow this. GCC has bugs related to volatile. Clang often follows the C++ standard, where it is different from C, so probably does not conform to the standard here (as for some other things).
The new C++ standard will have UB "barrier", i.e. std::observable that will limit the effect of UB to things before this barrier.
Thanks for pointing to this. I'm surprised the specification verbiage differs so much between the two standards. However I think the language for C is aspirational and there are cases where compiler optimizations will fail this.
There are a few rare cases where compiler optimization will fail this in GCC. There are also cases where the C spec has defined behavior that optimizers break. Where I know about them, I file bugs. It would be up for users to insist that those get fixed. This includes optimization issues that affect Rust as well.
The compilers are mostly doing the right thing (not always). The C standard specifies what is considered an effect on the outside world, i.e. file I/O and volatile accesses. For concurrent programming, there is also a memory model that specifies what other threads can see.
Here, the issue seems that compilers can reload variables. If this is a bug, then you already have a data race in your program which you can prevent with correct use of locks and/or atomics.
Sort of. Its idea of “the outside world” is very restrictive. Simple example: write into a pointer, then free it. From the compiler’s perspective, this write can’t be observed and can be removed. It’s not legal to read memory after it’s been freed, so there’s no legal side effect from that write. But in reality, we can use a dangling pointer or a debugger and read that memory just fine.
The compiler does everything that the language spec allows it to get away with. The language spec itself is a communication protocol between users and compilers, and that includes the definition of the abstract machine.
Regarding compiler deleting memset, there are multiple memory-zeroing functions that are guaranteed not to be optimized away:
https://cppreference.com/c/string/byte/memset (memset_s)
https://cppreference.com/cpp/string/byte/memset (memset_explicit)
Also see notes section for various other functions that people were using before memset_s became standard.
A key problem is that compilation operates on an implicit (compiler writers have this in the back of their heads) notion of correctness which is very roughly “preservation of observable behaviors” where “observable” is sequences of system calls and then return value. That is, the final output of a compiler should never add new sequences of observable behaviors.
Security properties on the other hand are very often about the relationship between these sequences. For example we like to say that an external observer/attacker can’t distinguish internal state by the external observations (confidentiality) which requires that two observable traces given different hidden values don’t have different observations from the same starting observable state.
If you’re an LTS nerd you know this difference as trace properties vs hyper properties. Compilers try to preserve the former but not the latter.
Separately there is the problem of “what is observable?” For example, if you include timing in your observable behaviors suddenly the kinds of compiler passes that are able to preserve observations tends to zero rather quickly.
Title should be "Your C compiler can undo your security checks".
Any compiler can do it -- LLVM definitely does even for languages that aren't C
weird f*ing site. apparently doesn't like hn visitors and doesn't want me opening the console. i know when im not wanted
> doesn't want me opening the console
Whoa. That's not even something it should be able to detect. What's the gaping security hole that lets it do that?
The compiler is allowed to transform your code if it can prove that the result would interact with the outside world in exactly the same way as your original code would, right? You can optimize on "I already checked the value of X" if you can prove that nothing could have changed X.
Well, it sounds like a lot of compilers are making unjustified assumptions about what the outside world is allowed to affect or observe. Maybe with the encouragement of specs, maybe not.
I don't think that's the problem, and I think your "if it can prove..." isn't really accurate.
The compile can transform your code if the result of the computations it makes is the same (plus any ordering guarantees you've encoded with the correct primitives, etc.). "Interact with the outside world in exactly the same way" is way too strong a guarantee.
The canonical example is the constant-time comparison. You want to compare a provided password hash with the one in your database in such a way that every comparison, regardless of success or failure, completes in exactly the same amount of time. The compiler does not care about this desire of yours, though, and can and will try to optimize things so it will stop the comparison as soon as it knows they don't match, which can take different amounts of time depending on the input. This is perfectly valid and reasonable, but breaks security sometimes.
Another example is to allocate some memory, write to it, and free it, without reading it. The compiler is free to optimize the entire thing away. We have `volatile` in C because sometimes merely writing to a memory address has side-effects that aren't visible to the compiler, but if you don't use it, the compiler can do what it wants.
> The compile can transform your code if the result of the computations it makes is the same (plus any ordering guarantees you've encoded with the correct primitives, etc.). "Interact with the outside world in exactly the same way" is way too strong a guarantee.
The word "result" is doing a lot of work there. `printf ("%d\n", 2+2);` isn't interesting because 4 appears in a memory cell; it's interesting because 4 appears on stdout. Which one is the "result"?
If you're going to make assumptions about what's "inside" the program and what's "outside", you have to make them explicit. And they have to be reasonable assumptions. An assumption that memory is "inside" has to be justified in the presence of shared memory, virtual memory, debuggers, or whatever. You have to actually explain what you mean in a lot more detail than I think the average spec has a chance of doing.
If I create an unlinked temp file, and the compiler can observe that I'm holding the only FD open on that file, should that file be seen as "inside the computation", or as "a collection of results and inputs"?
Without reading the specs, I can be 95 percent sure that they don't nail down all the issues... and 100 percent sure that if they do nail down all the issues, or even all the possibly important issues, the corner cases are unknown to almost all actual programmers. Which means either that it's not appropriate for the compiler to rely on just any rule regardless of what the spec says, or that it's not reasonable to write code in the language.
> The compiler is allowed to transform your code if it can prove that the result would interact with the outside world in exactly the same way
Well, in C/C++, as soon as your program has one UB bug, the compiler has absolutely no obligation whatsoever.
Well yes, that's what UB means. It's a singularity, you run into it and there's no longer a specified requirement for the behavior that will follow.
Rust also has UB, btw, https://doc.rust-lang.org/reference/behavior-considered-unde..., so I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you.
If you want to write code for a VAX, then you can use the K&R C compiler where it had defined outcomes for everything. If you want to write portable C code for modern CPUs then it's fair to ask what the C language standard is supposed to define for each of those CPUs and OSes and ABIs.
And a bunch of people were nice enough to do that for you and I, but because they are not deities, there are things that they had to leave out to make the language useful, so they did.
> Rust also has UB, btw
Not at all the same. C and C++ are full of hazards and I get the impression it’s genuinely difficult to avoid entirely in normal code bases, and typically impossible to avoid statically. Whereas in Rust it’s all gated behind the unsafe keyword, and if you don’t use it (and most code bases never need to use it), you cannot encounter UB; and that scoping makes it far easier to control and handle correctly.
Sure Rust is better on it, my point is that it is there, despite that community making it a Big Freakin' Deal to have all the memory safe they could design into the compiler and language.
If even they had to add escape hatches despite the presences of powerful language primitives like types, traits, borrow-checking, maybe the people charged with making it all work with 80s compiler technology weren't the literal Antichrist for also having UB as Rust does.
For what it's worth, C and C++ are much different in terms of hazard, so when you bucket them together it makes me wonder how familiar you are with the actual risk of UB in practice.
Nowadays it generally stems from doing weird things.... but no one is making you do weird things, any more than people are making you use Rust's unsafe keyword.
I think this is a bit exaggerated. I mostly find it not difficult to avoid UB in C. There are mainly five areas where you can have problems: type safety issues, signed integer overflow, out-of-bounds accesses, use-after-free, and race conditions. Type safety is generally not a problem if you avoid unsafe casts (and casts are easy to screen for just like "unsafe"), signed overflow one can protect against via sanitizers or one can rule it out statically, and out-of-bounds accesses you can avoid by using safe buffer and string abstractions and never doing open-coded pointer arithmetic.
Use-after-free and race conditions are the areas where Rust has a clear advantage. Here one needs to have a clear strategy and enforce it manually (or using tools, but we lack good open-source tools for this). Valgrind and similar tools also help.
As someone who coded C++ (15+ years) and later Rust (about 4 years now) for my dayjob: there are more than those you listed I have seen commonly. Unaligned accesses is a perrenial favourite, as is ODR violations and reliance on the undefined order of static constructors between translation units. In C++ I didn't see much of unsafe casts, except related to enums (always specify an underlying type to mitigate this).
Rust protects against all of these, but if you think Rust is only about memory safety, I don't believe you have seriously tried it. It does a lot of things in std API design as well to steer you away from bugs. Some examples:
- The pervasive use of Result and Option makes it impossible to forget to handle (or forward) the error case.
- Because of usage of RAII (C++ has this too, but not as pervasively, C doesn't except using some very new GCC extension) it is very hard to forget to free resources such as files, sockets, database connections, mutexes, etc.
- Enums can carry payload in their variants (C devs: think tagged unions, but safe, C++ devs: think std::variant but with match/case rather than bulky visitor pattern), which means you can make API designs that cannot represent invalid states.
- The typestate pattern is a bit hard to explain briefly, but it allows a state machine with types at compile time, to make sure you dont misuse an API. For example it can be used to prevent forgetting setting required fields in a builder before building. Or in embedded microcontrollers to make sure you can't hand out the same GPIO pin to different parts of the code base.
I often find that my code in Rust works first try, while that almost never happen in C++ for non-trivial code. It is what all those Haskell devs were talking about all these years, but in a systems language (no GC is critical to my day job in hard realtime control systems) and without the incomprehensible abstract math lingo.
I was only speaking for C not C++ (I fled C++ a long time ago). My code usually works first try in C, but my experience also working with students is that you need to learn to use safe patterns and strategies first. I can imagine that Rust enforces those.
You can do a lot more in C too: You can design safe interfaces based around incomplete structure types. This also should allows what you call typestate pattern (if I understand it correctly). You can build a decent option type / result type. You can have a bounds safe vector type. You can have safe string types. One can have type-safe dynamic casts. One can annotate return values so that they can't be ignored. One can use many different tools for safety. People coming from C++ often think that one can not do this in C because "it lacks abstractions", but this is not really true.
It has been a long time since I coded C. Last I did I remember hating C strings and the standard functions for dealing with them. So easy to get buffer overflows. I'd rather have the language design be so that I dont constantly have to think about not tripping over various things, instead I want to focus on the hard and interesting domain specific problems.
EDIT: Also, errno is an awful design. Forgetting to check for errors, or screw up which error you report is so easy in C. Exceptions in C++ are also bad, it is very easy to have no idea what exceptions are possible 5 layers down and end up with unhandled exceptions.
True, if you do pointer arithmetic on C strings or similar low-level buffer operations without introducing safe abstractions, there is basically no way to do this safely.
And I think that illustrates my point well. Yes there is MISRA C++ and CERT for C when you write safety critical code. But that is a lot of extra rules to follow and remember (and have linting tools check where possible). It is basically a entirely separate dialect of the parent languages. And if you aren't doing safety critical you won't be dealing with these but have to come up with your own (company specific or individual) rules. (You dont want to write MISRA C++ unless you have to, large parts of it are quite miserable).
In Rust I get good defaults, and a language that guides me in the right direction. The rules for safety critical rules are somewhat still under development but so far they look a lot shorter (you still need the "don't allocate in hard realtime tasks except at startup" and similar rules for example). And if you aren't doing safety critical you can safely use all of the language as long as you stay away from unsafe.
And for most code you dont need unsafe, and even when you do someone else has likely done the hard work for you already, providing safe abstractions on top. (The exception is FFI to other languages, it is impossible to avoid unsafe when calling code in another language that the compiler can't reason about, you should build a safe Rust API on top of the raw bindings. For popular libraries this often already exists.)
And my point is that "do not do low-level pointer arithmetic" is not really much harder to follow in practice than do not use "unsafe". In safety critical systems you may also care about panics, memory leaks, etc. I am not sure this is so simply in Rust as well.
That you do not get safe libraries out-of-the-box in C is a major problem. But I also see the supply chain situation in the Rust world as highly problematic.
Two counterpoints: Searching for "unsafe" is a lot easier when you want to audit the code (there is even a lint you can enable to forbid all unsafe in a crate (library)). And there is a pervasive culture to avoid unsafe where possible and document all the unsafe you do have explaining why it is in fact ok.
There are also experiments in formalizing the safety comments with attributes. To me the current prototypes look halfway towards formal verification, with unsafe functions specifying named requirements that must be upheld when calling them and the callsites needing to "discharge" them by name. Time will tell if this is a good idea for general code and it it catches on.
Almost but not quite. You are promised that safe Rust doesn't have Undefined Behaviour, but that's cultural, not technological. The technology is just enabling Rust's culture to deliver what they promised, but it would be very easy to purposefully (indeed there are known bugs where it does happen, look for "Rust soundness bug" if you want examples) inject UB which happens in your safe Rust.
The extent to which this is about culture should not be underestimated, to me that's the most hilarious part of Bjarne Stroustrup's big rant on memory safety a few years ago. The C word appears exactly once in his slides, in a quote from somebody else about what needs fixing. But Bjarne never addresses this once, even though it's the actual problem.
> I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you.
They did. Most other languages, the vast majority of which are also memory safe languages, go out of their way to do the opposite, and give meaning even to erroneous programs. Some things slip through the cracks, and generally language designers and implementers work hard to get rid of UB.
C/C++ is the only ecosystem that has fully embraced UB as a way of life. They are the only compilers that make full use of "UB is bad and cannot ever happen" as a core tenet in how optimizations are designed. Rust UB is at least a little different. Rust UB can only be the result of unsafe code and is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster.
C/C++ have a surprisingly large set of UB, too. Thankfully, the rest of the software world is rising up and the committees are starting to make things like gasp signed arithmetic overflow into defined behavior.
But don't hold your breath.
For, C we have already removed at lot of UB from the working draft for C2y. My hope is that only the UB is left that is difficult to remove without requiring extensive changes to code or compilers, and that this can then be addressed by a technical specification that defines a safe subset of C.
But note that UB also does not necessarily mean your program has no meaning. A good compiler can do something reasonable by defining the UB. There is no mandate in the specification that a compiler has to break things. It is also the user's responsibility to put pressure on compiler developers to do something reasonable.
C++26 and C++29 are in the process of turning a lot of that UB into erroneous behaviour, basically what safer languages have been doing for ages.
However, how many years it will take until those versions become widely deployed across major compilers, and used by developers?
> Rust UB […] is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster.
This isn't true. Rust UB is meant to only be possible to trigger via `unsafe` code, but if you do trigger it, the compiler is free to do whatever it wants, and in practice it will make full use of that freedom. Rustc uses LLVM, it shares most of its optimizations with Clang!
If you don't want UB to mean "Absolutely anything might happen" which necessarily has to include "... forever" then you need Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead.
To some extent in C and even more in C++ there's a much worse problem, IFNDR [Ill-formed No Diagnostic Required]. Programs which the language specification insists mean nothing at all, but your tools won't (in many cases can't) notice so the result might do anything. It's not Undefined Behaviour, your program never had any defined behaviour at all.
> Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead
It’s worth being clear here that this is not what Fil-C does, it still has UB, and can still explode in many of the same ways as C and all (after all, it’s a clang fork). Fil-C takes one particular class of allocation related bugs and UB off the table, but leaves many of them behind.
Whatever happens for the other UB remains bounded memory safely in Fil-C. (according to a definition of memory safety that excludes protection of subobject bounds, but Rust also redefines memory safety to what the Rust compiler can do, e.g. excludes memory leaks).
This is not quite correct in C. ISO C at least requires that observable behavior until this point is preserved.
This is important. Undefined Behaviour is a behaviour, and so if we can ensure the behaviour doesn't happen, our problem is averted. For example if there's UB when a zero size file is loaded by our software, we can instruct operators to check the file has a non-zero size and we're preventing whatever horrible UB would arise.
We can even rope off whole parts of the software. If the Postscript printing code has UB, simply instruction operators only to use the HP inkjet printers for which we know Postscript is not used can prevent this UB from happening.
I don't think this is true. You're going to have to point to the exact place in the spec that says this, because optimizations in the presence of UB in most compilers make absolutely no assumptions.
It follows from the definition of UB:
undefined behavior: "behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this document imposes no requirements" The C++ spec at some point changed this to explicitly allow changing anything in the program not just the specific behavior implied by the "for which". The C spec never did this.
Because people were confused about this, in C23 we added the following note. "Note 3 to entry: Any other behavior during execution of a program is only affected as a direct consequence of the concrete behavior that occurs when encountering the erroneous or non-portable program construct or data. In particular, all observable behavior (5.1.2.4) appears as specified in this document when it happens before an operation with undefined behavior in the execution of the program."
Compilers mostly follow this. GCC has bugs related to volatile. Clang often follows the C++ standard, where it is different from C, so probably does not conform to the standard here (as for some other things).
The new C++ standard will have UB "barrier", i.e. std::observable that will limit the effect of UB to things before this barrier.
Thanks for pointing to this. I'm surprised the specification verbiage differs so much between the two standards. However I think the language for C is aspirational and there are cases where compiler optimizations will fail this.
There are a few rare cases where compiler optimization will fail this in GCC. There are also cases where the C spec has defined behavior that optimizers break. Where I know about them, I file bugs. It would be up for users to insist that those get fixed. This includes optimization issues that affect Rust as well.
Some of those cases have been fixed. For example, Clang used to reorder UB around volatile loads, but the latest version no longer does so.
This is good news! Clang representatives were very basically the only ones against this clarification. Do you have a pointer?
https://llvm.org/docs/LangRef.html#volatile-memory-accesses:
> Volatile operations are permitted to trap. The compiler may not assume that execution will continue after a volatile operation.
The compilers are mostly doing the right thing (not always). The C standard specifies what is considered an effect on the outside world, i.e. file I/O and volatile accesses. For concurrent programming, there is also a memory model that specifies what other threads can see.
Here, the issue seems that compilers can reload variables. If this is a bug, then you already have a data race in your program which you can prevent with correct use of locks and/or atomics.
Sort of. Its idea of “the outside world” is very restrictive. Simple example: write into a pointer, then free it. From the compiler’s perspective, this write can’t be observed and can be removed. It’s not legal to read memory after it’s been freed, so there’s no legal side effect from that write. But in reality, we can use a dangling pointer or a debugger and read that memory just fine.
Compiler does what I tell it. Not the other way around.
The compiler does everything that the language spec allows it to get away with. The language spec itself is a communication protocol between users and compilers, and that includes the definition of the abstract machine.