>> implementing these features for a smart-pointer type with a malicious or broken Deref (the trait that lets a programmer dereference a value) implementation could break the guarantees Rust relies on to determine when objects can be moved in memory. (…) [In] keeping with Rust's commitment to ensuring safe code cannot cause memory-safety problems, the RFC also requires programmers to use unsafe (specifically, implementing an unsafe marker trait) as a promise that they've read the relevant documentation and are not going to break Pin.

To the uninformed this seems like crossing the very boundary that you wanted Rust to uphold? Yes it’s only an impl Trait but still… I can hear the C devs now. ‘We pinky promise to clean up after our mallocs too!’

You'd be tempted, for performance, to use `get_unchecked` methods that skip the boundary checks. After all, the trait says that this should always succeed.

However, if the user passes in a type that is not indexable with integers smaller than 100, whose fault it is if the program segfaults? The users? But they managed to get the program to segfault _without_ writing unsafe code. The provider of the library? They are using `unsafe` to call `get_unchecked` after all.

Bingo. It's the library dev's fault. The API they provide is not sound. However, they can make it sound by marking the _trait_ unsafe to implement. Then the user needs to type `unsafe` when implementing the trait for their type. "I solemnly swear that this type is actually indexable with all integers smaller than 100." That shifts the blame to the mistaken implementation, and the user is to blame.

It's the same situation here. Deref is not unsafe to implement. That's why if you need to uphold a trust boundary, you need an unsafe trait.

So, the whole thing doesn't account to crossing the boundary willy nilly, a big point of Rust's unsafe is for the compiler to force documenting and checking for accountability: who is required to do what, and who is allowed to rely on that.

The scope of an unsafe block has always been interpreted to include guaranteeing things about it's surroundings up to other things in the same module. E.g. if I'm implementing `Vector::push` I'm going to rely on the fact that `self.capacity` really is the size of the allocation behind `self.ptr` without verifying it, and I'm going to feel good about that because those fields aren't public and everything within the module doesn't let you violate that constraint, so it's not possible for external safe code to violate it.

The same applies to marker traits. If I'm writing `unsafe impl Send for MyStruct {}` I'm promising that the module exposes an interface where `MyStruct` with will always comply with the requirements of `Send` no matter what safe external code does (i.e. sending MyStructs across threads is safe given the exposed API). With this proposal if I write `unsafe impl PinCoerceUnsized for MyStruct {}` I'm promising the same with respect to that (whatever the documentation for that trait ends up saying, which should essentially be that I've implemented `Deref for MyStruct` in the same module and I don't expose any way for safe external code to change what reference `Deref` returns).

The punchline here, so to speak, is that for all Rust’s claims to revolutionize safety, it simply(!) formalizes the same unwritten social contract C developers have been meandering along with for decades. The uniqueness boils down to “we still trust the devs, but at least now we’ve made them swear on it in writing”.

For those projects that don't use any unsafe, we can say -- absent compiler bugs or type system unsoundness -- that there will be no memory leaks or data races or undefined behavior. That's useful! Very useful!

For projects that do need unsafe, that unsafe code can be cordoned off into a corner where it can be made as small as possible, and can be audited. The rest of the code base is just as safe as one with no unsafe at all. This is also very useful!

Now, sure, if most projects needed to use unsafe, and/or if most projects had to write a significant amount of unsafe, then sure, I'd agree with you. But that's just not the reality for nearly all projects.

With C, everything is unsafe. Everything can have memory leaks or data races or undefined behavior. Audits for these issues need to examine every single line of code. Compilers and linters and sanitizers can help you here, but they can never be comprehensive or guarantee the absence of problems.

I've been writing C for more than 20 years now. I still write memory leaks. I still write NULL pointer dereferences. I still struggle sometimes to get my data ownership (and/or locking) right when I have to write multithreaded code. When I get to write Rust, I'm so happy that I don't have to worry about those things, or spend time with valgrind or ASAN or clang's scan-build to figure out what I've done wrong. Rust lets me focus more on what I actually care about, the actual code and algorithms and structure of my program.

Exactly this, and very well put!

I'd just like to add one small but important detail. It's one of the things that is so obvious to one group that they rarely even mention it, but at the same time so obscure to the others that they are completely oblivious to it.

While the unsafe code is cordoned off into a corner its effects are not. A bug in an unsafe block in one part of your program can trigger an outcome in a completely different and safe part of your program, that normally safe Rust should prevent.

To put it more metaphorically, Rust restricts the places where bombs can be placed, it does not limit the blast radius in case a bomb goes off.

This is still huge progress compared to C/C++, where the bombs can and usually are everywhere and trying to write it safely feels a lot like playing minesweeper.

The magnitude matters.

1) I have a more ergonomic/precise/local contract to satisfy safety

2) Since this unsafe block is local, it is easier to set up its testing conditions for various scenarios. Otherwise, testing for bigger unsafe block (e.g. unsafe language) would also have to handle coupling between api from which ub originates and the rest of the code.

At the end of the day this isn’t a technical argument I’m trying to make, it’s a philosophical one. I think that the more we normalize eroding the core benefits the language safety features provide, one enhancement proposal at a time, one escape hatch added each year for special interfaces, the less implicit trust you can have in rust projects without reviewing them and their dependencies for correctness.

I think that trust has enormous value, and I think it would suck to lose it. (reflect: what does seeing “written in rust” as a suffix make you think about a project’s qualities before you ever read the code)

If one claims otherwise, I say they have no understanding of Rust. But also, if one helds that against Rust's value promise, I, again, say that they have no understanding of Rust.

In a more practical sense, all software, even Python programs, ultimately call C functions that are unsafe.

It's like that saying "all abstractions are wrong, some are useful".

> what does seeing “written in rust” as a suffix make you think about a project’s qualities before you ever read the code

By itself, that tells me very little about a project. Same thing if I see a project written in Python or Go, which are nominally memory safe programming languages. I perceive a statistically significant likelihood that software written in these languages will not segfault on me, but it's no guarantee. If I see two programs with the same functionality, where one is written in Python and another one in Rust, I also have some expectation that the one written in Rust will be more performant.

But you cannot draw general conclusions from that piece of information alone.

However, as a programmer, Rust is a tool that makes it easier for me to write code that will not segfault or cause data races.

- Memory leaks are not just possible in Rust, they're easy to write and mildly encouraged by the constraints the language places on you. IME I see more leaks in Rust in the wild than in C, C#, Python, C++, ...

- You can absolutely have data races in a colloquial sense in Rust, just not in the sense of the narrower definition they created to be able to say they don't have data races. An easy way to do so is choosing the wrong memory ordering for atomic loads and stores, including subtle issues like those arising from mixing `seq_cst` and `acquire`. I think those kinds of bugs are rare in the wild, but one project I inherited was riddled with data races in Safe rust.

- Unsafe is a kind of super-unsafe that's harder to write correctly than C or C++, limiting its utility as an escape hatch. It'll trigger undefined behavior in surprising ways if you don't adhere to a long list of rules in your unsafe code blocks (in a way which safe code can detect). The list changes between Rust versions, requiring re-audits. Some algorithms (especially multi-threaded ones) simply can't even be written in small, easily verifiable unsafe blocks without causing UB. The unsafeness colors surrounding code.

The Rustonomicon [1] serves as a decent introduction to what you can or can't do in unsafe code, and none of that changed to my knowledge.

I agree that it's sometimes challenging to contain `unsafe` in a small blast zone, but it's pretty rare IME.

[1]: https://doc.rust-lang.org/nomicon/intro.html

Instead of the whole codebase being suspect, and hunting for unsafety being like a million-line "Where's Waldo?", it reduces the problem to just verifying the `unsafe` blocks against safety of their public interface, "is this a Waldo?". This can still be tricky, but it has proven to be a more tractable problem.

In C, the type system does not express things like pointer validity, so you have to consider the system as a whole every time something goes wrong. In Rust, because the type system is sound, you can consider each part of the program in isolation, and know that the type system will prevent their composition from introducing any memory safety problems.

This has major implications in the other direction as well: soundness means that unsafe code can be given a type signature that prevents its clients from using it incorrectly. This means the set of things the compiler can verify can be extended by libraries.

The actual practice of writing memory-safe C vs memory-safe Rust is qualitatively different.

Unfortunately, it's not. Now I do think it will be eventually fixed, but given how long it has taken it must be thorny. https://github.com/rust-lang/rust/issues/25860

There has also not been a single case of this bug being triggered in the wild by accident.

There was a Git repository demontrating unsoundness issues in the compiler, but I seem to be unable to find it anymore :/. It seemed like there would be more than one underlying issue, but I can't really remember that.

Yes, because this is a discussion about the value of "unsafe", so we're only talking about the wrongs that are enabled by "unsafe".

> and also ignore that unsafe parts violating invariants can make safe parts of Rust to be wrong.

If I run a line of code that corrupts memory, and the program crashes 400 lines later, I don't say the spot where it crashes is wrong, I say the memory corrupting line is wrong. So I disagree with you here.

Something that people often ponder is why you can't just solve the null safety problem by forcing every pointer dereference to be checked, with no other changes. Well of course, you can do that. But actually, simply checking to make sure the pointer is non-null at the point of dereference gets you surprisingly little. When you do this, what you're (ostencibly) trying to do is reduce the number of null pointer dereferences, but in practice what happens now is that you just have to explicitly handle them. But, in a lot of cases, there's really nothing particularly sensible to do: the pointer not being null is an invariant that was supposed to be upheld and it wasn't, and now at the point of dereference, at runtime, there's nothing to do except crash. Which is what would've happened anyways, so what's the point? What you really want to do isn't actually prevent null pointer dereferences, it's to uphold the invariants that the pointer is non-null in the first place, ideally before you leave compile time.

Disallowing "unsafe" operations without marking them explicitly unsafe doesn't give you a whole lot, but what you can do is expand the number of explicitly safe operations to cover more of what you want to do. How Rust, and many other programming languages, have been accomplishing this is by expanding the type system, and combining this with control flow analysis. Lifetimes in Rust are a prime example, but there are many more such examples. Nullability, for example, in languages like TypeScript. When you do it this way, the safety of such "safe" operations can be guaranteed, and while these guarantees do have some caveats, they are very strong to a lot of different situations that human code reviews are not, such as an unsafe combination of two otherwise-safe changesets.

It's actually totally fine that some code will probably remain unable to be easily statically verified, the point is that we want to reduce the amount of code that can't be easily statically verified to be as small as possible. In the future we can use much less easy approaches to statically verify unsafe blocks, such as using theorem provers to try to prove the correctness of "unsafe" code. But even just reducing the amount of not-necessarily-memory-safe code is an enormous win, for obvious reasons: it dramatically reduces the surface area for vulnerabilities. Moreover, time and time again, it is validated that most new vulnerabilities come from relatively recent changes in code, which is another huge win: a lot of the unsafe foundations actually don't need to be changed very often.

There is absolutely nothing special about code written in Rust, it's doing the same shit that C code has been doing for decades (well, on the abstract anyway; I'm not trying to downplay how much more expressive it is by any means). What Rust mainly offers is a significantly more advanced type system that allows validating many more invariants at compile-time. God knows C developers on large projects like the Linux kernel care about validating invariants: large amounts of effort have been poured into static checking tools for C that do exactly this. Rust is a step further though, as the safe subset of Rust provides guarantees that you basically can't just tack onto C with only more static checking tools.

If you were only afraid of the undefined behavior, you could augment the compiler to insert runtime checks anywhere undefined behavior could occur (which obviously can be done with Clang sanitizers.) However, the undefined behavior problem is really just a symptom of incorrect code, so it'd be even better if we could just prevent that instead.

In high level languages like Java and Python there is just as much, if not more, interest in preventing null reference exceptions, even though they are "safe".

Perhaps the most important exception is when the optimizer assumed the pointer was non-null, so optimized it in a way that produces completely unexpected behavior when it is null.

Also use-after-free and use of uninitialized pointers is more likely to point to incorrect, but mapped, locations.

The kernel is one such exception.

Crashing is the lucky case! Specifically in the kernel, there can be valid memory at address 0, and there are exploits that capitalise on the friction between memory address 0 sometimes being and C's null pointer being full of undefined behaviour.

Furthermore, Rust doesn’t turn off all checks in unsafe, only certain ones.

,

Obviously if you want to get formal guarantees beyond that property, you have to reason about safe code also.

(Also, the comparison in this entire chain is against C, and the latter is better than Rust in this regard… how?)

The difficult bit with Rust is still the sound use of unsafe, but it is quite feasible to do that by hand. It does, sadly, require looking at the entire module that contains the unsafe code.

I wrote this GCC plugin for this exact reason. Let's see whether the kernel team is interested in adopting it.

People need to learn the niceness of safety and perfect execution is a continuum of tolerances and flimsy guarantees from unmarked silicon that could be made in US, but is most likely a knock off made in China that will fail in 1/3rd of the expected time and gives a false reading if you so much as look at it the wrong way.

[0] https://www.usenix.org/system/files/1311_05-08_mickens.pdf

When I was doing some embedded development using rust it was actually a great experience, with hal and pac crates available already for a lot of hardware or easy to generate.

What applicable skills would someone writing a kernel driver gain from reciting a memory map? Abstractions exist for a reason.

The skill is in creating useful an performant abstractions.

Exactly! Kids like stonethrowaway with their C. Real engineers write their compilers in assembly.

(Less snarky, why stop at C, and not complain about even lower level stuff?)

Not sure where this misconception comes from. The engineering department mostly relies on Verilog twiddling, shoddy Spice models, debugging prototype boards with poorly crimped jumper wires, charged capacitors scattered around with no adults in the room, and freshly minted junior EEs who forget spice models and Verilog hacks aren’t the real thing.

You have the wrong department. Software development is down the hall to the left. Those folks down there don’t even have an engineering degree.

I've recently done a bit of work on Rust in something like embedded systems. Only instead of real hardware, we are running our software on Zero Knowledge VMs, ie on math.

So it's not that embedded Rust is bad. It's that developers who can't do their jobs without embedded Rust are usually very bad indeed. It's great when it's a choice. It's terrible when you lack the skills or perspective to work with what's under the hood.