Forget marker trait

[Ddystopia]

Add a Forget marker trait indicating whether it is safe to skip the destructor before the value of a type exits the scope and basic utilities to work with !Forget types. Introduce a seamless migration route for the standard library and ecosystem.

Rendered

Pre-RFC thread

Unresolved questions

• Unsafe guarantee that !Forget gives to the unsafe code is already fulfilled for all 'static types. Because of that, it doesn't really make a lot of sense to have !Forget + 'static, it is still sound to mem::forget that type. Should we force impl<T: 'static> Forget for T then? Won't that impl create some unexpected problems?

[clarfonthey]

This definitely seems to be a reasonably motivated RFC, although it's going to definitely be something I'll have to read through a lot more closely to fully comment on it. A few first impressions:

1. I don't think that Forget as a name is very good for a marker. Forgetting something as a verb is fine, but as… an adjective? Not really. I think that at least Forgettable would be a better name, although something like ForgetSafe would probably be more in line for this trait, in line with UnwindSafe.
2. While I appreciate the effort to be as accessible as possible by explaining everything, including concepts like RAII guards which not everyone might be familiar with, your desire to explain everything in advance of using them means that the RFC itself is difficult to follow. For example, the motivation section doesn't really get into the actual motivation until it provides multiple code blocks and examples. This makes the entire thing difficult to follow, since I can't easily look at the top of the section and understand what it's about. Chronological order is good for explaining the prior art, but not for the motivation, IMHO.
3. On that note, I don't think that the current guide-level explanation is very good for a guide-level explanation. It feels very structured like an FAQ, which I don't think is good for the guide level: you're explaining to someone what the feature is outright, and not just answering their questions about it.

Again, I do want to go through this a bit more closely before fully commenting on it, but I think that you definitely need to go back and make the primary motivation for this crystal clear: because of forget, invariants in types cannot be violated by borrowing wrappers. This was not a problem before async code, because you could effectively ensure that function calls happen to completion (minus poisoning semantics) and nothing is ever left in an invalid state. However, with async code, you now have the issue that you can "forget" to finish part of a full function call via forgetting its Future, which leaves things in an invalid state.

Generally, the only way to deal with this is to make these calls unsafe and tell people to pinky-promise they run the future to completion, but it would be nice to be able to have a safe way to do this instead.

[kennytm]

@clarfonthey We do use verbs as trait name when the only purpose of the trait is to support that verb, think of Send or Copy or Borrow or std::iter::Extend.

[kornelski]

This is definitely a much desired feature that would enable new safe design patterns in Rust.

Having a gradual migration path with default_generic_bounds sounds reassuring.

I appreciate all the links to the prior art.

[kornelski]

Do I understand correctly that all !Forget types need to have a lifetime bound to have any effect? Does it ever make sense to have a !Forget type that has no lifetime? (like an integer)

[Jules-Bertholet]

That’s correct. Non-Forget types must be dropped before their lifetime ends

[Ddystopia]

From the perspective of use cases and definitions, it really makes little sense to have !Forget and 'static together. But I am not opposed too, it may have some other benefits which I am not aware of / or implementational / migrational benefits.

[kornelski]

This makes me wonder whether PhantomNotForget should have a generic lifetime argument. Although it's not technically necessary, it would make it clearer that it's about lifetimes. I could work as self-documenting code which lifetime is the tricky one, and PhantomNotForget<'static> would be something to lint against.

[Ddystopia]

Yeah, I also think so, but I fear some decision makers may not like it

[Jules-Bertholet]

If there were to be a blanket impl<T: 'static> Forget for T, then the lifetime parameter would be necessary for the type to work at all. I think that alone justifies it. IIUC the lifetime parameter would have to be covariant

[Ddystopia]

I would mention it just in case, this is already in unresolved questions:

Maybe force impl Forget for T where T: 'static {} and add a generic to the PhantomNonForget? Use cases and unsafe guarantee are fine with it, and we already allow !Forget in static.

[zetanumbers]

This makes me wonder whether PhantomNotForget should have a generic lifetime argument. Although it's not technically necessary, it would make it clearer that it's about lifetimes. I could work as self-documenting code which lifetime is the tricky one, and PhantomNotForget<'static> would be something to lint against.

In my (not new) reference implementation of the Forget trait, I have added it on a wrapper type Unforget as a contravariant lifetime:

https://zetanumbers.github.io/leak-playground/leak_playground_std/marker/struct.Unforget.html

As to the magic 'static lifetime, seems like it made a lot of contention in its prior discussion:

#t-lang > The destruction guarantee and linear types formulation

[Ddystopia]

And RFC directly forbids Box::leak and talks about statics

I retract my comment with the correction in the typo of the RFC 😛

Please provide an example of unsoundness that exploits that loop {} can cause unsoundness.

The point has already been addressed here: #3782 (comment)

Great. Then you can find an explanation from @kornelski or me under that comment you linked.

[Ddystopia]

I have found this text to be unsubstantiated, and rather simply a "what if" exploration with examples and counterexamples. There are no new answers to questions I've faced from Rust major contributors, leaving progress on this feature still stuck.

Please note that the single case of “what if” you commented on earlier occurs in the guide-level explanation, which is intended to provide an intuitive understanding of the feature.

From the rust-lang/rfcs repository:

• Explaining the feature largely in terms of examples.
• Explaining how Rust programmers should think about the feature, and how it should impact the way they use Rust. It should explain the impact as concretely as possible.

The section you commented on explained concretely: there should be a borrow-checked lifetime between tx and rx handle, no matter the implementation details about where sent values are living - on the rx's stack, on the stack of the tx's and rx's parent, inside the shared allocation etc.

[zetanumbers]

I have found this text to be unsubstantiated, and rather simply a "what if" exploration with examples and counterexamples. There are no new answers to questions I've faced from Rust major contributors, leaving progress on this feature still stuck.

Please note that the single case of “what if” you commented on earlier occurs in the guide-level explanation, which is intended to provide an intuitive understanding of the feature.

From the rust-lang/rfcs repository:

• Explaining the feature largely in terms of examples.
• Explaining how Rust programmers should think about the feature, and how it should impact the way they use Rust. It should explain the impact as concretely as possible.

Doesn't change what I've said, nothing new.

The section you commented on explained concretely: there should be a borrow-checked lifetime between tx and rx handle, no matter the implementation details about where sent values are living - on the rx's stack, on the stack of the tx's and rx's parent, inside the shared allocation etc.

Discussed in review.

[zetanumbers]

There are major open problems, which do not allow this and default bounds RFC to be an effective proposal.

[zetanumbers]

Considering I couldn't find anything in the reference-level explanation closely related to the explanation on why rendezvous channels could not operate on unforgettable types, let me exactly describe why rendezvous channels create a major hole in the current design.

Let's start by exploring the new JoinGuard design. JoinGuard has to be an unforgettable type, but one of the obvious restrictions on unforgettable types is to restrict transfer of object's ownership to itself, rendering the destruction of an object to be avoidable while reaching the end of a lifetime. This means there should be no way to transfer ownership of the JoinGuard object to itself.

You could restrict JoinGuard to also be !Send type. Logic is simple: transfer of JoinGuard between threads is prohibited, thus transferring it from the source thread to the one this JoinGuard refers to is impossible. Graph of threads borrowing from other threads may only be a DAG (a tree in this case), meaning there is an order in which threads could terminate while not invalidating any reference.

Just like async drop, unforgettable types were intended to enable safe borrowing async tasks as part of an idea of the structured concurrency. Composition of borrowing async tasks means being able to hold one such task handle inside of another (borrowing) async task, and analogously designed non-thread-safe ScopedAsyncTask: !Send makes it impossible to store the inner task inside of the outer task without making the outer task !Send too, thus rendering structured concurrency unobtainable in today's rust.

One might speculate and try to fix some holes, for example by making JoinHandle: !Send, but this can only count as a workaround. If we look at the problem in depth, we can see that Forget is generally incompatible with Rc, as well as other APIs that can be expressed with its signature, because it creates a hidden self-reference.

A traditional approach to the message-passing cannot be applied to !Forget types - slightly different APIs should be developed, preserving a lifetime connection between tx and rx handles.

It is clear to a library developer that any synchronization channel with a buffer (for example mpsc) could easily create a self-reference by sending a receiver object to its own channel buffer, as such T: Forget bound is obvious. However your reasoning for a sound rendezvous channel having T: Forget bound too is incomplete, which is simply "because it makes thread-safe JoinGuard unsound as in combination can create a self-reference." If you cannot derive both of these APIs as sound from the language's semantics independently, it becomes fair to say this feature will make the language itself (more) unsound. And that is just a single example with rendezvous channels, how could you really know there're no other holes?

What you are actually trying prohibit is the transfer of (any) JoinGuard between threads, while one of them is handled by a JoinGuard too. You have stated that solution with JoinGuard: !Send is a "workaround", which I've shown is a good enough for JoinGuard alone, while the proposal with channels and lifetimes to me seems as vague as possible and wouldn't be clear to any library developer unless there's a clear elaboration on it, otherwise in currently described terms being a patchwork.

Is what I'm talking about clear now? If this issue with rendezvous channels wasn't there I would have already published this RFC. For now unforgettable types proposal have only been discussed on the rust zulip. Here's a thread about this exact problem:

#wg-async > Structured parallel tasks require !Send in Send coroutines

EDIT: If you think your RFC have already addressing this concern, please leave a citation.

[Ddystopia]

as such T: Forget bound is obvious

However your reasoning for a sound rendezvous channel having T: Forget bound too is incomplete

Do we have a misunderstanding? You absolutely can and should send T: !Forget types using channels! RFC is not claiming that rendezvous channels must have T: Forget, it is saying that to be able to transfer T: !Forget, tx and rx should be connected via some lifetime (borrow the allocation, not both ownlike Arc), even if phantom, and are !Forget themselves - it has no ergonomic downsides. They anyway share some data via some allocation or the stack where new() was called, how would tx know where rx's stack is? And, RFC is stating things about rendezvous channels to help them to adapt to the new change, not the other way around (it is elaborated later).

If your question is "your reasoning is incomplete because you ban tx and rx without a lifetime connection not based of the language semantics but based on the broken use case" - no. Reference Level Explanation gives a guarantee that values borrowed by !Forget types will remain borrowed untill drop is executed. Current channels do not fulfill it, they will "weaken" the state and remove the borrow if combined with anything like Box<dyn Trait>, while drop is not executed. There is nothing special about scoped and JoinHandle, see guide level explanation. By giving tx and rx lifetime (borrowing from a single owner instead of being like Arc with a shared allocation ownership) you can have sound code. And generally, tx and rx not having a lifetime is a consequence of the fact that you can't pass them to tasks/threads because those require 'static closures. It is a vicious cycle of Arc and 'static we are part of. If not 'static requirements, channels didn't need to have Arc inside for shared state, and we wouldn't even have to have that discussion today about channels without lifetimes.

The text later was written before I spot this misunderstanding, but I think it still should be relevant. I do not enjoy long scrolls, so I would put in the details.

Long response to the previous comment

The reason this RFC is published by me is because I fundamentally disagree with you on that point, and what is new. Moving JoinGuard to another thread is a problem only if you are trying to move it inside the same thread - and borrow checker is already fine-tuned around detecting any kinds of self references, it is already enforcing DAGs you mention. But tx and rx without a lifetime, which is only possible with some unsafe code involved, like Arc, can sidestep those checks.

The most important thing is the cost-benefit ratio. Benefits are immense, motivation is huge. From the side of the costs, you don't loose anything by requiring tx and rx to be connected via lifetime and be !Forget themselves, except the ability to create that unsoundness. With async scope you can pass them into other tasks freely. Rendezvous use case is covered by the design, as well as all others. There are literally no downsides in that approach except the migration, do you want me to do a formal soundness proof like RustBelt?

And please note, we are not talking about structured concurrency alone in this RFC, there are plenty of other use cases.

What I am trying to say in this RFC is that everything should be connected via lifetime - ergonomics is still at the highest level, but borrow checker is able to control you code better. Rc-like APIs are using unsafe code to introduce dynamic lifetimes, they are directly sidestepping the borrow checker.

simply "because it makes thread-safe JoinGuard unsound as in combination can create a self-reference."

The fact that you can exploit that unsoundness by sending JoinGuard into itself is not the motivation, it is just that it is literally unsound and you can reveal that unsoundness by placing a lifetime, which was removed and made the borrow checker blind.

To give more intuition: disconnected tx and rx for !Forget type is literally unsound, !Forget is not 'static, but send() has weakening signature (after that function T is gone for the borrow checker), and borrow checker stops any protection. This is why we disallow channels, not because we want JoinGuard be Send. But with a lifetime between tx and rx borrow checker is not making such assumption, because it sees where the value may have been moved, with a lifetime.

---

To reiterate, the requirement to have a lifetime between tx and rx is the statement of this RFC. When writing unsafe code, it is your responsibility to assess, would it be compatible with the language rules or not (if you have use case to read from the foreign *const i32 while &mut i32 to the same memory exists, your use case is not compatible with Rust). RFC does not leave any way to get unsoundness with std by using already existing borrow checker, while other libraries should audite themselves according to the language rules. This RFC made additional effort (in order to be more attractive) to help the ecosystem and already solved many uses cases for them, those listed in the motivation, including those rendezvous channels. Even if there exists another way to solve this by making JoinGuard: !Send, this is just another way to make things sound (I am not claiming that you will not find more holes with that directed patch like JoinGuard: !Send, but let's assume it), you can analyze which approach has better cost-benefit ratio to understand why RFC took another approach.

If your question is "Why require lifetime between tx and rx instead of forcing JoinGuard: !Send, it is because the former has no downsides and "feels natural" and uses the borrow checker while the latter has some downsides and "feels like a hack".

---

You certainly do have a grasp on this proposal. I would truly appreciate if you would be open-minded and instead of purely destructive actions (for now you just brought up some cases and speculations where you personally believe RFC should be discarded, without considering const-benefit ratio) put some work into improving the RFC. For example, looking for additional costs, investigating how you can use safe code to create unsoundness (which happens from time to time even in the stable Rust due to human factor), or thinking about real use cases to decide on 'static + !Forget situation using cost-benefit ratio, improving the migration plan, which is really the only pain point with this RFC.

I'm sorry if I've offended you in any way or if you think I've stolen your credits - I'm not claiming anything, I just believe that you're not correct and I'm willing to spend the time and effort to improve the language. If you feel like I answered your concerns, please mark your comments as resolved, thank you.

[Jules-Bertholet]

Wouldn't be enough if the proposed Forget trait prevent memory being invalid? Then I think these assumptions are enough to solve the problems using Forget trait.

As for any T If

1. T is 'static and Forget then It is safe to Box::leak and forget.
2. T is 'static and !Forget then It is safe to Box::leak, but not safe to forget.
3. T is non 'static but Forget then It is safe to Box::leak and forget.
4. T is non 'static and !Forget then It is not safe to Box::leak or forget.

After considering this for a bit, I don’t think category 2 makes sense. If you have an owned value (on the stack or in a Box), you can always invalidate the memory simply by moving the value out to some other location. If the value has to stay in a particular location, the proper mechanism is Pin.

[storycraft]

After considering this for a bit, I don’t think category 2 makes sense. If you have an owned value (on the stack or in a Box), you can always invalidate the memory simply by moving the value out to some other location. If the value has to stay in a particular location, the proper mechanism is Pin.

If the value has to be pinned in particular place, than using Pin is correct. Maybe my first sentence was a bit misleading because I thought that moving values are not invalidating memory(place for the value still exists, although it's moved). So category 2 is more like movable and unforgettable 'static type(not sure if it would be useful or not). If it leaks, the backing memory would be still valid to use. But forgetting value on the stack deallocate memory without dropping. Pin has almost same drop guarantee too. But only for 'static type(for non 'static type references can be dangling). Forget trait prevents these cases.

[teor2345]

This seems like a useful change, but it would be easier to understand with some tweaks to the ordering of paragraphs, and some wording clarifications.