Skip to content
Tutorials Center
My learning

Topics

Lesson 106 of 132 ยท Advanced Rust

Pin, Unpin and Self-Referential Types

Learn why Rust has Pin, Unpin and PhantomPinned: how moves break self-referential structs, Pin<Box<T>>, projection and async state machines.

Pin has a reputation for being the hardest type in the standard library to understand. The reputation is partly deserved, because the problem it solves is subtle. But the idea fits in one sentence: Pin is a promise that a value will stay at the same address until it is dropped. Almost nothing needs that promise. A few things do: some hand-written data structures, and, more importantly, every async fn and async block, which the compiler turns into a state machine that may point into itself.

This lesson builds the idea from the ground up. We show what goes wrong when a struct holds a pointer to its own field and then moves, introduce Pin, Unpin and PhantomPinned, build a safe-to-use self-referential type, learn why pinned fields need projection, and finish with the async state machines that are the reason Pin exists. You will want to be comfortable with ownership and moves, Box and Unsafe Rust. Async itself is introduced in the Async Rust chapter; here we use only a tiny hand-made executor.

The problem: moves change addresses

In Rust, moving a value is a plain byte copy to a new location, followed by forgetting the old one. For most types that is perfectly fine, because nothing else knows where the value lives. Trouble starts when a value contains a pointer to itself:

Rust A self-referential struct breaks when it moves

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

After the swap, a holds b's old bytes: its value is 20, but its pointer still holds the address of the other box's value field, which now stores 10. The program prints a reads 10 next to a.value = 20: the struct lies about itself. Here both addresses stay valid, so it merely gives wrong answers; if the other struct had been freed, the pointer would dangle, which is undefined behaviour. (Putting the struct in a Box first matters: returning a struct by value from a function also moves it, so a pointer taken before the return would already be stale.)

Safe Rust cannot create this situation with ordinary references. The compiler refuses a struct that borrows its own field:

Rust Safe references cannot point into yourself (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The error is error[E0505]: cannot move out of `holder` because it is borrowed: holder.first borrows holder.text, and moving holder would invalidate that borrow. That is the borrow checker doing its job. The catch is that some Rust code needs exactly this shape, and async code generates it automatically. We need a way to say "this value will never move", and let the type system enforce it.

Pin: a value that cannot move

Pin<P> wraps a pointer type P (such as &mut T, Box<T> or Rc<T>). While a value is pinned, safe code cannot get a plain &mut T to it, because with &mut T you could call mem::swap or mem::replace and move it. Pin<P> only hands out Pin<&mut T>, whose API has no way to move the pointee.

Three things to remember:

  1. Pin does not allocate and does not stop anything from being moved; it simply restricts what you can do with the pointer, and so makes moves of the pointee impossible in safe code.
  2. Moving the pointer itself is fine. Moving a Pin<Box<T>> copies the box's pointer, and the T on the heap stays where it was.
  3. The pin lasts until the value is dropped. After that the memory may be reused.

The usual way to pin a value is on the heap with Box::pin:

Rust Pin<Box<T>> keeps the target in place

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The last lines work only because String is Unpin. That is the next piece.

Unpin and PhantomPinned

Unpin is an auto trait (like Send and Sync) that means "it is fine to move this type even after it has been pinned". The compiler implements it automatically for any type whose fields are all Unpin. That is nearly everything: integers, String, Vec<T>, Box<T>, references, your ordinary structs. For an Unpin type, Pin does nothing special: Pin::new(&mut x) works and Pin<&mut T> implements DerefMut, so you can use it like a normal &mut T.

A type opts out by containing std::marker::PhantomPinned, a zero-sized marker. Futures generated from async blocks also opt out when they might hold a self-borrow. For a !Unpin type, safe code can no longer get &mut T out of the pin:

Rust PhantomPinned blocks safe mutable access (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

You get error[E0277]: `PhantomPinned` cannot be unpinned. Pin::new is only available when the target is Unpin; the note suggests the pin! macro, or Box::pin if the value must outlive the current scope. The fix is to pin the value properly. The std::pin::pin! macro (stable since Rust 1.68) pins a value on the stack for the rest of the enclosing scope:

Rust pin! pins on the stack, Box::pin on the heap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

pin! gives you a Pin<&mut T>; as_ref() turns it into Pin<&T> for a shared look. If you need a pinned value to outlive the current function, or to put it in a collection, use Box::pin.

A self-referential type, done right

Now we can build the earlier Gauge safely. The recipe:

  1. Add PhantomPinned so the type is !Unpin.
  2. Return it only as Pin<Box<Self>>, constructed in two steps: allocate, then set the self-pointer once the final address is known.
  3. Expose methods that take self: Pin<&Self> or Pin<&mut Self>.
Rust A pinned self-referential struct

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The pointer always agrees with the field, even after the boxes are pushed into a vector, because only the box's pointer moved. Now try to cheat and swap two pinned gauges:

Rust Pin stops the swap that broke Gauge (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The compiler rejects &mut *a with error[E0596]: cannot borrow data in dereference of `Pin<Box<Gauge>>` as mutable, with the help line that DerefMut is required to modify through a dereference but is not implemented for Pin<Box<Gauge>> (it needs Gauge: Unpin). That compile error is the whole point of Pin: the unsafe invariant (the value never moves) is guarded by the type system, and unsafe is needed only in the few places such as get_unchecked_mut where you take responsibility.

Inside an unsafe block you can still break pinning (for example with get_unchecked_mut followed by mem::swap). The contract is: after a value is pinned, you must never move it or invalidate its memory until its destructor has run. Document every unsafe use with a SAFETY comment.

Pin projection: reaching a field of a pinned struct

Suppose a struct holds a pinned future in a field. Given a Pin<&mut Wrapper>, how do you get a Pin<&mut Inner> for that field? Plain field access gives &mut Inner, which would let you move it. The operation is called pin projection, and the standard library offers Pin::map_unchecked_mut to do it, which is unsafe because you must guarantee the rule:

  • Structural pinning: if you project to a field as pinned, that field must never be moved out of the struct afterwards, Drop for the struct must not move it, and the struct must be !Unpin whenever the field is.
  • A field you do not treat as pinned (like a plain counter) can be accessed normally, as long as the whole struct stays Unpin-safe in your design.

Here is a wrapper future that counts how often its inner future is polled. It projects to the inner future (structural) and to the counter (non-structural). The Future trait and the tiny executor appear in the next section; for now read poll as "make progress once":

Rust Projecting to a pinned field by hand

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The future returns Pending twice and Ready on the third poll, so the wrapper reports 3 polls. Because CountPolls<F> is Unpin only when F is, the compiler propagates the !Unpin-ness of async blocks automatically. Writing projections by hand is error-prone, so real code uses the pin-project crate (or pin-project-lite), which generates the projection methods and checks the rules at compile time:

Rust
// Read-only: needs the pin-project crate
use pin_project::pin_project;

#[pin_project]
struct CountPolls<F> {
    #[pin]
    inner: F,      // projected as Pin<&mut F>
    polls: u32,    // projected as &mut u32
}

// in poll():
// let this = self.project();
// *this.polls += 1;
// this.inner.poll(cx)

Why async needs Pin

Consider this function:

Rust
async fn example() -> usize {
    let data = [1, 2, 3];
    let reference = &data;           // borrows a local
    some_future().await;             // suspends here
    reference.len()                  // uses the borrow AFTER the suspension
}

The compiler turns it into a state machine, an enum-like struct that stores every local that is alive across an .await. Here that includes both data and reference, and reference points at data, inside the same struct. That is exactly our Gauge problem. If the future could be moved between two polls, reference would dangle. So all futures from async code are !Unpin, and Future::poll takes self: Pin<&mut Self>:

Rust
pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

The executor must pin the future once (to the heap with Box::pin or to the stack with pin!) and then poll it repeatedly in the same place. This is why spawning APIs ask for Pin<Box<dyn Future<Output = T> + Send>>, and why Future objects you put in a Vec need Box::pin. See how it plays out with an actual borrow across an .await:

Rust An async block that borrows across an await

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

You see answer 60 after 1 pending poll(s), then the two task results. The future is suspended exactly once (inside Yield) and resumed in the same memory location, so first_two stays valid. Waker::noop() (stable since Rust 1.85) is a waker that does nothing: perfect for a toy executor that simply polls in a loop. A real executor sleeps until the waker is called.

Practical rules

  • Most code never writes Pin. You .await a future and the runtime pins it. Hand-written Future impls are the main place you meet Pin<&mut Self>.
  • If your type is Unpin (the default), Pin costs nothing and you can use Pin::new and DerefMut as usual.
  • To store or send a future: Box::pin(fut) gives Pin<Box<dyn Future<Output = T> + Send>>.
  • To poll a future on the stack: std::pin::pin!(fut).
  • To write a self-referential type: PhantomPinned, Pin<Box<Self>>, set pointers after pinning, document the unsafe. Before reaching for this, consider indices instead of pointers (an offset into a Vec), or the ouroboros/self_cell crates, which wrap the unsafe part for you.
  • Do not implement Drop for a type with structurally pinned fields unless you are careful not to move them in drop.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does Pin<P> guarantee?

    Pin withholds &mut T for !Unpin types, so safe code cannot call mem::swap or replace on the pinned value.
  2. 2. Why is a plain struct with a raw pointer to its own field dangerous?

    A move is a byte copy. The self-pointer is copied unchanged and now points at the old location.
  3. 3. Most types such as String, Vec<T> and i32 are Unpin. What does that mean for Pin<&mut String>?

    Unpin says moving is harmless even after pinning, so Pin::new works and DerefMut is available.
  4. 4. Why do async blocks produce !Unpin futures?

    A local that is borrowed across a suspension point lives in the future and is pointed to by another field of the same future.
  5. 5. Which call pins a value on the heap so you can store it in a Vec?

    Box::pin allocates the value and returns Pin<Box<T>>, whose pointer you can move around freely while the value stays put. Pin::new requires Unpin.

Summary

  • Moving a value copies its bytes to a new address; a pointer from the value into itself would then dangle, which is why safe references cannot do this.
  • Pin<P> wraps a pointer and withholds &mut T for !Unpin types, so safe code cannot move the pointee until it is dropped.
  • Unpin is automatic and true for almost every type; PhantomPinned and async-generated futures are !Unpin.
  • Pin on the heap with Box::pin, on the stack with std::pin::pin!; moving a Pin<Box<T>> moves only the pointer.
  • Reaching a field of a pinned struct (projection) is unsafe by hand, so use the pin-project crate in real code.
  • Future::poll takes Pin<&mut Self> because async state machines may borrow their own locals across .await.

Next: Type System Techniques