The ring inside your process

Workflow · for a Rust developer who wants the shared-memory ring and the envelope store without the daemon; about fifteen minutes, with the standard alone

You will have: one program that creates a ring and a store in shared memory under a name, publishes a Pulse that points at an envelope, opens both again by name the way a second process would, and reads the Pulse and the envelope back, verified; the ring’s and the store’s own test suites run; and the edges seen rather than read about: a ring opened with the wrong capacity, a ring opened by name alone, two envelopes with the same content, a fetch of an envelope never stored, and the names gone once their creator drops.

You need: the standard’s repository set up as Start here describes, and the Rust toolchain, on Linux or macOS: the ring and the store are POSIX shared memory.

The daemon is a convenience on top of two primitives. The ring carries 128-byte Pulses through shared memory with no lock, and the store beside it holds payloads under their content identifiers. Both come from the core-ring and core-store crates, and any process can use them.

Hand this to your agent

Run the workflow "The ring inside your process" from book/src/workflows/ring-in-process.md in the pulse-mesh repository:
1. Run cargo run --release -p example-03-embed-ring-rust and show me its output. Expect one line beginning "OK — published one Pulse referencing envelope b6a5ee1e601cf94a3bd3ba244ddc0ee8".
2. Read examples/03-embed-ring-rust/src/main.rs and explain in plain words what crosses the ring and what sits in the store, and how the reader knows its copy of a slot is whole.
3. Beside the repositories, make a crate (cargo new ring-edges) with path dependencies core-ring, core-store and pulse-canon on ../pulse-mesh/core/core-ring, ../pulse-mesh/core/core-store and ../pulse-mesh/pulse/pulse-canon; save the edges program from the page as src/main.rs and run it. Expect: a capacity-mismatch error for the open with the wrong capacity, 64 from open_existing, "same id: true", "None" for the fetch, 0 publishes before any, and an shm_open error with errno 2 after the creator dropped.
4. Run cargo test -p core-ring --release and cargo test -p core-store --release in pulse-mesh and show me the result lines; they hold the ring and the store to the rules the specification states.
Stop and tell me if anything fails, and say which step.

Or do it by hand

1. Run the example

From the repository root:

cargo run --release -p example-03-embed-ring-rust

OK — published one Pulse referencing envelope b6a5ee1e601cf94a3bd3ba244ddc0ee8 and read both back through a second handle.

The envelope’s identifier is the same on every run and every machine, because it is a hash of the envelope’s content and the content is fixed.

2. Create the ring and the store, and publish

The program is examples/03-embed-ring-rust; these two listings are its main, apart from choosing a name and the final line.

#![allow(unused)]
fn main() {
    // One writer creates the ring: a 64-byte header plus 64 slots of 128 bytes.
    // Under a name that already holds a ring, `create` attaches to that ring
    // as it stands instead.
    let mut ring = PulseRing::create(&name, 64).expect("create ring");
    let store = EnvelopeStore::create_or_open(&store_name, 16).expect("create store");

    // The payload travels as an envelope, addressed by its content.
    let envelope = PulseEnvelope::typed("event.fired", b"payload".to_vec());
    store.store(&envelope).expect("store envelope");

    // The Pulse is the signal: a Data instruction whose envelope_ref names the envelope.
    let mut pulse = Pulse::zeroed();
    pulse.instruction = Instruction::Data as u16;
    pulse.monotonic_nanos = 1;
    pulse.set_envelope_id(&envelope.id());
    let write_number = ring.publish(&pulse);
    assert_eq!(write_number, 1);
}

PulseRing::create makes a shared-memory object named /pms-example-<pid>, sizes it for a 64-byte header plus 64 slots of 128 bytes, and maps it; under a name that already holds a ring it attaches to that ring as it stands instead, and a handle that attached does not unlink the name when it drops. EnvelopeStore::create_or_open does the same for the store. store writes the envelope once, under its identifier. The Pulse is the signal: a Data instruction whose envelope reference set_envelope_id sets to the envelope’s identifier. publish marks the next slot’s counter odd, copies the Pulse in, marks the counter even, and advances the cursor; it returns 1 because this is the ring’s first publish.

3. Open both by name and read

#![allow(unused)]
fn main() {
    // Another process opens both by name; the capacity must match the creator's.
    let reader = PulseRing::open(&name, 64).expect("open ring");
    let reader_store = EnvelopeStore::create_or_open(&store_name, 16).expect("open store");

    // The newest slot is one behind the cursor. read_slot copies it and keeps
    // the copy only if the slot's counter was even and unchanged across the copy.
    let newest = (reader.cursor_position() - 1) % reader.capacity();
    let seen = reader.read_slot(newest).expect("a consistent Pulse");
    assert_eq!(seen.instruction, Instruction::Data as u16);

    // The envelope comes back by identifier, from the store beside the ring.
    let id = seen.envelope_id().expect("the signal names an envelope");
    let found = reader_store.fetch(&id).expect("the referenced envelope");
    assert!(found.verify(), "the identifier matches the content");
    assert_eq!(found.type_name(), Some("event.fired"));
}

PulseRing::open attaches to the existing object, checks its size and then the header the creator wrote, and refuses a ring whose header records another capacity or slot size, so a mismatch is an error rather than a fault; PulseRing::open_existing opens a ring by name alone and takes the geometry from the header. The newest slot is one behind the cursor. read_slot copies it and keeps the copy only if the slot’s counter was even before the copy and unchanged after it; it retries up to a hundred times and returns None if the slot was never written or a writer is stuck on it. envelope_id reads the reference back as an identifier, fetch looks the envelope up by it, and verify recomputes the identifier from the content.

The creator unlinks the shared-memory names when it drops, so running the program twice leaves nothing behind.

4. See the edges

Beside the repositories, cargo new ring-edges, with these under [dependencies]:

core-ring = { path = "../pulse-mesh/core/core-ring" }
core-store = { path = "../pulse-mesh/core/core-store" }
pulse-canon = { path = "../pulse-mesh/pulse/pulse-canon" }

and this as src/main.rs:

use core_ring::PulseRing;
use core_store::EnvelopeStore;
use pulse_canon::PulseEnvelope;

fn main() {
    let name = format!("/pms-edges-{}", std::process::id());
    let ring = PulseRing::create(&name, 64).expect("create");

    if let Err(e) = PulseRing::open(&name, 32) {
        println!("open with the wrong capacity: {e}");
    }
    let by_header = PulseRing::open_existing(&name).expect("open_existing");
    println!("open_existing took the capacity from the header: {}", by_header.capacity());

    let store = EnvelopeStore::create_or_open(&format!("{name}-env"), 16).expect("store");
    let a = PulseEnvelope::typed("event.fired", b"payload".to_vec());
    let b = PulseEnvelope::typed("event.fired", b"payload".to_vec());
    println!("two envelopes with the same content, same id: {}", a.id() == b.id());
    store.store(&a).expect("store a");
    let c = PulseEnvelope::typed("event.fired", b"payloaD".to_vec());
    println!("fetch of an envelope never stored: {:?}", store.fetch(&c.id()).map(|e| e.id()));

    println!("reader sees {} publish(es) before any", ring.cursor_position());
    drop(ring);
    if let Err(e) = PulseRing::open_existing(&name) {
        println!("open after the creator dropped: {e}");
    }
}

cargo run --release

open with the wrong capacity: ring header capacity mismatch: the region has 64, the caller expects 32
open_existing took the capacity from the header: 64
two envelopes with the same content, same id: true
fetch of an envelope never stored: None
reader sees 0 publish(es) before any
open after the creator dropped: shm_open failed: errno 2

5. Run the rules under test

cargo test -p core-ring --release
cargo test -p core-store --release

Every test result line reads ok. The ring’s suites include the torn-read test: one writer as fast as it can against four readers, counting reads assembled from two publishes, and the count is zero.

Check it worked

If create fails with RegionTooSmall or LayoutMismatch, the name holds something other than a ring of that capacity, likely left by a program that did not exit cleanly; create never resizes or reformats an object, so pick another name or remove it. A leftover ring of the same capacity is attached to as it stands, old slots and cursor included, and a handle that attached does not unlink it. On Linux, NoSpace at creation means /dev/shm cannot hold the ring or the store; the pages are reserved when the object is made, so this is an error then rather than a fault at the first touch.

What happened

• The ring is the signal, the store is the content. A 128-byte Pulse carries the envelope’s 16-byte identifier; the envelope sits in the store under that identifier, written once. Any process that maps both by name reads the Pulse, fetches the envelope, and verifies it by recomputing the identifier from the content.
• A reader never sees a torn Pulse. Each slot carries a counter that is odd while a write is in progress and even when the slot is stable. A reader copies the slot and keeps the copy only if the counter was even before and unchanged after; otherwise it retries, up to a hundred times, and reports nothing for a slot whose writer died mid-write. Readers never write, so the writer never waits for a reader. The SeqLock ring states the protocol; The SeqLock memory model says why it holds.
• Geometry is checked, not assumed. The ring’s header records its capacity and slot size, and open refuses a header that disagrees with what the caller expects; open_existing takes the geometry from the header and needs no capacity.
• The creator owns the names. A handle that created an object unlinks its name when it drops; a handle that attached does not. Running a program twice leaves nothing behind, and a second process must open the names while the creator lives.
• Identity is content. Two envelopes with the same type name and payload have the same identifier, so the store holds one copy per distinct envelope, and a fetch by an identifier nothing was stored under is None.

What these two do not do: the ring overwrites the oldest slot when it is full, so a reader more than a capacity behind loses publishes; one that walks positions with read_position is told which, as lost (the ring chapter, readers that fall behind); the store is shared memory and vanishes with its creator, so it is not persistence (the daemon’s basket is the record); and neither routes, so two programs that share a ring agree on its name out of band, which is what the daemon does for them.

Make it yours

• Two real processes. One program creates the ring and the store and publishes; another opens them by name with PulseRing::open_existing and EnvelopeStore::open_existing (or open(name, capacity)) and reads; a reader must not create_or_open the store, or one that starts after the creator has gone makes a fresh, empty store instead of failing. Keep the creator running while the reader opens.
• Walk every publish. PulseRing::read_position(position) returns each position’s Pulse once, as itself, reports a position the writer has not reached as not written, and reports a lapped position as lost; it is what the daemon’s runtime polls with.
• Many readers, no copy per reader. HFT: one feed, many readers does this in process memory; across processes, PulseRing::create_with_readers(name, capacity, max_readers) gives each reader a cursor and publish_blocking keeps the slowest registered reader’s pace.
• Payloads too large for a store slot. Robotics: a 4 MiB scan through shared memory puts the payload in a pool and a 16-byte reference in the Pulse.

Next

• The SeqLock ring and The envelope store: the chapters these crates are held to.
• Why a 128-byte atom and Why content addressing: the two decisions underneath.
• Two programs talk through the daemon: the same two primitives behind a socket, with routing.

Spoken walkthrough

A script for the video and audio version of this page, about ninety seconds.

Take the daemon away, and two things are left: a ring and a store, both in shared memory, both opened by name.

The program creates them, writes an envelope into the store, and publishes one Pulse into the ring. The Pulse is a hundred and twenty-eight bytes, and sixteen of them are the envelope’s identifier, a hash of its content.

Then it opens both again by name, the way a second process would, reads the newest slot, fetches the envelope by the identifier the Pulse carries, and verifies it by recomputing the hash. One line: OK.

How does the reader know its copy of the slot is whole? Each slot has a counter. Odd while a write is happening, even when it is done. The reader copies, checks the counter was even before and unchanged after, and otherwise copies again. The writer never waits for anyone.

The second program walks the edges. Open the ring with the wrong capacity, and it tells you what the header says. Open it by name alone, and it takes the geometry from the header. Make two envelopes with the same content, and they have the same identifier. Ask for one you never stored, and you get none. Drop the creator, and the names are gone. Everything the daemon does for you, you have just done by hand.