Freshness and Replay

Freshness describes how current the read daemon’s projections are relative to the commit log. Replay is the mechanism that keeps projections synchronized.

Problem this concept solves

In event-sourced systems, writes and reads are separated:

The write daemon commits events immediately
The read daemon must process those events to update its view

This creates a freshness gap: the time between when a commit is acknowledged and when it appears in read queries. Understanding this gap is essential for:

Building applications that need consistent reads
Diagnosing apparent data “staleness”
Tuning system performance

Core ideas

Commit Log

The commit log is the durable sequence of event batches:

Append-only: New batches are added at the end
Immutable: Once written, batches don’t change
Sequenced: Each batch has a monotonic sequence number

The write daemon appends to the log; the read daemon tails it.

Projections

Projections are in-memory views of state:

Built by applying events from the commit log
Optimized for query performance
Published atomically via snapshot swap

Projections are derived data. If lost, they can be rebuilt from the commit log.

Checkpoints

Checkpoints record progress through the commit log:

Write daemon: Tracks last committed sequence
Read daemon: Tracks last applied sequence

Checkpoints enable:

Fast startup (resume from checkpoint, not beginning)
Crash recovery (replay from checkpoint)
Freshness calculation (compare checkpoints)

Freshness Metadata

Read responses include freshness information:

{
  "freshness": {
    "checkpoint_seq": 42,
    "commit_log_seq": 45
  }
}

checkpoint_seq: The sequence the read daemon has processed
commit_log_seq: The latest sequence in the commit log

The difference is the lag: how many commits haven’t been applied yet.

Replay Process

When the read daemon tails the commit log:

Fetch next batch after checkpoint
Apply each event via L1 setters
Update checkpoint
Publish new snapshot

Events are idempotent: applying the same event twice produces the same state. This makes replay safe to retry.

Recovery Flow

After a crash:

Load checkpoint from disk
Fetch events from checkpoint position
Replay events to rebuild state
Resume normal tail loop

Because events carry post-state, replay doesn’t need to re-execute business logic. It simply sets the final values.

How it fits into the system

Write Path

Client → Write Daemon → Commit Log
                         ↓
              [checkpoint updated]

The write daemon updates its checkpoint after successful persistence.

Read Path

Commit Log → Read Daemon → Projections → Client
    ↓
[checkpoint updated after apply]

The read daemon publishes snapshots before updating its checkpoint. This ensures:

Queries always see consistent state
Crashes don’t lose applied but uncheckpointed work

Freshness Reporting

The health endpoint reports lag:

{
  "commit_log": {
    "end_seq": 100,
    "checkpoint_seq": 98,
    "lag": 2
  }
}

Monitor this to detect:

Normal operation (lag near 0)
Temporary burst (lag spikes then recovers)
Systematic issues (lag grows continuously)

Key invariants and guarantees

Eventual Consistency

Projections will eventually reflect all committed events:

The read daemon continuously tails the log
Lag may spike during bursts but recovers
No committed event is permanently invisible

Publish-Before-Checkpoint

Snapshots are published before checkpoints are updated:

Queries see data that is guaranteed applied
Crashes don’t cause data to “disappear”
Recovery replays from last safe point

Deterministic Replay

Replay produces identical state:

Events carry post-state values
No business logic during replay
Same events → same state

Checkpoint Safety

Checkpoints are persisted atomically:

No partial checkpoint writes
Recovery always finds valid checkpoint
Progress is never lost across restarts