Consistency Model

Primatomic views process events asynchronously. This document specifies the consistency guarantees and how to achieve read-after-write consistency.

Consistency Guarantees

Log Storage Guarantees

• Events within a log are assigned strictly monotonically increasing sequence numbers starting at 1.
• Each event is stored exactly once with a unique sequence number.
• The service never reorders or skips events in the log.

View Processing Guarantees

• Views process events in sequence order.
• View execution is at-least-once: events may be reprocessed after failures or leader changes.
• Views need to be deterministic. Applying the same event sequence must produce the same state.

Read Consistency

Mode	Guarantee
With after parameter	Response reflects all events up to and including that sequence
Without after parameter	Response may be arbitrarily stale

The Consistency Problem

When you append an event and immediately query a view, the view may not have processed your event:

Client                    Log                     View
  |                        |                       |
  |-- append event ------->|                       |
  |<-- sequence: 5 --------|                       |
  |                        |                       |
  |-- query view --------------------------------->|
  |<-- stale result (processed up to seq 3) ------|

Read-After-Write Consistency

To guarantee consistency, pass the after parameter when querying:

# Append returns sequence number
curl -X POST https://api.primatomic.com/logs/$LOG_ID/append \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/octet-stream" \
  -d '{"action":"increment"}'
# Response: {"success": true, "sequence": 5}

# Query with after waits for view to catch up
curl -X POST "https://api.primatomic.com/logs/$LOG_ID/views/my-view/query?after=5" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @query.bin
# Response: binary data (format defined by view)

When the after parameter is provided:

Client                    Log                     View
  |                        |                       |
  |-- append event ------->|                       |
  |<-- sequence: 5 --------|                       |
  |                        |-- event 5 ---------->|
  |                        |                       |
  |-- query (seq=5) ------------------------------->|
  |                        |                       | (waits)
  |                        |                       | (processes)
  |<-- result (includes event 5) ------------------|

The service does not return a response until the view has processed all events up to the requested sequence.

Consistency Modes

Stale Reads (Default)

curl -X POST https://api.primatomic.com/logs/$LOG_ID/views/my-view/query \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @query.bin

Use stale reads when:

• Reading dashboards or analytics where eventual consistency is acceptable
• Polling for updates
• Latency is more critical than freshness

Consistent Reads (With After)

curl -X POST "https://api.primatomic.com/logs/$LOG_ID/views/my-view/query?after=5" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @query.bin

Use consistent reads when:

• Displaying results of a user action
• Building UI that reflects recent mutations
• Requiring deterministic results for testing

Client Implementation Requirements

Clients implementing read-after-write consistency should:

1. Store the sequence number returned from append operations
2. Pass the sequence number to subsequent queries
3. Handle timeout errors with retry logic

Reference Implementation

async function appendAndQuery(
  logName: string,
  viewName: string,
  event: Uint8Array,
  query: Uint8Array
): Promise<Uint8Array> {
  // 1. Append event and capture sequence
  const appendResult = await api.appendLog(logName, event);
  const sequence = appendResult.sequence;

  // 2. Query with sequence for consistency
  const result = await api.queryView(
    logName,
    viewName,
    query,
    sequence  // Required for consistency
  );

  return result;
}

Append Idempotency

The Idempotency-Key header enables safe retries for append operations using NATS JetStream’s message deduplication.

Semantics

Behavior	Description
Deduplication window	Keys are deduplicated within the stream’s duplicate window (configured via nats.duplicate_window_secs, default: 1 hour in production)
Outside window	The same key may append again after the window expires
Payload mismatch	If a client reuses the same key with a different payload inside the window, JetStream still treats it as a duplicate (“first payload wins”, no error returned)

First Payload Wins

The Idempotency-Key does NOT verify payload equality. If you need to detect payload mismatches and return 409 Conflict, you must implement stronger idempotency at the application layer (e.g., store the key-to-payload mapping externally).

Single Event Append

The Idempotency-Key header is optional for single appends:

curl -X POST https://api.primatomic.com/logs/$LOG_ID/append \
  -H "Authorization: Bearer $TOKEN" \
  -H "Idempotency-Key: evt-abc123" \
  -H "Content-Type: application/octet-stream" \
  -d '{"action":"increment"}'

If the request times out and the client retries with the same Idempotency-Key, the duplicate is silently ignored and the original sequence number is returned.

Batch Append

The Idempotency-Key header is required for batch appends. Each event in the batch gets a per-event key in the format {base}:{index}:

curl -X POST https://api.primatomic.com/logs/$LOG_ID/append_batch \
  -H "Authorization: Bearer $TOKEN" \
  -H "Idempotency-Key: batch-xyz789" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @events.bin

This creates three events with keys: batch-xyz789:0, batch-xyz789:1, batch-xyz789:2.

Why this matters: If a network failure occurs after 7 of 10 events are published, retrying the entire batch is safe. JetStream deduplicates the first 7 events and only appends the remaining 3.

When to Use Idempotency Keys

Scenario	Recommendation
Critical business events	Use idempotency keys
Batch operations	Required (enforced)
Analytics/telemetry	Optional if duplicates are acceptable
Retryable operations	Use idempotency keys

Generating Idempotency Keys

Keys should be:

• Unique per logical operation (e.g., UUID, request ID)
• Deterministic for retries (same key on retry)
• Scoped appropriately (per-user, per-session, or global)

// Good: UUID per operation
const key = crypto.randomUUID();

// Good: Deterministic from operation context
const key = `user-${userId}-order-${orderId}`;

// Bad: Timestamp (not deterministic on retry)
const key = Date.now().toString();

Fallback: View-Level Deduplication

If your application requires stronger guarantees (e.g., detecting payload mismatches), include a unique identifier in your event payload and deduplicate in your view:

fn append(event: Vec<u8>) {
    let event: Event = serde_json::from_slice(&event).unwrap();
    if self.processed_ids.contains(&event.event_id) {
        return; // Already processed, skip
    }
    self.processed_ids.insert(event.event_id.clone());
    // Process event...
}

Sequence Number Specification

Property	Specification
Starting value	Sequence numbers start at 1
Increment	Each append increments the sequence by exactly 1
Gaps	Sequence numbers have no gaps
Scope	Sequence numbers are scoped to a single log
Uniqueness	Each sequence number is assigned to exactly one event

Timeout Behavior

Timeout Specification

Parameter	Value
Default timeout	4 seconds
Client override	Not supported (server-enforced)
HTTP status on timeout	504 Gateway Timeout
Scope	Per-request, leader-local

If the view cannot reach the requested sequence within the timeout period, the service returns:

HTTP Status: 504 Gateway Timeout

{
  "error": "Timeout waiting for view {view_key} to catch up (target: {sequence}, current: {current})"
}

Note

The timeout is enforced on the leader node only. If a leader change occurs during the wait, the request may fail with 502 Bad Gateway instead.

Common Causes

This error may occur when:

• The view is processing a large backlog
• The leader node is overloaded
• Network issues exist between nodes
• The requested sequence does not exist (higher than log high watermark)

Handle this error with exponential backoff:

async function queryWithRetry(
  logName: string,
  viewName: string,
  query: Uint8Array,
  after: number,
  maxRetries: number = 5
): Promise<Uint8Array> {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await api.queryView(logName, viewName, query, after);
    } catch (error) {
      // Retry on timeout or leader change
      const isRetryable = error.status === 504
        || error.status === 502;
      if (isRetryable && i < maxRetries - 1) {
        await sleep(100 * Math.pow(2, i));
        continue;
      }
      throw error;
    }
  }
  throw new Error('Max retries exceeded');
}

View Statistics

Check view processing progress:

curl .../logs/my-log/views/my-view/stats

{
  "view_name": "my-view",
  "processed_sequence": 42,
  "leader_status": "ready"
}

Field	Description
processed_sequence	The highest sequence number the view has processed
leader_status	"ready" indicates the view is caught up; null indicates no active leader

Use this endpoint to monitor view lag before querying with high sequence numbers.