This document describes the feature system available when creating ledgers. Features allow you to customize ledger behavior to optimize for different use cases (high write throughput, full audit trail, etc.).

Overview

Each ledger can be configured with a specific set of features at creation time. Features are immutable after ledger creation - you cannot change them once the ledger exists.

When you create a ledger without specifying features, all features are enabled with their default values.

Features Summary

Feature Details

MOVES_HISTORY

Values: ON | OFF

Default: ON

When enabled, the ledger tracks every individual fund movement for each account/asset pair. This creates detailed records in the moves table.

What it stores

Each move record contains:

• account_address - The account involved

• asset - The asset being moved

• amount - The amount moved

• is_source - Whether this account is the source (debit) or destination (credit)

• insertion_date - When the transaction was inserted

• effective_date - The logical date of the transaction (can be backdated)

• post_commit_volumes - Account volumes after the move (by insertion order)

• post_commit_effective_volumes - Account volumes after the move (by effective date order)

Use cases

When enabled (ON):

• Full balance history available at any point in time

• Ability to query historical balances

• Support for point-in-time queries

• Required for effective volumes calculation

When disabled (OFF):

• Only current balances are available

• Better write performance (fewer database inserts)

• Lower storage requirements

MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES

Values: SYNC | DISABLED

Default: SYNC

This feature maintains the post_commit_effective_volumes column in moves, which tracks volumes ordered by effective date rather than insertion date.

Dependency: This feature depends on MOVES_HISTORY being ON. If MOVES_HISTORY is OFF, this feature has no effect.

How it works

When a transaction is created with a backdated timestamp, the effective volumes for all subsequent moves (by effective date) need to be updated. This feature enables automatic maintenance of these volumes.

Example scenario:

1. Transaction T1 is created at timestamp 2024-01-01 (100 USD to account A)

2. Transaction T2 is created at timestamp 2024-01-03 (50 USD to account A)

3. A backdated Transaction T3 is inserted with timestamp 2024-01-02 (25 USD to account A)

With SYNC enabled:

• T3’s effective volumes are computed based on T1

• T2’s effective volumes are automatically updated to include T3’s impact

The database uses triggers to:

1. Compute the effective volumes for the new move based on the previous move (by effective date)

2. Update all moves with an effective date greater than the new move

Impact on Transaction Response

When this feature is enabled, transaction responses include postCommitEffectiveVolumes:

{
  "id": 1,
  "postings": [...],
  "postCommitVolumes": {
    "user:123": {
      "USD": { "input": 100, "output": 0 }
    }
  },
  "postCommitEffectiveVolumes": {
    "user:123": {
      "USD": { "input": 100, "output": 0 }
    }
  }
}

When this feature is disabled, the postCommitEffectiveVolumes property disappears from the transaction response:

{
  "id": 1,
  "postings": [...],
  "postCommitVolumes": {
    "user:123": {
      "USD": { "input": 100, "output": 0 }
    }
  }
  // postCommitEffectiveVolumes is NOT available
}

Use cases

When enabled (SYNC):

• postCommitEffectiveVolumes available on transactions

• Accurate historical balance queries at any effective date

• Support for backdated transactions with correct volume tracking

• Essential for audit trails with historical accuracy

When disabled (DISABLED):

• postCommitEffectiveVolumes not available on transactions

• Backdated transactions won’t update effective volumes in moves

• Better write performance (no cascade updates on backdated transactions)

• Use when transactions are always inserted in chronological order

HASH_LOGS

Values: SYNC | ASYNC | DISABLED

Default: SYNC

This feature provides cryptographic integrity verification for the ledger’s log chain. Each log entry is hashed using SHA-256, incorporating the previous log’s hash to create an immutable chain (similar to a blockchain).

How hashing works

The hash is computed from:

• Log type (e.g., NEW_TRANSACTION, SET_METADATA)

• Log data (the memento/payload)

• Timestamp

• Idempotency key (if provided)

• Previous log’s hash

This creates a chain where any modification to historical logs would break the hash chain, making tampering detectable.

SYNC Mode

When set to SYNC, hashes are computed synchronously during each log insert within the same database transaction.

Mechanism:

┌─────────┐     ┌───────┐     ┌──────────┐
│ Ledger  │────▶│ Store │────▶│ Database │
└─────────┘     └───────┘     └──────────┘
                    │
                    ▼
              1. Acquire advisory lock
              2. Get last log hash
              3. Compute new hash
              4. Insert log with hash
              5. Release lock (on commit)

The advisory lock (pg_advisory_xact_lock) prevents concurrent writes to ensure sequential hashing. This is released automatically when the transaction commits.

Pros:

• Hash is immediately available in the log record

• Integrity verification possible at any time

• No additional worker required

Cons:

• Performance bottleneck: only one log can be inserted at a time per ledger

• Higher latency on high-throughput ledgers

ASYNC Mode

When set to ASYNC, logs are inserted without a hash, and a background worker periodically computes hashes in batches called “blocks”.

Mechanism:

┌─────────────────────────────────────────────────────────────────────┐
│                        LOG INSERTION                                │
├─────────────────────────────────────────────────────────────────────┤
│  Logs are inserted WITHOUT hash (no advisory lock needed)           │
│                                                                     │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐    │
│  │  Log 1  │  │  Log 2  │  │  Log 3  │  │  Log 4  │  │  Log 5  │ ...│
│  │(no hash)│. │(no hash)│. │(no hash)│. │(no hash)│. │(no hash)│    │
│  └─────────┘  └─────────┘  └─────────┘  └─────────┘  └─────────┘    │
└─────────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                    ASYNC BLOCK RUNNER                               │
├─────────────────────────────────────────────────────────────────────┤
│  Worker runs on a CRON schedule (default: every minute)             │
│                                                                     │
│  1. Query all ledgers with HASH_LOGS = "ASYNC"                      │
│  2. For each ledger:                                                │
│     - Get last processed block                                      │
│     - Fetch next batch of logs (up to max_block_size)               │
│     - Compute block hash from all logs in batch + previous hash     │
│     - Store block in `logs_blocks` table                            │
│                                                                     │
│  Block structure:                                                   │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │ Block N                                                     │    │
│  │ ├── from_id: first log ID in block                          │    │
│  │ ├── to_id: last log ID in block                             │    │
│  │ ├── hash: SHA-256(previous_block_hash + all_log_data)       │    │
│  │ ├── previous: reference to previous block                   │    │
│  │ └── date: when block was created                            │    │
│  └─────────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────────┘

Block hashing algorithm:

-- Hash is computed as:
SHA256(
  previous_block_hash ||
  CONCAT(
    log1.type || log1.memento || log1.date || log1.idempotency_key || log1.id,
    log2.type || log2.memento || log2.date || log2.idempotency_key || log2.id,
    ...
  )
)

Worker configuration:

The async block runner is configured via command-line flags on the worker process:

Pros:

• No locking during log insertion

• Much higher write throughput

• Batched hashing is more efficient

Cons:

• Hashes are not immediately available

• Requires a separate worker process

• Blocks (not individual logs) are hashed

• Slight delay before integrity verification is possible

DISABLED Mode

When set to DISABLED, no hashing occurs at all.

Pros:

• Maximum write performance

• No locking overhead

Cons:

• No integrity verification possible

• Cannot detect tampering

Comparison

ACCOUNT_METADATA_HISTORY

Values: SYNC | DISABLED

Default: SYNC

When enabled, every change to account metadata is historized with revision tracking.

How it works

The accounts_metadata table stores every revision of metadata:

Triggers automatically create history entries:

• On INSERT: Creates revision 1 with initial metadata

• On UPDATE: Creates a new revision with incremented number

Use cases

When enabled (SYNC):

• Query account metadata at any point in time (using PIT queries)

• Full audit trail of metadata changes

• Support for $pit parameter in account queries

When disabled (DISABLED):

• Only current metadata is stored

• Historical metadata queries return current values

• Better write performance

Impact on Point-in-Time (PIT) Queries

Scenario: Account user:123 metadata changes over time:

With ACCOUNT_METADATA_HISTORY: "SYNC":

# Query metadata as of 2024-01-10 (before verification)
curl "<http://localhost:3068/v2/my-ledger/accounts/user:123?pit=2024-01-10T00:00:00Z>"

Response:

{
  "data": {
    "address": "user:123",
    "metadata": {
      "status": "pending"
    }
  }
}

# Query metadata as of 2024-01-20 (after verification, before upgrade)
curl "<http://localhost:3068/v2/my-ledger/accounts/user:123?pit=2024-01-20T00:00:00Z>"

Response:

{
  "data": {
    "address": "user:123",
    "metadata": {
      "status": "verified",
      "tier": "basic"
    }
  }
}

With ACCOUNT_METADATA_HISTORY: "DISABLED":

PIT queries always return the current metadata, regardless of the pit parameter:

# Query with any PIT date - always returns current metadata
curl "<http://localhost:3068/v2/my-ledger/accounts/user:123?pit=2024-01-10T00:00:00Z>"

Response:

{
  "data": {
    "address": "user:123",
    "metadata": {
      "status": "verified",
      "tier": "premium"
    }
  }
}

TRANSACTION_METADATA_HISTORY

Values: SYNC | DISABLED

Default: SYNC

When enabled, every change to transaction metadata is historized with revision tracking.

How it works

Similar to account metadata history, the transactions_metadata table stores every revision:

Use cases

When enabled (SYNC):

• Query transaction metadata at any point in time

• Full audit trail of metadata changes

• Support for $pit parameter in transaction queries

When disabled (DISABLED):

• Only current metadata is stored

• Better write performance

Impact on Point-in-Time (PIT) Queries

Scenario: Transaction 42 metadata changes over time:

With TRANSACTION_METADATA_HISTORY: "SYNC":

# Query transaction metadata as of 2024-01-03 (before status was added)
curl "<http://localhost:3068/v2/my-ledger/transactions/42?pit=2024-01-03T00:00:00Z>"

Response:

{
  "data": {
    "id": 42,
    "metadata": {
      "order_id": "ORD-001"
    }
  }
}

# Query transaction metadata as of 2024-01-07 (processing status)
curl "<http://localhost:3068/v2/my-ledger/transactions/42?pit=2024-01-07T00:00:00Z>"

Response:

{
  "data": {
    "id": 42,
    "metadata": {
      "order_id": "ORD-001",
      "status": "processing"
    }
  }
}

With TRANSACTION_METADATA_HISTORY: "DISABLED":

PIT queries always return the current metadata:

# Query with any PIT date - always returns current metadata
curl "<http://localhost:3068/v2/my-ledger/transactions/42?pit=2024-01-03T00:00:00Z>"

Response:

{
  "data": {
    "id": 42,
    "metadata": {
      "order_id": "ORD-001",
      "status": "completed"
    }
  }
}

Setting Features at Ledger Creation

Features are set when creating a ledger via the API. You only need to specify features you want to override - unspecified features receive their default values.

API Endpoint

POST /v2/{ledger}

Request Body

{
  "bucket": "optional-bucket-name",
  "metadata": {},
  "features": {
    "FEATURE_NAME": "VALUE"
  }
}

Examples

Create ledger with all defaults (full features)

curl -X POST <http://localhost:3068/v2/my-ledger>

Result:

{
  "data": {
    "name": "my-ledger",
    "bucket": "_default",
    "features": {
      "MOVES_HISTORY": "ON",
      "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "SYNC",
      "HASH_LOGS": "SYNC",
      "ACCOUNT_METADATA_HISTORY": "SYNC",
      "TRANSACTION_METADATA_HISTORY": "SYNC"
    }
  }
}

Create high-throughput ledger (minimal features)

curl -X POST <http://localhost:3068/v2/high-throughput> \\
  -d '{
    "features": {
      "MOVES_HISTORY": "OFF",
      "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "DISABLED",
      "HASH_LOGS": "DISABLED",
      "ACCOUNT_METADATA_HISTORY": "DISABLED",
      "TRANSACTION_METADATA_HISTORY": "DISABLED"
    }
  }'

Create ledger with async hashing

curl -X POST <http://localhost:3068/v2/async-hashing> \\
  -d '{
    "features": {
      "HASH_LOGS": "ASYNC"
    }
  }'

Override specific features only

curl -X POST <http://localhost:3068/v2/custom> \\
  -d '{
    "features": {
      "HASH_LOGS": "DISABLED",
      "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "DISABLED"
    }
  }'

Result (unspecified features get defaults):

{
  "data": {
    "name": "custom",
    "features": {
      "MOVES_HISTORY": "ON",
      "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "DISABLED",
      "HASH_LOGS": "DISABLED",
      "ACCOUNT_METADATA_HISTORY": "SYNC",
      "TRANSACTION_METADATA_HISTORY": "SYNC"
    }
  }
}

Feature Sets

Default Features (All enabled)

Recommended for most use cases requiring full audit capabilities:

{
  "MOVES_HISTORY": "ON",
  "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "SYNC",
  "HASH_LOGS": "SYNC",
  "ACCOUNT_METADATA_HISTORY": "SYNC",
  "TRANSACTION_METADATA_HISTORY": "SYNC"
}

Minimal Features (Performance optimized)

For high-throughput scenarios where audit trail is not required:

{
  "MOVES_HISTORY": "OFF",
  "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "DISABLED",
  "HASH_LOGS": "DISABLED",
  "ACCOUNT_METADATA_HISTORY": "DISABLED",
  "TRANSACTION_METADATA_HISTORY": "DISABLED"
}

Async Hashing (Balanced)

For high-throughput with eventual integrity verification:

{
  "MOVES_HISTORY": "ON",
  "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "SYNC",
  "HASH_LOGS": "ASYNC",
  "ACCOUNT_METADATA_HISTORY": "SYNC",
  "TRANSACTION_METADATA_HISTORY": "SYNC"
}

Feature Impact Summary

Important Notes

1. Features are immutable: Once a ledger is created, its features cannot be changed. Plan your feature set carefully.

2. ASYNC hashing requires a worker: If using HASH_LOGS: "ASYNC", you must run the ledger worker process with appropriate configuration.

3. Feature dependencies: MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES depends on MOVES_HISTORY. Disabling MOVES_HISTORY makes the effective volumes feature ineffective.

4. Performance vs. auditability: There’s a trade-off between write performance and audit capabilities. Choose based on your use case requirements.