Skip to content

Storage Backends

Hardy uses a dual storage model: one backend for bundle metadata (tracking state and timestamps) and another for bundle payload data (the actual bytes). The two backends can be mixed independently.

When no storage is configured, the server defaults to SQLite metadata and local-disk bundle data: a stock build persists to disk out of the box, writing to the platform default paths listed below, and will fail to start if those locations are unwritable (read-only filesystem, full volume). The non-persistent memory backends are never chosen by default — they must be selected explicitly, and the server logs a warning at startup when they are in use. The sqlite-storage and localdisk-storage build features that provide the default backends are enabled by default; a build that disables either feature refuses to start while the corresponding storage section is unconfigured (the backend must then be selected explicitly, e.g. type: memory), so the default never silently shifts to non-persistent storage.

storage — Cache Settings

Key Valid Values Default Description
lru-capacity Positive integer 1024 Maximum bundles in the in-memory LRU cache.
max-cached-bundle-size Positive integer (bytes) 16384 Maximum bundle size eligible for caching. Bundles larger than this bypass the cache.

storage.metadata — Metadata Backend

Key Valid Values Default Description
type memory, sqlite, postgres sqlite Metadata storage engine.

Memory

No persistence — metadata is lost on restart, and the server logs a warning at startup to say so. Suitable for testing only, and must be selected explicitly.

Key Valid Values Default Description
type memory - Selects the in-memory backend.
max-bundles Positive integer 1048576 Maximum entries held before the oldest are evicted. 
storage:
  metadata:
    type: memory

SQLite

The default for single-node deployments. Zero external dependencies. Uses WAL mode for concurrent read/write access. The database is created automatically on first run; migrations are applied automatically on upgrade.

Key Valid Values Default Description
type sqlite - Selects the SQLite backend.
db-dir Directory path OS-dependent (see below) Directory for the database file.
db-name Filename metadata.db Database filename.

Default db-dir by platform:

Platform Default path
Linux $HOME/.cache/hardy-sqlite-storage (XDG)
macOS $HOME/Library/Caches/dtn.Hardy.hardy-sqlite-storage
Windows C:\Users\<user>\AppData\Local\Hardy\hardy-sqlite-storage\cache
Container (no $HOME) /var/spool/hardy-sqlite-storage

Example:

storage:
  metadata:
    type: sqlite
    db-dir: /var/lib/hardy/db

PostgreSQL

Recommended for production and multi-container deployments where the BPA container needs to be stateless. The database and tables are created automatically on first connection.

Key Valid Values Default Description
type postgres - Selects the PostgreSQL backend.
database-url PostgreSQL connection string DATABASE_URL env var Standard postgresql://user:pass@host:port/db connection URL; absent falls back to the DATABASE_URL environment variable, and startup fails if neither is set.
max-connections Positive integer 20 Maximum pooled connections. Scale deployments should size this to worker_threads * 2 or higher.
min-connections Non-negative integer 2 Idle connections kept alive in the pool.
connect-timeout Duration string (e.g. 30s) 30s How long to wait when acquiring a connection before erroring; must be greater than zero.
idle-timeout Duration string (e.g. 10m) 10m How long a connection may sit idle before it is closed; must be greater than zero.
max-lifetime Duration string (e.g. 30m) 30m Maximum lifetime of a pooled connection; must be greater than zero.
poll-page-size Positive integer 64 Rows fetched per page in keyset-paginated poll queries.

Example:

storage:
  metadata:
    type: postgres
    database-url: "postgresql://hardy:secret@db.internal/hardy"

storage.bundle — Bundle Data Backend

Key Valid Values Default Description
type memory, localdisk, s3 localdisk Bundle data storage engine.

Memory

No persistence — bundle data is lost on restart, and the server logs a warning at startup to say so. Suitable for testing only, and must be selected explicitly.

Key Valid Values Default Description
type memory - Selects the in-memory backend.
capacity Positive integer (bytes) 268435456 (256 MiB) Total bytes held before least-recently-used bundles are evicted.
min-bundles Positive integer 32 Bundle count never evicted below, even when over the byte capacity. Must be at least 1. 
storage:
  bundle:
    type: memory

Local Disk

Stores bundles as individual files in a structured two-level hash directory layout (xx/yy/).

Key Valid Values Default Description
type localdisk - Selects the local disk backend.
store-dir Directory path OS-dependent (see below) Root directory for bundle files.
fsync true, false true Flush all changes to disk after each write. Set false for higher throughput at the risk of data loss on power failure.

Default store-dir by platform:

Platform Default path
Linux $HOME/.cache/hardy-localdisk-storage (XDG)
macOS $HOME/Library/Caches/dtn.Hardy.hardy-localdisk-storage
Windows C:\Users\<user>\AppData\Local\Hardy\hardy-localdisk-storage\cache
Container (no $HOME) /var/spool/hardy-localdisk-storage

Example:

storage:
  bundle:
    type: localdisk
    store-dir: /var/spool/hardy/bundles
    fsync: true

Amazon S3

Stores bundles in any S3-compatible object store (AWS S3, Google Cloud Storage, MinIO, Ceph, etc.). Credentials are provided via standard AWS mechanisms: environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), IAM instance roles, or shared credentials file.

Key Valid Values Default Description
type s3 - Selects the S3 backend.
bucket Bucket name Required S3 bucket for bundle storage.
prefix Key prefix (none) Key prefix for all objects (no leading or trailing slash), for buckets shared with other applications.
endpoint-url URL AWS default S3 endpoint URL. Set for non-AWS S3-compatible stores (MinIO, GCS, etc.).
region AWS region string AWS_DEFAULT_REGION / AWS_REGION env vars AWS region for the bucket.
force-path-style true, false false Path-style addressing (http://host/bucket/key), required for MinIO and some S3-compatible stores.
multipart-threshold Non-negative integer (bytes) 8388608 (8 MiB) Bundle size above which multipart upload is used. Must be at least the part size, and at most 5 GiB (the S3 PutObject limit).
multipart-part-size Positive integer (bytes) 8388608 (8 MiB) Size of each multipart part. Must be between 5 MiB and 5 GiB (the S3 part bounds).

Example (AWS S3):

storage:
  bundle:
    type: s3
    bucket: hardy-bundles
    region: eu-west-1

Example (MinIO / self-hosted):

storage:
  bundle:
    type: s3
    bucket: hardy-bundles
    endpoint-url: "http://minio:9000"
    region: us-east-1
    force-path-style: true

Tip

S3 storage makes the BPA container fully stateless when combined with PostgreSQL metadata — ideal for auto-scaling Kubernetes deployments.

Choosing a Backend

Deployment Metadata Bundle Data Notes
Development / testing Memory Memory No persistence, zero setup
Single node SQLite Local Disk Default, no external dependencies
Production (single node) SQLite Local Disk + fsync Durable, simple operations
Cloud / multi-node PostgreSQL S3 Stateless BPA, scalable
Hybrid PostgreSQL Local Disk When S3 latency is unacceptable

See also: