Operating a node

This is the operator’s page: what runs where, how the node is reached, how it is backed up, and what to do when something is lost. Design rationale lives in SPEC.md; this page only tells you what to run.

Layout

Path (in data_dir, /data in the container)	What
hdtp.db	the SQLite store: accounts, contacts, messages, integrations, audit chain
blobs/	content-addressed media
keyring.key	the keyring master key (0600). Every account’s private key is sealed under it.
hdtp.lock	held while serve runs; offline commands refuse to run while it is held
admin.sock	the admin socket the CLI talks to while the node is running

Configuration: env HDTP_* > config.json > defaults (hdtp-gateway doctor prints the resolved result). The deployment mode is derived from the tunnel adapter, never declared (SPEC §10.1).

Reachability

You have	Use	Mode
a public port / static IP / VPS	tunnel: direct (default)	direct
a Tailscale account	tunnel: tailscale — tsnet Funnel, free, ports 443/8443/10000	direct
an frps you run on any VPS	tunnel: frp — raw SNI/tcp passthrough	direct
a paid ngrok plan	tunnel: ngrok — tls:// endpoint	direct
a Cloudflare account	cloudflare edge adapter (P4-05) — Cloudflare terminates TLS	edge
a domain of your own	run a second hdtp-gateway in the ingress role on a VPS (below)	direct (passthrough) or edge (terminate)
nothing inbound at all	one of the tunnels above, or a provider hosting the identity under a leaf you issue (HDTP §9). There is no relay role: it went on 2026-09-18, because a store-and-forward gateway sees every sender, recipient and timestamp	—

Direct mode keeps mTLS end to end: callers’ client certificates reach the node. Edge mode cannot — a third party terminates TLS — so the node forces seal=required and client_cert=off, and callers are identified by their sealed-envelope signatures. hdtp-gateway doctor reports the derived mode and probes your advertised endpoint; a wrong_cert verdict means something between the caller and the node is terminating TLS that should not be.

Every carrier still sees metadata (who talks to whom, sizes, timing). Where that is itself sensitive, use direct on infrastructure you control (SPEC §13).

Ingress role (own domain)

On the VPS run the ingress; on the node pair with a one-time token:

1. ingress: hdtp-gateway ingress serve --domain example.com (P5-05 wires the command; the library is internal/ingress) and mint a token from its portal.
2. node: portal → Settings → Ingress — paste the token, pick the subdomain and mode:
   • passthrough — the ingress routes on SNI and forwards raw TLS; your node’s own certificate is what callers see (direct mode).
   • terminate — the ingress holds an ACME certificate, terminates the public TLS session, and re-originates a mutually-pinned mTLS leg to your node (edge mode for your node). Both keys are pinned at pairing; nothing else can deliver traffic.
3. The node connects outbound (embedded frp client) — no inbound port at home.

Call budgets

Every call counts against a budget of the account it is addressed to, sized by how many contacts that account may hold (HDTP §12). The budgets are not the node’s to decide: the limits sidecar, hdtp-limitd, holds their numbers and their counters and decides each call with hdtp-identity’s hdtp-limits crate, the decision the hosted cloud makes (SPEC §5.7). The node asks it over a unix socket, limits_socket (HDTP_LIMITS_SOCKET, default <data_dir>/limits.sock), on one connection it keeps open; nothing but the charge — the account, the caller’s root and tier, its address, the contact cap — crosses it.

Its numbers are its configuration file, the rules document of hdtp-identity’s contract, shipped as deploy/limitd/limits.json (in the image at /etc/hdtp-limitd/limits.json):

Member	Budget	Keyed by
contact_calls_per_second, contact_burst	a contact	account, contact root
identity_capacity_per_second	every contact together: limit.contacts × the contact rate, burst one second of it, at most this	account
guest_calls_per_hour	a guest (a proven root that is not a contact)	account, root, address
guest_source_calls_per_hour	an address alone (nothing proven)	account, address
stranger_calls_out_per_hour	calls OUT to strangers (request_contact, redeem_invite, the answers to a request)	account
integration_calls_per_hour	one integration’s tools, per contact	account, integration, contact
pending_in_cap	requests waiting on the owner that strangers may write	account
guest_total_calls_per_hour	every caller the open does not prove a contact, together (below)	account

guest_total_calls_per_hour is checked before the envelope is opened (the owner’s decision of 2026-09-29). Nothing unopened says who sent a sealed call, so the node asks the sidecar first, before it reads a key: while the account’s total holds a call, every sealed call goes on to be opened; once it holds none, only a call from a known address does — one that carried an active or pending_out contact’s call to the account in the last hour, which the sidecar remembers — and everything else is refused rate_limited, in the clear, with the total’s retry_after. Asking spends nothing. The total is spent after the open by every call that is opened and does not prove an active or pending_out contact: a guest, a blocked or superseded root, a root whose only row is the request it left, a small form naming a leaf nobody pinned, and an envelope_invalid or certificate_renewed the open found. A proven contact never spends it; a plaintext call opens nothing and never spends it.

The known cost: during a flood, a contact calling from an address it has not used in the last hour is refused before the open, as a stranger is, with retry_after, until the total refills. The known addresses live in the sidecar’s memory beside the counters, an hour each and a bounded number an account (the oldest going first: KNOWN_SOURCE_TTL_MS and KNOWN_SOURCES_CAP, cmd/hdtp-limitd/src/lib.rs), so a sidecar restart forgets them too.

A known address is an address: every caller arriving from it shares its standing. Behind a carrier that delivers every caller from one address of its own — frp, ngrok and tailscale from the node’s host, a terminate-mode ingress from its own, none of which names the client’s address — one contact’s call makes that one address known, and the check before the open lets every stranger through to be opened (and refused after it). The check does its work where each caller arrives with an address of its own: behind Envoy (below), or the cloudflare adapter. (TestAStrangerFloodDrainsTheTotalThenIsRefusedBeforeTheOpenAndAKnownContactGetsThrough shows a stranger at the contact’s known address opened and refused.)

Calls out to a contact spend that contact’s rate and the account’s aggregate, in buckets of their own. A refusal is a rate_limited tool error carrying retry_after — the whole seconds until the bucket holds a call again — and an audited rate_limited row naming the bucket, not a dropped connection, so the caller’s agent can read it and back off. A call out that is refused never leaves the node. A request past pending_in_cap is answered unavailable: no wait empties a list only the owner can.

The shipped identity_capacity_per_second is measured, and it is below what 500 contacts ask for. One node on an Apple M2 Max served sealed send_message from 500 contacts at up to 280 calls a second in every run, and broke between 300 and 450 a second from run to run (TestMeasureAccountCapacity, internal/node; the method and the numbers are in its file), and the shipped figure is the lowest knee with a margin. The figure is the node’s, and every identity on the node shares it.

Changing a number is editing the file and restarting the sidecar, which reads it once and refuses one it cannot enforce (a rate of zero, a burst under one call, a contact bucket that takes longer than the hour an idle row is kept to refill), saying which member and why. The node needs no restart: get_card asks the sidecar for the numbers it advertises on every call.

When the sidecar is down, the node refuses. Every sealed call, call out, request and integration call is answered unavailable until it answers again, which the node notices by itself. /healthz answers 503 and names the socket, so the container’s healthcheck fails; hdtp-gateway doctor prints FAIL limits with the reason, and the serve banner says limits: NOT ANSWERING. Start the sidecar (hdtp-limitd -config <file>; the compose file runs it) and the node serves again with no restart.

The counters live in the sidecar’s memory, one set for every node process on the host. A restart of the sidecar refills every bucket, which is the trade-off for not writing to a database on every call — the budget is there to blunt abuse, not to meter usage, and an attacker who can restart your sidecar has already won. Memory is bounded: a bucket idle for an hour is full whatever it budgets, so it is dropped (checked once a minute), and a caller cycling addresses or fingerprints cannot grow the table past who called in the last hour.

The one budget setting that is the node’s is how many contacts each identity may hold, which sizes the aggregate:

Setting	Environment	Default	Meaning
limit.contacts	HDTP_LIMIT_CONTACTS	500	contacts each identity may hold — active contacts plus the requests it sent — and the size of its call budget

It is an ordinary knob: the portal’s Settings page edits it under Security, the config file carries it as limit_contacts, and the environment pins it above both (SPEC §12.2), in which case the page shows it locked and says why. It is read per use, so a change takes effect with no restart. Empty, zero or unparseable restores 500 rather than removing the cap. The cap is enforced wherever a contact is added — approving a request, unblocking a contact, accepting somebody’s invite, sending a request, a peer redeeming an auto-accept invite, approving a contact at a new address, and an import — and nothing already held is removed when it is lowered.

Connection bounds

The public listener holds at most 1,024 connections open (SPEC §5.7); one more is closed before its TLS handshake, and the audit trail gets one listener_full row a minute while that continues, with how many there were. Requests have 10 s for their headers, 60 s in all, 75 s for the answer, and an idle connection is kept 120 s. Rate limits per address or for the whole node are not the node’s: put them where the traffic arrives — the edge, or a proxy in front of the node, such as the one below.

Behind Envoy

deploy/envoy/ is the node behind a proxy of its own, the first of the two layers of its rate limits (the second is the sidecar above): docker compose -f deploy/envoy/compose.yaml up -d runs Envoy, the node and the limits sidecar, and only Envoy publishes a port. Before it: make identity-proxy limitd-vendor (the image build), a certificate for the node’s public name at deploy/envoy/tls/cert.pem and key.pem, and HDTP_PUBLIC_URL, that name, in the environment.

What Envoy does (deploy/envoy/envoy.yaml, whose numbers are its own and nowhere else):

• terminates the caller’s TLS with that certificate, which is what a caller now sees — WebPKI for the node’s name, as behind a terminating edge — and asks for the caller’s certificate chain, accepting any, since there is no authority above the person;
• limits each source address per path — the MCP endpoints, the invite landing, everything else, each with a bucket of that address’s own — and answers 429 past it, before the node sees the request; and holds the listener’s connection cap and timeouts;
• forwards the caller’s chain in X-Forwarded-Client-Cert and the address its socket saw in X-HDTP-Client-Address, replacing whatever the caller sent in either.

The node reads those two headers only from Envoy’s address, proxy_address (HDTP_PROXY_ADDRESS, an IP; the compose file gives Envoy a fixed one on its network and the node that one). From anywhere else they are a caller’s own claim and prove nothing, and with no proxy_address they are never read. The node still opens every envelope: Envoy sees the MCP requests, never what a sealed one carries. internal/integrationtest/envoy_test.go holds the two files to what the node relies on, on every commit, and the harness’s S23 runs them with the image and floods them, with a control.

The store, when it is large

Nothing here needs setting. It is written down so that what the node does to its database is not a surprise, and so that the one knob that exists is findable.

• SQLite (the default) is opened in WAL mode with synchronous=FULL, write transactions that take their lock at BEGIN, and a pool of four connections. Each was measured against a store of a million messages and the reasons are beside the code (internal/core/store/sqlite.go). synchronous=FULL is the one that costs speed on purpose: a commit here is a message a peer was told was delivered, and it is not given up to a power cut for a faster write.
• Postgres (store_engine: postgres) uses pgx’s pool with its default size, the larger of four and the number of CPUs. The DSN is where it changes: append pool_max_conns=16 (and pool_min_conns, pool_max_conn_lifetime) to postgres_dsn.
• Every hour the node removes what has outlived its own window, whatever retention an account has set: idempotency records of sealed calls, which a node would otherwise keep one of for every call it ever took, owner sessions nobody came back to, and change-log rows older than a week. With a retention window set it also removes the messages, threads and media past it.
• Every statement the store can run is checked at build time to have an index on any table that grows (TestEveryQueryHasAPlan, both engines). To see the numbers on your own hardware:

HDTP_SCALE_DB=/tmp/hdtp-scale.db go test ./internal/core/store/ -run '^$' -bench '^BenchmarkScale' -benchtime 20x

The first run seeds the file — 10,000 contacts, a million messages, a million audit rows — and takes about a minute. make scale runs these benchmarks over a scratch file, and times an import of 40,000 threads against one of 10,000 (HDTP_EXPORT_SCALE), three rounds interleaved, failing if the best of three takes more than 6 times as long for 4 times the threads, in reading or in writing (linear is 4); the pre-push hook runs make scale last, after every other step.

More than one process

Several serve processes can share one store, each with its own internal_bind and public_bind behind whatever balances between them (SPEC §11.1):

• SQLite: on one host, all with the same data_dir. Not across hosts, and not on a network filesystem: SQLite’s locking does not hold there. One limit of this profile: Scrub, the checkpoint that clears the write-ahead log after a leaf key is destroyed, cannot finish while another process is reading the file. It says so — a warning on an install or a signing request, an error on a retirement or a leave — and the next scrub that finishes clears the log; until then the destroyed key’s bytes may remain in it. Postgres has no scrub at all (SPEC §3.9).
• Postgres: on any hosts, each with a data_dir of its own and the same postgres_dsn. Give every host the same master key in HDTP_MASTER_KEY (a host that generates a keyring.key of its own cannot open a key another sealed), and point blob_dir (HDTP_BLOB_DIR) at storage every host mounts, or media stored through one host is missing on the others.

The first serve on an idle data dir migrates; the others check that the schema is the one they were built for and refuse to start if it is not. To migrate, stop every process on the data dir (on Postgres, every process) and start the new binary. migrate, export, import and the audit commands refuse to run while any serve holds the data dir. One process serves the admin socket and serve prints admin: ... is served by another hdtp-gateway process on the others; the setup URL a first run prints works on the portal of the process that printed it. The outbound retries and the hourly retention pass run on one process at a time: the one holding the work’s lease in the store, renewed every 10 s; a holder that stops lets it go at once, and one that crashes is replaced within 30 s.

The call budgets are shared by every process on a host: they are the limits sidecar’s, and every process names the same limits_socket. A unix socket does not cross hosts, so on Postgres each host runs a sidecar of its own, and each host grants the whole budget.

What each process still keeps to itself, and so what is not yet shared between them:

• integrations: every process connects each one itself, so a stdio integration runs a child per process. An OAuth token is one for them all: the store holds it, and an expired one is refreshed by the one process holding that integration’s refresh lease while the others wait for it.

Export and import

hdtp-gateway export --config config.json --slug alina --out alina.zip
hdtp-gateway import alina.zip --config config.json --slug alina          # review: writes nothing
hdtp-gateway import alina.zip --config config.json --slug alina --yes    # writes it

Both are offline commands: stop the node first (they refuse while hdtp.lock is held).

An export is one identity’s contacts, chats and files, and nothing else (SPEC §3.10, HDTP §9.2): one unencrypted zip of manifest.json, contacts.csv, threads.csv, messages.jsonl and media/<sha256>, the same format the cloud and the hdtp CLI read and write. export says, before it writes, that the file is not encrypted: anyone who gets it can read the contact list and every conversation and file, though it holds no key and cannot be used to speak as anyone. The file is created 0600 and never replaces an existing one. A stranger’s request that was never accepted, and its conversation, stay behind.

What is not in it, because it is the host’s and not the person’s: every key, saved settings, integration credentials, owners, passkeys, sessions, tokens, invites, the audit chain, the ledger of leaves, and a contact’s preset, trust flag and card. There is no flag that adds any of them.

import checks the WHOLE file before it writes anything, and refuses it whole at the first fault, naming the member, row or line. Without --yes it shows what it would write and stops. Into a slug that is not here it creates the identity keyless — its root and nothing more; into the slug the file belongs to it merges, keeping every pin this host already holds. The rows go in under one transaction, the files after it. An undelivered outbound message arrives as failed: delivering it was the old host’s job, under the old host’s leaf.

Every import ends the same way: a request for a new leaf, which the import mints itself — move for an identity that arrived, renew for one that was here — and prints with how to complete it (the portal’s /identity/<slug>/wallet for the web wallet, or the request itself for the CLI wallet). Installing the leaf sends every imported contact this host’s handshake (account announce reports it). With no public_url set there is no address to ask for, and it names account csr instead.

This is not a backup of the node, and the node has none: what a host accumulates beyond contacts and chats is rebuilt, not restored. After a lost machine: import, the setup wizard for a passkey, reconnect integrations, one certificate per identity.

Recovery

Lost	Consequence	Do
the node, with an export per identity	identities are not served until re-certified; settings, integrations, passkeys and the audit history are not in an export	import each, serve, the setup wizard; then one account csr / install-leaf per identity. Contacts keep their pins
keyring.key only	sealed leaf keys, saved settings and integration credentials are unreadable. No identity is lost — the root is in the wallet. If nothing on the node opens under the master key it was given, serve refuses to start and says why; if anything does, it starts and prints NOT SERVED for each account that does not	put the key back if it was kept anywhere, and nothing is lost. If it is gone for good: export -slug each identity, then import each into a fresh data directory — an export never needed the master key, because it never held anything sealed under it. For a single NOT SERVED account on a running node, a renewal alone does it: the install retires the key it cannot open and says so
a leaf simply ran out (nobody renewed it)	the account stops being served within the hour and its key is destroyed; contacts keep their pins, and doctor warns before it happens	account csr --slug me -purpose renew, the wallet signs, account install-leaf
a card or certificate on file that the identity core no longer reads (its reading got stricter with a release of the identity library: the identity core 0.4.2 refuses a character outside base64url in a card’s certificate that 0.4.1 skipped, and what intake accepted then is still on file)	the core refuses it where it reads it, and the node says so only then: a contact whose card does not read cannot be written to (node.PeerOf reads the card first and refuses — a refresh of that contact included), and a call from a contact whose pinned leaf does not parse, in either form, is this node’s unreadable state (identity_state_unreadable on the trail, envelope_invalid to the caller). The serve banner’s store: line and hdtp-gateway check store read every such field first and name each that does not read — table, row, field, reason — and change nothing; check store exits 1 on one, and runs beside a serving node	for a contact’s card: this node cannot ask for one; the contact’s own next update_contact writes one that reads, or remove the contact and re-add it from a new card of theirs. For a pin’s leaf: remove the contact and re-add it, since nothing it sends can be decided against that pin. Run hdtp-gateway check store before and after a binary that bumps the identity library
a contact in a state neither a pin nor a request has (the schema admits only active, pending_in, pending_out and blocked — both engines — so such a row is a hand-edited store’s or a later binary’s)	the node hands the identity core no pin for it, where the core would refuse the state as unreadable: a call from its holder is decided as a stranger’s — the small form chain_required, the chain form a guest’s — and no effect reaches the row. The serve banner’s store: line counts such rows and a NO PIN line names each by account, root and status; hdtp-gateway check store prints the same and exits 1	remove the contact (remove_contact ends a relationship in any state) and re-add it from a new card of theirs
one account’s leaf key (compromised)	the thief speaks as that host until the leaf expires or is outranked	hdtp-gateway account csr --slug me -purpose renew, have the wallet sign it, account install-leaf: the newer leaf outranks the stolen one with every contact it reaches (HDTP §14.3)
nothing: the person moved an identity to another host	this node goes on serving it, with its key, until told	hdtp-gateway account leave -slug me -yes once the new host has told the contacts (without -yes it shows what it would erase): every record and leaf key of the identity erased, its address reserved until its last leaf expires (SPEC.md §3.11)
the wallet’s root	the identity itself; this node cannot help	the wallet’s own recovery, if it has one (HDTP §9, §14.5)
the audit chain shows a break	someone altered history	audit verify names the first bad row; treat the store as untrusted from there

No telemetry leaves the node, ever; the audit chain is yours alone.