Skip to content
Documentation menu · version 0.1.0-alpha.4

High availability

YARMA API replicas are stateless apart from live connections. Run two or more replicas behind a load balancer with cluster.enabled: true (YARMA_CLUSTER_ENABLED=true).

What runs where

Concern Mechanism
Agent gRPC streams Each agent holds one stream to any replica. The replica owns the NATS subject yarmarelay.agent.<device>; other replicas send commands, session requests and disconnects through it.
Browser signaling sockets The replica holding a technician or end-user WebSocket owns yarmarelay.hub.<session>.<role>; pushes from the agent's replica are relayed.
Command and elevation results The replica waiting for a result owns yarmarelay.command.<id> / yarmarelay.elevation.<id> until it arrives or times out.
Schedulers, sweepers, monitoring, reports, license revalidation One leader elected with a PostgreSQL advisory lock on a dedicated connection. If the leader dies or loses its database connection, another replica takes over within about 10 seconds.
Rate limits Redis (redis.addr); shared by all replicas.
Events NATS JetStream (SESSIONS, DEVICES, AUDIT, ALERTS streams).
Files and recordings S3-compatible object storage.

A request to a subject nobody owns fails immediately (NATS "no responders"), so offline devices and closed browsers are detected without waiting for timeouts.

Requirements

  • PostgreSQL with streaming replication and automatic failover (e.g. Patroni, CloudNativePG, or a managed service). The API reconnects after failover; the leader is re-elected because advisory locks die with their connection.
  • NATS cluster of three nodes with JetStream (replicas: 3). The relay carries control messages between replicas, so NATS must be on a private network with TLS and authentication (user/password, NKeys or JWT); never expose it to agents or browsers.
  • Redis with Sentinel or a managed HA offering.
  • Object storage with versioning or replication.
  • Load balancer: HTTP(S) for the API and WebSockets (no stickiness required); TCP or HTTP/2 pass-through for agent gRPC (:18443 by default, mTLS terminates at the API).

Failure behaviour

Failure Effect
API replica lost Its agents reconnect to other replicas (exponential back-off with jitter, starting at 1 s, capped at 5 minutes); active sessions on it end and can be restarted. Background work moves to another replica if it was the leader.
NATS unavailable Single-replica paths keep working; cross-replica relaying and event publishing fail until NATS returns.
PostgreSQL failover Requests fail for the failover window; the API reconnects automatically.
Redis unavailable Sign-in rate limits fail open (identity providers and local lockout still apply); support-code joins fail closed until Redis returns.

See backup and disaster recovery for data protection and the Helm chart for a Kubernetes deployment.

Documentation for Yarma 0.1.0-alpha.4. Something unclear? .