Deployment

SkyFollower is a single monorepo, but it deploys across several independent hosts — each one brings up one or more Docker Compose files, either from a full repo clone or, for a pre-built-image deployment, just the file(s) it needs (see Getting Started for both paths). Every role brings up exactly one compose file except the core host, which brings up two (docker-compose.core.yaml and docker-compose.management-ui.yaml), since management-ui's only dependency is Redis and there's no reason to entangle its lifecycle with the rabbitmq/redis/runner stack's compose project. See Getting Started for the commands to actually bring a host up once you know which compose file(s) it runs.

Compose Files

Each compose file below is meant to run on its own host, with the exception of docker-compose.core.yaml and docker-compose.management-ui.yaml, which are typically co-located on the same host but kept as separate compose projects (management-ui's only dependency is Redis, so it can move to a different host later without disturbing rabbitmq/redis/runners). The message processor is designed to scale both by adding more hosts, each running the same compose file, and by running more processors on one host (see Architecture's Scaling Message Processors section). A second receiver instance is optional and can be deployed separately from the host running the local RTL-SDR hardware, e.g. to ingest a remote feed. Archive compaction runs on its own ofelia instance alongside the archive processor; the one-shot job itself sits behind the compaction Compose profile so only its scheduler, never the job, comes up with docker compose up -d. Get the relevant file(s) onto each host and write its .env — scripts/install.sh does both (see Getting Started) — then bring the host up:

File	Role	Services
`docker-compose.receiver.yaml`	ADS-B reception (Raspberry Pi); also any additional receiver instance (same file, own `.env`, its own auto-generated identity)	`skyfollower-receiver-{name-slug}`, one per instance, generated into `docker-compose.instances.yaml` by `scripts/install.sh`
`docker-compose.core.yaml`	Message bus + enrichment data	`rabbitmq`, `redis`, `ofelia`, all runners
`docker-compose.management-ui.yaml`	Rules/areas API (co-located with `docker-compose.core.yaml`)	`management-ui`
`docker-compose.message-processor.yaml`	Flight state + rules (scale by adding hosts, or by running more than one processor on a host)	`skyfollower-message-processor-{id}`, one per fleet-wide ID, generated into `docker-compose.instances.yaml` by `scripts/install.sh`
`docker-compose.archive.yaml`	Long-term storage	`archive-processor`, its own `ofelia` instance; `archive-compaction` (the one-shot job, behind the `compaction` Compose profile — see Archive Compaction)
`docker-compose.map.yaml`	Live real-time aircraft map (own standalone host, given the expected UDP-ingest + WebSocket-fan-out load)	`map`, `map-redis` (dedicated, no-persistence Redis instance — never core Redis)

Components

Container	Description	Default port
`receiver`	Reads raw ADS-B frames from readsb TCP streams; routes to RabbitMQ queues	—
`receiver` (additional instance, optional)	Same image, its own auto-generated identity, deployed independently on its own host	—
`skyfollower-message-processor-{id}`	Consumes ADS-B messages, maintains flight state, enriches from Redis, runs rules engine — one container per fleet-wide ID, several may run on one host	—
`archive-processor`	Receives completed flights from RabbitMQ, writes gzipped JSON to S3	—
`archive-compaction`	Daily job (its own `ofelia` instance, alongside the archive processor) consolidating each day's per-flight Parquet index files into one file per partition	—
`rabbitmq`	Message broker between receiver, message processors, and archive	5672, 15672 (mgmt)
`redis`	In-memory enrichment store (aircraft, operators, airports, flight O/D, rules, areas)	6379
`ofelia`	Cron scheduler that runs runner containers on a schedule	—
`management-ui`	FastAPI backend + React frontend for rules and areas editing	HTTPS `MANAGEMENT_UI_HTTPS_PORT` (default 18443, self-signed by default); plain HTTP on that port redirects to HTTPS
`map`	Backend for the live real-time aircraft map: UDP listener, REST snapshot, WebSocket relay	UDP `MAP_LISTEN_PORT`, HTTPS `MAP_HTTPS_PORT` (default 19443, self-signed by default; plain HTTP if no cert is present). Single port — no plain-HTTP redirect listener
`map-redis`	Dedicated, no-persistence Redis instance backing `map`'s live aircraft state — never core Redis	6379 (not published to the host; reached only by `map` on the same compose file)
`mictronics` runner	Imports global aircraft registration data into Redis	—
`us-faa-registry` runner	Imports US FAA detailed registration data into Redis	—
`ca-transport-canada-registry` runner	Imports Transport Canada detailed registration data into Redis	—
`ourairports` runner	Imports airport metadata into Redis	—

...plus the two other global runners (vrs-standing-data, airportwebcams-special-liveries) and 36 more country-specific registration runners — see Data Runners for the full list.

redis in docker-compose.core.yaml sets explicit, non-default tuning (--save "", --no-appendfsync-on-rewrite yes, a raised RediSearch fork-GC interval) — these are deliberate choices for a constrained host (Raspberry Pi CM4008032, 8GB RAM/32GB eMMC), not an oversight.

Configuration

Every component reads its configuration from environment variables via shared/config.py's load_config(), interpolated by Compose from that host's .env. scripts/install.sh collects everything a role's .env needs interactively (see Getting Started); the tables below are the full reference, organized by role, for writing one by hand or checking what a generated one contains. LOG_LEVEL ("debug" or "info", default info) is accepted by every role and omitted from each table below.

Internal timing values — MQTT publish cadence, Redis heartbeat refresh and TTL, config-poll interval, reconnect backoffs, enrichment key TTLs — are not environment variables. They are fixed constants in shared/timing.py, documented in Timing and cadences. The one operator-tunable behavioural value, flight_ttl_seconds, lives in the config:flight_ttl_seconds Redis key, not in any .env.

Receiver

Variable	Required	Default
`RABBITMQ_HOST`	✅	—
`RABBITMQ_PORT`	❌	`5672`
`RABBITMQ_USERNAME`	✅	—
`RABBITMQ_PASSWORD`	✅	—
`MQTT_HOST`	❌	—
`MQTT_PORT`	❌	`1883`
`MQTT_USERNAME`	❌	—
`MQTT_PASSWORD`	❌	—
`REDIS_HOST`	❌	—
`REDIS_PORT`	❌	`6379`
`REDIS_PASSWORD`	❌	—

RECEIVER_NAME and RECEIVER_SOURCES aren't in this table: they're not read from .env. Each instance's values are in its own receivers/{name-slug}.env, written by scripts/install.sh (one shared receiver/ folder and .env per host, one env file and generated service per instance). See Receiver for what each variable means, RECEIVER_SOURCES's host:port:source format, and the multiple-receiver-instance pattern.

Core (`rabbitmq`, `redis`, `ofelia`, runners)

Variable	Required	Default
`RABBITMQ_USERNAME`	✅	—
`RABBITMQ_PASSWORD`	✅	—
`RABBITMQ_ADMIN_USERNAME`	❌	`skyfollower-admin`, generated by the installer
`RABBITMQ_ADMIN_PASSWORD`	❌	Generated by the installer
`REDIS_PASSWORD`	✅	—
`MQTT_HOST`	❌	—
`MQTT_PORT`	❌	`1883`
`MQTT_USERNAME`	❌	—
`MQTT_PASSWORD`	❌	—

RABBITMQ_USERNAME/PASSWORD and REDIS_PASSWORD are also what every other role's .env authenticates with, so the same values must be distributed to whichever hosts run message-processor, archive, and management-ui. RABBITMQ_ADMIN_USERNAME/PASSWORD are the opposite: they exist only for logging into the dashboard on 15672, are never referenced by any other role's .env, and are provisioned automatically by scripts/install.sh's provision_rabbitmq_users once RabbitMQ reports healthy, on install and on every --upgrade of the core role (see Architecture and docker-compose.core.yaml's comments on the rabbitmq service).

Management UI

Variable	Required	Default
`MANAGEMENT_UI_HTTPS_PORT`	❌	`18443`
`REDIS_HOST`	✅	—
`REDIS_PORT`	❌	`6379`
`REDIS_PASSWORD`	✅	—
`S3_BUCKET`	✅	—
`AWS_DEFAULT_REGION`	✅	—
`AWS_ACCESS_KEY_ID`	✅	—
`AWS_SECRET_ACCESS_KEY`	✅	—
`ATHENA_WORKGROUP`	❌	`skyfollower`
`ATHENA_DATABASE`	❌	`skyfollower`
`ATHENA_TABLE`	❌	`archive_flights`
`MQTT_HOST`	❌	—
`MQTT_PORT`	❌	`1883`
`MQTT_USERNAME`	❌	—
`MQTT_PASSWORD`	❌	—

management-ui is a separate Compose project from core even when co-located on the same host, so REDIS_HOST is typically localhost here rather than the redis service name core's own components use. MQTT is optional: set MQTT_HOST and the backend publishes a minimal Home Assistant presence (discovery, running version, start time) and nothing else -- no telemetry.

Message Processor

Variable	Required	Default
`LATITUDE`	✅	—
`LONGITUDE`	✅	—
`RABBITMQ_HOST`	✅	—
`RABBITMQ_PORT`	❌	`5672`
`RABBITMQ_USERNAME`	✅	—
`RABBITMQ_PASSWORD`	✅	—
`REDIS_HOST`	✅	—
`REDIS_PORT`	❌	`6379`
`REDIS_PASSWORD`	✅	—
`MQTT_HOST`	❌	—
`MQTT_PORT`	❌	`1883`
`MQTT_USERNAME`	❌	—
`MQTT_PASSWORD`	❌	—
`MAP_UDP_HOST`	❌	—
`MAP_UDP_PORT`	❌	—
`CAPTURE_RAW_FRAMES`	❌	`false`

CAPTURE_RAW_FRAMES enables opt-in forensic raw-frame capture (see the message processor's Raw Frame Capture section). install.sh writes it as false and --upgrade adds it to an existing .env that lacks it, never overwriting a value you set.

MESSAGE_PROCESSOR_ID isn't in this table: it's not read from .env at all. Each instance's ID is in its own message-processors/{id}.env, decided by scripts/install.sh's prompts (replacing an existing processor, or how many are currently implemented fleet-wide plus how many to add) at generation time — see Message Processor for the full flow.

Variable	Required	Default
`S3_BUCKET`	✅	—
`AWS_DEFAULT_REGION`	✅	—
`ARCHIVE_PROCESSOR_AWS_ACCESS_KEY_ID`	✅	—
`ARCHIVE_PROCESSOR_AWS_SECRET_ACCESS_KEY`	✅	—
`ARCHIVE_COMPACTION_AWS_ACCESS_KEY_ID`	✅	—
`ARCHIVE_COMPACTION_AWS_SECRET_ACCESS_KEY`	✅	—
`RABBITMQ_HOST`	✅	—
`RABBITMQ_PORT`	❌	`5672`
`RABBITMQ_USERNAME`	✅	—
`RABBITMQ_PASSWORD`	✅	—
`REDIS_HOST`	✅	—
`REDIS_PORT`	❌	`6379`
`REDIS_PASSWORD`	✅	—
`MQTT_HOST`	❌	—
`MQTT_PORT`	❌	`1883`
`MQTT_USERNAME`	❌	—
`MQTT_PASSWORD`	❌	—

Map

Variable	Required	Default
`MAP_LISTEN_HOST`	❌	`0.0.0.0`
`MAP_LISTEN_PORT`	✅	—
`MAP_HTTP_HOST`	❌	`0.0.0.0`
`MAP_HTTPS_PORT`	❌	`19443`
`MAP_REDIS_HOST`	✅	—
`MAP_REDIS_PORT`	❌	`6379`
`MAP_REDIS_PASSWORD`	❌	—
`MAP_CENTER_LATITUDE`	❌	—
`MAP_CENTER_LONGITUDE`	❌	—
`MQTT_HOST`	❌	—
`MQTT_PORT`	❌	`1883`
`MQTT_USERNAME`	❌	—
`MQTT_PASSWORD`	❌	—

MQTT is optional: set MQTT_HOST and the service publishes a minimal Home Assistant presence (discovery, running version, start time) and nothing else -- no telemetry loop.

MAP_REDIS_HOST is this role's own dedicated map-redis (bundled in docker-compose.map.yaml), never core Redis -- scripts/install.sh defaults the prompt to map-redis (the bundled container's service name) rather than reading a REDIS_HOST value from another role's .env. MAP_REDIS_PASSWORD is optional: map-redis ships with no auth by default (same "trusted network" posture as management-ui), so this is only set when pointing at an external, already-secured Redis instead. MAP_LISTEN_PORT must match whatever message-processor's MAP_UDP_PORT (above) is set to. MAP_CENTER_LATITUDE/MAP_CENTER_LONGITUDE are the frontend's "center" marker/recenter reference point -- both optional (together, or neither), read at runtime and served to the frontend over GET /api/config rather than baked into the frontend bundle, so changing them takes effect on the next page load with no image rebuild.

Migration note: these were named MAP_HOME_LATITUDE/MAP_HOME_LONGITUDE before this rename. There is no dual-read period -- the old names are no longer recognized at all, so an existing deployment must switch to the new names for the center marker/recenter to keep working. Running scripts/install.sh --upgrade does this automatically, rewriting an existing .env's MAP_HOME_LATITUDE/MAP_HOME_LONGITUDE lines to MAP_CENTER_LATITUDE/MAP_CENTER_LONGITUDE in place and preserving whatever value was set; no manual edit is required.

The frontend's own build-time configuration (VITE_MAP_API_BASE_URL) is not part of this table -- see Map's "Frontend Configuration" section. It's a Vite import.meta.env.VITE_* value baked into the frontend's static bundle at npm run build time, read from map/frontend/.env, not a runtime container environment variable scripts/install.sh or docker-compose.map.yaml ever touches. That same section also covers VITE_CENTER_LATITUDE/VITE_CENTER_LONGITUDE, which only matter for npm run dev -- they have no effect on a production build or the published image.

See the component pages for everything beyond configuration: Receiver, Message Processor, Archive Processor, Archive Compaction, and Data Runners (logging convention, plus one page per runner).

Upgrading and Rolling Back

SKYFOLLOWER_VERSION in each role's .env pins the tag every ghcr.io/brentio/skyfollower-* image resolves to. scripts/install.sh --upgrade re-resolves the latest release tag, re-fetches each role directory's docker-compose.*.yaml and any config/*.example templates (so a service, label, or port mapping added to a compose file since the last install/upgrade actually reaches an existing deployment), and runs docker compose pull && up -d in every role directory found under the install root — no prompting. SKYFOLLOWER_VERSION is rewritten only once that role's pull and up -d succeed; a failed role keeps its old value and the run continues with the remaining roles, then prints a per-role summary and exits non-zero so you know what to re-run. Every compose file is replaced, including docker-compose.message-processor.yaml and docker-compose.receiver.yaml; the per-instance env files and generated docker-compose.instances.yaml next to them are left alone. An install that still holds per-instance service blocks inside those two files is converted first (each block becomes an env file and a generated service, the original compose file is kept as *.bak, container and data directory names are unchanged). A config/* file already derived from its .example template is never clobbered — same no-clobber behavior a first install uses. Rolling back is the same operation in reverse: edit SKYFOLLOWER_VERSION to an older release tag by hand and run docker compose pull && docker compose up -d in that role's directory.

To install or upgrade to a development build instead of a release, set the single branch variable — see Testing a Dev Build.

Maintenance

Each component has different fault-tolerance characteristics, so the safe procedure for taking one down — OS patching, a host reboot, a container image update — depends on what it is and what depends on it.

Receiver — no draining needed. It's the origin of the data, not a consumer of anything upstream, so stopping it is simply a coverage gap in the ADS-B feed itself; every downstream component (RabbitMQ, message processors, archive) is unaffected. Stop it, restart it, done. An optional second receiver instance (same docker-compose.receiver.yaml, its own host and .env) is maintained identically and independently of the SDR-hosting instance, with its own independently-generated identity. Notably, a message processor resize (adding, removing, or restarting any number of them) never touches a receiver at all — see Architecture's Scaling Message Processors section.

Core (rabbitmq, redis, ofelia, runners) — stop ofelia first, so a scheduled runner isn't killed mid-write to Redis, and let any currently-running runner finish (or stop it). Stopping message processors before taking RabbitMQ/Redis down isn't strictly required — they retry their connections and, once reconnected, resume exactly where they left off — but doing so avoids noisy reconnect-retry logging during the maintenance window. The archive processor is the same story: it also depends on Redis now, for split-flight stitching, but that dependency fails soft — a Redis outage doesn't block or fail an archive write, it just means stitching quietly stops working, and any flight archived during the window that would have merged onto a recent segment stays as a separate, un-merged record instead (no data loss, just a permanent miss for that pair, since there's no later backfill). Stopping the archive processor first avoids that miss and the log noise, but isn't required for correctness. Bring everything back in this order: Redis, then RabbitMQ, then ofelia. RabbitMQ's consistent-hash exchange and every message processor's queue binding are durable state held by the broker, so they survive a RabbitMQ restart intact; only losing RabbitMQ's data directory entirely re-establishes slot order from scratch on the next round of bindings.

Upgrading an existing host to Redis authentication — Redis now starts with requirepass and rejects unauthenticated connections. Every host that already has a .env predating this change needs REDIS_PASSWORD added to it before pulling the new images, on every host that talks to Redis: core, message processor, archive, and management-UI. Set the same value everywhere Redis is reached across the network — there is one password, not one per component. Upgrading core first and everything else after (or in any order) both fail the same way in between: whichever side updates first simply can't authenticate to the other until the last host catches up, the same as rotating any shared credential. There's no partial or silent failure mode to worry about — a client missing REDIS_PASSWORD fails loudly at startup (shared/config.py treats it as required), rather than connecting unauthenticated or hanging.

Upgrading an existing host to the split RabbitMQ application/admin users — re-running scripts/install.sh --role core (or scripts/install.sh --upgrade) against an already-running broker is the entire migration: provision_rabbitmq_users demotes the application user (RABBITMQ_USERNAME) to SkyFollower's own scoped, tag-less permissions regardless of whether it's still full-admin from before this existed, creates any of the application, admin and monitoring users the broker is missing, and resets every existing one's password to the value in .env. All of it is idempotent, so this is also just what happens on every ordinary re-run or --upgrade of the core role (an unreachable broker only warns) — nothing else in this section's drain order changes.

A message processor: stopping vs. unbinding. These look like the same action from the operator's side and are not. Stopping is free — the same recovery path as any restart (a deliberate stop, a crash, a host reboot): RabbitMQ retains its queue (skyfollower-message-processor-{id}, durable) and it simply grows while the processor is down, and the active flight store is durable with recovery driven by message timestamps rather than wall-clock time, so flights in progress when it stopped resume correctly instead of being archived just because time passed. See the Message Processor page's Fault Tolerance section for the full recovery behavior. Unbinding — deleting the queue — is a resize, not a stop: it removes that processor's slot and renumbers every slot after it, reshuffling aircraft per the table in Architecture's Scaling Message Processors section (~20% removing the last-bound processor, ~68% removing one from the middle). Stopping a container never unbinds its queue by itself; unbinding is a deliberate, separate action (see the runbooks below). A node running several processors follows the same rule per-instance: stopping the node leaves every one of its bindings intact, and nothing renumbers until a queue is actually deleted.

Archive processor — stop it. Message processors keep publishing completed flights to the durable skyfollower-archive RabbitMQ queue (or their own local fallback if RabbitMQ is also unavailable at the time), which simply grows while the archive processor is down. Restart it and it drains normally — already fault-tolerant by design, no special handling needed.

Archive compaction — no live process to stop; it's a one-shot job its own ofelia instance launches on a daily schedule, not a long-running container. Taking the host down between scheduled runs just means the next run picks up wherever the watermark was left — see Archive Compaction's own Fault Tolerance characteristics (parity checks, watermark catch-up).

Management UI — stop it. It's a stateless REST API with no queue and no background work; nothing writes to config:rules/config:areas while it's down, and message processors keep running against whatever rules/areas were last saved. Restart it and it's immediately usable — nothing to drain or resync.

Runbooks

Add a message processor to a node. Re-run scripts/install.sh --role message-processor against that node: answer "no" to "replacing an existing message processor," give the fleet-wide count currently implemented and how many to add on this host, confirm, then docker compose up -d. This adds the new env file(s) and service(s) to the node's message-processors/ folder and docker-compose.instances.yaml without touching already-running instances. Nothing else on any host is touched — no receiver restart, no other node involved.

Remove a message processor permanently. Stop the highest-numbered instance on its node, then unbind it — delete its skyfollower-message-processor-{id} queue in the RabbitMQ management UI (or rabbitmqctl delete_queue) — and remove its data directory and its service block from that node's docker-compose.message-processor.yaml. Removing anything other than the last-bound processor reshuffles most in-flight aircraft, producing split flight records; the archive processor's split-flight stitching merges them back together automatically. This is degraded archive quality for a few minutes, not data loss.

Cut over to consistent-hash routing (a one-time sequence for a deployment still on the pre-x-consistent-hash exchange). See Receiver's Cutover section for the full operational sequence, including the deliberate binding-order step — the only moment slot order can be chosen.

Upgrade and rollback. See Upgrading and Rolling Back above.

Deployment ​

Compose Files ​

Components ​

Configuration ​

Receiver ​

Core (rabbitmq, redis, ofelia, runners) ​

Management UI ​

Message Processor ​

Archive ​

Map ​

Upgrading and Rolling Back ​

Maintenance ​

Runbooks ​