Skip to main content

Add an HA controller

For production, run multiple controllers with Raft consensus. Three-node clusters are the common production minimum, with 2-of-3 quorum surviving a single host loss.

After your single-controller deployment is healthy, follow the steps below for each additional controller you want to add.

Important

Additional requirements for adding an HA controller:

  • The new controller host needs docker and docker compose (same as the bootstrap host) plus 7zip (p7zip-full on Debian/Ubuntu, p7zip on RHEL/Fedora) to extract the encrypted join package.
  • The new host does not need the kziti binary. The join package is self-contained and includes the installer.
  • You will be transferring two things to the new host: the encrypted join package, and the extraction password. These should travel through different channels.
  • The new controller needs its own DNS A record (e.g. ziti2.example.com) and its router needs one too (e.g. ziti2-router.example.com), with ports 1280/tcp, 6262/tcp, and 3022/tcp open inbound.

On the existing controller​

Generate a join package for the new controller:

kziti deploy ha create-join-package \
  --output /tmp/join.zip \
  --node-name ziti-c-2 \
  --controller-host ziti2.example.com \
  --router-host ziti2-router.example.com \
  --admin-password '<your-admin-password>'

Notes on the flags:

  • --node-name is the cluster member name. It must be unique within the cluster.
  • --controller-host and --router-host are the FQDNs of the new controller and its router — not the existing one.
  • Add --controller-ip <new-controller-ip> if the new controller's hostname does not resolve from inside the existing controller container. This injects an extra_hosts entry into a docker-compose override on the existing host so it can reach the new node by name during the join.

kziti produces an AES-encrypted zip at the path you supplied with --output, and prints a randomly generated extraction password.

Transfer the zip to the new host:

scp /tmp/join.zip user@ziti2.example.com:/tmp/

Send the extraction password through a separate secure channel — a password manager, an encrypted message, or similar. Do not put the password in the same email or the same chat as the zip.

On the new controller host​

Install 7zip if it is not already present:

# Debian / Ubuntu
apt-get install -y p7zip-full

# RHEL / Fedora
dnf install -y p7zip

Extract the encrypted package and run the bundled installer:

7z x -p<extraction-password> /tmp/join.zip -o/tmp/kziti-join
bash /tmp/kziti-join/install.sh

If the bootstrap controller's hostname does not resolve from this new host, add --existing-controller-ip <bootstrap-ip> to the install.sh invocation so the new node can reach the bootstrap controller during join.

The new controller starts as a non-voter, syncs Raft state from the existing cluster, and self-promotes to a voting member once it is caught up.

Verify cluster health​

On any controller host, list cluster members:

docker compose -f /opt/kziti/docker-compose.yml exec ziti-controller \
  ziti agent cluster list

The new node should appear with voter: true once promotion completes (typically within a minute of join).