Configuration Guide

8.2.1 Process User and Privilege Drop

8.2.1.1 --user

Description OS username to drop privileges to after the HTTPS and HTTP listeners are ready. Must be started as root (UID 0) for this option to take effect. If omitted and started as root, replication-manager continues running as root with a log warning.
Type String
Default ""
Example "repman"

replication-manager supports a privilege-drop model: it starts as root to bind the HTTPS API port (which may be 443 or another low-numbered port), then immediately drops to a less-privileged OS user once the listeners are accepting connections.

7.2.1.1.1 Drop sequence

1. Start as root
2. Bind HTTPS API port and HTTP dashboard port
3. Wait until both listeners are ready
4. chown -R <user>:<group> {monitoring-datadir}     ← ownership hand-off
5. syscall.Setgid(<gid>)                             ← drop group
6. syscall.Setuid(<uid>)                             ← drop user (irreversible)
7. Adjust any config dirs that are no longer writable under the new user
8. Start cluster monitoring goroutines

Steps 5–6 use syscall.Setuid / syscall.Setgid, which are irreversible on Linux — the process cannot regain root privileges after this point.

7.2.1.1.2 File ownership requirements

Before starting, the target user must own (or have write access to) the key directories:

# Create a dedicated system user
useradd -r -s /sbin/nologin repman

# Give it ownership of config and data directories
chown -R repman:repman /etc/replication-manager
chown -R repman:repman /var/lib/replication-manager

# The encryption key (generated as root) should remain root-readable only
chmod 600 /etc/replication-manager/.replication-manager.key

replication-manager will automatically chown the working directory (monitoring-datadir) to the target user before dropping, so files written during the root phase (encryption keys, initial config) are accessible afterwards. If any writable config sub-directories are not accessible under the new user, they are transparently remapped to paths under the target user's home directory (~/.config/replication-manager/).

7.2.1.1.3 systemd unit example

[Unit]
Description=replication-manager
After=network.target

[Service]
Type=simple
# Start as root so the process can bind privileged ports, then drop to repman
ExecStart=/usr/bin/replication-manager monitor --user repman --config /etc/replication-manager/config.toml
Restart=on-failure

[Install]
WantedBy=multi-user.target

If replication-manager does not need to bind a port below 1024, you can skip the privilege drop entirely and start directly as the target user:

[Service]
User=repman
Group=repman
ExecStart=/usr/bin/replication-manager monitor --config /etc/replication-manager/config.toml

In this case the --user flag is ignored (only root can call setuid).

7.2.1.1.4 Docker rootless alternative

The official Docker images provide rootless variants that run as repman (UID/GID 10001) from the start, removing the need for any privilege drop:

docker run -u repman \
  -v /etc/replication-manager:/etc/replication-manager \
  -v /var/lib/replication-manager:/var/lib/replication-manager \
  signal18/replication-manager:latest-rootless

8.2.2 Configuration File Security

replication-manager provides password obfuscating security by implementing AES encryption.

An encryption key must be generated by running replication-manager keygen as root. This ensures that no unprivileged user can read the contents of the encryption key.

With the key now generated, you can create encrypted passwords using replication-manager password. Example:

# replication-manager password secretpass
Encrypted password hash: 50711adb2ef2a959577edbda5cbe3d2ace844e750b20629a9bcb

You can now replace your password in the configuration file using this encrypted hash:

user = "root:50711adb2ef2a959577edbda5cbe3d2ace844e750b20629a9bcb"

When an encryption key is detected at replication-manager monitor start, the encrypted passwords will be automatically decrypted by the application. There is no further configuration change required.


8.2.3 API Security Configuration

replications-manager-cli clients and API use JWT TLS protocol over https.

The REST API is secured using encrypted token that is used to validate user:password credential of the API, the client will use same default password and so enter the API without asking a password but if the configuration api-credentials is changed, all client will prompt for the password unless given the correct parameter user and password flags.

api-credentials (2.0), user (1.1)
Item Value
Description Rest API credential in [user]:[password] format
Type string
Default Value "admin:repman"

At startup of the monitor some x509 certificates are loaded from the replication-manager share directory to ensure TLS https secure communication.

Replace the files with your own certificate to make sure your deployment is truly secured.

`monitoring-ssl-cert (2.1)
Item Value
Description HTTPS & API TLS certificate
Type string
Default Value ""
`monitoring-ssl-key (2.1)
Item Value
Description HTTPS & API TLS key
Type string
Default Value ""
# Key considerations for algorithm "RSA" ≥ 2048-bit
openssl genrsa -out server.key 2048

# Key considerations for algorithm "ECDSA" ≥ secp384r1
# List ECDSA the supported curves (openssl ecparam -list_curves)
openssl ecparam -genkey -name secp384r1 -out server.key
openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650

In addition at startup replication-manager monitor will generate in memory extra self signed RSA certificate to ensure token encryption exchange for JWT


8.2.4 Database Security Configuration

db-servers-tls-ca-cert (2.0)
Item Value
Description Path to the database connection TLS authority certificate.
Type string
Default Value ""
db-servers-tls-client-cert (2.0)
Item Value
Description Path to the database connection TLS client certificate.
Type string
Default Value ""
db-servers-tls-client-key (2.0)
Item Value
Description Database TLS client key.
Type string
Default Value ""
replication-use-ssl (1.0)
Item Value
Description Replicas connect to their primary over TLS (MASTER_SSL=1 in every CHANGE MASTER).
Type boolean
Default Value false
replication-ssl-ca (3.1)
Item Value
Description With replication-use-ssl, the CA file on the database hosts that signed the primaries' certificates (MASTER_SSL_CA). Named only when the primary runs a configured certificate. Empty: the CA shipped with the ssl tag on provisioned databases, else the replica's own ssl_ca when it is the primary's too.
Type string
Default Value ""

On MariaDB, the CA a replica verifies its primary with cannot come from any option file: ssl-ca/ssl-cert/ssl-key configure the server side. It lives only in master.info, set by CHANGE MASTER, which keeps every parameter it does not name. replication-manager therefore decides the TLS material of every CHANGE MASTER (switchover, failover, rejoin, reseed), reading the primary live:

Primary Emitted
Unreachable nothing: the channel keeps what it has
No TLS (have_ssl not YES), replication-use-ssl off MASTER_SSL=0 — a MariaDB 11.4+ replica defaults to TLS and would not connect
Auto-generated certificate (MariaDB 11.4+, ssl_cert empty) MASTER_SSL_CA='' — verified through the replication password (zero-configuration TLS); any CA would break it
Configured certificate, replication-use-ssl on MASTER_SSL_CA from replication-ssl-ca, else the shipped CA (ssl tag, provisioned), else the replica's ssl_ca when equal to the primary's; otherwise the channel keeps its CA

Measured behaviour (MariaDB 12.3 and 10.11), which these rules follow:

  • a CA that did not sign the primary's certificate cuts the channel with error 2026 — with or without MASTER_SSL_VERIFY_SERVER_CERT; a CA file that does not exist is error 2026 too;
  • a primary on its auto-generated certificate is verified with no CA, the replication user authenticating with mysql_native_password, ed25519 or parsec;
  • MASTER_SSL_CERT/MASTER_SSL_KEY are never emitted, MASTER_SSL_VERIFY_SERVER_CERT is never forced.

Two security states watch every replication channel (also those set by hand):

State Opens when
WARN0235 the channel is encrypted, has no CA, and verification is off or the release has no zero-configuration TLS (MySQL, MariaDB before 11.4): any certificate of the system trust store is accepted as the primary
WARN0236 the TLS handshake fails (Last_IO_Errno 2026): the CA is missing or did not sign the primary's certificate

The monitor only reports: a switchover or a rejoin issues the CHANGE MASTER that carries the decision. By hand, on the replica: STOP SLAVE; CHANGE MASTER TO MASTER_SSL_CA='<ca>', MASTER_SSL_VERIFY_SERVER_CERT=1; START SLAVE.

The ssl tag makes provisioned servers use the certificates replication-manager generates (CA shipped at /etc/mysql/ssl/ca-cert.pem, server certificate naming every database host). It does not enforce TLS on client connections: that is the separate strictssl tag (require_secure_transport).


8.2.4.1 Passwords of the accounts replication-manager manages

Available since: replication-manager v3.1.44

replication-manager creates or changes database accounts in a few cases: the accounts of app databases (app-db-auto-create), the sponsor user at provisioning, the password rotation (rotate-passwords), a monitoring credential changed in Vault config store v2 mode, and the Spider shard proxy passwords.

On MariaDB, the hash is sent, never the password. MariaDB writes an account statement to the binary log as it receives it, so a password sent in clear would reach the binlogs, the replicas' relay logs and the binlog backups. replication-manager sends IDENTIFIED BY PASSWORD '<hash>' (the mysql_native_password hash) instead. Consequences:

  • The account always uses the mysql_native_password authentication plugin, whatever the server's default (an ed25519 default is not applied to these accounts).
  • Password validation plugins (simple_password_check, cracklib_password_check) cannot check a password given as a hash. With strict_password_validation=ON the server refuses the hash (error 1290): replication-manager then sends the password, which the plugins check, and which reaches the binary log as before.
  • The binary log holds the hash: an unsalted double SHA1, which can be attacked offline. Protect the binlogs and their backups like the accounts themselves.

MySQL and Percona Server rewrite password statements in their binary log themselves: replication-manager sends them the password and their default authentication plugin applies.

Passwords are masked in what replication-manager shows. Texts read from the servers can carry a statement with its password: a replica's replication error, the processlist, the error, audit, SQL error and slow logs, the lost-events viewer. replication-manager replaces every password literal in them with ***** before storing, logging or serving them in the API and the dashboard (IDENTIFIED BY '...', PASSWORD('...'), SET PASSWORD = '...', MASTER_PASSWORD='...', password=...). Accounts created by earlier releases keep their passwords in the old binlogs and backups: rotate them.

8.2.5 Vault Security Configuration

replication-manager provides password obfuscating security by using Vault services.

To access your vault server, you must specify its address in the parameter vault-server-addr flag. Here, the documentation to setup a Vault server.

vault-server-addr (2.3)
Item Value
Description Vault server address
Type string
Default Value ""

To authenticate and access your data stored in your Vault server, you need to use an authentication method. For now, it is only possible to connect using the Approle method. Here, the documentation for creating an Approle role. Be carefull to create the necessary policies for your auth role, you can find documentation here

vault-auth (2.3)
Item Value
Description Vault auth method
Type string
Default Value "approle"

Specify your role and secret id obtained by creating your approle role with the parameters vault-role-id and vault-secret-id flags.

vault-role-id (2.3)
Item Value
Description Vault role id
Type string
Default Value ""
vault-secret-id (2.3)
Item Value
Description Vault secret id
Type string
Default Value ""
[mycluster]
vault-server-addr="http://vault.infra.svc.cloud18:8200"
vault-auth="approle"
vault-role-id="bgsg2f1af-7ce4-e938-46d0-4fbuzibfe4r0d"
vault-secret-id="91b1a039-1597-54b7-f304-c703gyct9d4e"

Vault allow multiple services to store secret data. Replication-manager provides two modes to use Vault services. Vault can be used to store keys in a secret on a Vault server, the config_store_v2 mode. Vault also provides a feature for automatic password rotation, implemented in the database_engine mode.

vault-mode (2.3)
Item Value
Description Vault mode
Type string
Default Value "config_store_v2"

8.2.5.1 Config_store_v2

In this case, the secret path to the remote secret must be specified in the parameters db-server-credential flag for the database credential and replication-credential flag for the replication-manager credential as user:password. The key of your secret, which is store at your secret path, has to be db-server-credential for your database credentials and replication-credential for the replication credential.

Exemple :

secret path : kv/applications/repman
key : db-server-credential
secret key : user:password

That means you can store different keys at the same secret path!

We also need to specify the mount directory of the secret in the vault-mount variable.

vault-mount (2.3)
Item Value
Description Vault mount for the secret
Type string
Default Value "secret"

Exemple of configuration for a secret store at /kv/applications/repman secret Vault path.

[mycluster]
vault-mode="config_store_v2"
db-servers-credential = "applications/repman"
replication-credential = "applications/repman"
vault-mount="kv"

In this case, we store database server credential and replication credential at the same secret path, in two different key named by their respective parameters flags.

Be carefull to create the necessary policies to acces for your secret Vault path, you can find documentation here

8.2.5.2 Database_engine

Vault also allow to perform automatic password rotations with a management service of database credential. It allow to create a vault user that will performs password rotations on the specified database(s) at the desired regular time interval. First, you have to create a database role in which you will specify the connection url of your database, the username and password which allows Vault to access the database, the allowed role for the database and a SQL statement that will perform the rotation password. Here, the documentation to create a database role. Then, you have to create a static-role, with this documentation. A static-role is define by a username, a password and a rotation period.

Exemple of a database and static-role configuration :

#Database configuration
vault write database/config/my-mysql-database \
    plugin_name=mysql-database-plugin \
    connection_url="{{username}}:{{password}}@tcp(proxysql1.emma.svc.cloud18:3306)/" \
    allowed_roles="repman-monitor, repman-replication" \
    username="vaultuser" \
    password="vaultpass"

#SQL statement to perform password rotation
tee rotation.sql <<EOF
ALTER USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';
EOF

#Static-role configuration
vault write database/static-roles/repman-monitor \
    db_name=my-mysql-database \
    [email protected] \
    username="repman" \
    rotation_period=600

vault write database/static-roles/repman-replication \
    db_name=my-mysql-database \
    [email protected] \
    username="repman-rep" \
    rotation_period=600

Once these roles are set up, Vault will perform automatic password rotations for all roles defined in the database role configuration. To allow the repman to access the created roles, you have to specify the secret path to access the roles must be specified in the parameters db-server-credential flag for the database credential role and replication-credential flag for the replication-manager credential role.

Exemple of configuration for two static-role store at "database/static-creds/repman-monitor" and "database/static-creds/repman-replication" secret Vault path.

[mycluster]
vault-mode="database_engine"
db-servers-credential = "database/static-creds/repman-monitor"
replication-credential = "database/static-creds/repman-replication"

8.2.5.3 Rotation of Database Credentials

So far, replication-manager is taking care of the monitoring and the replication credentials. When an authentication error state is trigger, replication-manager will presume that the password has been rotated and will fetch Vault secret path again. Also, we provide a password rotation service which can be accessible from the replication-manager interface or with an api call. This features is available in both Vault mode. In the config_store_v2 mode, replication-manager will generate a new password and update it in the vault secret and the databases. In the database_engine mode, replication-manager will send a resquest of rotation password to vault services, which will generate all the rotation password in the databases.

Rotation without Vault (from release 3.1.44). The same action (rotate-passwords, the dashboard's Rotate Database Credentials, the MCP tool cluster-rotate-passwords) also rotates the passwords when no Vault is configured: replication-manager generates the new passwords, changes them on the databases and keeps them encrypted. It is refused, with the reason, when the new passwords would have no durable place:

  • monitoring-secret-versioning is off (the default; backup-encryption turns it on too): the new passwords would live only in replication-manager's memory, and a restart after the rotation would lose access to every server. Turn it on, or use Vault.
  • no encryption key (.replication-manager.key), or monitoring-save-config off;
  • db-servers-credential or replication-credential set in a configuration file (immutable);
  • a database down, or replication stopped on a replica.

The API answers 409 with the reason; the dashboard greys out the action and shows the reason on hover.