Redis Sentinel for Failover

A Redis master-replica setup gives you read scaling and a backup copy of your data, but it doesn’t give you automatic recovery: if the master crashes, a human has to notice, pick a replica, promote it, and reconfigure every client and every other replica. Redis Sentinel is a separate, purpose-built system that watches your master and replicas around the clock, decides when the master is truly down, and performs that promotion for you — with no human in the loop. It’s the standard way to get high availability out of classic Redis replication (as opposed to Redis Cluster, which bundles HA with sharding).

Overview / How Sentinel Works

Sentinel is not a mode of your existing Redis server — it’s a distinct process. You start it with the redis-sentinel binary (or redis-server --sentinel), pointed at its own configuration file. A single Sentinel process is not enough for production: you run a cluster of Sentinel processes (three is the typical minimum), usually spread across different hosts or availability zones, all configured to watch the same master. This matters because Sentinel’s failure detection is quorum-based — it deliberately avoids trusting any single observer.

Each Sentinel continuously sends PING commands to the master and to every replica it discovers (Sentinel learns about replicas automatically via the master’s INFO output, and about other Sentinels via a pub/sub channel — you only ever configure the master’s address). If a monitored instance doesn’t reply within down-after-milliseconds, that one Sentinel marks it SDOWN (Subjectively Down) — “subjective” because it’s just one node’s opinion, and could be caused by a network blip between that Sentinel and the master rather than a real outage. The Sentinel then asks the other Sentinels for their opinion. Once a configured number of Sentinels (the quorum) agree the master is unreachable, the state is upgraded to ODOWN (Objectively Down), and a failover is triggered.

Because more than one Sentinel could try to run the failover at once, the Sentinels first hold a leader election (using a variant of the Raft consensus algorithm) so exactly one Sentinel acts as the failover leader. That leader picks the best replica to promote — preferring higher replica-priority, then more up-to-date replication offset, then the lowest run ID as a tiebreaker — sends it REPLICAOF NO ONE to make it a master, reconfigures the remaining replicas to replicate from the new master, and publishes the new topology over pub/sub so every other Sentinel (and every subscribed client) learns the new master’s address. When the old master eventually comes back online, Sentinel reconfigures it as a replica of the new master rather than letting two masters coexist.

Because each Redis instance is single-threaded, every command a Sentinel issues (a PING, a REPLICAOF) is applied atomically with no interleaving from other clients — there’s no risk of a promotion command racing a write mid-execution. The key design idea to internalize: clients should never hardcode “the master’s IP” — they should ask Sentinel for it, so that when Sentinel changes the master, every client transparently follows.

Syntax

Starting a Sentinel process:

redis-sentinel /etc/redis/sentinel.conf

Core sentinel.conf directives:

Directive	Meaning
sentinel monitor <name> <ip> <port> <quorum>	Start watching a master under a logical name, with the minimum number of Sentinels that must agree it’s down
sentinel down-after-milliseconds <name> <ms>	How long a master/replica must be unresponsive before this Sentinel calls it SDOWN
sentinel failover-timeout <name> <ms>	Time budget for the whole failover process before it’s retried or aborted
sentinel parallel-syncs <name> <n>	How many replicas resync from the new master simultaneously (keep low to limit load)
sentinel auth-pass <name> <password>	Password Sentinel uses to authenticate to the monitored master/replicas

Once Sentinels are running, you talk to them with plain redis-cli, pointed at a Sentinel’s port (default 26379) instead of a normal Redis port, using the SENTINEL command family:

Command	Purpose	Time complexity
SENTINEL MASTERS	List every monitored master and its current state	O(N)
SENTINEL MASTER <name>	Details for one monitored master	O(1)
SENTINEL SLAVES <name>	List replicas of a monitored master	O(N)
SENTINEL SENTINELS <name>	List the other Sentinels watching this master	O(N)
SENTINEL GET-MASTER-ADDR-BY-NAME <name>	Return the current master’s ip/port — what clients should call	O(1)
SENTINEL CKQUORUM <name>	Check whether enough Sentinels are reachable to reach quorum	O(N)
SENTINEL FAILOVER <name>	Force a failover immediately, without waiting for ODOWN	O(1) to trigger

Examples

Example 1 — a Sentinel doesn’t change how you talk to Redis itself; ordinary commands and introspection still work exactly the same on the master. ROLE is the cheap, scriptable way to check whether an instance you’ve connected to is currently a master or a replica — useful in health checks:

SET user:1001:name "Ada"
GET user:1001:name
ROLE

Output:

OK
"Ada"
1) "master"
2) (integer) 0
3) (empty array)

The third line of the ROLE reply would list connected replicas and their offsets if any were attached; here there are none, so it’s empty. In production you generally wouldn’t read this directly — you’d let Sentinel track it for you and just ask Sentinel who the master is.

Example 2 — a minimal sentinel.conf monitoring one master named mymaster, requiring 2 of the Sentinels to agree before declaring it down, then starting the process:

port 26379
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1

redis-sentinel /etc/redis/sentinel.conf

This file isn’t typed at a redis-cli prompt — it’s saved to disk and loaded when the Sentinel process starts. With three Sentinels running this same config (pointed at the same master), quorum 2 means any two of them agreeing is enough to call it ODOWN and start a failover.

Example 3 — once Sentinels are running, applications and operators query them for the live topology instead of hardcoding an address:

SENTINEL GET-MASTER-ADDR-BY-NAME mymaster
SENTINEL MASTERS

Output:

1) "127.0.0.1"
2) "6379"
1) 1) "name"
   2) "mymaster"
   3) "ip"
   4) "127.0.0.1"
   5) "port"
   6) "6379"
   7) "flags"
   8) "master"
   9) "num-slaves"
  10) "2"
  11) "quorum"
  12) "2"
  ... (additional fields omitted)

GET-MASTER-ADDR-BY-NAME is the call your client library makes on every connection (or on connection error) so it always talks to whichever node is currently master, even right after a failover swaps it out.

How Failover Works Step by Step

Walking through what happens when a master actually dies:

• Every Sentinel PINGs the master (and replicas) roughly once per second.
• The master stops responding. Each Sentinel that notices, independently, waits out down-after-milliseconds and then marks it SDOWN.
• An SDOWN Sentinel asks its peer Sentinels, via SENTINEL is-master-down-by-addr, whether they see the same thing.
• Once the number of agreeing Sentinels reaches the configured quorum, the state becomes ODOWN and a failover is authorized.
• The Sentinels run a Raft-style leader election among themselves to pick exactly one Sentinel to drive the failover.
• The leader ranks candidate replicas by priority, then replication offset (most up-to-date wins), then lowest run ID, and picks a winner.
• The leader sends REPLICAOF NO ONE to the winning replica, promoting it to master.
• The leader sends REPLICAOF <new-master-ip> <new-master-port> to every other replica.
• The new configuration is broadcast over Sentinel’s pub/sub channels so every Sentinel — and any subscribed client — learns the new master’s address.
• If the old master returns, Sentinel demotes it to a replica of the new master instead of allowing a split-brain with two masters.

Common Mistakes

Mistake: running a single Sentinel. One Sentinel has no one to reach quorum with, and it’s itself a single point of failure — if that one process dies, you lose automatic failover entirely. Always run at least three Sentinels, ideally each on separate hosts or availability zones.

Mistake: talking to a normal Redis port with SENTINEL commands. SENTINEL is only understood by a process actually running in Sentinel mode; sending it to a regular redis-server fails:

SENTINEL MASTERS

Output:

(error) ERR unknown command 'SENTINEL', with args beginning with: 'MASTERS', 

Fix: connect with redis-cli -p 26379 (or whatever port your Sentinel processes listen on), not the Redis master’s port.

Mistake: hardcoding the master’s IP in application config. This defeats the entire point of Sentinel — after a failover, the app keeps writing to a node that’s now a plain replica (or gone) and every write fails or is silently lost. Fix: have the client ask Sentinel for the current address via SENTINEL GET-MASTER-ADDR-BY-NAME on startup and on reconnect (production client libraries with Sentinel support do this automatically), rather than pointing directly at a fixed host.

Mistake: setting quorum too low, or never testing failover. A quorum of 1 means a single Sentinel’s network hiccup can trigger a needless failover; a team that has never actually run a failover in staging tends to discover configuration mistakes exactly when it matters most, during a real outage. Rehearse it with SENTINEL FAILOVER mymaster in a non-production environment before you depend on it.

Best Practices

• Run an odd number of Sentinels (3, 5, …) so quorum and leader election always have a clear majority.
• Spread Sentinels across different hosts, racks, or availability zones — don’t co-locate all of them with the master they’re watching.
• Use a client library with native Sentinel support, or always resolve the master via SENTINEL GET-MASTER-ADDR-BY-NAME; never hardcode a Redis endpoint in application config.
• Tune down-after-milliseconds deliberately: too low causes false failovers on transient network blips, too high delays recovery during a real outage.
• Keep parallel-syncs low on large datasets so a mass resync after failover doesn’t overload the new master.
• Protect Sentinel with the same authentication as your Redis instances (sentinel auth-pass) if requirepass is set.
• Sentinel gives you availability, not durability — still configure RDB/AOF persistence so promoted replicas don’t just have a faster copy of nothing.
• Periodically rehearse failover in staging so the first real failover isn’t the first time anyone has seen the process run.

Practice Exercises

1. Write a sentinel.conf that monitors a master named orders-master at 10.0.0.5:6379 with quorum 2, a 3-second down-after threshold, and a 30-second failover timeout.

2. Assume three Sentinels are running and you connect to one on port 26379. Write the command you’d run to find the current master’s address for a monitored group named cache-master, and the command to check whether quorum is currently reachable.

3. Explain, in your own words, why a setup with two Sentinels and quorum 2 is worse than three Sentinels with quorum 2 — what specific failure does the third Sentinel protect against?

Summary

• Sentinel is a separate process (redis-sentinel) that monitors master-replica groups and automates failover; it is not built into a normal redis-server instance.
• Failure detection is quorum-based: one Sentinel’s SDOWN becomes an actionable ODOWN only once enough Sentinels agree.
• A Raft-style election picks one Sentinel to lead each failover, avoiding conflicting promotions.
• The winning replica is promoted with REPLICAOF NO ONE; other replicas are repointed with REPLICAOF to the new master.
• Clients should resolve the master via SENTINEL GET-MASTER-ADDR-BY-NAME, never a hardcoded address.
• Run at least three Sentinels on separate failure domains, and actually test failover before you need it in production.