Consumer Groups (XREADGROUP)

A Redis consumer group lets several clients cooperatively read from the same stream so that each entry is handed to exactly one member of the group, instead of every client seeing every message. It turns a stream from a plain append-only log into something closer to a work queue, with load balancing across workers, tracked delivery, and a recovery path if a worker crashes mid-task. This is what XREADGROUP and its companion commands (XACK, XPENDING, XCLAIM, XAUTOCLAIM) are for, and it’s the pattern you reach for whenever multiple workers need to split up jobs, events, or notifications without duplicating effort or silently losing one when a process dies.

Overview: How Consumer Groups Work

A stream (created with XADD, itself O(1) per entry) is just a log of entries, each identified by an ID of the form milliseconds-sequence. Reading it with plain XREAD is stateless: every caller sees every entry from whatever ID it asks for, and it’s entirely the caller’s job to remember where it left off. A consumer group changes that. XGROUP CREATE attaches a named, durable cursor to a stream, and Redis itself remembers, per group, the last-delivered ID plus a structure called the Pending Entries List (PEL) — a record of which entries have been handed to which named consumer, when, and how many times, until they are explicitly acknowledged with XACK.

When a consumer calls XREADGROUP ... STREAMS key >, the special ID > means “give me entries never yet delivered to this group.” Each entry returned this way is added to the PEL under that consumer’s name and will never be handed out again to a different consumer’s > read — but it is not deleted from the stream, and it doesn’t disappear from the PEL either. It stays there, still the responsibility of the consumer it was given to, until that consumer calls XACK. If you call XREADGROUP with an explicit ID (commonly 0) instead of >, you get that specific consumer’s own already-delivered-but-unacknowledged backlog back — useful for a worker resuming after a restart to see what it still owes an acknowledgment for.

Group creation matters: XGROUP CREATE key group id with id set to 0 makes the group start at the very beginning, so the first > read hands out the stream’s entire existing backlog. Setting id to $ starts the group at the current end of the stream, so only entries appended after creation are ever delivered — the usual choice for “only process new events from now on.” XGROUP CREATE requires the target key to already exist as a stream; add MKSTREAM to have Redis create an empty stream on the spot rather than erroring.

Because Redis executes one command at a time on its single main thread, the entire “pick the next undelivered entries, stamp them into the PEL under this consumer, advance the cursor” sequence inside one XREADGROUP call is atomic. Two workers calling XREADGROUP ... > at the same moment can never both receive the same entry — there’s no separate locking step for you to get wrong.

Consumer groups give you at-least-once delivery, not exactly-once. If a worker reads an entry and crashes before calling XACK, nothing automatically redelivers it — it just sits in the PEL, still assigned to a consumer that’s gone, with its idle time climbing. XPENDING lets you inspect exactly which entries are stuck and who “owns” them; XCLAIM (or the more efficient XAUTOCLAIM, added in Redis 6.2) lets a healthy consumer take ownership of entries whose idle time exceeds a threshold you choose, so you don’t steal work from someone still legitimately processing it. That claimed entry can then be processed and acknowledged. This manual-but-supported recovery loop is exactly why the guarantee is “at least once”: a message can in principle be processed twice (if a worker dies after finishing but before acking), but it is never silently dropped.

Syntax

XGROUP CREATE key group id|$ [MKSTREAM]
XREADGROUP GROUP group consumer [COUNT count] [BLOCK milliseconds] [NOACK] STREAMS key [key ...] id [id ...]
XACK key group id [id ...]
XPENDING key group [[IDLE min-idle-time] start end count [consumer]]
XCLAIM key group consumer min-idle-time id [id ...] [IDLE ms] [TIME ms-unix-time] [RETRYCOUNT count] [FORCE] [JUSTID]
XAUTOCLAIM key group consumer min-idle-time start [COUNT count] [JUSTID]

group — the consumer group’s name, unique per stream key.
consumer — a name you choose for the caller; created implicitly the first time it’s used in XREADGROUP.
id|$ in XGROUP CREATE — 0 (or any ID) to start from that point in the backlog, or $ to start from “only new entries.”
> in XREADGROUP — deliver only entries never given to this group before; any other ID re-reads that consumer’s own pending history.
COUNT — maximum entries to return per stream in this call.
BLOCK milliseconds — wait up to this long for new entries instead of returning immediately empty (long-polling).
NOACK — skip adding the entry to the PEL entirely; use only if you don’t need delivery tracking for that read.
min-idle-time (XCLAIM/XAUTOCLAIM) — only claim entries that have been pending at least this many milliseconds.
start (XAUTOCLAIM) — cursor to resume scanning the PEL from; pass 0 on the first call.

Command	Purpose	Time Complexity
`XGROUP CREATE`	Attach a new consumer group to a stream	O(1)
`XREADGROUP`	Read undelivered or pending entries as a named consumer	O(N) per stream, N = entries returned
`XACK`	Remove entries from the group’s PEL	O(N) for N IDs acknowledged
`XPENDING` (summary)	Overview of pending entries per group	O(1)
`XPENDING` (extended)	List individual pending entries with detail	O(N), N = entries returned
`XCLAIM`	Reassign specific pending entries to another consumer	O(log N + M), N = PEL size, M = entries claimed
`XAUTOCLAIM`	Scan and reassign stale pending entries in bulk	O(1) plus O(M) for M entries scanned per call
`XINFO GROUPS` / `XINFO CONSUMERS`	Inspect group/consumer state (lag, pending count, idle time)	O(1) / O(N) consumers

Examples

Example 1: Creating a group and processing a message

XADD orders:stream 1-1 order_id 1001 status pending
XADD orders:stream 2-1 order_id 1002 status pending
XGROUP CREATE orders:stream order_processors 0
XREADGROUP GROUP order_processors worker-1 COUNT 10 STREAMS orders:stream >
XACK orders:stream order_processors 1-1

Output:

"1-1"
"2-1"
OK
1) 1) "orders:stream"
   2) 1) 1) "1-1"
         2) 1) "order_id"
            2) "1001"
            3) "status"
            4) "pending"
      2) 1) "2-1"
         2) 1) "order_id"
            2) "1002"
            3) "status"
            4) "pending"
(integer) 1

Two orders are added with explicit IDs so they’re deterministic for this walkthrough (in production you’d normally use * and let Redis stamp the current timestamp). XGROUP CREATE ... 0 starts the group at the beginning of the stream, so the very first > read by worker-1 receives both existing entries and immediately marks them pending under its name. The final XACK removes only ID 1-1 from the PEL — order 2-1 is still outstanding until acknowledged separately.

Example 2: Two consumers splitting the work

XADD tasks:stream 1-1 job resize_image
XADD tasks:stream 2-1 job send_email
XADD tasks:stream 3-1 job generate_report
XGROUP CREATE tasks:stream task-workers 0
XREADGROUP GROUP task-workers worker-a COUNT 2 STREAMS tasks:stream >
XREADGROUP GROUP task-workers worker-b COUNT 2 STREAMS tasks:stream >
XPENDING tasks:stream task-workers

Output:

"1-1"
"2-1"
"3-1"
OK
1) 1) "tasks:stream"
   2) 1) 1) "1-1"
         2) 1) "job"
            2) "resize_image"
      2) 1) "2-1"
         2) 1) "job"
            2) "send_email"
1) 1) "tasks:stream"
   2) 1) 1) "3-1"
         2) 1) "job"
            2) "generate_report"
1) (integer) 3
2) "1-1"
3) "3-1"
4) 1) 1) "worker-a"
      2) "2"
   2) 1) "worker-b"
      2) "1"

This is the load-balancing behavior that makes consumer groups useful: worker-a asks for up to 2 entries and gets the first two (1-1, 2-1), which advances the group’s delivery cursor. When worker-b then asks for up to 2, only one undelivered entry (3-1) remains, so that’s all it gets — the two workers never receive the same entry. XPENDING with no range arguments returns the summary form: total pending count, the lowest and highest pending IDs, and a per-consumer breakdown.

Example 3: Recovering from a crashed consumer with XCLAIM

XADD alerts:stream 1-1 level critical message "disk full"
XGROUP CREATE alerts:stream alert-workers 0
XREADGROUP GROUP alert-workers worker-1 COUNT 1 STREAMS alerts:stream >
XPENDING alerts:stream alert-workers - + 10
XCLAIM alerts:stream alert-workers worker-2 0 1-1
XACK alerts:stream alert-workers 1-1

Output:

"1-1"
OK
1) 1) "alerts:stream"
   2) 1) 1) "1-1"
         2) 1) "level"
            2) "critical"
            3) "message"
            4) "disk full"
1) 1) "1-1"
   2) "worker-1"
   3) (integer) 0
   4) (integer) 1
1) 1) "1-1"
   2) 1) "level"
      2) "critical"
      3) "message"
      4) "disk full"
(integer) 1

worker-1 reads the alert but, in this scenario, crashes before acknowledging it. XPENDING alerts:stream alert-workers - + 10 is the extended form (- to + covers the full ID range, up to 10 results) and lists each pending entry with its consumer, idle time in milliseconds, and delivery count — here idle time will show as a very small number since almost no time has passed; in a real crash it would be large. XCLAIM ... worker-2 0 1-1 reassigns the entry to worker-2 (a min-idle-time of 0 means “claim regardless of how idle it is,” fine for a demo but normally you’d use a real threshold like 30000 so you don’t grab work from a consumer still actively processing it). worker-2 then finishes the job and calls XACK itself.

Example 4: Bulk recovery with XAUTOCLAIM

XADD queue:stream 1-1 task cleanup
XGROUP CREATE queue:stream workers 0
XREADGROUP GROUP workers worker-1 COUNT 1 STREAMS queue:stream >
XAUTOCLAIM queue:stream workers worker-2 0 0

Output:

"1-1"
OK
1) 1) "queue:stream"
   2) 1) 1) "1-1"
         2) 1) "task"
            2) "cleanup"
1) "0-0"
2) 1) 1) "1-1"
      2) 1) "task"
         2) "cleanup"
3) (empty array)

XAUTOCLAIM key group consumer min-idle-time start scans the PEL starting from start (use 0 the first time) and claims, in a single atomic call, every entry idle at least min-idle-time milliseconds — no need to call XPENDING first and then XCLAIM each ID individually. It replies with three parts: a cursor to pass into the next call if there was more to scan ("0-0" means the scan is complete), the array of claimed entries, and (since Redis 7.0) an array of any entry IDs that were pending but have since been deleted from the stream itself.

How It Works Step by Step

When XREADGROUP ... STREAMS key > runs, Redis: (1) compares the group’s stored last-delivered-id against the stream and finds entries after it; (2) selects up to COUNT of them; (3) for each selected entry, adds an item to the group’s PEL recording that consumer’s name, the current time as the delivery timestamp, and a delivery count of 1; (4) advances the group’s last-delivered-id to the newest entry just handed out; (5) returns the entries to the caller. All of this happens as one atomic step on Redis’s single command-processing thread, so no other client can observe or interleave with a partially-completed delivery. XACK simply deletes the matching entries from the PEL — there is nothing left to track once acknowledged. XCLAIM/XAUTOCLAIM instead rewrite an existing PEL entry in place: the consumer field changes to the new owner, the delivery timestamp resets to now, and the delivery count increments, while the entry itself remains in the PEL until someone eventually calls XACK on it.

Common Mistakes

Mistake 1: Forgetting MKSTREAM on a stream that doesn’t exist yet

XGROUP CREATE missing:stream mygroup 0

If missing:stream doesn’t already exist, this fails with (error) ERR The XGROUP subcommand requires the key to exist. Note that for CREATE you may want to use the MKSTREAM option to create an empty stream automatically. — XGROUP CREATE refuses to silently create a stream unless you say so explicitly. Add MKSTREAM:

XGROUP CREATE missing:stream mygroup 0 MKSTREAM

This creates an empty stream and the group in one step, which is the usual pattern when a consumer’s startup code can’t guarantee the stream already has producers writing to it.

Mistake 2: Never calling XACK

XADD jobs:stream 1-1 task send_invoice
XGROUP CREATE jobs:stream billing-group 0
XREADGROUP GROUP billing-group worker-1 COUNT 1 STREAMS jobs:stream >
XPENDING jobs:stream billing-group

Nothing here errors, which is exactly the trap: after reading, XPENDING still reports (integer) 1 pending entry, permanently owned by worker-1, because it was read but never acknowledged. Left this way, the PEL only grows over time — every message ever delivered stays resident in memory until it’s acked, whether or not the worker actually finished the job. The fix is to acknowledge as the very last step of successful processing:

XADD jobs:stream 1-1 task send_invoice
XGROUP CREATE jobs:stream billing-group 0
XREADGROUP GROUP billing-group worker-1 COUNT 1 STREAMS jobs:stream >
XACK jobs:stream billing-group 1-1
XPENDING jobs:stream billing-group

Now XPENDING reports zero pending entries, because the PEL only ever holds work that’s genuinely still in flight.

Mistake 3: Re-creating a group that already exists

XGROUP CREATE events:stream mygroup 0 MKSTREAM
XGROUP CREATE events:stream mygroup 0

The second call fails with (error) BUSYGROUP Consumer Group name already exists — group names must be unique per stream, and Redis won’t silently reset an existing group’s delivery cursor for you (that would drop track of what’s already pending). If your application might start up more than once against the same stream, check first with XINFO GROUPS, or catch the BUSYGROUP error in your client code and treat it as “already set up, continue”:

XGROUP CREATE events:stream mygroup 0 MKSTREAM
XINFO GROUPS events:stream

XINFO GROUPS returns each group’s name, consumer count, pending count, last-delivered-id, and lag — a safe, side-effect-free way to check what already exists before deciding whether to create anything.

Best Practices

Always decide deliberately between XGROUP CREATE ... 0 (process the existing backlog) and XGROUP CREATE ... $ (only new entries) — this choice is easy to get backwards and hard to notice until you’re missing data or replaying far more than intended.
Treat “read but not yet acked” as still your responsibility: call XACK only after the work is durably done, not right after reading.
Periodically monitor each group’s PEL with XPENDING or XINFO CONSUMERS so a crashed or hung consumer’s backlog doesn’t grow unnoticed.
Prefer XAUTOCLAIM over manually pairing XPENDING with XCLAIM for bulk recovery — it’s a single atomic call and avoids the small race between listing pending entries and claiming them one by one.
Give consumers stable, meaningful names (such as hostname plus process ID) so that after a restart, a worker can find and resume its own prior pending entries by reading with an explicit ID instead of >.
Set a real min-idle-time when claiming (not 0) so you don’t steal work from a consumer that’s simply still processing it normally.
Use BLOCK on XREADGROUP for long-polling workers instead of a tight loop of immediate reads — it’s far cheaper on both the client and the server.
Size COUNT to what a consumer can actually process promptly; handing out a huge batch it can’t get through quickly just inflates the PEL and delays other consumers waiting on the same backlog.

Practice Exercises

1. Build a two-worker pipeline. Create a stream, add three entries, create a consumer group starting from the beginning, and have two differently-named consumers each call XREADGROUP with COUNT 2 in turn. Confirm with XPENDING that all three entries are accounted for and that no entry was delivered to both consumers.

2. Simulate and recover from a crash. Read an entry with one consumer but deliberately skip XACK. Use XPENDING to find it, then reassign it to a second consumer with XCLAIM (or XAUTOCLAIM) and finish by acknowledging it. Verify the PEL is empty afterward.

3. Inspect group health. Create a group, deliver a few entries without acking them, then run XINFO GROUPS and XINFO CONSUMERS on the group. Identify which fields tell you how many entries are outstanding and which consumer is holding them.

Summary

A consumer group is a durable, named cursor over a stream that ensures each entry (read via >) goes to only one consumer in the group.
XGROUP CREATE key group 0|$ [MKSTREAM] creates the group; 0 replays the existing backlog, $ starts from new entries only.
Every entry delivered via > is recorded in the group’s Pending Entries List (PEL) until explicitly removed with XACK — forgetting to ack leaks memory and leaves work permanently “in flight.”
XPENDING inspects the PEL; XCLAIM and the more efficient XAUTOCLAIM reassign stale entries from a dead or stuck consumer to a healthy one.
Redis’s single-threaded execution makes each XREADGROUP call atomic, so concurrent consumers never receive duplicate entries from the same > read.
Consumer groups provide at-least-once delivery, not exactly-once — design processing to be idempotent where possible.

Consumer Groups (XREADGROUP)

Overview: How Consumer Groups Work

Syntax

Examples

Example 1: Creating a group and processing a message

Example 2: Two consumers splitting the work

Example 3: Recovering from a crashed consumer with XCLAIM

Example 4: Bulk recovery with XAUTOCLAIM

How It Works Step by Step

Common Mistakes

Mistake 1: Forgetting MKSTREAM on a stream that doesn’t exist yet

Mistake 2: Never calling XACK

Mistake 3: Re-creating a group that already exists

Best Practices

Practice Exercises

Summary

Table of Contents