Design Real-Time Chat Notifications and Presence

Purpose and Outcome

Real-time chat feels simple when a friend appears online, a typing indicator flashes, and a message notification lands on the right device. The design work underneath is less simple: the system must track short-lived connections, distribute message events to the right subscribers, suppress duplicate alerts, and recover when phones sleep, browser tabs disappear, or network partitions hide the truth. In a system design course, this lesson is a compact case study in managing fast state that is useful even though it is temporary.

The outcome is a design vocabulary and implementation model for chat notifications and presence. You should be able to separate durable chat data from ephemeral presence data, decide when to use WebSockets or mobile push, explain how heartbeats and leases make online status approximate, and reason about fanout, idempotency, latency, privacy, and failure recovery.

Core Mechanism

A typical chat system has four related streams. The message stream records durable facts such as message id, room id, sender id, body, timestamp, and attachments. The realtime delivery stream carries events to active clients through WebSockets, Server-Sent Events, or a similar bidirectional channel. The notification stream creates offline alerts through Apple Push Notification service, Firebase Cloud Messaging, email, or in-app badge counters. The presence stream tracks connection, heartbeat, idle, typing, and last-seen events.

The important internal distinction is ownership of truth. Messages usually live in a database or append-only log because users expect history to survive restarts. Presence usually lives in memory, Redis, or another low-latency store because online state changes frequently and becomes stale quickly. A user can be online from multiple devices, so presence should be modeled as user plus device or user plus connection. The user is online if at least one non-expired session exists. If every session disconnects or misses its lease renewal, the user becomes offline.

Realtime notification begins when the chat write path accepts a message. After authorization and persistence, the service publishes a message-created event to a broker or internal bus. A fanout worker looks up room membership, removes the sender unless the product wants self-echo on other devices, and decides the delivery path for each recipient. Connected recipients receive the event over their gateway connection. Disconnected recipients receive push notifications, but often only after a short delay so a reconnecting client does not get both a WebSocket event and a push alert.

The gateway is a stateful edge process. It terminates WebSocket connections, authenticates tokens, maps connection ids to users and rooms, sends ping frames or application heartbeats, and forwards events from backend streams to local sockets. Because each gateway only knows its local sockets, a shared registry is needed when fanout workers are separate from gateways. That registry can be Redis sets, a distributed presence service, or broker topics partitioned by user id.

API Anatomy

A practical API has explicit commands and events. Client commands include connect with an access token, subscribe room, send message, ack message, heartbeat, typing start, typing stop, and set activity state. Server events include message created, message edited, message deleted, notification count changed, presence changed, typing changed, and delivery receipt changed. Commands should be authenticated and authorization-checked; events should carry ids that allow clients to deduplicate after reconnect.

Common fields are connection_id, device_id, user_id, room_id, event_id, sequence, server_time, and expires_at. A sequence number per room or per user mailbox lets clients ask for missed events: reconnect after sequence 105 and fetch everything greater than 105. Presence events need a time-to-live because an offline event can be lost. Typing events should expire even faster, often in a few seconds, because they describe intent rather than durable state.

Example 1: Multi-Device Presence

The first example shows why presence is not a boolean column on the user row. User u1 connects from a phone and a laptop. The phone disconnects, but the laptop is still active, so the user remains online. The deterministic output is a single online user.

from dataclasses import dataclass

@dataclass(frozen=True)
class PresenceEvent:
    user_id: str
    device_id: str
    action: str
    happened_at_ms: int

sessions = {}
events = [
    PresenceEvent('u1', 'phone', 'connect', 1000),
    PresenceEvent('u1', 'laptop', 'connect', 1500),
    PresenceEvent('u1', 'phone', 'disconnect', 3000),
]
for event in events:
    key = (event.user_id, event.device_id)
    if event.action == 'connect':
        sessions[key] = event.happened_at_ms
    else:
        sessions.pop(key, None)

online_users = sorted({user_id for user_id, device_id in sessions})
print(online_users)

This model scales into a presence store by using keys such as presence:u1:phone and presence:u1:laptop with short expirations. Disconnect removes one key. Heartbeat refreshes one key. A query for u1 returns online when at least one key exists. The trade-off is that exact global counts become harder; the benefit is that normal device churn does not incorrectly flip a user offline.

Example 2: Choosing Realtime or Push Delivery

The second example separates recipients from delivery channels. The sender should not receive an ordinary notification for their own message. Active recipient u3 can receive a WebSocket event. Inactive recipient u2 needs a push notification or a later badge update. The expected output is one push decision and one WebSocket decision.

rooms = {
    'r1': {'members': {'u1', 'u2', 'u3'}, 'active': {'u1', 'u3'}},
    'r2': {'members': {'u1', 'u4'}, 'active': {'u4'}},
}
message = {'id': 'm9', 'room_id': 'r1', 'sender': 'u1', 'text': 'ship it'}
room = rooms[message['room_id']]
recipients = sorted(room['members'] - {message['sender']})
for user_id in recipients:
    channel = 'websocket' if user_id in room['active'] else 'push'
    print(f"notify {user_id} through {channel}")

Real systems add notification preferences, mute rules, mention priority, blocked users, quiet hours, and device tokens. Still, the core fanout decision remains the same: compute authorized recipients, classify each recipient by current reachability and preferences, then enqueue delivery work with a stable message-recipient id.

Example 3: Heartbeat Leases

Disconnect events are helpful but unreliable. A laptop can lose Wi-Fi, a process can crash, and a mobile operating system can suspend the app without a clean close frame. Presence therefore uses leases: a session is online only while recent heartbeats keep it alive. Here, users with heartbeat age below the timeout are online; others are offline.

last_seen = {'u1': 1000, 'u2': 2300, 'u3': 2400}
now_ms = 5200
presence_timeout_ms = 3000
state = {}
for user_id, heartbeat_ms in last_seen.items():
    age = now_ms - heartbeat_ms
    state[user_id] = 'online' if presence_timeout_ms > age else 'offline'
print(state)

The output marks u2 and u3 online while u1 is offline. Timeout selection is product-sensitive. A five-second timeout feels responsive but can flicker on weak networks. A one-minute timeout is stable but makes departed users look online. Many products display richer states such as online, idle, recently active, and last seen, which communicates uncertainty better than a hard binary.

Example 4: Idempotent Notification Sends

Queues retry. Workers restart. Brokers can redeliver. Notification workers must therefore treat send requests as at-least-once input and make visible effects idempotent per message and recipient. This example skips the duplicate send for m1 and u2.

deliveries = set()
queue = [
    ('m1', 'u2', 10),
    ('m1', 'u2', 11),
    ('m1', 'u3', 12),
]
for message_id, user_id, attempt in queue:
    key = (message_id, user_id)
    if key in deliveries:
        print(f"skip duplicate {message_id} for {user_id}")
        continue
    deliveries.add(key)
    print(f"send {message_id} to {user_id} on attempt {attempt}")

In production, the delivery key may be stored in Redis, a database table with a unique constraint, or a notification ledger. The ledger can also store status such as queued, sent, failed, suppressed, or acknowledged. Without this guard, a retry storm can send the same phone alert repeatedly even though the chat message itself was stored only once.

Design Choices and Trade-Offs

WebSockets are the common choice for interactive chat because they allow low-latency server-to-client events and client commands on one connection. Server-Sent Events can work for one-way event delivery from server to browser, but the client still needs HTTP calls for commands. Long polling is simpler operationally and can cross strict proxies, but it increases request overhead and latency. Mobile push is not a realtime transport for active chat screens; it is a wake-up and notification mechanism controlled by platform policies.

Fanout can happen on write or on read. Fanout-on-write creates per-user inbox entries when a message is sent, which makes reconnect and badge counts fast but increases write amplification for large rooms. Fanout-on-read stores the message once and computes visibility when users fetch, which reduces write cost but makes unread counts and offline notification targeting more complex. Many chat systems use hybrid designs: direct and small group chats fan out eagerly, while huge channels use topic subscriptions and read-time catch-up.

Presence can be centralized or sharded. A centralized Redis cluster is simple to reason about and gives fast TTL operations, but it becomes a critical dependency and hot users or giant rooms can stress it. Sharding by user id spreads load, but room-level presence queries require gathering data from many shards or maintaining separate room indexes. For very large rooms, approximate presence counts may be better than exact member lists.

Failure Modes and Troubleshooting

One common symptom is users appearing online long after closing the app. The cause is usually missed disconnect cleanup combined with a TTL that is too long or not refreshed consistently. Diagnostic steps are to inspect the connection registry for stale device keys, compare last heartbeat timestamps with gateway logs, and verify expiration policies. The correction is to make heartbeat renewal explicit, set bounded TTLs on all session records, and run a small sweeper only as a backup rather than the main cleanup path.

A second symptom is duplicate push notifications for one message. The cause is often broker redelivery, worker timeout after the platform accepted the push, or using attempt id instead of message-recipient id as the deduplication key. Diagnose by tracing one message id through the outbox, queue, worker logs, and provider response. Correct it with an idempotency ledger, unique constraints, and careful retry classification. Retry network timeouts, but do not blindly retry permanent errors such as invalid device tokens.

A third symptom is a user missing messages after reconnect. The cause may be that WebSocket delivery was treated as durable delivery. A socket write only proves bytes were handed to the connection, not that the user has a recoverable event history. Diagnose by checking whether events have monotonic sequence numbers and whether the client sends last-seen sequence on reconnect. Correct it by storing messages durably, exposing a catch-up endpoint, and requiring client acknowledgements for read or delivered receipts.

Security, Performance, and Reliability

Presence leaks behavior, so authorization is part of the feature. A user should only see presence for contacts, room peers, or users allowed by privacy settings. Last-seen timestamps can be sensitive; coarse states or opt-outs may be required. Gateway authentication should bind the token to a user and optionally a device id, and subscription commands must check room membership. Never trust the client to announce someone else’s presence or unread count.

Performance pressure appears in fanout and connection management. Gateways need backpressure so a slow client does not accumulate unbounded buffers. Message events should be compact and refer to media by id or URL rather than embedding large payloads. Presence broadcasts should be coalesced; if a user reconnects three times in two seconds, peers do not need every transient edge. Reliability improves when message persistence and event publication use an outbox pattern, so a committed message is eventually published even if the process crashes after the database write.

Hands-On Lab

Prerequisites: Python 3, a terminal, and no external services. The lab simulates a tiny gateway and notification worker in memory. Step 1: copy the examples into a scratch file and run them to confirm the printed outputs. Step 2: change the presence timeout in Example 3 from 3000 to 5000 and observe that u1 becomes online because its heartbeat age is now within the lease. Step 3: add user u5 to room r1 but not to active, then rerun Example 2 and verify that u5 is routed to push. Step 4: add a second duplicate tuple for m1 and u3 in Example 4 and verify that only the first send is emitted for that user.

Verification is deterministic: Example 1 should print [‘u1’]; Example 2 should print push for u2 and websocket for u3 before your modification; Example 3 should initially print u1 offline, u2 online, and u3 online; Example 4 should print one send for each unique message-recipient pair and skip duplicates. Cleanup is simply deleting the scratch file. In a real environment, rollback would also remove test device tokens, test room memberships, and temporary notification ledger rows.

Assessment Exercises

1. Design the reconnect flow for a browser tab that was offline for two minutes. Specify which sequence number it sends, which endpoint it calls, and how it deduplicates events that arrive both through catch-up and the socket.
2. A group chat with 80,000 members creates a notification spike for every message. Choose fanout-on-write, fanout-on-read, or a hybrid, and justify the choice for unread counts and push alerts.
3. Presence for a user’s phone flaps online and offline every ten seconds. List two likely causes and propose one gateway-side and one client-side mitigation.
4. Write a deduplication key for notification sends. Explain why using only message id is insufficient when a message has many recipients.
5. Decide whether typing indicators should be stored in the message database, the presence store, or nowhere durable. Explain the failure behavior users should see.

Summary

Real-time chat notifications and presence combine durable message history with ephemeral connection state. The central mechanisms are gateway-managed sockets, TTL-backed presence leases, event fanout, offline push routing, catch-up by sequence number, and idempotent notification sends. Good designs admit that online status is approximate, that socket delivery is not durable history, and that retries will happen. Once those constraints are explicit, the architecture can provide fast chat interaction without turning transient presence into an unreliable source of truth.