projects / 04
notify
distributed notification infra
A notification pipeline split into four Go services. The API produces each event to Kafka keyed by user_id and serves notification history from Postgres. A dispatcher consumer group checks the event against a gRPC preferences service, deduplicates it, stores it and publishes it to Redis, where stateless WebSocket gateways pick it up and push it to that user's open sockets.
how it works
- 01
Ingest
POST /api/notifications assigns an event id if the caller didn't, produces the event to the notifications.events topic keyed by user_id, and returns 202 once Kafka acks.
- 02
Check preferences
A dispatcher in the dispatcher-workers consumer group asks the preferences service over gRPC whether this user accepts this event type. Blocked events are acknowledged and skipped.
- 03
Deduplicate and store
It sets idempotency:<id> in Redis with SETNX and a 24h TTL, then inserts into Postgres with ON CONFLICT DO NOTHING. If the insert genuinely fails, the Redis key is removed so the retry can go through.
- 04
Publish
The stored event is published to the notifications.realtime Redis channel.
- 05
Deliver
Every WebSocket gateway subscribes to that channel and forwards the event to the sockets registered for that user_id. Clients that were offline fetch their last 50 notifications from the API.
design decisions
Kafka keyed by user_id
Events for one user land on one partition, so they are processed in order, and the API can answer as soon as the event is durable instead of after delivery.
Two-tier idempotency
Kafka delivers at least once. Redis SETNX drops most duplicates cheaply; the Postgres primary key catches anything that slips past, such as an expired Redis key.
Stateless gateways behind pub/sub
No gateway needs to know where a user is connected. Each one receives every event and delivers only to its own sockets, so gateways scale horizontally.
highlights
- The API returns 202 as soon as Kafka acks; processing and delivery live in other services
- Every event is checked against a gRPC preferences service before it is stored or delivered
- Two-tier idempotency: a 24h Redis SETNX key, backed by Postgres ON CONFLICT DO NOTHING
- Redis pub/sub fans events out to horizontally scaled WebSocket gateways, routed by user_id
known limitations
- The preferences service is a stub: it only blocks the marketing.spam event type.
- Gateways trust the user_id query parameter; there is no authentication on the socket yet.
- The Kafka broker address is hardcoded to localhost:9092.
stack
- go
- kafka
- grpc
- redis
- postgres
- websockets