projects / 05
linklytics
url shortener & click analytics
A high-throughput, event-driven URL shortening and click-analytics platform. The redirect hot path is isolated from the database: clicks are published to Kafka fire-and-forget and micro-batched into Postgres by a separate consumer.
- p50 cache hit
- 15ms
- p50 cache hit
- errors over 22k req
- 0.00%
- errors over 22k req
- negative cache ttl
- 60s
- negative cache ttl
how it works
- 01
Resolve
GET /:slug on the Express redirect service, running on Bun, reads Redis first. On a miss it queries Postgres and caches the result for 24 hours; unknown slugs are cached as NOT_FOUND for 60 seconds.
- 02
Redirect first
The 302 is sent before anything else happens, so analytics can never slow a redirect down.
- 03
Emit
After responding, the service publishes the click (link id, IP, user agent, timestamp) to the click-events Kafka topic without awaiting it.
- 04
Ingest
A worker in the click-ingester-group consumer group processes whole batches: it parses the user agent into browser, OS and device, replaces the IP with a salted SHA-256 visitor hash, and inserts the batch in one statement, falling back to row-by-row if that fails.
- 05
Report
The Next.js dashboard aggregates in Postgres with GROUP BY, COUNT and COUNT DISTINCT on visitor hashes, grouping by day for the 7-day chart.
design decisions
A separate redirect service
The redirect path started inside Next.js with BullMQ; moving it to a small Express service removed framework overhead and connection-pool pressure from the hot path.
Fire-and-forget clicks
Kafka buffers traffic spikes and the consumer batches inserts, so a burst of clicks becomes a few multi-row writes instead of thousands of single ones.
Hashes instead of IPs
Unique visitors are counted from a salted SHA-256 of IP and user agent, salted daily by default, so raw IP addresses are never stored.
highlights
- Cache-aside in Redis: links cached for 24h, unknown slugs cached as NOT_FOUND for 60s
- Redirect returns 302 before the click is published; Kafka absorbs the spikes
- Consumer parses the user agent and stores a salted SHA-256 visitor hash, never the raw IP
- Dashboard aggregates in SQL (GROUP BY, COUNT DISTINCT) instead of in front-end memory
known limitations
- Benchmarks were run on a local Intel i3 with 8 GB RAM, with Kafka, Redis and Postgres in Docker on the same machine.
- A failed Kafka publish is only logged, so that click is lost.
- City and country columns exist, but the redirect service doesn't send geo data yet.
stack
- bun
- express
- kafka
- redis
- postgres
- drizzle
- next.js