The Problem: Monolithic Backends Don't Scale for Real-Time Events

When a fintech client approached SAMStack Tech with an event-driven platform that had to handle 100,000+ transactions per second across three continents, we immediately knew that a traditional Node.js monolith wasn't going to cut it. Latency requirements were brutal: sub-50ms globally, 99.99% uptime, and zero cold-start tolerance.

This article documents the complete architecture we built, the mistakes we made, and the lessons every senior engineer should internalize before building the next high-throughput system.

Why Serverless at the Edge?

Traditional serverless functions (AWS Lambda, Vercel Functions) still run in a single region. When a user in Karachi hits your Lambda deployed in us-east-1, they are adding 200–300ms of round-trip latency before your function even executes. For real-time financial data, that is catastrophic.

Edge computing changes the model entirely. Platforms like Cloudflare Workers deploy your function code to 310+ data centers worldwide. Every request is served from the data center geographically closest to the user. This is the only viable approach when sub-50ms is a hard requirement.

The Core Architecture

Here is the complete stack we chose and why:

• Cloudflare Workers — Edge runtime for all request handling. Zero cold starts, V8-isolated execution, runs in 310+ locations globally.
• Upstash Redis (Global Replication) — Our state layer. Upstash multi-region Redis replication means writes in one region propagate within 50ms globally. Perfect for rate-limiting, caching, and session state.
• Cloudflare Durable Objects — For stateful coordination (WebSocket connections, auction rooms, order books) — each Durable Object guarantees strong consistency in a single location.
• Cloudflare R2 — Object storage for event payloads too large for Redis. No egress fees, accessible directly from Workers.
• Upstash Kafka — For durable, ordered event streaming between services. Integrates natively with Cloudflare Workers via HTTP.

Handling 100k Events/Second: The Ingestion Layer

The ingestion Worker receives raw events from client SDKs. Each event goes through:

1. Schema validation — We use Zod at the edge. If the payload is malformed, we reject it immediately with a 400.
2. Deduplication via Upstash Redis — We store a SHA-256 hash of each event ID in a Redis SET with a 5-minute TTL. Duplicate events are dropped before any further processing.
3. Kafka publish — Valid unique events are written to Upstash Kafka. This is our durability guarantee.
4. Rate limiting — Per-tenant rate limits enforced via Upstash Redis sliding window algorithm in under 2ms.

The Fan-Out Consumer Layer

Consuming from Kafka triggers multiple downstream Workers in parallel: an Analytics Worker, a Webhook Delivery Worker, and a Real-Time Push Worker — all running concurrently with no single point of contention.

Achieving Sub-50ms Latency: The Reality

After deploying our first version, median latency was 38ms globally. But the 99th percentile was 120ms. The culprits were Zod parsing overhead, Redis connection latency, and Kafka publish blocking. After applying micro-batching, connection pooling, and a custom validator for hot paths, our final P99 latency reached 47ms globally.

Key Takeaways

• Serverless at the edge is the only architecture for true global sub-50ms requirements.
• Deduplication must happen before any state mutation — use Redis at the edge, not a database.
• Separate durability (Kafka) from caching (Redis) — never use Redis as your primary event store.
• Durable Objects are the most underrated primitive for stateful coordination at the edge.
• Always measure P99 and P999 latency — median latency is a vanity metric.