Building Resilient AI Infrastructure on Cloudflare's Edge

When we started DevBlock, we were deploying servers the old-fashioned way — VPS boxes, SSH, manual deploys, one environment per product. It worked until it didn't. Every product had its own failure modes, its own scaling story, its own way of breaking.

Today, every DevBlock system runs on Cloudflare's edge. Workers for compute, D1 for data, Vectorize for semantic search, and our own Sage Inference gateway for AI. This post is about why we standardized on the edge — and the patterns we use to keep everything fast and reliable.

Why the Edge

The edge model collapses deployment complexity. A Worker is a single script that runs in 300+ locations, with zero cold starts for requests that hit warm isolates. There's no server to patch, no cluster to scale, no region to choose. We ship code and Cloudflare figures out where it should run.

That means sub-50ms API responses from anywhere in the world, automatic scaling to zero when traffic drops, and a deployment model where "push to deploy" is genuinely that simple. The Console's deploy speed — under three seconds from commit to production — is a direct result of this architecture.

"Push to deploy. 300+ edge locations, zero cold starts, automatic scaling. That's the entire infrastructure story in one sentence."

D1 and the Data Layer

D1 is Cloudflare's serverless SQLite, and it powers most of our transactional data — credits, sessions, subscribers, project state. D1 gives us SQL semantics with sub-millisecond reads and no connection management. For write-heavy or relational workloads it has been more than sufficient, and because it's global, reads land near the user.

We use Vectorize for semantic search — things like knowledge-base lookup in MesageAI and document similarity in Bigglesworth. Storing embeddings at the edge lets us do similarity search without shipping data to a separate vector database.

Reliability Patterns

Edge infrastructure removes a lot of operational burden, but it doesn't remove the need for defensive design. Three patterns carry most of the weight:

Circuit breakers. Sage Inference tracks provider health and trips open on sustained errors, routing traffic to healthy providers automatically. This turned provider outages from incidents into non-events.

Autonomous recovery. MesageAI detects disconnected WhatsApp sessions and reconnects within 30 seconds. Self-healing, not paging.

Credits and quotas. Every system that calls an LLM goes through Sage Inference, which enforces per-key rate limits and per-model caps. No runaway costs, no accidental abuse.

What We Learned

The biggest lesson: infrastructure is a product decision, not a backend afterthought. Choosing the edge meant our latency, scaling, and failure behaviour were set from day one. New products start with the same base — Workers, D1, Sage Inference — and inherit a decade of hardening for free.

The second lesson is to distrust uptime numbers and design for failure anyway. Providers report 99.9% availability; our circuit breakers assume they'll fail. Planning for the worst case is what lets us deliver 99.9% ourselves.