fizz.today

Deterministic naming means your teardown has a cache invalidation checklist

Destroyed a tenant. Tore down the infrastructure, cleaned up 48 SSM parameters. Recreated it from scratch — new UUID, new org code, fresh databases. Can’t log in. The app hangs on the OAuth callback.

The tenant’s subdomain is derived from a deterministic hash of its name. “Ridgeback Security” always produces tenant-ridgeb-7ebfdce288. That hash becomes the hostname:

app.tenant-ridgeb-7ebfdce288.ramparts.dev

Same tenant name, same hash, same subdomain. That determinism is load-bearing — it’s what makes terraform apply idempotent and DNS predictable. But it also means every cache layer that ever referenced that hostname still holds state from the previous incarnation.

The checklist

When your naming scheme is deterministic, server-side teardown is the easy part. The edges are where the old tenant survives. Here’s what you need to invalidate on a destroy/create cycle:

Browser cookies. OAuth session cookies scoped to the subdomain survive server teardown. The browser sends stale session tokens to the new tenant’s auth service, which receives a token it never issued from an OAuth provider org that no longer exists. The redirect loop hangs silently. Fix: clear application cookies for the domain, or set short expiry with max-age on auth cookies to shrink the window.

DNS resolver caches. TTL means the old IP persists in recursive resolvers even after you update Route53. A 300-second TTL means 5 minutes of requests hitting the old address after teardown. Fix: drop TTL to 60s before teardown, wait one TTL period, then destroy.

CDN edge caches. CloudFront (and every other CDN) caches responses keyed by hostname. The new tenant’s origin serves fresh content, but edge nodes keep serving stale responses from the previous incarnation until cache expiry. Fix: issue an invalidation on the distribution after recreating the origin, or use cache-busting headers on tenant-specific responses.

HSTS state. Browsers remember HTTPS-only policy per domain. If the old tenant set Strict-Transport-Security with a long max-age and the new tenant’s cert isn’t ready yet, the browser refuses to connect over HTTP even for initial setup. Fix: keep max-age short during development (3600, not 31536000), or use includeSubDomains only when cert issuance is automated.

Service worker caches. Registered per origin, persist across sessions. A service worker from the old tenant intercepts requests to the new one and serves cached responses from a dead app. Fix: version your service worker URL or scope, and include an unregister step in teardown.

The principle

Deterministic naming is a deliberate tradeoff. You get idempotent infrastructure and predictable DNS. You pay for it with a cache invalidation surface that grows with every layer that touches the hostname. The teardown isn’t done when the server is gone. It’s done when every cache that ever saw that name has been flushed or expired.

If you use deterministic naming, maintain this checklist. Run it on every destroy/create cycle. The server forgets immediately. The edges remember.

#multi-tenant #kubernetes #dns #operations #debugging