ElastiCache won’t let you PING

I deployed a new tenant to our multi-tenant EKS platform. Login worked. The auth flow completed end to end. Then /api/v2/userinfo returned 500 and the apiserver logs showed nothing — just health checks returning 200.

The tenant has a per-tenant ElastiCache RBAC user with key-prefix isolation. The operator creates it with this access string:

on ~tenant-momcor-3f8a2bc901-* +@read +@write -@dangerous

I tested Redis manually from inside the pod:

AUTH tenant-momcor-3f8a2bc901 <password>
+OK

SET tenant-momcor-3f8a2bc901-test hello
+OK

GET tenant-momcor-3f8a2bc901-test
$5
hello

PING
-NOPERM this user has no permissions to run the 'ping' command

AUTH succeeds. SET works. GET works. PING is denied.

PING lives in a different neighborhood

PING is not in @read or @write. It’s in @connection — a separate Redis command category that covers AUTH, CLIENT, ECHO, PING, QUIT, and SELECT. These are connection management commands, not data commands. Redis treats them as a distinct permission surface.

ElastiCache normalizes the access string to -@all +@read +@write -@dangerous internally. The positive grants override the blanket deny. But if you never grant +@connection, connection management commands stay denied.

The error that doesn’t exist

The app’s Redis client sends PING as a health check when it opens a connection. When PING returns NOPERM, the client treats the connection as dead. But the HTTP request that triggered the Redis connection has already returned — the Redis call happens in an async path. No error in the HTTP response, no error in the request logs. The artifact just sits in a broken state.

SET and GET working made this harder to find. The Redis user authenticates fine, reads and writes data fine. The only command that fails is the one nobody thinks to test because it’s infrastructure plumbing.

on ~tenant-momcor-3f8a2bc901-* +@read +@write +@connection -@dangerous

One word: +@connection. Five commands unlocked. The app works.