SSM Parameter Store version history as incident forensics

Friday afternoon I merged a PR — SSM parameter seeding for a dev tenant. The workflow ran, wrote 48 parameters, and I moved on. Over the weekend, the other engineer couldn’t authenticate. Redis connections from the auth service were failing. He was blocked for two days.

Monday morning I pulled up the parameter history:

aws ssm get-parameter-history \
  --name "/ramparts/dev/tenants/momcorp/auth-service/REDIS_PASSWORD" \
  --with-decryption \
  --query 'Parameters[].{Version:Version,Value:Value,Modified:LastModifiedDate,User:LastModifiedUser}' \
  --output table

What the versions showed

REDIS_PASSWORD — momcorp/auth-service
v9  2026-02-18  u88XbnSx...  (operator-created, correct)
v10 2026-03-07  M3rXL7Ph...  (my overwrite — wrong credential)
v11 2026-03-10  u88XbnSx...  (restored from v9)

The tenant operator had created the correct REDIS_PASSWORD at v9 — a credential generated specifically for this tenant’s ElastiCache RBAC user. My seeding workflow overwrote it at v10 with a shared auth token that had nothing to do with Redis. The service tried to authenticate with the wrong password and failed silently.

I didn’t need to ask anyone what the old value was. I didn’t need to check operator logs or cross-reference ElastiCache. Version 9 was right there — exact value, exact timestamp, exact author. I read it, confirmed it matched the ElastiCache user, and wrote it back as v11.

What I assumed wrong

I assumed my seeding workflow was filling in empty parameters. It was overwriting populated ones. The operator had already created the correct credentials days earlier, and my workflow stomped them with values from a flat env file that didn’t distinguish between app secrets and infrastructure-generated credentials.

SSM behaved correctly. Every put-parameter call creates a new version. SSM kept all 11 versions of that parameter, each with its value, timestamp, and the IAM principal that wrote it. The version history was a complete audit trail of the damage — who changed what, when, and what the correct value was before I touched it.

20 minutes, 48 parameters, zero guesswork

For each of the 48 parameters, I ran get-parameter-history --with-decryption and compared the pre-overwrite version against what the service actually needed. Most of my values were correct. Three were wrong — credentials the operator had generated that my env file didn’t have. I restored each one from its previous version.

Total recovery time: 20 minutes. No guesswork, no coordination, no “does anyone remember what this password was?”

You already have incident forensics

get-parameter-history --with-decryption is the command. It returns every version of every parameter — values, timestamps, IAM principals. If you store secrets in SSM, you already have incident forensics. You don’t need to set it up. You just need to know to look.