Checkout is returning 500s during every deploy

Support has had a handful of complaints during each of the last four deploys. Errors spike for about twenty seconds and then stop on their own. Every pod is Running, every pod is Ready, nothing has restarted, and the rollout reports success. The database is fine.

real · about 12 minutes

Pods — in creation order

Every pod in the cluster, in the order it was created — never sorted by phase or by readiness
Name	Phase	Ready	Node
checkout-1t48z9-wbbjd	Ready	1/1	node-2
checkout-1t48z9-idfrc	Ready	1/1	node-2
checkout-1t48z9-8dsjf	Ready	1/1	node-1

Requests — simulated, by cause

Requests by the cause recorded when each was emitted, in the order the causes were declared
ok	6032
refused: pod terminating	15
500: not actually ready	1153

Events — in the order the cluster produced them

0.3sdeploymentScalingReplicaSetScaled up replica set checkout-56p08a from 0 to 3
0.5sreplicasetSuccessfulCreateCreated pod: checkout-56p08a-kgft0
0.6sschedulerScheduledSuccessfully assigned default/checkout-56p08a-kgft0 to node-1
0.7sreplicasetSuccessfulCreateCreated pod: checkout-56p08a-mbl5g
0.8skubeletPulledContainer image "checkout:v1" already present on machine
0.8sschedulerScheduledSuccessfully assigned default/checkout-56p08a-mbl5g to node-2
0.8sreplicasetSuccessfulCreateCreated pod: checkout-56p08a-cc11j
0.9skubeletCreatedContainer created
0.9skubeletPulledContainer image "checkout:v1" already present on machine
1.0sschedulerScheduledSuccessfully assigned default/checkout-56p08a-cc11j to node-1
1.1skubeletStartedContainer started
1.1skubeletCreatedContainer created
1.1skubeletPulledContainer image "checkout:v1" already present on machine
1.3skubeletStartedContainer started
1.3skubeletCreatedContainer created
1.4skubeletStartedContainer started
182.0sdeploymentScalingReplicaSetScaled up replica set checkout-1t48z9 from 0 to 0
182.0sdeploymentScalingReplicaSetScaled up replica set checkout-1t48z9 from 0 to 1
182.2sreplicasetSuccessfulCreateCreated pod: checkout-1t48z9-wbbjd
182.3sschedulerScheduledSuccessfully assigned default/checkout-1t48z9-wbbjd to node-2
182.5skubeletPulledContainer image "checkout:v2" already present on machine
182.7skubeletCreatedContainer created
182.8skubeletStartedContainer started
183.1sdeploymentScalingReplicaSetScaled down replica set checkout-56p08a from 3 to 2
183.1sreplicasetSuccessfulDeleteDeleted pod: checkout-56p08a-cc11j
183.2sdeploymentScalingReplicaSetScaled up replica set checkout-1t48z9 from 1 to 2
183.2sreplicasetSuccessfulCreateCreated pod: checkout-1t48z9-idfrc
183.3skubeletKillingStopping container api
183.3sschedulerScheduledSuccessfully assigned default/checkout-1t48z9-idfrc to node-2
183.4skubeletPulledContainer image "checkout:v2" already present on machine
183.6skubeletCreatedContainer created
183.8skubeletStartedContainer started
183.9sdeploymentScalingReplicaSetScaled down replica set checkout-56p08a from 2 to 1
183.9sdeploymentScalingReplicaSetScaled up replica set checkout-1t48z9 from 2 to 3
183.9sreplicasetSuccessfulCreateCreated pod: checkout-1t48z9-8dsjf
183.9sreplicasetSuccessfulDeleteDeleted pod: checkout-56p08a-mbl5g
184.1skubeletKillingStopping container api
184.1sschedulerScheduledSuccessfully assigned default/checkout-1t48z9-8dsjf to node-1
184.3skubeletPulledContainer image "checkout:v2" already present on machine
184.4skubeletCreatedContainer created
184.6skubeletStartedContainer started
184.8sdeploymentScalingReplicaSetScaled down replica set checkout-56p08a from 1 to 0
184.8sreplicasetSuccessfulDeleteDeleted pod: checkout-56p08a-kgft0
184.8skubeletKillingStopping container api

Diagnosis

Which field is responsible? The fix here is always a field — if you find yourself wanting to change the application, you are in the wrong gym and debug.liter8.sh is the right one.