The ordered kubectl toolkit we reach for when a pod misbehaves, with the five commands we run first and what each one actually tells you.
kubectl Commands for Debugging Any Pod
Pods break in about four ways: they won't start, they start and crash, they run but can't talk to anything, or they run and eat the node alive. Over the years the same handful of kubectl commands has covered nearly every one of those cases. This is the toolkit, in the order I actually reach for it.
If you want the broader story on reading Kubernetes failures, our Kubernetes troubleshooting guide goes wider. This post stays close to the terminal.
Before any theory, here are the five commands that resolve most of my pod problems before I've finished my coffee:
kubectl get pods -o wide
kubectl describe pod <pod>
kubectl logs <pod> --previous
kubectl get events --sort-by=.lastTimestamp
kubectl top pod <pod>
get pods -o wide adds the node name and pod IP to the usual output. Knowing which node a pod landed on has saved me more times than I can count, because half of "random" failures are one bad node. Everything after this is drilling into whichever of those five pointed at something odd.
Pending, ContainerCreating, ImagePullBackOff, Init:0/1. The pod exists but never gets to Running.
Start with describe. The Events block at the bottom is where the real message lives:
kubectl describe pod <pod>
Failed scheduling usually means no node has the CPU or memory you asked for, or a taint or node selector nobody can satisfy. ImagePullBackOff means a wrong tag, a private registry with no pull secret, or a typo in the image name. For init containers stuck at Init:0/1, read that specific container:
kubectl logs <pod> -c <init-container-name>
If describe isn't enough, widen out to the namespace and let the events tell the story in order:
kubectl get events --sort-by=.lastTimestamp
Sort matters. The default ordering is close to useless; --sort-by=.lastTimestamp puts the freshest failure at the bottom where you're already looking. When a pod is Pending because of resources, confirm what's actually free:
CrashLoopBackOff, or a Running pod that keeps restarting. The restart count in get pods is your tell.
The container that crashed is already gone, so live logs show you the new attempt, not the death. Ask for the previous one:
kubectl logs <pod> --previous
kubectl logs <pod> -c <container> --previous
--previous is the single most underused flag in kubectl. It pulls logs from the last terminated container, which is exactly the one that failed. For a multi-container pod, -c picks the container. To watch a fresh restart live:
describe still earns its place here too. The Last State and Reason fields tell you if the kernel OOM-killed the process (Reason: OOMKilled) versus the app exiting on its own (a plain nonzero exit code). Those two point at completely different fixes: one is a limits problem, the other is a code or config problem.
When logs are ambiguous, get inside a running instance and poke around:
kubectl exec -it <pod> -- /bin/sh
Some hardened images ship no shell at all. That's where ephemeral containers come in:
kubectl debug -it <pod> --image=busybox --target=<container>
This attaches a throwaway debug container that shares the target's process namespace, so you get a shell and tools without rebuilding the image. For a node-level problem, the same command debugs the host:
kubectl debug node/<node> -it --image=busybox
The pod is healthy, and something still can't reach it. This is where people burn hours.
Check whether the Service actually has backends:
kubectl get endpoints <service>
An empty endpoints list means the Service selector doesn't match your pod labels. No amount of DNS debugging fixes a selector typo, so rule it out first. Then test the pod directly, bypassing the Service:
kubectl port-forward <pod> 8080:80
Now localhost:8080 hits the pod. If that works but the Service doesn't, the problem is the Service or ingress, not the app. If you need a file off the pod to inspect a config that got mounted wrong:
kubectl cp <pod>:/etc/app/config.yaml ./config.yaml
Slow, throttled, or getting killed under load.
kubectl top pod <pod>
kubectl top pod --sort-by=memory
top needs metrics-server installed, and it's worth installing on day one. Compare live usage against the limits set on the pod:
kubectl get pod <pod> -o yaml
The full YAML shows requests, limits, volume mounts, env vars, and the live status conditions. It's verbose, so pull just what you need with jsonpath:
kubectl get pod <pod> -o jsonpath='{.spec.containers[*].resources}'
jsonpath is worth ten minutes of learning. It turns "scroll through YAML" into "answer one question." If you forget a field's shape, ask the API:
kubectl explain pod.spec.containers.resources
Sometimes the pod is fine and the rollout is wrong. Watch it land:
kubectl rollout status deployment/<name>
If a bad image shipped and pods are crashing, roll back before you debug further:
kubectl rollout undo deployment/<name>
Get the cluster stable, then investigate the broken revision at your own pace.
Reach for describe and --previous logs before anything fancier. Between those two and get events --sort-by, you've explained most pod failures without ever opening a dashboard. Save kubectl debug for the shell-less images and node problems, and lean on jsonpath so you're asking one question at a time instead of squinting at a wall of YAML. The tooling around Kubernetes keeps growing, but the pod that's down at 2am still answers to these same commands.
