kubectl Commands for Debugging Any Pod

Pods break in about four ways: they won't start, they start and crash, they run but can't talk to anything, or they run and eat the node alive. Over the years the same handful of kubectl commands has covered nearly every one of those cases. This is the toolkit, in the order I actually reach for it.

If you want the broader story on reading Kubernetes failures, our Kubernetes troubleshooting guide goes wider. This post stays close to the terminal.

The five I run first #

Before any theory, here are the five commands that resolve most of my pod problems before I've finished my coffee:

bash.bash

kubectl get pods -o wide
kubectl describe pod <pod>
kubectl logs <pod> --previous
kubectl get events --sort-by=.lastTimestamp
kubectl top pod <pod>

get pods -o wide adds the node name and pod IP to the usual output. Knowing which node a pod landed on has saved me more times than I can count, because half of "random" failures are one bad node. Everything after this is drilling into whichever of those five pointed at something odd.

Scenario: it won't start #

Pending, ContainerCreating, ImagePullBackOff, Init:0/1. The pod exists but never gets to Running.

Start with describe. The Events block at the bottom is where the real message lives:

bash.bash

kubectl describe pod <pod>

Failed scheduling usually means no node has the CPU or memory you asked for, or a taint or node selector nobody can satisfy. ImagePullBackOff means a wrong tag, a private registry with no pull secret, or a typo in the image name. For init containers stuck at Init:0/1, read that specific container:

bash.bash

kubectl logs <pod> -c <init-container-name>

If describe isn't enough, widen out to the namespace and let the events tell the story in order:

bash.bash

kubectl get events --sort-by=.lastTimestamp

Sort matters. The default ordering is close to useless; --sort-by=.lastTimestamp puts the freshest failure at the bottom where you're already looking. When a pod is Pending because of resources, confirm what's actually free:

bash.bash

kubectl top node

Scenario: it starts and crashes #

CrashLoopBackOff, or a Running pod that keeps restarting. The restart count in get pods is your tell.

The container that crashed is already gone, so live logs show you the new attempt, not the death. Ask for the previous one:

bash.bash

kubectl logs <pod> --previous
kubectl logs <pod> -c <container> --previous

--previous is the single most underused flag in kubectl. It pulls logs from the last terminated container, which is exactly the one that failed. For a multi-container pod, -c picks the container. To watch a fresh restart live:

bash.bash

kubectl logs -f <pod>

describe still earns its place here too. The Last State and Reason fields tell you if the kernel OOM-killed the process (Reason: OOMKilled) versus the app exiting on its own (a plain nonzero exit code). Those two point at completely different fixes: one is a limits problem, the other is a code or config problem.

When logs are ambiguous, get inside a running instance and poke around:

bash.bash

kubectl exec -it <pod> -- /bin/sh

Some hardened images ship no shell at all. That's where ephemeral containers come in:

bash.bash

kubectl debug -it <pod> --image=busybox --target=<container>

This attaches a throwaway debug container that shares the target's process namespace, so you get a shell and tools without rebuilding the image. For a node-level problem, the same command debugs the host:

bash.bash

kubectl debug node/<node> -it --image=busybox

Scenario: it runs but networking is broken #

The pod is healthy, and something still can't reach it. This is where people burn hours.

Check whether the Service actually has backends:

bash.bash

kubectl get endpoints <service>

An empty endpoints list means the Service selector doesn't match your pod labels. No amount of DNS debugging fixes a selector typo, so rule it out first. Then test the pod directly, bypassing the Service:

bash.bash

kubectl port-forward <pod> 8080:80

Now localhost:8080 hits the pod. If that works but the Service doesn't, the problem is the Service or ingress, not the app. If you need a file off the pod to inspect a config that got mounted wrong:

bash.bash

kubectl cp <pod>:/etc/app/config.yaml ./config.yaml

Scenario: it's eating resources #

Slow, throttled, or getting killed under load.

bash.bash

kubectl top pod <pod>
kubectl top pod --sort-by=memory

top needs metrics-server installed, and it's worth installing on day one. Compare live usage against the limits set on the pod:

bash.bash

kubectl get pod <pod> -o yaml

The full YAML shows requests, limits, volume mounts, env vars, and the live status conditions. It's verbose, so pull just what you need with jsonpath:

bash.bash

kubectl get pod <pod> -o jsonpath='{.spec.containers[*].resources}'

jsonpath is worth ten minutes of learning. It turns "scroll through YAML" into "answer one question." If you forget a field's shape, ask the API:

bash.bash

kubectl explain pod.spec.containers.resources

When the deploy itself is the problem #

Sometimes the pod is fine and the rollout is wrong. Watch it land:

bash.bash

kubectl rollout status deployment/<name>

If a bad image shipped and pods are crashing, roll back before you debug further:

bash.bash

kubectl rollout undo deployment/<name>

Get the cluster stable, then investigate the broken revision at your own pace.

The call we'd make #

Reach for describe and --previous logs before anything fancier. Between those two and get events --sort-by, you've explained most pod failures without ever opening a dashboard. Save kubectl debug for the shell-less images and node problems, and lean on jsonpath so you're asking one question at a time instead of squinting at a wall of YAML. The tooling around Kubernetes keeps growing, but the pod that's down at 2am still answers to these same commands.

kubectl Commands for Debugging Any Pod

The five I run first#

Scenario: it won't start#

Scenario: it starts and crashes#

Scenario: it runs but networking is broken#

Scenario: it's eating resources#

When the deploy itself is the problem#

The call we'd make#

The five I run first #

Scenario: it won't start #

Scenario: it starts and crashes #

Scenario: it runs but networking is broken #

Scenario: it's eating resources #

When the deploy itself is the problem #

The call we'd make #

#Kubectl

kubectl Commands for Debugging Any Pod