# Find What's Eating Your RAM on Linux URL: https://www.devopsness.com/blog/find-what-is-using-memory Author: Kiril Urbonas Published: 2026-07-20 Updated: 2026-07-20 Categories: Linux Tags: Linux, Performance, Troubleshooting, Sysadmin, Memory Summary: A practical method for reading free, top, and ps correctly so you attribute Linux memory to a real cause instead of guessing. --- Someone pages you: "the box is out of memory." You run `free -h`, see `free` sitting at 200 MB, and panic. Nine times out of ten nothing is wrong. The other time, you need to know exactly which process or subsystem is holding the pages. This is how to tell the two apart without guessing. ## Start with free -h and read the right column ```bash $ free -h total used free shared buff/cache available Mem: 31Gi 8.2Gi 412Mi 1.1Gi 22Gi 21Gi Swap: 8.0Gi 128Mi 7.9Gi ``` The trap is the `free` column. It shows 412 MiB and people assume the machine is starving. It is not. Linux deliberately fills unused RAM with page cache (recently read files, filesystem metadata) because idle RAM is wasted RAM. That cache is under `buff/cache`, and most of it is reclaimable the instant a process needs it. The column that matters is **available**. It is the kernel's own estimate of how much memory a new workload can grab without swapping, and it already accounts for reclaimable cache. Here 21 GiB is available, so the box is healthy despite `free` looking tiny. Rule of thumb: judge memory pressure by `available`, not `free`. If `available` is a small fraction of `total` and swap is climbing, then you have a real problem worth chasing. ## Per-process the honest way Sort processes by memory and look at the top of the list: ```bash $ ps aux --sort=-%mem | head -6 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND postgres 2417 1.2 18.4 9420112 5900312 ? Ss Jul18 42:11 postgres: writer java 3810 6.0 12.1 8123400 3880220 ? Sl Jul19 88:02 java -Xmx4g -jar app.jar redis 1902 0.4 3.2 1120044 1030880 ? Ssl Jul17 9:44 redis-server prom 2201 2.1 2.8 2455010 905122 ? Ssl Jul18 30:15 prometheus ``` Two columns describe each process: **VSZ (virtual size):** everything the process has mapped, including memory-mapped files it never touched, shared libraries, and reserved-but-unused heap. It is almost always huge and almost always misleading. Ignore VSZ for "who is using RAM." **RSS (resident set size):** the pages actually in physical RAM right now. Closer to the truth, but it has one big flaw: RSS counts shared pages in full for every process that maps them. Twenty PHP workers sharing 200 MB of the interpreter each report that 200 MB in their RSS. Sum the column and you will "find" more memory used than the machine has. ### PSS: stop double-counting Proportional set size (PSS) splits shared pages fairly. If a 200 MB library is shared by 20 workers, each is charged 10 MB. The per-process PSS values sum to something that actually reflects physical usage. The kernel exposes it per process: ```bash $ grep -E '^(Rss|Pss|Shared|Private)' /proc/3810/smaps_rollup Rss: 3880220 kB Pss: 3120044 kB Shared_Clean: 540210 kB Shared_Dirty: 210880 kB Private_Clean: 98110 kB Private_Dirty: 3030120 kB ``` Here RSS is 3.7 GiB but PSS is 3.0 GiB; the difference is shared libraries counted against everyone. `Private_Dirty` (3.0 GiB) is the interesting number: pages this process alone owns and dirtied. Kill the process and that is what you get back. For a fleet-wide view, `smem -tk -c "name pss rss"` (from the `smem` package) reads smaps for every process and gives you a PSS-sorted table without the arithmetic. ## What top's columns actually mean In `top` (press `M` to sort by memory) the relevant fields are `RES`, `SHR`, and `%MEM`: - **RES** is the same resident set as `ps` RSS, shared pages included. - **SHR** is the portion of RES that is shareable (libraries, mapped files, shared memory segments). - `RES - SHR` roughly approximates the private footprint, which is a decent quick proxy for PSS when you do not want to read smaps. If a process shows RES 4 GiB and SHR 3.5 GiB, it is mostly touching shared pages and is not your leak. A process with RES 4 GiB and SHR 40 MB is holding 4 GiB of private memory and deserves your attention. ## Containers live in cgroups, not free Inside a container, `free` reports the host's totals, which is useless. The container's real limit and usage live in its cgroup. On cgroup v2: ```bash $ cat /sys/fs/cgroup/memory.current 4187553792 $ cat /sys/fs/cgroup/memory.max 4294967296 $ grep -E '^(anon|file|slab|sock)' /sys/fs/cgroup/memory.stat anon 3980210176 file 178954240 slab 24117248 sock 2097152 ``` `memory.current` (about 3.9 GiB) against `memory.max` (4 GiB) tells you the container is near its limit and one bad allocation from the OOM killer. `memory.stat` breaks it down: `anon` is process heap and stacks, `file` is page cache the cgroup is charged for. A container getting OOM-killed with high `anon` is a genuine leak or an undersized limit. If yours is dying at the limit, see [the Linux OOM killer](/blog/fix-out-of-memory-linux) for how that decision gets made. ## When the culprit is the kernel Sometimes `ps` and `top` add up to far less than `used`, and no cgroup explains it. The kernel itself is holding memory in slab caches (dentries, inodes, network buffers). `slabtop` shows it: ```bash $ sudo slabtop --once --sort=c | head -8 Active / Total Size (% used) : 3120.44M / 3350.10M (93.1%) OBJS ACTIVE USE OBJ SIZE SLABS NAME 8912340 8900112 99% 0.19K 424397 dentry 1204880 1201002 99% 1.05K 80325 ext4_inode_cache 980220 975003 99% 0.10K 25130 buffer_head ``` Millions of `dentry` objects usually means something is stat-ing or opening enormous directory trees (a backup job, a misbehaving find, a container churning files). This memory is reclaimable under pressure, but a runaway can still squeeze real workloads. `cat /proc/meminfo` and check `Slab`, `SReclaimable`, and `SUnreclaim` for the split that matters. ## Swap thrashing: the smoking gun A machine can have "used" swap and be perfectly fine. What hurts is active swapping: pages moving in and out constantly because the working set no longer fits. ```bash $ vmstat 2 4 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 3 2 6820112 210044 40122 880210 4820 6110 5200 6300 8100 12200 14 9 20 57 0 2 3 6910220 198100 39980 861200 5100 6402 4900 6800 8400 12900 12 8 18 62 0 ``` Watch `si` and `so` (swap in / swap out per second). Sustained non-zero values in both, combined with high `wa` (I/O wait, 57-62% here), is thrashing. The CPU is idle because everything is blocked waiting on disk to page memory back and forth. That is when latency falls off a cliff. A one-time `so` spike during a backup is normal; continuous churn is not. ## A method that actually attributes memory 1. `free -h`: is `available` genuinely low, or does `free` just look scary? If available is healthy, stop, nothing is wrong. 2. If available is low, check swap with `vmstat 2`. Sustained `si`/`so` plus high `wa` confirms real pressure. 3. `ps aux --sort=-%mem | head` to find candidates. Treat RSS as a hint, not a verdict. 4. Confirm the top suspect with `/proc//smaps_rollup` (PSS and Private_Dirty) or `smem`. Private_Dirty is the memory you reclaim by killing it. 5. If processes do not add up to `used`, check `slabtop` and `/proc/meminfo` for kernel/slab growth. 6. In containers, ignore `free` entirely and read `memory.current` / `memory.max` / `memory.stat`. ## The call we'd make Trust `available` over `free`, and trust PSS over RSS. Most "out of memory" pages are page cache doing its job, and the fix is to close the ticket. When it is real, the honest number is Private_Dirty (or per-cgroup `anon`), because that is the memory that actually disappears when you kill the offender. Everything else is either shared, reclaimable, or a rounding error. For the broader playbook, see the [Linux troubleshooting guide](/blog/linux-troubleshooting-guide).