Our proxy topped out at 40k connections while the CPU sat half-idle. The bottleneck was kernel defaults tuned for 2009, not the hardware.
Linux TCP Tuning for High-Throughput Services
Our reverse proxy refused to go past about 40,000 concurrent connections. CPU was at 45%, memory was fine, the network card was rated for far more, and yet new connections started timing out at that ceiling. The hardware wasn't the limit. The Linux kernel ships with TCP defaults chosen to be safe on a machine with a tenth of our RAM, and those defaults were the wall we kept hitting.
Before changing anything, measure. Blind tuning is how you turn one problem into three. ss -s gives a connection summary, and netstat -s (or nstat) shows the counters that tell you what's actually being dropped.
ss -s
nstat -az | grep -E 'TcpExtListenDrops|TcpExtListenOverflows|TcpExtTCPBacklogDrop'
ListenOverflows climbing meant our accept queue was full. That was the first real clue.
When connections arrive faster than your app calls accept(), they queue. Two limits govern this: the app's listen() backlog argument and the kernel's net.core.somaxconn. If somaxconn is smaller than what the app asks for, the kernel silently caps it. The old default was 128, absurdly small for a busy server.
sysctl -w net.core.somaxconn=65535
sysctl -w net.ipv4.tcp_max_syn_backlog=65535
Then confirm your application actually passes a large backlog to listen(). Nginx caps it via listen ... backlog=65535; and it does not inherit somaxconn automatically. We set the kernel value and saw no change for an hour until we realized nginx was still calling listen(sockfd, 511). Both sides have to agree.
For a proxy making outbound connections to backends, you can run out of source ports. Each outbound connection needs a local port, and the default range is narrow.
sysctl -w net.ipv4.ip_local_port_range="1024 65535"
That gives you roughly 64k ports per destination tuple. Connections in TIME_WAIT hold their port for 60 seconds after closing, and under high churn they pile up fast. The right fix is not the dangerous one people copy from old forum posts, tcp_tw_recycle, which was removed in kernel 4.12 because it broke NAT clients. Enable reuse instead, which is safe for outbound connections.
sysctl -w net.ipv4.tcp_tw_reuse=1
tcp_tw_reuse lets the kernel reuse a TIME_WAIT socket for a new outbound connection when it's provably safe using timestamps. It only affects the connecting side, so it's the correct knob for a proxy dialing backends. Better still, use keepalive connection pools to your backends so you're not churning connections at all, but when you must, tw_reuse is the safe lever.
On a high-bandwidth, higher-latency link, small socket buffers cap throughput because the sender can't keep enough data in flight to fill the pipe. This is the bandwidth-delay product. Let the kernel autotune within a larger ceiling.
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"
The three numbers are min, default, and max. The kernel autotunes between default and max based on conditions, so you're raising the ceiling, not pinning a fixed size. On a cross-region link with 60ms RTT, raising the max from the 4MB default to 16MB roughly doubled our single-stream throughput because the window could finally grow large enough to keep the link full.
The default congestion control, CUBIC, backs off aggressively on packet loss, which tanks throughput on links with any loss. BBR models the bottleneck bandwidth instead and holds throughput up far better. It's in-tree since kernel 4.9.
sysctl -w net.core.default_qdisc=fq
sysctl -w net.ipv4.tcp_congestion_control=bbr
BBR needs the fq queueing discipline to pace packets correctly, so set both. On a lossy transcontinental path we measured throughput go from around 12 Mbit/s per stream under CUBIC to over 90 Mbit/s under BBR, same hardware, same link, just the algorithm. Confirm it took:
sysctl net.ipv4.tcp_congestion_control
# net.ipv4.tcp_congestion_control = bbr
sysctl -w is lost on reboot. Put the final set in a file under /etc/sysctl.d/99-tcp-tuning.conf and apply with sysctl -p. Then verify the behavior changed, don't just trust that you wrote the file. When we suspected retransmits were still high, tcpdump on the wire showed us the actual retransmitted segments:
tcpdump -i eth0 'tcp[tcpflags] & tcp-syn != 0' -c 20
nstat -az | grep -i retrans
Watching retransmit counters before and after each change is the only way to know a knob helped rather than hurt. After the full set, our proxy passed 200,000 concurrent connections on the same box, and the CPU was finally the limiting resource, which is where you want to be.
Change one thing at a time and watch nstat counters between changes, because half these knobs interact and a blind copy-paste of someone's sysctl.conf will bite you. If you only touch two settings, make them net.core.somaxconn (raise it, and match your app's listen() backlog) and switching congestion control to BBR with fq. Those two fixed the connection ceiling and the throughput ceiling for us, and everything else was fine-tuning on top. Never enable tcp_tw_recycle, it's gone for good reason, and prefer connection pooling over any TIME_WAIT trickery when you can.
