Load Balancing Algorithms Explained
KU
Kiril Urbonas
20 hours ago • 7 min read•0 views
A practical tour of the core load balancing algorithms, how each distributes traffic, and when to reach for one over another.
A load balancer's job sounds simple: take incoming requests and spread them across a pool of backends. The hard part is deciding which backend gets the next request. That decision is the algorithm, and picking the wrong one shows up as one server pegged at 100% CPU while its neighbors idle, or as users randomly logged out because their session landed on a different node.
This post walks through the algorithms you'll actually encounter, how each behaves, and the situations where each one earns its keep. If you want the broader context first, start with our networking fundamentals guide, and if the terminology around proxies is fuzzy, the proxy vs reverse proxy vs load balancer breakdown clears it up.
Before the algorithms, know what layer you're balancing at.
L4 (transport): The balancer routes based on IP and port. It never reads the request body, so it's fast and protocol-agnostic. AWS Network Load Balancer (NLB) and HAProxy in TCP mode work here. Good for raw throughput, databases, or non-HTTP protocols.
L7 (application): The balancer terminates the connection, reads HTTP headers, paths, and cookies, then routes accordingly. AWS Application Load Balancer (ALB), Nginx, and HAProxy in HTTP mode work here. This is where path-based routing, header inspection, and cookie-based stickiness become possible, at the cost of more CPU per request.
Most algorithm choices apply at both layers, but session-aware tricks generally need L7.
How it works: Requests go to backends in rotation: 1, 2, 3, 1, 2, 3. No state, no math, dead simple.
Weighted Round Robin assigns each backend a weight so a beefier box takes proportionally more traffic. A weight-3 server gets three requests for every one the weight-1 server sees.
When to use it: Uniform backends serving short, stateless requests where every request costs roughly the same. Static asset serving, stateless API tiers behind their own datastore. Weighted variant when your fleet is a mix of instance sizes.
Where it breaks: Requests with wildly uneven cost. Round robin counts requests, not work. One slow request per rotation and a backend backs up while the counter cheerfully keeps sending it more.
How it works: Send the next request to whichever backend currently has the fewest active connections. It adapts to real load instead of assuming every request is equal.
Weighted Least Connections factors capacity in, comparing connection count relative to each backend's weight.
When to use it: Long-lived or uneven connections. WebSockets, streaming, database proxies, or any workload where request duration varies a lot. This is the safe default when you're not sure request cost is uniform.
Here's an Nginx upstream using it:
upstream api_backend {
least_conn;
server 10.0.1.10:8080 weight=3;
server 10.0.1.11:8080;
server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
}
server {
listen 80;
location / {
proxy_pass http://api_backend;
}
}
The max_fails and fail_timeout settings above are passive health checks, covered below.
How it works: Hash a key (client IP, a header, a URL) and map the result to a backend. The same key always lands on the same backend as long as the pool is stable. This is how you get session affinity (sticky sessions) without a shared session store.
Plain modulo hashing has a nasty flaw: add or remove one backend and hash % N reshuffles nearly everything. Consistent hashing places backends on a ring so adding or removing a node only remaps its neighbor's slice, keeping most keys where they were. That property is why it's the backbone of distributed caches.
When to use it: Cache locality (route the same key to the same cache node to maximize hit rate), or session affinity when you can't move state into Redis or a database. Nginx offers both ip_hash and the more flexible hash ... consistent:
upstream cache_backend {
hash $request_uri consistent;
server 10.0.2.10:6379;
server 10.0.2.11:6379;
server 10.0.2.12:6379;
}
A caution: Sticky sessions are a crutch. If one client IP sits behind a corporate NAT, thousands of users hash to a single backend. Prefer stateless services with external session storage; reach for hashing when cache locality genuinely pays off or when a legacy app leaves you no choice.
How it works: Route to the backend with the lowest combination of active connections and measured latency. It's least connections with a responsiveness signal layered on, so a backend that's technically free but slow (GC pause, noisy neighbor) gets skipped.
When to use it: Latency-sensitive services with backends whose performance varies at runtime. HAProxy exposes this as balance leastconn plus its own timing logic; Nginx offers least_time in the commercial Plus build.
How it works: Pure random spreads load statistically well at scale with zero coordination. The upgrade is power of two choices: pick two backends at random, send the request to whichever has fewer connections. That one extra comparison avoids the pathological "everyone piled onto the unlucky server" tail that pure random suffers, and it does so without global state.
When to use it: Very large fleets or distributed balancers where maintaining a global least-connections count is expensive. It gets you most of the benefit of least connections at a fraction of the coordination cost.
How it works: Backends report real metrics (CPU, memory, custom load) and the balancer routes toward the least-loaded. It's the most accurate picture of capacity and the most operationally demanding, since it needs an agent or metrics feed on each backend.
When to use it: Heterogeneous workloads where connection count is a poor proxy for actual strain, and you already have the telemetry pipeline to feed it.
No algorithm helps if it keeps routing to a dead backend.
Passive health checks watch live traffic and pull a backend after N failures, as the max_fails/fail_timeout example did. Cheap, but a few users eat the failures first.
Active health checks poll a /health endpoint on an interval and remove a backend before real traffic hits it. HAProxy's check directive, ALB target group health checks, and Nginx Plus all do this. Use active checks anywhere an error budget matters.
Work down this list:
Map it to your tool: Nginx least_conn / ip_hash / hash consistent, HAProxy balance roundrobin|leastconn|source, ALB (round robin or least outstanding requests) and NLB (flow hash). Cloud LBs hide much of this, but the same tradeoffs apply underneath.
For a typical stateless HTTP service, start with Least Connections plus active health checks. It handles uneven request cost without you having to prove requests are uniform, and it degrades gracefully when a backend gets slow. Add consistent hashing only where cache hit rate or unavoidable session state demands it, and push session data into Redis so you can drop stickiness the day you no longer need it. Round robin is fine, but "fine" assumes uniformity you rarely actually have.
Get the latest tutorials, guides, and insights on AI, DevOps, Cloud, and Infrastructure delivered directly to your inbox.
Explore more articles in this category
A developer-focused walkthrough of the TLS 1.3 handshake, certificate trust, forward secrecy, and how to debug the errors you actually hit.
A practical, step-by-step guide to putting Nginx in front of your app for TLS, routing, and load balancing.
You don't need a CCNA to ship reliable services, but you do need the core ideas. This is the map: DNS, TCP, TLS, proxies, and CDNs, minus the jargon.
Evergreen posts worth revisiting.
