# Load Balancing Algorithms Explained

URL: https://www.devopsness.com/blog/load-balancing-algorithms
Author: Kiril Urbonas
Published: 2026-08-07
Updated: 2026-10-31
Categories: Infrastructure
Tags: Load Balancing, Networking, Infrastructure, Scalability, Nginx

Summary: A practical tour of the core load balancing algorithms, how each distributes traffic, and when to reach for one over another.

---
# Load Balancing Algorithms Explained

A load balancing algorithm picks which backend receives the next request. Round robin rotates through servers, least connections picks the least busy one, hashing pins a key to one server, and power of two choices samples two servers and takes the lighter. For a typical HTTP service, use a load-aware method plus active health checks.

> [!VERIFIED]
> nginx 1.30.6 stable and 1.31.8 mainline, HAProxy 3.4.7 LTS, against the nginx.org upstream module reference, the HAProxy 3.4 configuration manual and the AWS Application Load Balancer target group docs. Checked October 2026.

Picking wrong shows up as one server pegged while its neighbors idle. If the vocabulary is fuzzy, start with the [networking fundamentals guide](/blog/networking-fundamentals-guide) and the [proxy vs reverse proxy vs load balancer](/blog/proxy-vs-reverse-proxy-vs-load-balancer) breakdown.

## L4 vs L7: where the decision happens

**L4 (transport)** balancers route on IP and port, never read the request, and balance connections. AWS Network Load Balancer and HAProxy `mode tcp` work here.

**L7 (application)** balancers terminate the connection, read headers and cookies, and route per request: AWS Application Load Balancer, nginx `proxy_pass`, HAProxy `mode http`.

At L4, one HTTP/2 connection carrying a thousand streams counts as one unit of load.

## The algorithms, side by side

| Algorithm | Picks by | State needed | Best for | Main failure mode |
|---|---|---|---|---|
| Round robin (weighted) | Rotation | None | Uniform, short, stateless requests | Ignores request cost |
| Least connections | Fewest active connections | Per-backend counters | Long-lived or uneven requests | Counts connections, not CPU |
| Least response time | Latency plus connections | Timing data | Backends with variable speed | Chases noisy measurements |
| Hash / consistent hash | Hash of a key | None (ring) | Cache locality, affinity | Hot keys, NAT hotspots |
| Power of two choices | Lighter of two random picks | Local counters | Large fleets, many balancers | Weak on tiny pools |

### Round robin and weighted round robin

Requests go 1, 2, 3, 1, 2, 3. Weighted round robin sends proportionally more to bigger boxes: weight 3 gets three requests for every one a weight-1 server sees. nginx's default weight is 1, and round robin is nginx's default method when you name none. HAProxy's `roundrobin` is dynamic (weights can change at runtime) and its manual caps it at 4,095 active servers per backend; `static-rr` removes that limit but ignores runtime weight changes.

It breaks when request cost varies: a backend stuck on slow requests keeps getting its full share.

### Least connections

The next request goes to the backend with the fewest active connections, weighted by capacity. nginx's `least_conn` breaks ties with weighted round robin. It is the right default for WebSockets, streaming, database proxies and anything where duration varies a lot.

One honest caveat from the HAProxy manual: it recommends `leastconn` "where very long sessions are expected, such as LDAP, SQL, TSE" and calls it "not very well suited for protocols using short sessions such as HTTP." With very short requests, connection counts are mostly zero or one and carry little signal. For short HTTP on HAProxy, `random(2)` is the better load-aware choice.

### Least response time

Route to the backend with the lowest average response time and fewest active connections, skipping servers that are free but slow. nginx's `least_time header|last_byte` does this, and the nginx docs note it was commercial-only before 1.31.0, so it is now in open-source mainline but not yet in the 1.30 stable branch. HAProxy has no direct equivalent in its `balance` list; the closest is `random(2)` combined with `hash-balance-factor`.

### Hashing and consistent hashing

Hash a key (client IP, URI, a header) and map it to a backend, so the same key lands on the same server while the pool is stable. Plain `hash % N` reshuffles almost every key when N changes. Consistent hashing puts servers on a ring so adding or removing one remaps only its slice; nginx's `hash ... consistent` uses the ketama method for this.

nginx's `ip_hash` keys on the first three octets of an IPv4 address, so a whole /24, or every user behind one corporate NAT, lands on one backend.

### Random and power of two choices

Power of two choices picks two servers at random and sends the request to the less loaded one, avoiding the unlucky pile-ups of pure random. HAProxy's manual says the default of 2 draws "generally shows very good distribution," and suggests 3 or 4 for large farms with very low load. nginx has it as `random two least_conn`. ALB's closest option is `weighted_random`.

## Configs you can paste

nginx, with least connections, weights and passive health checks:

```nginx
# /etc/nginx/conf.d/api.conf
upstream api_backend {
    least_conn;
    server 10.0.1.10:8080 weight=3;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

server {
    listen 80;
    location / {
        proxy_pass http://api_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_next_upstream error timeout http_502 http_503;
    }
}
```

For cache nodes, swap the first line for `hash $request_uri consistent;`. For power of two choices, use `random two least_conn;`. Then validate and reload:

```
$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
$ sudo nginx -s reload
```

HAProxy, with power of two choices and active health checks:

```
# /etc/haproxy/haproxy.cfg
backend be_api
    mode http
    balance random(2)
    option httpchk
    http-check send meth GET uri /healthz
    http-check expect status 200
    default-server inter 2s fall 3 rise 2
    server api1 10.0.1.10:8080 check weight 30
    server api2 10.0.1.11:8080 check weight 10
    server api3 10.0.1.12:8080 check weight 10
```

```
$ haproxy -c -f /etc/haproxy/haproxy.cfg
Configuration file is valid
$ sudo systemctl reload haproxy
```

AWS ALB, switching a target group to least outstanding requests:

```
$ aws elbv2 modify-target-group-attributes \
    --target-group-arn "$TG_ARN" \
    --attributes Key=load_balancing.algorithm.type,Value=least_outstanding_requests
```

The ALB docs list three algorithms: `round_robin` (the default), `least_outstanding_requests`, and `weighted_random`. Neither of the last two works with slow start mode, `weighted_random` cannot be combined with sticky sessions, and only `weighted_random` supports Automatic Target Weights anomaly mitigation.

## Health checks tie it together

**Passive** checks watch real traffic: nginx's `max_fails` defaults to 1 and `fail_timeout` to 10 seconds, and both are ignored for a single-server group. A few users eat the failures first. **Active** checks poll an endpoint before users hit it. In nginx they are a commercial (NGINX Plus) feature; HAProxy's `check` does it in the open-source build, with `inter` defaulting to 2000 ms. ALB target groups always run active checks, and fail open (send to all targets) when too few are healthy.

## Error strings and what they mean

| Error | Where | Usual cause |
|---|---|---|
| `no live upstreams while connecting to upstream` | nginx error log | Every server marked failed by `max_fails`; check the backends, then `fail_timeout` |
| `upstream timed out (110: Connection timed out) while reading response header from upstream` | nginx error log | Slow backend; round robin keeps feeding it; consider `least_conn` and `proxy_read_timeout` |
| `backend be_api has no server available!` | HAProxy log | All servers failed `check`; verify the `/healthz` path and expected status |
| HTTP 503 from ALB | Client | Target group has no registered targets, or an AZ is empty with cross-zone balancing off |

## Where this breaks

None of these algorithms fixes a capacity problem; they only spread what you have. Per-balancer state also misleads when you run many balancer instances: each nginx worker or ALB node sees only its own connections unless configured otherwise (nginx needs a `zone` for shared state). Hash-based affinity collapses under hot keys. And for gRPC or HTTP/2 behind an L4 balancer, connection-level balancing pins whole streams to one backend, so balance at L7 instead; the [nginx reverse proxy guide](/blog/nginx-reverse-proxy-guide) covers that setup.

## Questions people ask

### What is the best load balancing algorithm for HTTP APIs?

A load-aware one with active health checks. On nginx that is `least_conn`, on AWS ALB `least_outstanding_requests`, and on HAProxy `random(2)`, because HAProxy's manual says `leastconn` suits long sessions better than short HTTP. Plain round robin is fine only when every request costs about the same and backends are identical, which is rarer than people assume.

### What is the difference between least connections and round robin?

Round robin hands requests out in a fixed rotation and ignores how busy each server is. Least connections looks at current active connections and picks the backend with the fewest, weighted by capacity. Round robin is simpler and fair for uniform requests; least connections adapts when some requests take seconds and others milliseconds, so slow servers stop getting buried.

### When should I use consistent hashing?

When the same key should keep hitting the same backend: cache nodes where locality drives hit rate, or legacy apps with in-memory sessions. Consistent hashing remaps only a small slice of keys when a server joins or leaves, unlike `hash % N`. Avoid hashing on client IP behind NAT, where thousands of users share one address and one backend.

## The call we'd make

For a stateless HTTP service, start with a load-aware algorithm and active health checks: `least_conn` on nginx, `least_outstanding_requests` on ALB, `random(2)` on HAProxy. Add consistent hashing only where cache hit rate or unavoidable session state demands it, and move session data into Redis so you can drop stickiness later. Round robin is fine, but "fine" assumes a uniformity you rarely have.

## Sources

- [nginx: ngx_http_upstream_module reference](https://nginx.org/en/docs/http/ngx_http_upstream_module.html)
- [HAProxy 3.4 Configuration Manual (balance, check, inter)](https://docs.haproxy.org/3.4/configuration.html)
- [AWS: Edit target group attributes for your Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/edit-target-group-attributes.html)