# vLLM vs SGLang vs TGI vs TensorRT-LLM: Which LLM Server in 2026

URL: https://www.devopsness.com/blog/llm-serving-vllm-vs-sglang-vs-tgi
Author: Kiril Urbonas
Published: 2026-10-31
Updated: 2026-10-31
Categories: AI
Tags: Vllm, LLM, AI Infrastructure, Inference, Gpu

Summary: Hugging Face TGI is archived and read-only, so the real 2026 choice is vLLM, SGLang or TensorRT-LLM. A feature table, a launch command for each, and the one we default to.

---
# vLLM vs SGLang vs TGI vs TensorRT-LLM: Which LLM Server in 2026

Use vLLM unless you have a specific reason not to. SGLang is the strongest alternative for agent and RAG traffic that reuses long prompts, and TensorRT-LLM is for NVIDIA-only fleets chasing peak throughput. Hugging Face TGI is out: its repository is archived, and Hugging Face itself now points users to vLLM and SGLang.

> [!VERIFIED]
> vLLM 0.32.0, SGLang 0.5.23, TensorRT-LLM 1.2.1 (1.4.0rc2 pre-release), TGI 3.3.7, against the vLLM, SGLang and TensorRT-LLM docs, their GitHub releases and PyPI pages, and the archived TGI repository README. Checked October 2026.

## TGI is in maintenance mode, and that settles one question

The TGI README now opens with "text-generation-inference is now in maintenance mode", and GitHub shows the repository as archived and read-only; the last commit landed on March 21, 2026. The last release is v3.3.7 (December 19, 2025), although the README's examples still use the `3.3.5` image. It still starts and still serves, but it will not get support for next quarter's model architectures or fixes for the next CVE in its dependency tree.

If you run TGI today, nothing is on fire. Put the migration on this quarter's plan: every engine below exposes an OpenAI-compatible API, so the client side of the move is usually a base URL change.

## The feature table

| Engine | Continuous batching | Prefix caching | Structured output | Quantization | Multi-node | OpenAI-compatible API |
|---|---|---|---|---|---|---|
| vLLM 0.32 | Yes | Yes, `--enable-prefix-caching` | xgrammar or guidance, `response_format` json_schema | AWQ, GPTQ, FP8, INT8, INT4, GGUF, bitsandbytes, ModelOpt | Ray or native multi-node, TP plus PP | Yes, plus `/v1/responses` and Anthropic `/v1/messages` |
| SGLang 0.5.23 | Yes | Yes, RadixAttention | XGrammar default, Outlines, llguidance | FP8, FP4, MXFP4, AWQ, GPTQ, INT8, GGUF | Yes, `--nnodes` and `--dist-init-addr` | Yes |
| TensorRT-LLM 1.2 | In-flight batching | Paged KV cache with block reuse | Guided decoding with constraints | FP8 on H100 and later, FP4 on B200 | Yes, tensor, pipeline and expert parallel | Yes, via `trtllm-serve` |
| TGI 3.3.7 | Yes | Not listed in README | Guidance/JSON | bitsandbytes, GPTQ, EETQ, AWQ, Marlin, FP8 | No, tensor parallel on one host | Yes, but archived |

Two caveats sit behind that table. TensorRT-LLM runs on NVIDIA GPUs only, per its own overview. And vLLM removed its old `guided_*` request fields in v0.12.0, so clients written against older vLLM need the `structured_outputs` field instead.

## Launch commands for each

vLLM listens on port 8000 by default:

```
$ pip install "vllm==0.32.0"
$ vllm serve Qwen/Qwen3-8B \
    --enable-prefix-caching \
    --max-model-len 32k \
    --tensor-parallel-size 1
```

SGLang defaults to port 30000 and binds to `127.0.0.1` unless you pass `--host`:

```
$ docker run --gpus all --shm-size 32g -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    lmsysorg/sglang:latest \
    sglang serve Qwen/Qwen3-8B --host 0.0.0.0 --port 30000
```

TensorRT-LLM's `trtllm-serve` takes the model plus parallelism flags; its docs' client examples use port 8000:

```
$ trtllm-serve "Qwen/Qwen3-8B" --tp_size 1 --host 0.0.0.0 --port 8000
```

TGI, for completeness and for the migration test, from its README:

```
$ docker run --gpus all --shm-size 1g -p 8080:80 -v $PWD/data:/data \
    ghcr.io/huggingface/text-generation-inference:3.3.5 \
    --model-id Qwen/Qwen3-8B
```

The same request works against all four, which is how we would A/B them before cutting over:

```
$ curl -s localhost:8000/v1/chat/completions \
    -H 'Content-Type: application/json' \
    -d '{"model": "Qwen/Qwen3-8B",
         "messages": [{"role": "user", "content": "Name three Linux init systems."}],
         "response_format": {"type": "json_schema", "json_schema": {"name": "inits",
           "schema": {"type": "object", "properties": {"items": {"type": "array",
           "items": {"type": "string"}}}, "required": ["items"]}}}}'
```

Swap the port to 30000 for SGLang or 8080 for TGI. Schema-constrained output is the feature that most often behaves differently between engines, so test it explicitly; our notes on [schema-constrained generation](/blog/llm-output-validation-schema-constrained-generation) cover what to validate even when the engine enforces a grammar.

## Where each engine wins

**vLLM wins on breadth.** The widest model coverage, the longest quantization list, AMD and other accelerators alongside NVIDIA, and an API surface that now includes Responses, embeddings, reranking and an Anthropic-style messages endpoint. It ships often: 0.30, 0.31 and 0.32 landed between September 22 and October 18, 2026. If you came from Ollama on a laptop, the [Ollama vs vLLM comparison](/blog/ollama-vs-vllm) explains why this is the usual next step.

**SGLang wins on shared prefixes.** RadixAttention keeps a tree of cached prefixes, which is exactly the traffic shape of agents that resend a long system prompt and tool list on every turn, and of RAG with repeated context. Its structured-output path defaults to XGrammar with EBNF support.

**TensorRT-LLM wins when the hardware is fixed and NVIDIA.** FP8 on Hopper, FP4 on Blackwell, speculative decoding with EAGLE, MTP and NGram, and expert parallelism for MoE models. The cost is velocity: as of October 2026 the newest stable on PyPI is 1.2.1 from April, while the 1.4 line is still at release candidates.

**TGI wins nothing new.** Keep it only until the migration test passes.

## Failure modes to watch for

- **`guided_json` stops working after a vLLM upgrade.** The `guided_*` fields were removed in v0.12.0. Move to `response_format` or `extra_body={"structured_outputs": {...}}`.
- **`Connection refused` on SGLang from another host.** The server binds to `127.0.0.1` by default. Pass `--host 0.0.0.0` and publish the port.
- **Prefix caching that never hits.** A timestamp or request ID at the top of the system prompt makes every prefix unique. Put volatile fields at the end, the same rule as in [prompt caching for production LLM apps](/blog/prompt-caching-production-llm-apps).
- **Out of memory at startup.** A model's full context length can reserve more KV cache than the GPU holds. Cap `--max-model-len` in vLLM, or pass `auto` to fit the largest length the card allows.

## Where this breaks

We have not published benchmarks here, because public numbers for these engines go stale within one release and depend heavily on model, GPU, batch size and prompt shape. Run your own traffic replay before trusting any throughput claim, including a vendor's. Multi-node serving on any of them adds network and scheduler failure modes that single-node tests will not show.

## Questions people ask

### Is Hugging Face TGI deprecated?

Yes, effectively. The TGI repository README says it "is now in maintenance mode", and GitHub shows it archived and read-only, with the last commit on March 21, 2026. Hugging Face recommends vLLM and SGLang for serving, and llama.cpp or MLX for local use. Existing TGI 3.3.x deployments still run, but plan a migration, because new model architectures and fixes will not arrive.

### Is SGLang faster than vLLM?

It depends on traffic shape, so measure on your own prompts. SGLang's RadixAttention is built for workloads that reuse long prefixes, such as agents resending tool definitions or RAG with repeated context, and can win there. vLLM also supports prefix caching and has broader model and hardware support. Replay a day of real requests against both before deciding.

### Which LLM serving engine has an OpenAI-compatible API?

All four do. vLLM serves `/v1/chat/completions`, `/v1/completions`, `/v1/responses` and `/v1/embeddings` on port 8000. SGLang serves `/v1/chat/completions` on port 30000 by default. TensorRT-LLM's `trtllm-serve` exposes completions, chat completions and responses. TGI also has an OpenAI-style chat endpoint, but its repository is archived.

## The call we'd make

Default to vLLM, pinned to an exact version, with prefix caching on and `--max-model-len` set deliberately. Run SGLang alongside it for a week if your traffic is agentic or RAG-heavy and keep whichever serves your replayed traffic cheaper. Reach for TensorRT-LLM only when the fleet is NVIDIA and the throughput gain pays for a slower release cadence. Whatever you choose, get off TGI this quarter.

## Sources

- [Hugging Face TGI repository (maintenance mode notice)](https://github.com/huggingface/text-generation-inference)
- [vLLM docs: vllm serve CLI reference](https://docs.vllm.ai/en/latest/cli/serve/)
- [SGLang docs: Quickstart](https://docs.sglang.io/docs/get-started/quickstart)
- [TensorRT-LLM docs: trtllm-serve](https://nvidia.github.io/TensorRT-LLM/commands/trtllm-serve/trtllm-serve.html)