Skip to main content

Production Tutorials & Guides

13 articles tagged with Production.

A demo RAG app is easy; one users trust is not. This is the map for reliable retrieval-augmented generation: grounding, evaluation, retrieval quality, guardrails, and safe rollout.
••2 months ago

Production RAG Reliability — Making LLM Answers Trustworthy

A demo RAG app is easy; one users trust is not. This is the map for reliable retrieval-augmented generation: grounding, evaluation, retrieval quality, guardrails, and safe rollout.

KU
Kiril Urbonas·8
Read article
A prompt tweak or model bump can quietly wreck answers everywhere. Ship LLM changes the way you ship risky code: gate, shadow, canary, roll back.
••2 months ago

Shadow Testing and Canary Releases for LLM Changes

A prompt tweak or model bump can quietly wreck answers everywhere. Ship LLM changes the way you ship risky code: gate, shadow, canary, roll back.

KU
Kiril Urbonas·23
Read article
Prompt injection is not a prompt bug, it is an architecture problem. Here is how we design LLM apps so a poisoned document cannot hijack them.
••2 months ago

Prompt Injection Defense for LLM Apps

Prompt injection is not a prompt bug, it is an architecture problem. Here is how we design LLM apps so a poisoned document cannot hijack them.

KU
Kiril Urbonas·8
Read article
A RAG system that invents facts erodes trust fast. Here is how we ground answers, force citations, and catch the fabrications before users do.
••2 months ago

Hallucination Detection — Grounding and Citations for RAG

A RAG system that invents facts erodes trust fast. Here is how we ground answers, force citations, and catch the fabrications before users do.

KU
Kiril Urbonas·26
Read article
Users hit stop, but our server kept paying for tokens for another 40 seconds. Here's how we wired real cancellation and backpressure into an SSE streaming endpoint.
••2 months ago

Streaming LLM Responses: SSE, Backpressure, and Cancellation

Users hit stop, but our server kept paying for tokens for another 40 seconds. Here's how we wired real cancellation and backpressure into an SSE streaming endpoint.

KU
Kiril Urbonas·35
Read article
Everyone says Compose is for dev only. We ran it in production for two years on a single node and it was the right call, until the day it very much wasn't.
••2 months ago

Docker Compose in Production: When It Fits and When It Doesn't

Everyone says Compose is for dev only. We ran it in production for two years on a single node and it was the right call, until the day it very much wasn't.

KU
Kiril Urbonas·18
Read article
A user got our support bot to recite its system prompt and then draft a refund it wasn't authorized to give. Two layers of guardrails, one on input, one on output, closed both holes.
••2 months ago

Guardrails for Production LLMs: Input and Output Filtering That Holds

A user got our support bot to recite its system prompt and then draft a refund it wasn't authorized to give. Two layers of guardrails, one on input, one on output, closed both holes.

KU
Kiril Urbonas·19
Read article
Our node image shipped 240 CVEs, most from OS packages we never called. Moving to distroless dropped the count to single digits and cut image size by 70%.
••2 months ago

Distroless Docker Images: Smaller, Safer Production Containers

Our node image shipped 240 CVEs, most from OS packages we never called. Moving to distroless dropped the count to single digits and cut image size by 70%.

KU
Kiril Urbonas·5
Read article
Users kept asking the same questions in slightly different words, and we paid full price every time. Semantic caching cut our LLM bill by a third.
••2 months ago

Semantic Caching for LLM Apps: Cutting Cost on Repeated Queries

Users kept asking the same questions in slightly different words, and we paid full price every time. Semantic caching cut our LLM bill by a third.

KU
Kiril Urbonas·20
Read article
A long, stable system prompt re-billed on every request is money on fire. How prompt caching works, where the cache boundary belongs, and the structuring discipline that got us a big cost and latency cut without changing behavior.
••2 months ago

Prompt Caching for Production LLM Apps — Cutting Cost and Latency at the Token Layer

A long, stable system prompt re-billed on every request is money on fire. How prompt caching works, where the cache boundary belongs, and the structuring discipline that got us a big cost and latency cut without changing behavior.

KU
Kiril Urbonas·34
Read article
Node upgrades, autoscaler scale-downs, and spot reclaims all drain nodes. Without PDBs they can take all your replicas at once. The budgets, probes, and graceful-shutdown handling that keep voluntary disruptions invisible to users.
••2 months ago

Kubernetes Pod Disruption Budgets — Surviving Node Drains Without an Outage

Node upgrades, autoscaler scale-downs, and spot reclaims all drain nodes. Without PDBs they can take all your replicas at once. The budgets, probes, and graceful-shutdown handling that keep voluntary disruptions invisible to users.

KU
Kiril Urbonas·16
Read article
Parsing model output with a regex and a prayer doesn't survive contact with traffic. The validation layers that keep structured LLM output reliable — constrained decoding, schema validation, and the repair loop.
••2 months ago

LLM Output Validation — Schema-Constrained Generation in Production

Parsing model output with a regex and a prayer doesn't survive contact with traffic. The validation layers that keep structured LLM output reliable — constrained decoding, schema validation, and the repair loop.

KU
Kiril Urbonas·29
Read article