#LLM — Articles | DevOpsNess

#LLM

63 articles tagged with LLM.

Production RAG Reliability — Making LLM Answers Trustworthy

A demo RAG app is easy; one users trust is not. This is the map for reliable retrieval-augmented generation: grounding, evaluation, retrieval quality, guardrails, and safe rollout.

Kiril Urbonas·1

Read article

Page 1 of 6Next

••yesterday

Shadow Testing and Canary Releases for LLM Changes

A prompt tweak or model bump can quietly wreck answers everywhere. Ship LLM changes the way you ship risky code: gate, shadow, canary, roll back.

Kiril Urbonas

Read article

••3 days ago

Long Context vs RAG — When to Use Which

A million-token window doesn't retire your retrieval stack. Here's when to stuff the prompt, when to retrieve, and when to do both.

Kiril Urbonas·1

Read article

••4 days ago

Prompt Injection Defense for LLM Apps

Prompt injection is not a prompt bug, it is an architecture problem. Here is how we design LLM apps so a poisoned document cannot hijack them.

Kiril Urbonas·1

Read article

••5 days ago

RAG Evaluation Metrics — Faithfulness and Context Precision

A single answer-quality score hides where your RAG pipeline actually breaks. Split retrieval eval from generation eval and measure each one honestly.

Kiril Urbonas·1

Read article

••6 days ago

Hallucination Detection — Grounding and Citations for RAG

A RAG system that invents facts erodes trust fast. Here is how we ground answers, force citations, and catch the fabrications before users do.

Kiril Urbonas·1

Read article

••last week

Token Budgeting for Long-Context Prompts: What to Cut First

A 180k-token context window is not a license to stuff everything in. Here's how we cut prompt size 60% without hurting answer quality, and what to trim first.

Kiril Urbonas·3

Read article

••last week

Multi-Provider LLM Gateways: Routing, Fallback, and Cost Control

When our single LLM provider had a 40-minute outage, every AI feature went dark. A gateway with routing and fallback fixed that, and cut spend 30% as a bonus.

Kiril Urbonas·4

Read article

••last week

Streaming LLM Responses: SSE, Backpressure, and Cancellation

Users hit stop, but our server kept paying for tokens for another 40 seconds. Here's how we wired real cancellation and backpressure into an SSE streaming endpoint.

Kiril Urbonas·4

Read article

••last week

Choosing an Embedding Model: Dimensions, Cost, and MTEB Reality

The top model on the MTEB leaderboard made our search worse and our bill bigger. Here's how we actually picked an embedding model for a real RAG system.

Kiril Urbonas·2

Read article

••last week

Agent Memory: Short-Term, Long-Term, and When You Need Neither

Most agents that "need memory" actually need a smaller context window and a database. Here's how we cut a support agent's token bill by 60 percent by deleting memory.

Kiril Urbonas·2

Read article

••last week

Guardrails for Production LLMs: Input and Output Filtering That Holds

A user got our support bot to recite its system prompt and then draft a refund it wasn't authorized to give. Two layers of guardrails, one on input, one on output, closed both holes.