3 articles tagged with Backend.
Buffered SSR makes users wait for the slowest query before they see anything. Streaming from the edge flips that — send the shell now, fill the gaps as data lands.
Users hit stop, but our server kept paying for tokens for another 40 seconds. Here's how we wired real cancellation and backpressure into an SSE streaming endpoint.
Streaming LLM responses is easy until the client disconnects, the model stalls, or the user cancels. The patterns that keep streaming responsive without leaking spend.
