A reconnect is not a regenerate
BuilderHelp and RuleCaddie stream long answers with tool calls in the middle. A golf-course LTE blip or a tab refresh should not mean a second model run. The truncated-stream problem is labeling incomplete text when generation actually stopped. The reconnect problem is different: generation is still running, or should still be running, and the client needs a way back onto that work. A reconnect that quietly fires sendMessage again is a regenerate wearing a recovery costume.
The socket is not the generation
HTTP and SSE couple delivery to one connection. When that connection closes, the naive server stops writing and the naive client treats the close as done. For a short completion that is often fine. For an agent that has already retrieved a rule, started a second tool call, or spent twenty seconds on an owner-packet lookup, killing the producer because the browser blinked is the expensive failure mode.
The fix is to decouple producer from consumer. The model (or the workflow that wraps it) keeps writing tokens into a buffer the server owns. The original SSE response is one reader of that buffer. A later GET can be another. If nobody is listening for a few seconds, the producer does not care — it finishes writing, the buffer holds the bytes, and the next client drains what it missed.
That is the shape the Vercel AI SDK's resumable-stream path encodes: create the stream once, publish into Redis (or equivalent), keep the producer alive with after()/waitUntil so a serverless request return does not kill it, and expose a resume endpoint keyed by chat. Closing a tab becomes a disconnect. It is not, by itself, a cancel.
On a course with spotty signal, or on a field tablet that sleeps mid-answer, this is the difference between "still working" and "ask again and hope the tools do not double-fire." The product promise is continuity of one turn, not a polite retry loop.
Give the stream an identity the chat can find
Resume only works if the client can ask for the right stream. Persist an activeStreamId on the chat (or session) when generation starts, and clear it when the producer finishes or is explicitly stopped. On mount, if that id is set, GET the resume route. If the route returns 204, there is nothing to reattach — render the saved messages and wait for the next user turn.
Do not use the assistant message id as the stream id if that id is only assigned when generation completes. Assign a stream id at start. Persist partial assistant text as you go so a reconnect that arrives after completion still has a coherent transcript even if the Redis buffer has expired.
Authorize the resume route the same way you authorize the chat. A stream id in a response header or sessionStorage is not a credential. Anyone who can guess or intercept it should still fail the ownership check for that chat.
Clear activeStreamId when starting a new user message, before the new producer begins. Leaving the old id in place is how a late resume attaches to yesterday's stream, or how two tabs fight over which completion is live.
Stop has to be a separate verb
Once disconnect no longer cancels generation, the Stop button cannot be "abort the fetch." Aborting the fetch only closes this client's pipe. The producer keeps running, and resume will cheerfully reattach to work the user thought they killed.
Ship a stop endpoint that persists the partial assistant snapshot, cancels the producer, and clears activeStreamId — but only if it still points at the stream the client meant to stop. A stale stop that lands after a newer turn has started should no-op. Keep route-change and refresh cleanup off that endpoint; those are disconnects, not stops.
After an explicit stop, suppress automatic resume for that chat until the user sends again or chooses retry. Otherwise the client can reconnect into a stream that is mid-cancel and flicker between incomplete and gone.
Mark the retained partial as incomplete when stop wins — same rule as a true truncation. Resume is for continuing live work. Stop is for ending it on purpose and leaving an honest half-answer on screen.
What we check before calling resume done
Reload mid-stream on a slow answer and confirm the same assistant message continues — same citations, same tool cards, no second user bubble. Drop the network for ten seconds and come back; the buffer should replay without starting a parallel completion.
Press Stop and confirm the producer actually ends: no further token spend, activeStreamId cleared, partial text kept and marked incomplete if it never finished. Start a new message immediately after a stop and confirm the old stop cannot clear the new stream's id.
Open the same chat in two tabs briefly. Both may follow one stream; neither should spawn a second. If your product is single-device only, say so — but still do not regenerate on refresh.
If resume is not wired yet, do not fake it with an automatic regenerate. Show that the answer was interrupted, keep the partial, and let the user choose to continue or ask again. A honest incomplete state beats a silent second bill for the same question.
Questions
- How is this different from labeling a truncated stream?
- Truncation covers the case where generation actually stopped and the UI must not look finished. Resume covers the case where generation should continue (or already continued server-side) and the client must reattach instead of starting a new completion.
- Why not just call sendMessage again after a drop?
- A new send starts a new model run. You pay twice, tool calls may fire twice, and the UI can show two assistant threads for one question. Reattach to the active stream id instead.
- Can Stop still be a client-side abort?
- Not once streams are resumable. Client abort only closes the pipe. Use a dedicated stop endpoint that cancels the producer and clears activeStreamId; treat refresh and navigation as disconnects.
Sources
- Vercel AI SDK — Chatbot Resume Streams — useChat resume, consumeSseStream, and activeStreamId patterns.
- resumable-stream (Vercel) — Redis-backed producer that survives the original HTTP reader going away.
- Ably — AI chat stream resumption — When a Redis buffer is enough versus when you need a durable session across devices.