Building a Chatbot API From Scratch — Part 2: Streaming, Prompt Engineering and Docker

Part 4 actually of building a retail inventory API and then giving it a brain. In Part 3 I built the chatbot foundation: FastAPI, PostgreSQL, conversation memory, context trimming, rolling summarization, and 13 PRs worth of broken things. The API worked. It remembered what you said. It didn't fall over when the context got too long. That was enough to call it functional. But it didn't feel finished. No streaming. No real identity. No way to run it anywhere except my machine. Five PRs later, all of that changed. Some of it was clean. Some of it was not. PR 14 — Auto-Title Generation Small PR. Big quality-of-life improvement. Every new conversation started with the title "New Chat..." and stayed that way forever. I wanted it to generate automatically from the first message, without blocking the response. The approach: fire a background task after the conversation is created. if not request.title: "" asyncio.create_task( update_conversation_title(engine, conversation.id, request.user_mess