Chat boxes are the worst place to put an LLM

The default UI for a new LLM feature is a chat panel pinned to the right edge of the screen. It's the wrong default. A chat box assumes the user wants to talk to your software, and almost no one in the middle of a real job does. They want the software to quietly help with the form they were already filling, the note they were already typing, the ticket they were already triaging. The chat box is a feature we know how to build; it's rarely the feature anyone actually wanted.

I keep watching teams ship a chat panel as the wrapper for a perfectly good model, then puzzle over the usage numbers a month later. The model is fine. The panel is the problem. It's a context switch, a blank prompt, and a vague invitation. The user has to stop, name what they want, type it well enough for the model to understand, and then translate the answer back into the next step of their actual work. The model has to do real work; the user has to do real work just to ask the model to do real work.

The chat box is friction in a hat

A blank text field is intimidating in a way we under-appreciate. The user has to formulate a question, but they were thinking about the task in front of them, not about phrasing a query for a generic assistant. The chat box turns every assist into a two-stage problem: first articulate what you want, then judge what came back. That's two pieces of work where there was one.

The other quiet cost is novelty. A chat box is a new place in the app — a new tab, a new sidebar, a new modal — and new places have to be remembered. They get used in the first week because they're shiny, and then they slip out of muscle memory because they aren't on the path the user takes through the app. The path is the form, the ticket, the report, the radio chatter. The chat box is sitting in a corner waiting for an invitation that mostly never comes.

Forms are conversations already

The thing a form is doing — asking the user a series of questions in a fixed order — is exactly the thing a chat interface tries to do, except a form does it better. The questions are scoped. The answers are typed in the field where they're going to be used. The structure tells the user what's expected. A form is what a conversation looks like when both sides know what they're trying to accomplish.

So the right place to put the model is inside the form. Pre-fill the fields the model can be confident about. Suggest the fields it's less sure about, but leave the user in control of the final value. Surface the fields it thinks are wrong or contradictory and let the user confirm or override. The user never has to leave the form. The model never has to guess what was being asked. It's the same conversation; it just happens in the fields that were already there.

The pre-filled draft pattern

The pattern I keep coming back to is the pre-filled draft. When the user opens the report, the work order, the handover note, the model has already taken its first pass. The opening, the structured fields, the bullet points pulled from the underlying system are sitting there waiting. The user's job is to read, edit, and ship — not to start from a blank page or, worse, to summon an assistant from a chat panel and copy-paste its output into the real document.

A pre-filled draft has two huge advantages over a chat-based assist. First, the user is editing prose rather than prompting it — a much shorter cognitive distance to good output. Second, the model's work is visible from the moment the user shows up. There's no question of whether it's earning its keep; either the draft is useful enough to edit, or it isn't, and either way the user has the data they need to decide.

Inline suggestions, not modals

When the model wants to volunteer something mid-flow — a flag, a correction, a related ticket the user might not have seen — the right shape is an inline suggestion the user can take or dismiss without leaving the spot they're in. Not a modal. Not a banner that pushes the rest of the page down. A single line, where the work is happening, that the user can act on in one keystroke or ignore.

The pattern Google Docs and GitHub Copilot popularized for writing — ghost-text completions, inline tag chips, single-key accept — is a much better fit for industrial software than people realize. The reason it works isn't the interface trick; it's that the model is meeting the user inside the work, not pulling them out of it.

The chat box has its place

There's a category where a chat box is the right answer: open-ended exploration where the user genuinely doesn't know what they want yet. Research, debugging, talking to a knowledge base with a fuzzy question in mind. Those are real use cases and a chat panel is the right home for them.

But that's not most of what people do at work. Most of what people do at work is fill the form, file the ticket, write the note, ship the report. The model belongs there, in the form, doing its share of the typing. The chat box can sit in a corner for the days the user wants to ask a wandering question. The rest of the time, it should stay out of the way.