Idea #1
A typed UI contract — where the model selects named UI components from a fixed, versioned catalog rather than generating free-form text or HTML — is the right architectural primitive for production agentic apps. The mechanism is straightforward: you supply a context that describes exactly which components exist (a product card, a flight row, a carousel, a submit button) and the model outputs structured blocks that name those components and pass data into them. The model never invents new UI; it only picks from the menu you give it. This constraint is what makes the system safe and predictable: the client knows at compile time every component it will ever need to render, so there are no unknown component types to crash on. The contract should also encode layout rules — the talk gives the example of "one to three flights → swipable carousel; four or more → vertical list" — so the model handles layout decisions too without the client needing conditional logic. Crucially, the catalog must be versioned against client app versions: if a new flight card UI ships in version 2.0, the context engineering layer must only surface it to users on 2.0 or later. The model then gets a view of capabilities appropriate to the client it is actually talking to, preventing it from emitting components that older installs cannot render.
[from: Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing – Bala Ramdoss, Amazon Lens — YouTube · AI Engineer · https://youtu.be/maTp79FD9gI]
Idea #2
The model output and the user experience are two different things, and conflating them is the central failure mode of most AI product builds. When an LLM returns a correct, well-reasoned answer in plain text, the job is not done — the user still has to read, parse, and act on it, which means the cognitive burden stays with the human. The right framing is that the model handles intelligence (routing, reasoning, tool selection) while a separate rendering layer handles experience (what gets shown, how it's laid out, what actions are available). This separation matters because the problems in each layer are completely different in kind: the model failing is an AI problem, while the user experience being slow, confusing, or broken is a delivery and design problem. The talk opens with a concrete contrast: an AI assistant asked to help reserve a restaurant table returns a correct text answer with the phone number, hours, and a note about the oyster bar — but leaves all the actual booking work to the user. The same result rendered with a date picker, a time slot, and a "Book" button collapses that work to a couple of taps. Same model output, radically different outcome. The speaker's core argument is that "your agent output is not the CX — you build on it."
[from: Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing – Bala Ramdoss, Amazon Lens — YouTube · AI Engineer · https://youtu.be/maTp79FD9gI]
Idea #3
For AI features, the right performance metric is time to first meaningful chunk, not total response latency — and this shift requires redesigning how your loading and progress states work. Traditional apps make a request and show a spinner until a complete response arrives; with LLMs, total latency is high and mostly outside your control, so optimizing for it is a losing game. Streaming flips the user experience: you start rendering as soon as the first chunk arrives, show a skeleton, progressively fill it in, and complete it over 3–4 seconds in a way that feels fast even if the wall-clock time is long. The speaker notes that this also changes what you can do with wait time: instead of a spinner, you design the experience so the user is active. Amazon Lens Live, for example, lets users focus the camera on different objects and tap items of interest while results are loading — the wait is productive rather than passive. The talk also recommends surfacing agent-thinking state sparingly but explicitly: showing users "what the agent is doing" builds trust and makes a 10-second wait tolerable. The key design rule is that a generic loading spinner is incompatible with AI features — the gap between request and response is too long and too variable, and users who have moved past the "forgiving phase" now expect to see progress, not a blank spinner.
[from: Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing – Bala Ramdoss, Amazon Lens — YouTube · AI Engineer · https://youtu.be/maTp79FD9gI]
Idea #4
A Backend for Frontend (BFF) layer sitting between the model and the client is the component that makes server-driven UI practical at scale, and its job is broader than just formatting. The BFF absorbs the model's typed UI intent and translates it into fully hydrated, platform-specific render instructions: it handles Android vs. iOS differences, attaches action payloads to every tappable element (what a tap does, what deep link it opens), names the impression metrics to log, and carries conversational context across turns so subsequent model responses know what was already shown. By centralizing all this logic in the BFF, the client can stay "dumb" — it receives a complete, ready-to-render instruction set and draws it, full stop, with no branching logic of its own. This is especially non-negotiable for mobile, where you cannot meaningfully patch the client: with hundreds of millions of installs on an arbitrary mix of app versions, a client that needs to make decisions about unknown content types will crash, and it will keep crashing for days or weeks until installs update. The BFF absorbs all the unknowns server-side, where you can patch instantly. An additional practical benefit the speaker highlights: the BFF can reuse existing UI components already shipped in the app — the flight row, the product card — rather than building a new agentic visual language from scratch. The agent experience then feels native, on-brand, and familiar, because it literally is the same components the app always used.
[from: Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing – Bala Ramdoss, Amazon Lens — YouTube · AI Engineer · https://youtu.be/maTp79FD9gI]
Source: Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing – Bala Ramdoss, Amazon Lens (AI Engineer)
Text summarized and optimized using Anthropic’s models and reviewed by a human.