Why Custom AI Chatbots Lose Users

Noé Achache's AI Engineering team at Theodo Data & AI ships AI features to enterprise users across Europe. His finding: chatbots fail adoption because users only unlock 5–10% of what the product can do before hitting a wall.

Sep 16, 2026 26:07

Noé Achache

Expert Insights

Noé Achache led AI engineering at Theodo Data & AI, where a team of around a hundred people builds AI and data products for enterprise clients across Europe. That position gave him an unusually direct view of what happens after launch. He observed what makes users return to a product.

His conclusion is that the adoption problem isn't model quality, it's that a custom chatbot advertises unlimited capability and then refuses to answer the user’s questions.

The episode traces that failure back through interface design and context, then lands on a structural argument about how AI products should be built.

Hear him outline:

Why a company-specific chatbot loses users faster than ChatGPT
Why buttons and guided flows beat prompting
How a traditional interface removes cognitive load
Why memory is "a rich people problem," and what to focus on instead
The gap between explicit and implicit understanding of a user
The steps of evaluation-driven development
Why AI workflows turn into the problematic expert systems built 20 years ago

From the moment you're doing AI, you need to ask yourself: what is the cost of an error? Because an error will happen. It will always happen.

— Noé Achache, Head of AI Engineering at Theodo Data & AI

Monterail Team Analysis

Adoption failure is a scoping problem. Noé's 5–10% figure points at a mismatch of expectations. An open text box promises everything and can cause frustration in users. Constraining the surface with buttons or guided flows lowers expectations to something the product can meet.
Audit your AI features for frequency before choosing an interface. Anything a user does daily or hourly is a poor candidate for prompting. Noé's Shippeo example (a button that analyses why a shipment is late, with the agent running behind it) keeps the AI and drops the prompting.
Design for first-time context instead of long-term memory. Most users will only use your product once, so lean on what you already know (the tool they came from, where they are in the product) instead of what you'd need them to come back for.
Let the dataset choose the technique. Noé's warning about vendors selling "a graph improves performance 30%" is worth taking literally: build the cheapest version first, then let observed errors dictate what you add.
Know when your workflow is becoming overcomplicated. If the if/else tree is growing legs out of its legs, you've re-created the architecture the industry abandoned 20 years ago. That's the signal to move toward workflow-backed agents.
Cost of error is the gating question in regulated domains. In Noé's healthcare example, practitioners' trust was built over decades. AI introduces uncertainty into it. If an error can't be absorbed, the answer is adding constraints, such as structured outputs or having a human in the loop.
Treat LLM-as-judge as a monitoring layer, not your primary evaluator. Start with human evals to build intuition for the data. Where LLM-as-judge does earn its keep is at scale, in production: catching frustration signals like users rephrasing the same question repeatedly or insulting the assistant.