Every AI feature we have shipped gets something wrong. Not occasionally, and not because we picked a bad model. Summaries miss the point of an email. Classifiers put a booking in the wrong category. Extraction pulls the invoice date from the wrong line. This is the normal operating condition of these systems, and it is not going away with the next release.
What decides whether the feature survives contact with real users is almost never the error rate itself. It is what the interface does on the wrong answer. We have watched a feature at 92 percent accuracy get switched off and a feature at 80 percent become the thing customers mention in renewal calls. The difference was design.
Price the wrong answer before you design the screen
The first question in any AI feature scope should be what a mistake costs, and who pays for it. A wrong tag on a support ticket costs a few seconds of someone’s attention. A wrong figure carried into an invoice costs money and trust. A wrong summary sent automatically to a member costs the relationship.
Those three cases need completely different designs, and teams routinely build all three the same way. Cheap errors can be corrected after the fact and the feature can act on its own. Expensive errors need a person in the loop before anything happens. Irreversible errors, anything that sends, charges, publishes, or deletes, should not be automated at all in the first version, no matter how good the accuracy looks in testing.
Show the source, not just the conclusion
When a model tells a user something, the user’s immediate question is where that came from. If the interface cannot answer it, the user has two options: trust blindly or verify manually. Both destroy the value of the feature.
The fix is usually simple. Link the summary sentence to the message it came from. Highlight the region of the document the figure was extracted from. Show the three records the recommendation was based on. This costs a day of work and it converts an unverifiable claim into a claim the user can check in two seconds. It is the single highest return thing you can add to an AI feature.
Make correction cheaper than starting over
Watch what a user does when the output is wrong. If they delete everything and do the task by hand, your feature has negative value: it added a step and then cost them the time anyway.
Design the wrong path deliberately. Make every field editable in place. If the model picked a category, show the next two most likely ones as one-click alternatives rather than making the user open a dropdown of forty options. If a summary is off, let them regenerate just that paragraph. The target is that a wrong output still leaves the user better off than a blank page, and that is a much lower bar than being right.
Say what the system does not know
Models will answer confidently on input they should have refused. In our own products the most common real-world failure is not a wrong answer to a good question, it is a fluent answer to a question the data cannot support.
Handle that upstream rather than in the prompt. Check whether the required inputs are actually present before you call the model, and return an honest empty state when they are not. “We could not find enough activity in the last 90 days to summarise” is a better product than a plausible summary of nothing, and it is a check you write in ordinary code.
Treat every correction as product data
When a user fixes an output, that is the highest quality signal your product will ever generate. Log the original, the correction, the user, and the context.
Within a few weeks that log tells you where the feature actually fails, which is almost never where the team assumed. It gives you an evaluation set drawn from real usage rather than from examples you invented. And it gives you an honest accuracy number to report internally instead of a benchmark score that has no relationship to your users’ data. We build this logging before launch now, because retrofitting it means throwing away the first month of the only feedback that matters.
Set the expectation in the copy
The label on the feature does more work than most teams expect. “Suggested categories” and “Categorised automatically” describe the same output and produce entirely different user reactions to an error. The first invites a quick check. The second makes every mistake feel like a broken product.
Be accurate rather than modest. Overclaiming turns normal error rates into complaints, and underclaiming means nobody uses the feature at all.
If you are planning an AI feature and want a partner who has shipped these and lived with the failure modes, see how we work.