Most conversations about building AI into a product start with the model. Which one, how big, how much it costs per call. Those are real questions, but they are not usually the ones that decide whether the feature works.
The thing that decides it is your data. Not how much of it you have, which is the metric everyone quotes, but whether it is in a shape the feature can actually use. We have run this assessment on our own products and on client builds, and the pattern holds: teams with modest, well-structured data ship useful AI features quickly, and teams with enormous, messy data spend months discovering they cannot.
What the feature needs to know
The first question in any discovery session is not what data exists. It is what the feature would need to know to be right.
If a client wants an AI assistant that answers member questions, the honest version of the requirement is: a current, correct, retrievable answer to each question a member is likely to ask. If those answers live in three staff members’ heads and a PDF from 2023, no model will fix that. If they live in a help centre that is kept up to date, the build is straightforward.
Working backwards from the output to the required inputs takes about an hour and saves entire sprints. It also tends to reveal that the interesting AI feature depends on one specific data source nobody thought to mention.
The four questions we ask about a data source
For each source the feature depends on, we want to know:
Is it current? An AI feature that confidently returns stale information is worse than no feature, because it moves the error from “I could not find that” to “here is the wrong answer.” Anything that feeds a customer-facing response needs a known refresh cadence.
Is it structured enough to retrieve? Free text is fine, but you need something to filter and rank on. A thousand support tickets with no category, product or date attached are much harder to use well than three hundred that are tagged.
Is it consistent? The same concept named four different ways across systems is the most common real-world blocker we see. Reconciling that is unglamorous work, and it usually has to happen before anything else.
Who is allowed to see it? If the underlying records have permissions, the AI feature inherits them. Retrieval that ignores access rules will eventually surface something it should not, and that is the failure mode that ends projects.
Cleaning is part of the build, not a prerequisite
There is a tempting version of this advice that says fix all your data first, then build AI. That project never finishes, because “all your data” has no edge.
The better approach is to scope the feature narrowly enough that the data work is finite. Pick one use case, identify the sources it truly needs, and fix only those. On one of our own products, a support assistant that only answered billing questions needed a fraction of the cleanup that a general assistant would have, shipped in weeks rather than quarters, and told us which category to tackle next based on what users actually asked.
Narrow scope is not a compromise here. It is the mechanism that makes the data problem solvable.
Build the evaluation set while you assess
One practical habit worth adopting early: as you go through the data, collect thirty to fifty real examples of what the feature should produce. Actual questions with correct answers, actual documents with correct classifications.
That set costs an afternoon and becomes the thing you test every change against. Without it, judgements about whether the feature is working reduce to whoever tried it last and how they felt about it. With it, you can change a prompt, a model or a retrieval strategy and know within minutes whether you made things better.
What good readiness looks like
You are ready to build when you can point at specific sources, say how fresh they are, explain who can see what, and show a handful of examples of correct output. That is a lower bar than most teams expect and a more demanding one than “we have a lot of data.”
We would rather spend a week finding out a feature is not buildable yet than three months proving it. Almost always, the week produces a smaller version that is buildable now.
If you are working out whether your next product idea can support the AI feature you have in mind, Orana Software builds these for a living and is happy to talk it through.