Why AI Projects Fail at the Integration Layer
The demo works. Someone loads a handful of documents, asks the questions the team has been asking for years, and gets answers that are better than expected. The room agrees this is real. Then the project spends nine months not shipping.
This pattern is common enough now that it deserves a proper explanation. The failure is rarely in the model. It sits in the layer between the model and everything else the firm runs on, and that layer is where most of the actual engineering lives.
The demo runs on a copy of the world
A pilot usually operates on an extract. Forty documents pulled into a folder. A spreadsheet exported last Tuesday. A test account with permissions nobody scrutinised because it was only a pilot.
Every one of those simplifications is a decision that got deferred rather than made. Production has no extract. It has the document management system with fifteen years of accumulated structure, the shared drive that three departments use differently, the core system with an API that was written for batch reporting, and a mailbox that half the process still depends on. The pilot answered whether the model could do the reasoning. It did not answer whether the reasoning could be delivered to the person who needs it, at the moment they need it, using only what they are allowed to see.
Identity is the first wall
In a regulated firm, the question of what an AI system may read is not a technical convenience. It is the entire compliance posture of the deployment.
An assistant that retrieves across a document store inherits whatever access its service account has. If that account can see everything, the assistant can quote a matter to someone walled off from it, surface a client file to a team that should not have it, or expose a personnel record inside an answer about policy. The user never asked for any of that. The retrieval step simply did not know it was forbidden.
Solving this properly means the system carries the user's identity all the way through, and that every retrieval is filtered by that identity before anything reaches the model. It means honouring ethical walls, matter-level restrictions, client segregation, and the exceptions that exist for good reasons and are documented nowhere. In practice it also means discovering that permissions in the source system are messier than anyone believed, and that the AI project is now the thing forcing that conversation. Firms are often surprised by how much of the work is here. It is usually the single largest item in the build.
Data access is not the same as data retrieval
Getting to the data is one problem. Getting the data into a shape the model can reason over is another, and the two are frequently confused during scoping.
Consider what a mid-market firm actually holds. Scanned PDFs where the text layer is unreliable. Contracts stored as email attachments. Spreadsheets carrying meaning in cell colours. Records in a system whose API returns a page at a time and rate limits after four hundred calls. Three versions of the same policy with no indication of which one is current.
None of this is exotic. It is what real environments look like. Making it usable means extraction that handles bad scans, chunking that respects document structure instead of cutting through the middle of a clause, metadata that lets the system know a document is superseded, and a refresh strategy so that answers reflect this week rather than the day the index was built. Each of these has failure modes that only appear at volume, which is why they never show up in a pilot.
Workflow logic is where the value sits
An answer on a screen is not an outcome. The outcome is a completed review, a filed document, an approved exception, a claim moved to the next stage.
That means the system has to know where its output goes and what happens next. Who reviews it before it counts. What the process does when the model is uncertain, and what threshold makes it uncertain. Which cases must be routed to a human regardless of confidence. What gets written back to the system of record, and in what format, so the audit trail stays coherent.
These are business decisions dressed as technical ones. They cannot be answered by an engineer alone, and they cannot be deferred, because the process is not automated until they are settled. A great many AI projects stall precisely here, at the point where the interesting modelling work is finished and the unglamorous work of fitting into an existing process begins.
Reliability is a different problem from accuracy
Accuracy is whether the answer is right. Reliability is whether the system produces an answer at all, within the time the process allows, on the day the provider is having trouble.
Production systems need timeouts and retries that do not silently duplicate work. They need graceful behaviour when an upstream system is down, so the queue does not disappear. They need cost controls, because a workflow that costs four cents in testing and runs eighty thousand times a month is a budget line somebody will question. They need monitoring that tells you the quality has drifted before a user does.
They also need to survive the model changing underneath them. Providers deprecate versions and adjust behaviour. A system built with the specific quirks of one model baked into its prompts will break on that schedule. A system built to be evaluated and swapped will not.
Evidence, because someone will ask
In financial services, legal, and professional services, the question arrives eventually. A regulator, an auditor, a client, or an internal risk committee wants to know why the system produced a particular output on a particular date.
Answering that requires more than logs of inputs and outputs. It requires knowing which documents were retrieved, which version of the prompt was in force, which model handled it, who reviewed it, and what they changed. Retrofitting this after go-live is painful and often incomplete. Designing it in from the start costs very little and is the difference between a system the firm can defend and one it quietly stops using.
What this changes about scoping
If the integration layer is the hard part, then the project plan should look like it. A few practical consequences.
Treat the pilot as a question about feasibility, not a head start on delivery. Very little of it survives.
Scope the permissions work explicitly, as its own stream, with the people who own the source systems in the room.
Pick the workflow by how well its edges are understood, not by how impressive it would be. A narrow process with clear rules ships. A broad one with tacit rules does not.
Ask any vendor or team what happens when the source system is unavailable, when the model returns nothing usable, and when someone asks for the audit trail. The quality of those answers tells you whether they have been to production before.
Where this leaves you
The gap between a working demo and a working system is not a matter of effort or budget. It is a different kind of engineering, performed inside constraints that only exist in your environment: your identity model, your data, your process, your regulator.
That is the work Alppoint does. If you have a pilot that proved the idea and has not moved since, we are happy to have a conversation about what stands between it and production.
