What it actually costs to build the AI layer yourself

MMurphi.ai6 min read

The model is the part you do not have to build. The orchestration, validation, governance and observability underneath it is the part that takes eighteen months, and it is the same for everyone.

Every EHR roadmap has an AI section now. Most of them have a hiring plan quietly attached.

The reasoning behind the hiring plan is sound, which is what makes it worth examining rather than dismissing. If AI is going to touch clinical documentation inside your product, somebody on your team needs to understand model orchestration, structured output validation, confidence scoring, evaluation, audit logging, and the security posture that makes all of it survive a procurement review at a thousand-bed health system.

That is a real team. The question is whether it is your team.

The part that looks like the work, and the part that is

Ask a product leader what building AI into their EHR involves and the answer usually starts with the model. Which model, how it is prompted, how it is fine-tuned, how good the output is.

That part is genuinely the easiest. Models are a commodity you rent, they improve without you doing anything, and swapping one for another is a week of work if you built the seam correctly.

The hard part sits underneath, and it divides into four layers that behave very differently from one another.

The experience layer is what clinicians actually see. Components embedded into screens they already work in, or a fully white-labeled workflow where your users never encounter another vendor's name. This layer is where your product judgement matters most, and it is the one place where building it yourself is usually correct, because it is the part your customers are paying you for.

The AI orchestration layer handles multi-model orchestration, structured output validation and confidence scoring. This is where most of the engineering difficulty lives, and almost none of it is differentiating. Two EHR vendors solving it independently will arrive at substantially the same design, having each spent a year getting there.

The integration layer is REST APIs, webhooks, JSON, HL7 and FHIR, with field-level EHR mapping. Every vendor in this market has already built some of this for other reasons, which creates a persuasive illusion that the AI version is a small extension of work you have done. It is not, because the failure modes are different. An interface that drops a message is a support ticket. An interface that writes an unreviewed AI-generated value into a clinical record is a different category of problem.

The operations and governance layer carries audit logs, usage telemetry, performance monitoring and human-in-the-loop controls. This is the layer teams discover last, usually during a customer's security review, and it is the one that cannot be retrofitted cheaply because it has to be true of every call the system has ever made.

Three of those four are infrastructure. One of them is your product.

The eighteen months is not the hiring

The timeline objection to building is usually framed as a recruiting problem. It is worse than that, because the recruiting is the fast part.

Suppose you hire well and quickly. You still have to build the orchestration seam, then discover the validation edge cases that only appear on real clinical text, then build confidence scoring, then discover that confidence scoring is useless unless the output carries its provenance, then build provenance, then rebuild your review interface around provenance because the first version assumed the reviewer would read the chart.

That sequence is not a failure of planning. It is what the work is. Everybody who has built this has walked that path, and the path does not get shorter because you read about it first.

Meanwhile your customers have an AI section on their own roadmap, and they are being told by somebody that it is available now.

When building it yourself is the right call

This deserves a straight answer rather than a rhetorical one, and there are three cases where building is correct.

When the AI layer is your product. If you intend to sell the AI capability separately, to other platforms, then it is not infrastructure for you. It is the thing. Buy nothing.

When your clinical domain is genuinely unusual. If the documents your product handles have no analogue elsewhere, a general-purpose orchestration layer will fight you. Home health and hospice do not qualify. OASIS, HOPE, the plan of care and the certification of terminal illness are standardised precisely so that they are the same everywhere.

When you have the team already. If you employ people who have shipped this kind of system before and they are currently underused, the calculus changes entirely. That is rarer than roadmaps assume, and the honest test is whether you had the team before the AI section appeared on the roadmap.

Outside those three, the AI layer is infrastructure, and infrastructure is a buy decision for the same reason you did not write your own database.

What buying it looks like in practice

Murphi provides the orchestration, validation, security and observability layer so a platform partner does not build it in-house. Three ways in, depending on how much of your own surface you want to keep.

Embedded components that sit inside the screens your users already work in. Your navigation, your design, your workflow.

A documented API layer your own product calls, if you would rather own the entire experience and treat Murphi as a service behind it.

The full white-labeled platform under your brand, where clinicians never see the Murphi name.

The figure Murphi puts on it is 30 to 40 AI features within six months, without a big-bang implementation. Treat that as a vendor number, because it is one, and ask in a technical conversation what the first three features are and how long the first one takes. The answer to the small question tells you more than the answer to the large one.

There is also a commercial point worth knowing in advance, because it shapes how the build-versus-buy arithmetic actually works. Murphi carries the token and infrastructure cost rather than passing it through per call, which means the usage economics do not change shape as your customers' volume grows. Teams that model the build path often forget inference cost entirely until the first full month of production traffic.

The boundary that is not negotiable

One thing holds regardless of which path you choose, and your customers' compliance teams will ask about it before they ask about anything else.

Clinician review and approval happens before any EHR write-back. The EHR remains the system of record.

This matters architecturally rather than contractually. If a reviewer cannot evaluate an AI output quickly, review degrades into approval, and a dashboard reading one hundred percent reviewed tells you nothing about whether anybody read anything. That is why findings have to carry the evidence they came from, and why the review interface is part of the safety design rather than part of the presentation.

If you are embedding AI into a system of record, that boundary should already be in the architecture rather than on the roadmap. It is the part that is most expensive to add afterwards, and the part a buyer will probe hardest.

The actual question

The useful framing is not whether your team could build the AI layer. Competent teams can build most things.

It is whether the eighteen months that would take is the best eighteen months available to your product, given that the output of those months is a layer that looks substantially the same in every competing product that also built it.

Your product team should be building the thing only you can build. For almost every EHR in this market, the orchestration layer underneath the AI is not that thing.

← All articles

Keep reading

All articles →
Ambient AI & DictationWhat Is an AI Scribe for Home Health Nurses? A Plain-English GuideSeptember 4, 2026Read →Ambient AI & DictationWhat Is HOPE? A Plain-English Guide to Hospice DocumentationAugust 28, 2026Read →Ambient AI & DictationOASIS-E Functional Scores: How Ambient AI Keeps M1800–M1860 AccurateAugust 21, 2026Read →

Experience AI Automation at Scale

Tell us your care setting and we’ll show you exactly what Murphi.ai delivers for your organization — a live demo tailored to your workflows and your team.