The simple ask hid a serious problem. The moment a patient speaks their name into a phone, that audio becomes Protected Health Information. The moment we transcribe it, the transcript is PHI. The moment we send the transcript to a large language model to figure out what the patient wants, we have a regulated entity sending PHI to a third-party AI provider. One leaked transcript, one logged prompt, one cached response on the wrong server, and the clinic loses its HIPAA standing. Not a fine. The actual ability to operate.

This case study is about the decisions we made so that never happened. It is not a feature list. It is the set of architectural calls that determine whether a system like this is safe to put in production or quietly dangerous.

Decision 1: PHI never reaches the LLM

The first question we asked ourselves before writing any code: where exactly does patient data flow, and at which step does it touch something we do not control?

The naive architecture for a voice booking agent is: patient speaks, audio goes to a transcription service, transcript goes to an LLM, LLM figures out intent, system books appointment. Every box in that flow is a vendor. Every vendor is a potential PHI exposure.

We considered three approaches.

The first was to use only HIPAA-eligible vendors with signed Business Associate Agreements at every step. This is the standard playbook. It works for transcription (Deepgram offers a BAA) and telephony (Twilio offers a BAA). It does not work cleanly for general-purpose LLMs in the way most teams assume. Even with an enterprise agreement, sending raw PHI to a model adds a regulated data flow that has to be documented, audited, and defended in a compliance review. We did not want to add that surface area.

The second approach was to skip the LLM entirely and build the voice agent as a deterministic dialogue tree. Safer, but it gives up the natural conversation that was the entire point. Patients calling a dental clinic do not say “book appointment, provider Patel, Tuesday, 2pm.” They say “I think I cracked a tooth and I really need to see someone before the weekend.” A dialogue tree handles the first sentence and falls apart on the second.

The third approach, the one we shipped, was a PHI de-identification layer that sits between transcription and the LLM. Every transcript is scanned for patient identifiers, names matched against the patient database, dates of birth, phone numbers, addresses, MRNs, before any text leaves our infrastructure. Each identifier is replaced with an opaque token. The LLM sees “[PATIENT-A] called about a cracked tooth and wants an appointment before Friday.” The LLM returns structured intent. Our server re-associates the tokens after the response comes back. The mapping is stored in an encrypted server-side session and never leaves the building.

The compliance posture this gives us is simple to explain to an auditor: we can produce an API log for every LLM call ever made by the system, and prove that no patient identifier was in any of them. The de-identification module is intentionally aggressive. A false positive (stripping a word that turned out not to be PHI) is harmless. A false negative is a violation. We tuned every threshold toward the conservative side.

This single decision is what makes the rest of the platform defensible.

Decision 3: The scribe cannot fabricate

This was the decision that took the longest to get right, and the one that matters most clinically.

An AI scribe that hallucinates a clinical finding is not an inconvenience. It is a patient safety incident waiting to become a malpractice case. A provider, tired at the end of a long day, approves a SOAP note that says “patient denies chest pain” when the patient never mentioned chest pain either way. That note goes into the EHR. Six months later something happens, and the legal record says the provider asked about chest pain. They did not.

We considered two ways to handle this.

One was to make the scribe extremely conservative, summarizing only verbatim quotes. This produces SOAP notes that read like court transcripts and miss the clinical reasoning that makes a note useful.

The other was to write the system prompt and the review workflow to make fabrication structurally hard, not just discouraged. This is what we shipped.

The prompt the model receives is explicit: extract a SOAP note from the conversation, never infer clinical findings that were not stated, mark missing sections as “Not discussed,” mark anything uncertain as “VERIFY.” The output is parsed and any section flagged “VERIFY” is highlighted in the provider’s review screen in a different color. Suggested ICD-10 codes are validated against the CMS code table locally before they are displayed, so the provider never sees a hallucinated code.

The interface is the other half of the defense. Every generated note carries a persistent banner: “AI-generated. Review before saving.” The note does not go to the EHR until the provider clicks approve. There is no auto-push, no background sync, no “approve all.” Every note is a conscious decision.

What this means for the clinic is that the scribe accelerates documentation without ever being the final word on what happened in the room. The provider stays the author. The AI is a fast first draft.

Decision 5: The audit log cannot be edited

HIPAA requires that you can produce, on demand, a record of who accessed which patient’s data, when, and what they did. Most systems implement this as a logging table that the application writes to. This is almost good enough. The gap is that if the application can write to the table, the application can also be tricked or compromised into modifying or deleting from it. An audit log you can edit is not an audit log.

We built the audit log as append-only at the database permission level. The application’s database role has INSERT permission on the audit table and nothing else. No UPDATE. No DELETE. No TRUNCATE. Even if an attacker compromised the application, the worst they could do is add fake entries, not remove real ones.

The Break Glass emergency access feature uses the same principle. A staff member who needs elevated access to a record they would not normally see (a covering provider in an emergency, an admin investigating a billing issue) must enter a written reason. The elevated access auto-revokes after a short window. The admin gets an immediate alert. Every minute of the elevated session is logged. The reason field is required, and the audit entry cannot be edited after the fact.

When the clinic eventually goes through a formal HIPAA audit, this is the kind of architecture that lets them answer “yes” to every control question without preparation. The controls are not policies. They are enforced by the database.

What the platform looks like in production

Patients call the clinic’s phone number and speak to a voice agent that handles booking, rescheduling, cancellation, and general inquiries. After hours, calls drop into a queue that staff see the next morning, with full transcripts and intent summaries. Patients can also book through a web portal with multi-factor authentication, and a chat interface for natural-language requests.

Providers see a daily timeline of appointments synced bi-directionally with athenahealth. They tap to update status, manage availability, and open the scribe interface for documentation. The scribe records the visit, transcribes it, generates a draft SOAP note in seconds, and pushes the approved note to athenahealth via FHIR.

Admins see clinic configuration, user management, audit log viewer, call analytics, and integration health. Everything is multi-tenant and isolated at the database layer with Row Level Security, so adding the next clinic location is a configuration change, not a deployment.

The core stack: FastAPI and PostgreSQL on the backend, Next.js on the frontend, Twilio for telephony, Deepgram for transcription, ElevenLabs for voice synthesis, Claude for reasoning, athenahealth FHIR R4 for EHR integration. All hosted on BAA-covered infrastructure in US regions.

How to talk to us about a build like this

If you are running a clinic, a digital health company, or a healthcare-adjacent platform, and you are trying to figure out whether to build something like this or buy it: we offer a paid one-week discovery sprint that produces an architecture document, a compliance risk map, and a build plan you own whether or not you continue with us. Most teams find this is the cheapest way to know what they actually need before committing to a build.