Course: Course 3 — LLM Fine-Tuning Masterclass
Module: FT21 — HIPAA and BAA Elimination
Duration: ~40 minutes (spoken at ~140 wpm)
Format: Verbatim transcript with [SLIDE N] cues. Read aloud or use as speaker notes.
[SLIDE 1 — Title]
Welcome to module FT twenty-one — HIPAA and BAA Elimination. This is Pillar seven, Sensitive Domains, and it is the module where the course thesis — the model steers, the harness bounds — becomes load-bearing in the most literal sense you have seen so far. The boundary is no longer a policy gate or a refusal direction. It is physical. The data does not leave the building.
The prerequisite is FT twenty, Serving Stacks. If you have not done FT twenty, you cannot make the move this module teaches, because the move requires you to be able to serve a model on your own metal. FT twenty gave you that capability — vLLM, TGI, Ollama, the quant formats, the VRAM math. This module is the regulatory payoff.
Let me give you the one sentence up front, then we will spend the next forty minutes defending it.
[SLIDE 2 — The core logic]
Here is the sentence. Under HIPAA, any vendor that processes Protected Health Information on a covered entity's behalf is a business associate, and it must sign a Business Associate Agreement — a BAA. Run the model on your own on-prem GPUs, and no third party processes the PHI. No business associate. No BAA. The regulatory hook disappears with the third party.
Read that twice. The BAA is not a magic compliance certificate that makes a vendor safe. It is a contractual instrument that flows HIPAA obligations down a chain of vendors. It exists because a third party is touching the data. Eliminate the third party and the instrument has nothing to attach to. This is structurally cleaner than any patched-up API arrangement, and it is the reason Pillar seven exists in this course at all.
[SLIDE 3 — Why a BAA is necessary but not sufficient]
Now the nuance. A signed BAA is necessary but it is not sufficient. Necessary, because it flows the obligations to the vendor and onward to its subcontractors — without it the vendor cannot lawfully process PHI. Not sufficient, because it does not guarantee every product surface honors zero-data-retention. The BAA is the floor, not the ceiling.
This sets up the most important teaching point of the module. You can get a BAA. You cannot get a BAA that covers everything. Coverage is endpoint-specific, not product-wide. The chat-completions endpoint might be covered by zero-retention. The image endpoint might not be. The stateful assistant product might not be. The batch pipeline, the embeddings call, the fine-tuning endpoint — each must be checked individually. A real clinical workflow almost always touches at least one uncovered surface.
This is the real driver for local. It is not that local is cheaper — it often isn't. It is not that local is easier — it is operationally harder. It is that local eliminates the coverage question entirely. There is no endpoint matrix to maintain, because there are no endpoints you do not own.
[SLIDE 4 — The four ways to run an LLM on PHI]
Now the taxonomy. There are four ways to run an LLM on PHI, ranked weakest to strongest by data-exposure posture.
Option one. Cloud endpoints under BAA, multi-tenant. You call an API. PHI leaves your network and lands on the vendor's shared infrastructure. The BAA and zero-retention are what make this permissible at all, not what make it safe. Weakest posture.
Option two. Dedicated capacity, single-tenant. You rent dedicated instances from the vendor. PHI still leaves your network, but it lands on infrastructure you have more control over. Fewer noisy neighbors, clearer tenancy. The BAA is still required. Stronger than one, weaker than local.
Option three. Self-hosted open weights, in your own cloud VPC. You run an open-weights model — Llama, Qwen, Mistral — on instances you control, served with vLLM or TGI. With careful configuration, the cloud provider can be argued to be a conduit, not a business associate. This is the crossover point. You own the software stack.
Option four. On-prem GPUs, your metal. You run the model on GPUs you physically own, in a building you control. PHI never leaves the premises. No cloud vendor in the data path. No API provider. For the LLM itself, there is no business associate at all. Strongest posture. This is the only option that eliminates the BAA requirement rather than managing it.
The four options are not a menu of equally valid choices. They are a gradient of exposure. Every step toward the right moves the boundary from contractual — a BAA you hope the vendor honors — to physical — a cable that is not plugged in. Pillar seven's job is to push you as far right as your constraints allow.
[SLIDE 5 — The API coverage gaps]
Let me make the coverage-gap point concrete, because this is where teams get burned. Per analyses like Protecto's breakdown of the OpenAI-BAA surface, and consistent with the vendor's own community discussion, zero-retention is honored on some endpoints and not others.
Image and multimodal inputs are frequently excluded. The text-in, text-out path may be covered. The image-in path may not be. A clinical workflow that sends a wound photo or a radiology snippet through the vision endpoint can be routing PHI through an uncovered surface — even with a signed BAA.
Stateful products like Assistants and Threads retain data by design. That is how they maintain state. Zero-retention is structurally in tension with a stateful product. Coverage claims must be read carefully and they change.
Batch, embeddings, and fine-tuning endpoints each have their own eligibility. A covered chat endpoint does not transitively cover them. And subprocessors can sit outside the BAA chain. A single un-BAA'd link is a breach surface.
The exposure here is silent. The request succeeds. The response comes back. And PHI has flowed through a path the BAA does not cover. You discover the gap during an audit, or during a breach. That is why this is the real driver for local.
[SLIDE 6 — No model is "HIPAA certified"]
Now the misconception I want to kill before you ever sit in a procurement meeting. There is no such thing as a HIPAA-certified LLM. HHS does not certify models, vendors, or software. The phrase "HIPAA compliant" is not a property of an artifact. It is a property of a deployment.
What is actually required for an LLM handling PHI? First, an unbroken BAA chain if any third party processes PHI — or the elimination of that chain via local serving. One or the other. There is no third option. Second, technical safeguards — encryption at rest, encryption in transit, access controls. Third, audit trails — every prompt, every response, every access, logged and retained. Fourth, administrative and physical safeguards — training, incident response, risk analysis, facility controls. Fifth, a documented, periodically refreshed risk analysis.
Notice what is not on that list. The model. Its weights. Its training. Its capabilities. None of them. The model is a component. Compliance is a property of the system around it. When a vendor says "our model is HIPAA compliant," they are either being loose with language or misrepresenting the regulatory structure. Your job is to look past the marketing at the actual deployment.
[SLIDE 7 — De-identification as defense-in-depth]
Now the subtlety that catches teams who got the local part right. A natural conclusion from the core logic is: we are local, so we can train on raw PHI. You can. You almost always shouldn't.
When you fine-tune a model on data, the model can memorize that data. This is not the FT-zero-zero steering thesis in tension. Steering is a form of low-rank memorization of behavioral patterns. When the training examples themselves contain PHI, the model can memorize the identifiers along with the patterns. The result is a model from which PHI can be extracted by prompting. Carlini and colleagues showed this in the foundational paper on training-data extraction from large language models. A model that has seen thousands of clinical notes containing patient names may, under the right adversarial prompt, emit one of those names verbatim.
The exposure scales with repetition and rarity. A name that appears once is unlikely to be extracted. A name that appears in fifty notes, or a rare identifier the model latches onto, is more extractable.
The correct posture is defense-in-depth. De-identify the fine-tuning data before training, even under local. Run the corpus through Presidio, or Tonic Textual, or a custom spaCy pipeline. Remove direct identifiers, quasi-identifiers, free-text PHI. The output is a PHI-scarce corpus. Train on that. The behavior transfers; the identifiers do not. Then evaluate for memorization before deploy — canary insertion, membership inference, extraction attacks. For high-sensitivity subsets, apply DP-SGD. Then serve under audit.
You de-identify because the model might memorize. You audit because de-identification might miss something. You run local because the API might leak. No single control is trusted alone. That is defense-in-depth. HIPAA's own structure encodes it — the Security Rule lists safeguard categories, not alternatives.
[SLIDE 8 — The local architecture]
Here is what the target architecture looks like. Inside the trust boundary — your building, your network — you have the data sources, the de-identification pipeline, the de-identified fine-tuning corpus, the training and serving stack, the clinician UI, the audit log store, and the encryption surfaces. PHI enters, gets de-identified for training, the model is trained and served inside the boundary, responses go to clinicians inside the boundary.
What crosses the boundary? Two things, neither carrying PHI. The open-weights model is downloaded once, with no PHI in it. And optionally, the audit log ships out to a SIEM — under BAA, if that SIEM is external. Everything else stays inside. No PHI egress. No business associate for the LLM.
This is the boundary enforced by topology rather than by contract. The course thesis — the harness bounds — becomes physical here. A policy gate can be misconfigured. A BAA can be breached. A cable you never plugged in cannot leak.
[SLIDE 9 — Anti-patterns]
Let me name the four ways teams get this wrong, in order of how often I see them.
First. Assuming the API BAA covers everything. "We signed the BAA, so we are fine to send PHI through any endpoint." No. The BAA covers the vendor relationship. It does not cover every surface. The image endpoint, the stateful assistant, the batch pipeline — each must be checked against the coverage matrix. Teams that skip this silently route PHI through uncovered paths and discover the gap during an audit.
Second. Fine-tuning on raw PHI without de-identification. "We are local, so we can train on the raw notes." The model may memorize identifiers, and they may be extractable. The local boundary protects against egress. It does not protect against memorization-then-extraction by an insider. De-identify first, always.
Third. No audit logging. "The model is local, we do not need to log every call." You do. The Security Rule requires audit controls. Every call to a clinical LLM is processing ePHI. Every call must be logged. A local deployment with no logging is less auditable than a logged API deployment — and auditability is a requirement, not a nice-to-have.
Fourth. Treating "local" as sufficient without the technical safeguards. "We are on-prem, so we are compliant." Local eliminates the BAA problem. It does not eliminate the Security Rule. You still need encryption at rest, encryption in transit, access controls, a risk analysis, workforce training, incident response. A local deployment that skips these is non-compliant and exposed. The worst of both worlds.
[SLIDE 10 — The bridge to FT22 and FT23]
Where this goes next. FT twenty-two takes the local posture to its extreme — government, military, air-gapped deployment. No network at all. The boundary becomes not just physical but absolute. If FT twenty-one is "the data stays in the building," FT twenty-two is "the building has no door."
FT twenty-three is the synthesis — the steered model inside the bounded harness. Everything in this course comes together there. The model you have steered toward your intent, deployed inside a harness that bounds what it may do, on infrastructure that keeps the data where you put it.
The thesis, one more time. The model steers. The harness bounds. In Pillar seven, the harness bounds physically. That is the strongest form of boundary there is.
[SLIDE 11 — What you can now do]
By the end of this module you can state the core HIPAA logic for LLMs and explain why local eliminates the business-associate relationship. You can rank the four deployment options by exposure posture and name local as the target. You can describe the API-BAA coverage gaps and explain why they are the real driver for local. You can correct the "HIPAA certified" misconception and enumerate what is actually required. You can apply de-identification as defense-in-depth, even under local. And you can design the architecture for a clinical-decision-support LLM and defend the choice.
The lab is a design lab, not a build lab. You produce an architecture diagram and a written defense for a clinical-decision-support system handling PHI. No GPU required. The judgment is the deliverable.
Next. FT twenty-two — Government, Military, and Air-Gapped Deployment.
# Teaching Script — Module FT21: HIPAA and BAA Elimination **Course**: Course 3 — LLM Fine-Tuning Masterclass **Module**: FT21 — HIPAA and BAA Elimination **Duration**: ~40 minutes (spoken at ~140 wpm) **Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes. --- [SLIDE 1 — Title] Welcome to module FT twenty-one — HIPAA and BAA Elimination. This is Pillar seven, Sensitive Domains, and it is the module where the course thesis — the model steers, the harness bounds — becomes load-bearing in the most literal sense you have seen so far. The boundary is no longer a policy gate or a refusal direction. It is physical. The data does not leave the building. The prerequisite is FT twenty, Serving Stacks. If you have not done FT twenty, you cannot make the move this module teaches, because the move requires you to be able to serve a model on your own metal. FT twenty gave you that capability — vLLM, TGI, Ollama, the quant formats, the VRAM math. This module is the regulatory payoff. Let me give you the one sentence up front, then we will spend the next forty minutes defending it. [SLIDE 2 — The core logic] Here is the sentence. Under HIPAA, any vendor that processes Protected Health Information on a covered entity's behalf is a business associate, and it must sign a Business Associate Agreement — a BAA. Run the model on your own on-prem GPUs, and no third party processes the PHI. No business associate. No BAA. The regulatory hook disappears with the third party. Read that twice. The BAA is not a magic compliance certificate that makes a vendor safe. It is a contractual instrument that flows HIPAA obligations down a chain of vendors. It exists because a third party is touching the data. Eliminate the third party and the instrument has nothing to attach to. This is structurally cleaner than any patched-up API arrangement, and it is the reason Pillar seven exists in this course at all. [SLIDE 3 — Why a BAA is necessary but not sufficient] Now the nuance. A signed BAA is necessary but it is not sufficient. Necessary, because it flows the obligations to the vendor and onward to its subcontractors — without it the vendor cannot lawfully process PHI. Not sufficient, because it does not guarantee every product surface honors zero-data-retention. The BAA is the floor, not the ceiling. This sets up the most important teaching point of the module. You can get a BAA. You cannot get a BAA that covers everything. Coverage is endpoint-specific, not product-wide. The chat-completions endpoint might be covered by zero-retention. The image endpoint might not be. The stateful assistant product might not be. The batch pipeline, the embeddings call, the fine-tuning endpoint — each must be checked individually. A real clinical workflow almost always touches at least one uncovered surface. This is the real driver for local. It is not that local is cheaper — it often isn't. It is not that local is easier — it is operationally harder. It is that local eliminates the coverage question entirely. There is no endpoint matrix to maintain, because there are no endpoints you do not own. [SLIDE 4 — The four ways to run an LLM on PHI] Now the taxonomy. There are four ways to run an LLM on PHI, ranked weakest to strongest by data-exposure posture. Option one. Cloud endpoints under BAA, multi-tenant. You call an API. PHI leaves your network and lands on the vendor's shared infrastructure. The BAA and zero-retention are what make this permissible at all, not what make it safe. Weakest posture. Option two. Dedicated capacity, single-tenant. You rent dedicated instances from the vendor. PHI still leaves your network, but it lands on infrastructure you have more control over. Fewer noisy neighbors, clearer tenancy. The BAA is still required. Stronger than one, weaker than local. Option three. Self-hosted open weights, in your own cloud VPC. You run an open-weights model — Llama, Qwen, Mistral — on instances you control, served with vLLM or TGI. With careful configuration, the cloud provider can be argued to be a conduit, not a business associate. This is the crossover point. You own the software stack. Option four. On-prem GPUs, your metal. You run the model on GPUs you physically own, in a building you control. PHI never leaves the premises. No cloud vendor in the data path. No API provider. For the LLM itself, there is no business associate at all. Strongest posture. This is the only option that eliminates the BAA requirement rather than managing it. The four options are not a menu of equally valid choices. They are a gradient of exposure. Every step toward the right moves the boundary from contractual — a BAA you hope the vendor honors — to physical — a cable that is not plugged in. Pillar seven's job is to push you as far right as your constraints allow. [SLIDE 5 — The API coverage gaps] Let me make the coverage-gap point concrete, because this is where teams get burned. Per analyses like Protecto's breakdown of the OpenAI-BAA surface, and consistent with the vendor's own community discussion, zero-retention is honored on some endpoints and not others. Image and multimodal inputs are frequently excluded. The text-in, text-out path may be covered. The image-in path may not be. A clinical workflow that sends a wound photo or a radiology snippet through the vision endpoint can be routing PHI through an uncovered surface — even with a signed BAA. Stateful products like Assistants and Threads retain data by design. That is how they maintain state. Zero-retention is structurally in tension with a stateful product. Coverage claims must be read carefully and they change. Batch, embeddings, and fine-tuning endpoints each have their own eligibility. A covered chat endpoint does not transitively cover them. And subprocessors can sit outside the BAA chain. A single un-BAA'd link is a breach surface. The exposure here is silent. The request succeeds. The response comes back. And PHI has flowed through a path the BAA does not cover. You discover the gap during an audit, or during a breach. That is why this is the real driver for local. [SLIDE 6 — No model is "HIPAA certified"] Now the misconception I want to kill before you ever sit in a procurement meeting. There is no such thing as a HIPAA-certified LLM. HHS does not certify models, vendors, or software. The phrase "HIPAA compliant" is not a property of an artifact. It is a property of a deployment. What is actually required for an LLM handling PHI? First, an unbroken BAA chain if any third party processes PHI — or the elimination of that chain via local serving. One or the other. There is no third option. Second, technical safeguards — encryption at rest, encryption in transit, access controls. Third, audit trails — every prompt, every response, every access, logged and retained. Fourth, administrative and physical safeguards — training, incident response, risk analysis, facility controls. Fifth, a documented, periodically refreshed risk analysis. Notice what is not on that list. The model. Its weights. Its training. Its capabilities. None of them. The model is a component. Compliance is a property of the system around it. When a vendor says "our model is HIPAA compliant," they are either being loose with language or misrepresenting the regulatory structure. Your job is to look past the marketing at the actual deployment. [SLIDE 7 — De-identification as defense-in-depth] Now the subtlety that catches teams who got the local part right. A natural conclusion from the core logic is: we are local, so we can train on raw PHI. You can. You almost always shouldn't. When you fine-tune a model on data, the model can memorize that data. This is not the FT-zero-zero steering thesis in tension. Steering is a form of low-rank memorization of behavioral patterns. When the training examples themselves contain PHI, the model can memorize the identifiers along with the patterns. The result is a model from which PHI can be extracted by prompting. Carlini and colleagues showed this in the foundational paper on training-data extraction from large language models. A model that has seen thousands of clinical notes containing patient names may, under the right adversarial prompt, emit one of those names verbatim. The exposure scales with repetition and rarity. A name that appears once is unlikely to be extracted. A name that appears in fifty notes, or a rare identifier the model latches onto, is more extractable. The correct posture is defense-in-depth. De-identify the fine-tuning data before training, even under local. Run the corpus through Presidio, or Tonic Textual, or a custom spaCy pipeline. Remove direct identifiers, quasi-identifiers, free-text PHI. The output is a PHI-scarce corpus. Train on that. The behavior transfers; the identifiers do not. Then evaluate for memorization before deploy — canary insertion, membership inference, extraction attacks. For high-sensitivity subsets, apply DP-SGD. Then serve under audit. You de-identify because the model might memorize. You audit because de-identification might miss something. You run local because the API might leak. No single control is trusted alone. That is defense-in-depth. HIPAA's own structure encodes it — the Security Rule lists safeguard categories, not alternatives. [SLIDE 8 — The local architecture] Here is what the target architecture looks like. Inside the trust boundary — your building, your network — you have the data sources, the de-identification pipeline, the de-identified fine-tuning corpus, the training and serving stack, the clinician UI, the audit log store, and the encryption surfaces. PHI enters, gets de-identified for training, the model is trained and served inside the boundary, responses go to clinicians inside the boundary. What crosses the boundary? Two things, neither carrying PHI. The open-weights model is downloaded once, with no PHI in it. And optionally, the audit log ships out to a SIEM — under BAA, if that SIEM is external. Everything else stays inside. No PHI egress. No business associate for the LLM. This is the boundary enforced by topology rather than by contract. The course thesis — the harness bounds — becomes physical here. A policy gate can be misconfigured. A BAA can be breached. A cable you never plugged in cannot leak. [SLIDE 9 — Anti-patterns] Let me name the four ways teams get this wrong, in order of how often I see them. First. Assuming the API BAA covers everything. "We signed the BAA, so we are fine to send PHI through any endpoint." No. The BAA covers the vendor relationship. It does not cover every surface. The image endpoint, the stateful assistant, the batch pipeline — each must be checked against the coverage matrix. Teams that skip this silently route PHI through uncovered paths and discover the gap during an audit. Second. Fine-tuning on raw PHI without de-identification. "We are local, so we can train on the raw notes." The model may memorize identifiers, and they may be extractable. The local boundary protects against egress. It does not protect against memorization-then-extraction by an insider. De-identify first, always. Third. No audit logging. "The model is local, we do not need to log every call." You do. The Security Rule requires audit controls. Every call to a clinical LLM is processing ePHI. Every call must be logged. A local deployment with no logging is less auditable than a logged API deployment — and auditability is a requirement, not a nice-to-have. Fourth. Treating "local" as sufficient without the technical safeguards. "We are on-prem, so we are compliant." Local eliminates the BAA problem. It does not eliminate the Security Rule. You still need encryption at rest, encryption in transit, access controls, a risk analysis, workforce training, incident response. A local deployment that skips these is non-compliant and exposed. The worst of both worlds. [SLIDE 10 — The bridge to FT22 and FT23] Where this goes next. FT twenty-two takes the local posture to its extreme — government, military, air-gapped deployment. No network at all. The boundary becomes not just physical but absolute. If FT twenty-one is "the data stays in the building," FT twenty-two is "the building has no door." FT twenty-three is the synthesis — the steered model inside the bounded harness. Everything in this course comes together there. The model you have steered toward your intent, deployed inside a harness that bounds what it may do, on infrastructure that keeps the data where you put it. The thesis, one more time. The model steers. The harness bounds. In Pillar seven, the harness bounds physically. That is the strongest form of boundary there is. [SLIDE 11 — What you can now do] By the end of this module you can state the core HIPAA logic for LLMs and explain why local eliminates the business-associate relationship. You can rank the four deployment options by exposure posture and name local as the target. You can describe the API-BAA coverage gaps and explain why they are the real driver for local. You can correct the "HIPAA certified" misconception and enumerate what is actually required. You can apply de-identification as defense-in-depth, even under local. And you can design the architecture for a clinical-decision-support LLM and defend the choice. The lab is a design lab, not a build lab. You produce an architecture diagram and a written defense for a clinical-decision-support system handling PHI. No GPU required. The judgment is the deliverable. Next. FT twenty-two — Government, Military, and Air-Gapped Deployment.