HIPAA-Safe AI in a Small Medical or Dental Practice: What's Actually Allowed

The blocker isn't the model, it's the business associate agreement. What ChatGPT and cloud AI can and can't touch inside a covered entity, and why running the model on the practice's own hardware changes the question.

Summary

A small practice that wants to use AI on patient data hits one wall before any technical question: the business associate agreement. The tools people reach for first — ChatGPT Free, Go, Plus, Pro, and Business (formerly Team) — will not sign a BAA for any of them, which means they cannot legally touch protected health information at all. The fix most practices never hear about is that the entire problem disappears when the model runs on hardware the practice owns and the PHI never leaves the building. That is not a compliance shortcut. You still owe HIPAA's Security Rule on whatever you build, and a local model is weaker than the frontier models on some clinical tasks. But it removes the single biggest blocker: a third party that holds your patients' data and refuses to be accountable for it.

The thing that decides it first: the BAA

HIPAA lets a covered entity hand PHI to another party only when that party is a business associate operating under a written business associate agreement (45 CFR 164.502 and 164.504). Any service that "creates, receives, maintains, or transmits" PHI on your behalf is a business associate by definition. So the first question about any AI tool is not "is it good" or "is it accurate." It is "will they sign a BAA, and is the product even eligible for one."

If the answer to either half is no, the tool is off the table for anything containing PHI — full stop. That rule applies to transcription services, to the dictation app on your phone, and to the chatbot a staff member pastes a progress note into. The model's capabilities are irrelevant to that first gate.

What the vendors actually offer

OpenAI's own eligibility matrix is the clearest example, because it contradicts what a lot of practices assume. From OpenAI's help documentation on HIPAA-eligible products and functionality and their BAA article:

• ChatGPT Free, Go, Plus, and Pro are not HIPAA-eligible. No BAA is available for them, under any circumstance.
• ChatGPT Business (called Team until August 2025) is also not BAA-eligible — this is the one that trips people up, because "Business" sounds like the compliance tier. It is not.
• ChatGPT Enterprise and Edu are BAA-eligible, but only on sales-managed accounts. You have to go through OpenAI's sales process and execute the agreement before PHI goes anywhere.
• The API requires an executed BAA before you process PHI through it. Even then, the default enterprise API retention settings need attention; the BAA is a precondition, not the whole story.

The pattern generalizes. Every cloud AI vendor has a different answer to the BAA question, and "we take privacy seriously" is not a BAA. Get it in writing before a single byte of PHI leaves the practice.

"HIPAA compliant" is not a certification

No government body certifies a product as "HIPAA compliant." The Department of Health and Human Services' Office for Civil Rights (OCR) does not run a certification program for software. The phrase is marketing, and seeing it on a landing page tells you almost nothing.

What HIPAA actually is: three rules — Privacy, Security, and Breach Notification — that impose obligations on you, the covered entity, and flow through to your business associates. The Security Rule (45 CFR Part 164, subpart C) requires administrative, physical, and technical safeguards for electronic PHI: a documented risk analysis, access controls, audit controls, integrity controls, and encryption where appropriate. A tool can't be "compliant" on its own; the question is whether the arrangement lets you meet your obligations while using it.

The de-identification escape hatch — and why it's narrow

There is one legal way to use a non-BAA tool without a BAA: make the data no longer PHI. De-identification under 45 CFR 164.514 has a safe harbor — remove 18 specific identifiers (names, dates more specific than year, geographic subdivisions smaller than a state, account numbers, and so on) and the data is no longer protected.

In practice this almost never works for the way practices want to use AI. A progress note with a patient name, a date of service, and a town is still PHI until all of that is stripped. A staff member pasting a live note into ChatGPT fails the test immediately, because the note is loaded with identifiers. De-identification is a legitimate workflow, but it is a process with a scrub step and a documented policy — not something that happens by accident.

The tracking angle people miss

OCR has also made clear that "using a cloud tool" includes things nobody thinks of as AI. Its bulletin on the use of online tracking technologies by HIPAA covered entities and business associates (issued December 2022, revised 2024) treats a tracker embedded in a patient portal or authenticated page that sends PHI to a vendor as a disclosure requiring a BAA. (In June 2024 a federal court vacated the bulletin's position on unauthenticated public pages; its treatment of authenticated pages and patient portals was not the vacated part.) The lesson carries over directly: the line between "the AI sees the data" and "the AI's vendor sees the data" is thinner than it looks, and it is exactly where practices get burned.

What running it locally actually changes

When the model runs on hardware the practice owns and PHI never transits a public network, there is no third party receiving PHI. No third party means no business associate relationship, which means no BAA is required for that component. That is a real, categorical difference, and it is the whole reason on-premises AI is interesting to small medical and dental practices.

But it is not a get-out-of-HIPAA-free card. The practice is still a covered entity, and the Security Rule still applies to the local system. A compliant local setup still needs:

• Physical and access controls. The machine lives in a locked space; only authorized staff have accounts, with per-user credentials and no shared logins.
• Audit logging. Who ran what query, when. If you ever need to answer an OCR inquiry, this is the first thing they ask for.
• Encryption at rest. Full-disk encryption (FileVault on macOS, LUKS on Linux) so a stolen or decommissioned drive isn't a breach.
• A documented risk analysis. The Security Rule's foundational requirement, whether your AI is on-prem or in the cloud.
• BAAs where they still apply. Your managed IT provider who can see the box, your EHR, your billing service — every one of those still needs its own BAA. Local AI fixes the AI component; it does not fix the rest of the compliance program.

We go through this exact checklist during an on-site audit, and the local setup work is designed so the PHI boundary is obvious and documented, not implied. The deployment packages step through the same checklist by tier.

The honest limits

Three things a practice should hear before it spends anything:

A local model is weaker than the frontier on some clinical tasks. Open-weight models are good at summarization, drafting, coding, and structured extraction. They are not as strong as the top closed models on open-ended clinical reasoning, and they will not magically read an MRI. Match the task to the model (the model-fit tool helps), and keep a human in the loop on anything clinical.

Compliance is a program, not an install. Buying a box does not make you HIPAA compliant. It removes the cloud-AI blocker. The policies, training, risk analysis, and BAAs are still yours to do.

This fixes the AI piece, not the whole stack. Telehealth, email, patient portals, and the EHR all still need their own vendor diligence. On-prem AI is one component of a compliance posture, and it is the one that happens to be the hardest to do any other way.

Key takeaways

1. The BAA is the gate, not the model. If a vendor won't sign one and isn't eligible for one, no amount of capability makes it usable on PHI.
2. ChatGPT Free, Go, Plus, Pro, and Business (formerly Team) are all no-BAA. Only sales-managed Enterprise and Edu, and the API under an executed BAA, can touch PHI — per OpenAI's own documentation.
3. "HIPAA compliant" is not a certification. Nothing certifies it. Ask instead whether the arrangement lets you meet the Privacy, Security, and Breach Notification rules.
4. De-identification is real but narrow. Stripping 18 identifiers is a documented process, not something that happens when you paste a note into a chat window.
5. On-prem removes the third party, not your obligations. No BAA for the model, but the Security Rule — access controls, audit logs, encryption, risk analysis — still applies to the box.
6. Treat it as one component. Local AI fixes the AI component of your compliance posture; every other vendor that touches PHI still needs its own BAA.

Sources

• U.S. Dept. of Health and Human Services — HIPAA Privacy, Security, and Breach Notification Rules, 45 CFR Parts 160 and 164: https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-C/part-164
• HHS Office for Civil Rights — Business Associate Contracts (BAA) guidance and sample provisions: https://www.hhs.gov/hipaa/for-professionals/covered-entities/sample-business-associate-agreement-provisions/index.html
• OpenAI Help Center — "HIPAA-eligible products and functionality": https://help.openai.com/en/articles/20001069-hipaa-eligible-products-and-functionality
• OpenAI Help Center — "How can I get a Business Associate Agreement (BAA) with OpenAI?": https://help.openai.com/en/articles/8660679-how-can-i-get-a-business-associate-agreement-baa-with-openai
• HHS OCR — "Use of online tracking technologies by HIPAA covered entities and business associates" (issued Dec 2022, revised 2024): https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/hipaa-online-tracking/index.html
• HHS — "Methods for De-identification of PHI" (the 45 CFR §164.514 safe harbor): https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html

This post describes the rules as we read them; it is not legal advice. HIPAA-compliance decisions should be made with a healthcare attorney or a qualified HIPAA consultant.