Top 7 HIPAA Risks in Third-Party AI Data Use

A signed BAA does not prove that an AI vendor protects patient data. Before I share protected health information (PHI), I check where it goes, who can access it, and whether the vendor’s settings match its contract as part of third-party vendor risk management.

My review covers seven risks:

• Too much access: Limit users, connectors, and support staff to the PHI their tasks need.
• Unclear sharing: Specify who receives PHI and which uses are allowed.
• Unapproved training: Check training, fine-tuning, evaluation, and human review separately.
• Retention and deletion gaps: Set deadlines and verify deletion across storage, backups, and vendor copies.
• Missing audit trails or PHI in logs: Record and review activity without copying patient details into logs.
• Cross-border processing: Check hosting, backup, and remote-access locations - not just the main data center.
• Subcontractor access: Verify downstream agreements, access limits, and deletion procedures.

I use one PHI data-flow map to check all seven, then test the controls before launch and after changes. <u>HIPAA sets no universal PHI retention period</u>, and business associates must report breaches of unsecured PHI without unreasonable delay - no later than 60 days after discovery.

Map PHI Across the AI Data Flow

Map every place PHI travels, starting with clinician prompts, attachments, audio, and electronic health record (EHR) connections. Document what enters the service, why it’s sent, who sends it, and how it moves. Include metadata and support tickets that could identify a patient. Then follow those inputs through the rest of the system.

Trace PHI through preprocessing, inference, APIs, outputs, logs, caches, vector databases, temporary files, storage, and backups. Include testing environments, human review, and any use for model training. Deleting a conversation does not delete every copy. For each location, record retention periods, deletion procedures, and who can access the data.

Identify every person and subcontractor with PHI access using common healthcare third-party risk assessment questions. Document their roles, processing locations, access controls, and the exact contract that governs their access. Require named accounts, multifactor authentication, a documented reason for access, and a time limit.

Give each point on the map an owner, a required safeguard, an evidence source, and a review date. Review the map after model or integration changes, security incidents, or subcontractor changes - not just during procurement. This ongoing oversight is a core component of HIPAA-compliant vendor risk management. This map provides the basis for the seven risks below.

The seven risks are connected, not ranked by severity. Unclear sharing terms can allow training; logs can hold extra copies; subcontractors can extend access across borders.

sbb-itb-535baee

1. Excessive or Unapproved PHI Access

Risk and PHI Exposure

Once PHI is mapped across the workflow, check who can access it. Access to an AI tool does not justify broad access to PHI. Broad connectors, shared service accounts, unrestricted prompts, vendor support tools, and weak multi-tenant isolation can expose more PHI than a task needs.

HIPAA Requirements

HIPAA’s minimum necessary standard generally requires reasonable, purpose-based limits on PHI uses, disclosures, and requests.^[12] Document any applicable exceptions; they are not permission for unrestricted access. The Security Rule also requires access controls and strong authentication for ePHI.^[7] The BAA must limit which users, connectors, and support sessions can access PHI.^[6][8]

Vendor Questions and Evidence

Request the permission matrix and architecture diagram. Check whether connectors can read, write, export, or delete records, and whether retrieval follows the source system’s patient and role permissions.

Ask for tenant-isolation test results covering vector indexes, caches, and support tools. Confirm that isolation is enforced server-side, not through user-supplied identifiers. Also verify support-session limits and authentication requirements for privileged access.

Technical Controls and Contract Terms

Use narrowly scoped, read-only connectors where the workflow allows. Limit prompts to approved fields and templates. Require secure API authorization, short-lived tokens, phishing-resistant MFA for privileged access, and approved, time-limited support sessions.

Specify permitted connectors, data categories, support-access rules, isolation duties, and audit rights in the BAA and security addendum. Then compare exposed fields with required fields, and verify session length and access-revocation time. Apply those same limits to any downstream use of the PHI.

2. Unclear Data Sharing and Secondary Uses

Risk and PHI Exposure

Once a vendor has access, check where it sends PHI and why. “Service improvement” language does not justify reusing PHI. Review whether the vendor shares data with named affiliates or uses it for analytics, advertising, research, benchmarking, or product improvement. Vague references to partners or business purposes can hide who receives PHI and what they do with it.

HIPAA Requirements

Each use must comply with the BAA and HIPAA, not just the vendor’s privacy policy.^[5][2] Marketing generally requires individual authorization, with limited exceptions. Depending on the facts, research may require authorization, a waiver or alteration of authorization, a limited data set with a data-use agreement, or properly de-identified data.^[9][16]

Treat benchmarking as a separate secondary use. Don’t assume it’s allowed. If the vendor de-identifies PHI, the BAA must authorize that step. Properly de-identified data is no longer PHI.^[13][14]

Vendor Questions and Evidence

Request a data-use matrix that lists each recipient’s legal entity, data categories - including prompts, outputs, embeddings, and telemetry - purpose, location, retention period, and legal basis. It should cover affiliate sharing, cross-customer analytics, commercialization, and research.

Check those answers against the contract and transfer logs. Assurances alone aren’t enough.

Technical Controls and Contract Terms

Block transfers to unapproved destinations and keep production PHI out of analytics workflows that lack approval.

Define service improvement narrowly, prohibit unrelated secondary uses, and spell out reporting duties for uses and disclosures without authorization, breaches, and security incidents.^[2] Require advance written notice and approval before any recipient or purpose changes. Include suspension and termination rights for unresolved violations.^[1]

If PHI can be reused beyond operations, check whether that same data also feeds model training.

3. PHI Used for Training Without Permission

Risk and PHI Exposure

After the secondary-use review, separate live inference from any path that reuses data for the model. Inference produces outputs. Training and fine-tuning can reuse PHI to change the model, while evaluation and human review can retain PHI outside live use. A no-training claim must address each activity separately: training, fine-tuning, evaluation, and human review.

Track prompts, outputs, evaluation sets, embeddings, and reviewer queues as PHI destinations in the workflow map. Set clear retention and deletion rules for any PHI used in these activities.

HIPAA Requirements

Fine-tuning, evaluation, and human review involving PHI must fall within the BAA and HIPAA.^[6] Training or review without approval turns a live service into an unapproved PHI reuse channel. Give reviewers access only to the smallest PHI set needed for evaluation or tuning.

Removing names alone does not de-identify PHI. Safe Harbor requires removing 18 identifier categories and having no actual knowledge that the remaining data could identify an individual. Expert Determination requires a qualified expert to document a very small re-identification risk.^[15][17]

Vendor Questions and Evidence

Request configuration exports that show whether training is disabled. Also request documentation showing that prompts, feedback, and evaluation samples stay out of shared training datasets.

For organization-specific fine-tuning, verify that model weights, checkpoints, and embeddings are isolated. Check reviewer access records and sample audit logs, then recheck these controls after any material product or workflow change.^[10]

Technical Controls and Contract Terms

Block unapproved training and evaluation reuse of PHI through both system settings and contract terms. Cover prompts, outputs, feedback, embeddings, and derived artifacts - not just uploaded files.

Require separate customer datasets, logged configuration changes, and subcontractor restrictions. Define how checkpoints containing PHI and evaluation copies will be returned or destroyed when the agreement ends.^[5]

4. Indefinite PHI Retention and Deletion Gaps

Risk and PHI Exposure

Deleting a conversation does not delete every copy of PHI. After training and review controls, retention is the next enterprise risk to address. Data can remain long after an interaction with a vendor ends. Prompts, outputs, embeddings, caches, logs, and backups may each have different retention schedules.

Track how long each copy stays and how it gets deleted, including copies in replicas and subcontractor systems. Longer retention gives people more time to access PHI without permission. Restoring backups can also bring deleted data back into active systems.

HIPAA Requirements

HIPAA does not set a universal PHI retention period. Its six-year documentation rules cover certain Privacy and Security Rule records - not all patient data or AI transactions.^[4][7] State medical-record laws, payer requirements, contracts, and legal holds may set different obligations.

The BAA should require the return or destruction of PHI when the agreement ends, where feasible.^[2][19] If destruction is infeasible, the agreement should maintain privacy and security protections. It should also limit further uses and disclosures to the purposes that make destruction infeasible.^[5][18]

Vendor Questions and Evidence

Request a retention schedule that lists each artifact’s deletion trigger, retention period, location, and deletion deadline. Use it to test whether deletion reaches every copy, not just the conversation users can see.

Ask for deletion procedures, recent test results, time-stamped deletion reports, and subcontractor confirmations. Organizations can streamline these requests using Censinet Connect™ Copilot to automate vendor responses. Require documented legal-hold exceptions that identify the affected data, the reason for the hold, the responsible owner, the review date, and the release process.

Technical Controls and Contract Terms

A retention schedule only works if systems and backups follow it. Set automatic expiration, alerts for failed deletion jobs, and backup lifecycle limits. Define return formats, export and deletion deadlines, and offboarding sign-off. Confirm that records have transferred before revoking access. Require deletion to include derived artifacts and subcontractor copies.^[2][19]

If destruction is infeasible, require a written explanation of what remains and where. Keep retained PHI encrypted, restrict access, and limit its use to the purpose that makes destruction infeasible.^[5][18] Isolate backups, reapply deletion controls after restoration, and require confirmation when retained copies expire or are destroyed.

5. Missing Audit Trails and PHI in Logs

Risk and PHI Exposure

Logs are both evidence and exposure. They can retain PHI even after ordinary deletion. Too little logging hides PHI access; too much creates another place for PHI to leak.

Login records alone may not show which records an AI system retrieved or where its outputs went. Prompts, responses, error traces, and support tickets can also copy PHI into logs and monitoring tools.

HIPAA Requirements

HIPAA’s Audit Controls requirement, 45 C.F.R. § 164.312(b), requires mechanisms to record and examine activity in systems that contain or use ePHI.^[7][20] Organizations must also regularly review audit logs, access reports, and security-incident tracking records. Collecting logs without reviewing them is not enough.^[22][23]

Vendor Questions and Evidence

Ask for a logging coverage matrix that covers the application, APIs, model host, data stores, support tools, logging services, and subcontractors. Check that logging covers user queries, administrator changes, model versions, tool calls, exports, and support-tool access.

Sanitized logs should show the actor, timestamp, resource, outcome, and correlation ID. Before deployment - and after material changes - test events using synthetic data to confirm end-to-end traceability and PHI redaction.

Technical Controls and Contract Terms

Log metadata rather than full clinical content, and redact PHI before logs reach external services. Protect logs with restricted access, administrator MFA, encryption, time synchronization, and tamper-resistant storage. Separate log-writing rights from log administration so people with access to the AI system cannot silently alter the evidence.

Contracts should define required events, usable log exports, subcontractor coverage, and notice of logging changes. Assign responsibility for monitoring, alert triage, evidence preservation, and incident reporting. Set clear review schedules and notification deadlines.

Test alerts for bulk exports, unusual PHI access, privileged-account activity, unexpected support-tool access, and disabled logging - not just failed logins. Track where logs are stored and processed, including monitoring locations. These locations affect the cross-border PHI processing review in the next section.

6. Cross-Border PHI Processing Without Risk Review

Risk and PHI Exposure

Once you’ve mapped PHI flow, check where it crosses borders. U.S. hosting does not guarantee U.S.-only access. Include every PHI touchpoint: hosting, backups, disaster recovery, support, monitoring, admin consoles, and subcontractor access - not just the primary data center.

For each touchpoint, record the country, provider, region, data type, access purpose, encryption status, and whether the location can change automatically. Overseas support staff may view PHI even when the database stays in the United States.^[26] Use this map to assess whether offshore processing changes the risk profile.

HIPAA Requirements

HIPAA does not bar offshore storage or processing. But HHS includes geography in risk analysis because foreign jurisdictions can weaken enforcement.^[5][27] Assess government-access powers, limits on legal requests, local threats, and whether contracts can be enforced. Also check state privacy laws, payer contracts, research agreements, and internal data-residency policies for stricter restrictions.

Vendor Questions and Evidence

Ask for a data-flow and data-residency diagram, a country-by-country list of production, backup, support, monitoring, and model-serving locations, and a list of subcontractors that may access PHI. Confirm that support tools, telemetry, crash reports, email, and logs cannot send PHI outside approved regions.

Request remote-access procedures, disaster-recovery test results, and a policy for government and court requests. Also ask for documentation explaining how the vendor will disclose material location changes in advance. Use this evidence to assess location-specific threats to ePHI.^[24] Check these locations when assessing who can view PHI remotely.

Technical Controls and Contract Terms

Use data minimization and de-identification before data leaves the organization whenever the use case permits. When PHI must be included, use encryption in transit and at rest, region-specific keys, country-based access restrictions, and tenant segmentation.

Keep prompts, outputs, embeddings, fine-tuning datasets, backups, and diagnostic logs within approved regions. Disable telemetry containing PHI where possible, and monitor administrative access for unusual cross-border transfers.

The BAA and service agreement should name approved countries or regions. They should prohibit relocation, replication, or remote access from unapproved jurisdictions without prior written approval. Apply these terms to prompts, outputs, embeddings, caches, backups, logs, support tickets, and derived datasets when they contain PHI.

Require advance notice of location or subcontractor changes, the customer’s right to object or terminate for material changes, and obligations that subcontractors must also follow.^[7][25] Include cooperation with location-risk assessments, notice of government and court requests where legally permitted, and requirements to return or destroy data at offboarding or after a region change.

Update location evidence at least annually and after material changes. Review privileged access quarterly. If a vendor cannot verify processing locations, pause expansion until it proves where PHI, logs, and backups are processed.

7. Subcontractor PHI Access Without Oversight

Risk and PHI Exposure

The PHI chain doesn’t stop at the primary AI vendor. Model providers, data-labeling services, and support contractors may also handle PHI. If downstream access goes undisclosed, your organization may be unable to trace unapproved use, investigate incidents, or confirm deletion. Identify every downstream tier that can touch PHI before deployment.

HIPAA Requirements

Before sharing PHI, a business associate must obtain a written agreement with each business associate subcontractor. Each agreement must carry forward the same applicable restrictions, conditions, and requirements for PHI.^[1][2][28] HHS identifies missing required subcontractor BAAs as a direct liability for the business associate.^[8] A signed BAA does not replace risk-based due diligence.^[6][11]

Vendor Questions and Evidence

Ask for a dated, version-controlled inventory of every subcontractor, including upstream model hosts and downstream support providers. For each provider, document its function, PHI access, processing locations, safeguards, retention periods, and applicable agreements. Separate routine access from access for troubleshooting, logs, and backups.

Request executed BAAs, security assessments, open remediation items, and incident-response test results. Review critical subcontractors at least annually and after material changes.

Technical Controls and Contract Terms

Require task-specific, time-limited access, multifactor authentication for subcontractor accounts, and logs that identify each subcontractor’s activity. Hold the primary vendor responsible for downstream compliance. Require advance notice of planned additions or expanded access, with the right to object or suspend transfers before unapproved access begins.

Require prompt incident escalation, preserved evidence, a coordinated investigation, and a single reporting path through the primary vendor. At offboarding, obtain deletion attestations from every downstream provider. These must identify covered systems, completion dates, and any exceptions. Verify that deletion reaches every subcontractor copy, not just the primary vendor’s systems.^[2][19]

Use these downstream controls in the vendor-chain checklist below.

HIPAA Risks and Controls at a Glance

Use this table to check each HIPAA duty against the vendor evidence and controls that support it. Complete each row using the same PHI data-flow map from above, and align the controls with your documented risk analysis. HIPAA does not require a single AI architecture.

HIPAA sets no universal retention period for vendor-held PHI, and a BAA alone does not permit model training. Check the purpose, contract terms, and handling controls.

Use the vendor evidence checklist below to test these controls before and after deployment.

Vendor Evidence Checklist Before and After Deployment

Use this checklist to confirm that the contract, configuration, and evidence match the risks above. Test each risk area against the live vendor setup, not just the paperwork.

• Scope: Match the BAA and service terms to the exact product, tenant, features, deployment model, region, and support model.^[1] Compare the vendor’s data-flow diagram with your actual configuration, including temporary storage and debugging flows.^[11] Align these checks with the seven risk areas above.
• Controls: Check that controls work in practice. Test access reviews, written limits on training and secondary use, deletion from indexes and caches, backup expiration, and audit-export speed.^[7]
• Vendor chain: Verify the full vendor chain, including evidence of actual processing regions and each PHI-handling subcontractor’s agreement.^[1] Review independent security evidence for the specific service - not just the parent company. Read exceptions and corrective actions. Certifications don’t replace risk analysis.^[11][10]
• Incident response: Before go-live, test the response process rather than relying on policy language. Record notification triggers, deadlines, incident contacts, evidence-preservation duties, and breach-assessment roles.^[19] Run a tabletop exercise to test vendor contact and access to investigation records. Reassess after security incidents, and repeat the same review after any material change.^[11][10]

Use Censinet RiskOps™ to centralize evidence review and remediation tracking.

Conclusion: Verify PHI Controls Across the Vendor Chain

Managing third-party AI risk reaches every entity that touches PHI, from cloud hosts and logging services to support subcontractors. A signed BAA does not establish compliance. PHI controls must work across the entire chain.^[1][2][5]

Compliance depends on both HIPAA duties and contract controls. Check that required safeguards and restrictions work throughout the vendor chain - not just on paper. Audit rights, deletion deadlines, and geographic limits are contract protections, not standalone HIPAA requirements.^[7][5][30]

For the seven risks above, limit access to the minimum needed, document permitted uses, and block unapproved training or secondary use. Pass required restrictions down to subcontractors that handle PHI.^[1][2][8] Verify retention, logging, and processing-location controls. Recheck them whenever access, data flow, logging, or subcontractors change.^[5][7][31]

FAQs

Which AI vendor risks should we address first?

Put safety impact and clinical criticality first. Clinical diagnostics, radiology AI, and sepsis scoring need more immediate, rigorous review than administrative tools.

Before approval, map the full supply chain: model owners, training data sources, cloud hosts, and subcontractors. If a vendor can’t disclose its model lineage, training data provenance, or subprocessor chain, move that review to the front of the line - or pause the deal until those gaps are resolved.

How can we test PHI safeguards before deployment?

Go beyond questionnaires. Ask for evidence that security controls are in place and work - not just documentation saying they do.

Use a non-production sandbox or staging environment to test EHR integrations, data flows, and access permissions. Run adversarial prompt injection tests, check input validation and output filtering, and conduct penetration tests and vulnerability scans focused on AI components.

Confirm that encryption, audit logging, and least-privilege access are implemented and work as documented in your threat model.

What should we do if an AI vendor violates its BAA?

Treat it as a compliance incident. Use the BAA’s breach-notification and incident-response provisions to require prompt notice and the facts. Immediately assess the impact on PHI access, uses and disclosures - including impermissible training - retention and deletion, and subcontractors.

Enforce the BAA’s remedies, including termination rights for noncompliance or safety risks. Require secure return or destruction, with proof. Use audit rights and independent validation to confirm corrective actions. Update governance and AI-specific contract clauses to prevent recurrence. ^[1][2][3]

Related Blog Posts

• AI Risks in HIPAA IT Compliance
• How AI Breaches Impact Healthcare Data
• AI in PHI Classification: Benefits and Risks
• The New Front Line in Healthcare Cybersecurity Is AI Governance