HIPAA compliance features

  • Article

Important

The ability for admins to add Enhanced Security and Compliance features is a feature in Public Preview. The compliance security profile and support for compliance standards are generally available (GA).

Databricks strongly recommends that customers who want to use HIPAA compliance features enable the compliance security profile, which adds monitoring agents, provides a hardened compute image, and other features. For technical details, see Compliance security profile.

It is your responsibility to confirm that each workspace has the compliance security profile enabled.

This feature requires your workspace to be on the Premium pricing tier.

Ensure that sensitive information is never entered in customer-defined input fields, such as workspace names, cluster names, and job names.

Which compute resources get enhanced security

The compliance security profile enhancements for HIPAA apply to compute resources in the classic compute plane and the serverless compute plane in all regions. For more information on the classic and serverless compute planes, see Azure Databricks architecture overview.

HIPAA overview

The Health Insurance Portability and Accountability Act of 1996 (HIPAA), the Health Information Technology for Economic and Clinical Health (HITECH), and the regulations issued under HIPAA are a set of US healthcare laws. Among other provisions, these laws establish requirements for the use, disclosure, and safeguarding of protected health information (PHI).

HIPAA applies to covered entities and business associates that create, receive, maintain, transmit, or access PHI. When a covered entity or business associate engages the services of a cloud service provider (CSP), such as Azure Databricks, the CSP becomes a business associate under HIPAA.

Does Azure Databricks permit the processing of PHI data on Azure Databricks?

Yes. Databricks strongly recommends enabling the compliance security profile and adding HIPAA during that configuration.

Enable HIPAA on a workspace

To process data regulated by the HIPAA compliance standard, Databricks recommends that each workspace have the compliance security profile enabled and add the HIPAA compliance standard.

You can enable the compliance security profile and add a compliance standard to a new workspace or an existing workspace using the Azure portal, or alternatively use an ARM template. For instructions and templates, see Configure enhanced security and compliance settings.

Important

  • You are wholly responsible for ensuring your own compliance with all applicable laws and regulations. Information provided in Azure Databricks online documentation does not constitute legal advice, and you should consult your legal advisor for any questions regarding regulatory compliance.
  • Azure Databricks does not support the use of preview features for the processing of PHI on the HIPAA on Azure platform, with the exception of the features listed in Preview features that are supported for processing of PHI data.

Preview features that are supported for processing of PHI data

The following preview features are supported for processing of PHI:

Shared responsibility of HIPAA compliance

Complying with HIPAA has three major areas, with different responsibilities. While each party has numerous responsibilities, below we enumerate key responsibilities of ours, along with your responsibilities.

This article use the Azure Databricks terminology control plane and a compute plane, which are two main parts of how Azure Databricks works:

Key responsibilities of Microsoft include:

  • Perform its obligations as a business associate under your BAA with Microsoft.

  • Provide you VMs under your contract with Microsoft that support HIPAA compliance.

  • Delete encryption keys and data when Azure Databricks releases the VM instances.

Key responsibilities of Azure Databricks include:

  • Encrypt in-transit PHI data that is transmitted to or from the control plane.
  • Encrypt PHI data at rest in the control plane
  • Limit the set of instance types to supported instance types for the compliance security profile. Azure Databricks limits the instance types both in the account console and through the API.
  • Deprovision VM instances when you indicate in Azure Databricks that they are to be deprovisioned, for example auto-termination or manual termination, so that Azure can wipe them.

Key responsibilities of yours:

  • Configure your workspace to use either customer-managed keys for managed services or the Store interactive notebook results in customer account feature.
  • Do not use preview features within Azure Databricks to process PHI other than features listed in Preview features that are supported for processing of PHI data
  • Follow security best practices, such as disable unnecessary egress from the compute plane and use the Azure Databricks secrets feature (or other similar functionality) to store access keys that provide access to PHI.
  • Enter into a business associate agreement with Microsoft to cover all data processed within the VNet where the VM instances are deployed.
  • Do not do something within a virtual machine that would be a violation of HIPAA. For example, direct Azure Databricks to send unencrypted PHI to an endpoint.
  • Ensure that all data that may contain PHI is encrypted at rest when you store it in locations that the Azure Databricks platform may interact with. This includes setting the encryption settings on each workspace’s root storage (ADLSgen2 for newer workspaces, Blob storage for older workspaces) that is part of workspace creation. You are responsible for ensuring the encryption (as well as performing backups) for this storage and all other data sources.
  • Ensure that all data that may contain PHI is encrypted in transit between Azure Databricks and any of your data storage locations or external locations you access from a compute plane machine. For example, any APIs that you use in a notebook that might connect to external data source must use appropriate encryption on any outgoing connections.
  • Ensure that all data that may contain PHI is encrypted at rest when you store it in locations that the Azure Databricks platform may interact with. This includes setting the encryption settings on each workspace’s root storage that is part of workspace creation.
  • Ensure the encryption (as well as performing backups) for your root storage (ADLSgen2 for newer workspaces, Blob storage for older workspaces) and all other data sources.
  • Ensure that all data that may contain PHI is encrypted in transit between Azure Databricks and any of your data storage locations or external locations you access from a compute plane machine. For example, any APIs that you use in a notebook that might connect to external data source must use appropriate encryption on any outgoing connections.

About customer-managed keys: