Configure virtual health data tables

  • Article
  • 9 minutes to read

The virtual health data tables feature in Microsoft Cloud for Healthcare optimizes your cloud hosting spend. This feature helps you virtualize your clinical reference data and other healthcare data when using our model-driven apps and other engagement solutions. Virtualizing your data removes the need for building expensive integrations that visualize the same data intensive tables multiple times across Microsoft and third-party applications like electronic health record (EHR) systems.

Prerequisites

Virtual health data tables target core functionality in connecting Microsoft Cloud for Healthcare to external FHIR servers. This article assumes you have a FHIR server, and the November 2021 release or a higher release of Microsoft Cloud for Healthcare. In addition, this article assumes you've completed the following steps:

  • Configure patients to flow into Dataverse using the Data Integration Toolkit.
  • Deploy Azure Logic App via the Data Ingestion ARM Template.
  • Set up a managed identity in Azure (client ID and client secret) with permissions to the FHIR server.
  • Configure the managed identity in Data Integration Toolkit.
  • Set up the virtual data source attributes (FHIR Server URL, Auth, and Resource).
  • Switch the Encounter and Observation data routes to virtual.

For more information about completing these steps, go to Overview of Data Integration Toolkit.

Data routes

Virtual health data tables take the best of the Microsoft Dataverse virtual tables (entities) solution and layer it on top of a data routing concept.

A diagram displaying the data routing concept.

One of the limitations with Dataverse virtual tables today is that there's no way to convert an existing physical table to virtual and vice versa, which causes the creation of net new tables. When the time comes to make the switch to either virtual or physical, you must then reconfigure your Dataverse applications to use the new table structure.

One of the challenges with Dataverse virtual tables today is that after virtualizing a table, end users are confronted with two possible tables to use when creating saved views and advanced finds.

Virtual health data tables solve this challenge by allowing you to establish data routes for your tables or entities.

A screenshot showing a sample data route.

Note

Unsure about entity versus table? Go to Developers: Understand terminology in Microsoft Dataverse.

Entity and attribute maps

Virtual health data tables leverage the same entity maps and attribute maps whether you decide to virtualize your data or keep Dataverse as the source. For more information, go to Entity maps and Attribute maps.

Implementation details

The tables that support virtual health data tables are configured to route to Dataverse by default. To switch the data store to virtual and route data requests to the FHIR server, follow these steps in Data Integration Toolkit:

  1. Configure security roles for admins and virtual table users.

    • Assign the new Healthcare Virtual Tables Admin security role to admins who will configure virtual tables. Make sure these admins also have the Sync admin for FHIR Config Administrator role, as detailed in Data Integration Toolkit security roles.

    • Assign the new Healthcare Virtual Tables User security role to everyone who will access the virtual table entities.

  2. Configure connectivity to the FHIR server with the following fields:

    • Provider client ID and client secret
    • Microsoft Healthcare data source

  3. Update data routes for the selected tables. Only Observation and Encounter related tables are in scope for virtual health data tables. These records have read-only access.

Note

If the core table Observation or Encounter is changed to virtual or Dataverse, then you should change all related supporting tables at the same time.

Other implementation details include the following:

  • Expand: Supports virtual Observation and Encounter expand tables.

    • Retrieve multiple query: When the expand tables are configured to route data from a virtual data provider, retrieve multiple is only supported if the query contains filters on the parent link attribute. For example, Observation Component is an expand entity of the Observation entity. The entity map for Observation Component is configured as shown in the following screenshot:

      A screenshot showing the observation component configuration.

      The retrieve multiple query for the Observation component must contain a filter on specific IDs of msemr_observation (parent link attribute).

      A screenshot displaying sample filters for the observation component.

    • Retrieve: Because the expand entries don't have a unique ID in FHIR, we don't support retrieving expand entries using an ID. The ID that appears when you select a record from a set of retrieve multiple results is temporary.

  • Filtering: Supports column filtering as defined by the HL7 FHIR specification. You can filter on linked entities for one level only.

  • Composite filtering: Supports limited composite filters for single table composite filters. The following composite filter definitions are supported:

    Composite filter Description
    code-value-concept Code and coded value parameter pair
    code-value-date Code and date/time value parameter pair
    code-value-quantity Code and quantity value parameter pair
    code-value-string Code and string value parameter pair
    combo-code-value-concept Code and coded value parameter pair, including in components
    combo-code-value-quantity Code and quantity value parameter pair, including in components

    For more information about composite filters, go to Composite Search Parameters in the HL7 FHIR documentation.

  • Linked entities: Supports linked entities filters using chained filtering as defined by the HL7 FHIR specification. The level of support depends on the Azure API for FHIR version. Unsupported filter conditions surface exceptions and return no results.

  • Sorting: Implemented as defined by the HL7 FHIR specification. The level of support depends on the Azure API for FHIR version. Unsupported sorting conditions will still return data.

  • Notifications and exceptions: Notifications in context of FHIR flex tables. User messages are available with limited sorting and filtering is based on the Azure API for FHIR version.

Additional supporting tables have been configured as part of virtual health data tables. These supporting records are:

  • Observation

    • Observation Based on
    • Observation Component
    • Observation Category
    • Observation Component Reference Range
    • Observation Performer
    • Observation Reference Range
    • Observation Reference Range Applies To

  • Encounter

    • Encounter Account
    • Encounter Class History
    • Encounter Diagnosis
    • Encounter Episode of Care
    • Encounter Hospitalization Arrangement
    • Encounter Hospitalization Courtesy
    • Encounter Hospitalization Diet
    • Encounter Location
    • Encounter Participant
    • Encounter Participant Type
    • Encounter Reason
    • Encounter Status History
    • Encounter Type

Note

If Data Provider to Use for Observation or Encounter is set to virtual, then you should change all the related supporting tables at the same time.

A screenshot showing the data provider to use value set to virtual.

Update FHIR server client ID and client secret

  1. In Data Integration Toolkit, navigate to Integration settings and select Virtual health data tables.

    A screenshot displaying the integration settings.

  2. Enter the Client ID and Client Secret for the FHIR server in the fields provided.

  3. Enter the Server URL and Authority URL for the target FHIR server.

  4. Select the desired value for Logging Level.

  5. Select Save to commit your changes.

Perform the same steps when updating a Client Secret when it's expired or newly generated.

Update data route to Dataverse or virtual

Follow these steps to switch the data route for tables from virtual to Dataverse, or vice versa.

  1. In Data Integration Toolkit, navigate to Data Routes under Map Setup.

  2. Select a core table, either Observation or Encounter.

    A screenshot showing the observation table.

  3. Change the value for Data Provider to Use. You'll be prompted with a confirmation.

    A screenshot displaying the data provider change confirmation message.

  4. After you select Confirm Change on the message, you can then select the entity map to use for this route.

    A screenshot showing the entity map selection.

  5. After you save the changes, repeat these steps for the supporting tables for your current selection.

Things to remember

The following section has some of the key implementation considerations to keep in mind while planning to enable the virtual health data tables feature. However, this isn't an exhaustive list.

For more information, go to Limitations of virtual tables.

Risk User experience Potential mitigation tactic
Existing saved views and dashboards All charts and dashboards that were created using physical entities that have since been virtualized will no longer function. Refactor saved views and dashboards to use the new virtualized entity.

Note the new Native text added to the front of legacy Dataverse versions of the virtual health data tables.

The virtual version of these tables will, for example, be named Encounters or Observations.

Communicate changes to end users. In addition to system views, users will need to refactor personal views.
Charts All charts will no longer function or be available for creation. Power BI or an alternative solution for visualizing this data is required. Model-driven charts won't render for virtualized data.

Communicate changes to end users. They'll no longer have charts in personal views and dashboards if they've been created prior to using physical entities.
Relevance search not supported Relevance search won't function for virtual health data tables. Communicate changes to end users. Assess if virtual entities can be leveraged in your deployment.

The new default search experience in model-driven Power Apps is built upon relevance search.
AI Builder not supported Any AI Builder insights that once leveraged physical Dataverse tables will no longer be available when those tables are virtualized. Consider other AI options from Microsoft.

The data sets you're considering virtualizing in Dataverse should likely be analyzed with Azure services such as Azure Synapse Analytics to uncover opportunities in your clinic or business.
Virtual tables feature a simplified security model, because they'll change to organization-level security. Security should be examined for your compliance requirements. If organization-wide security isn't a fit for your deployment, reconsider enabling the virtual health data tables feature.

Known limitations

Because the virtual health data tables feature is based on Dataverse's existing virtual table solution, it has the same limitations as that of virtual tables. Consider these limitations while you determine whether this feature would work for your needs.

The following limitations also apply:

  • This feature currently only supports connecting to Azure FHIR servers. For more information, go to What is FHIR service?

  • Support for search and sort is dependent on the version of the configured FHIR server. For more information, go to Overview of FHIR search.

  • For search and filtering, a single level of link entity is currently supported.

  • For search and filtering, a single level of expand entities is currently supported.

  • For virtual tables, relationships to non FHIR-based tables aren't supported.

Frequently asked questions

This section addresses some of the frequently asked questions about virtual health data tables.

Can I search on data tables set to virtual?

Search and sort are supported in a limited scope, depending on the FHIR server. Support is based on what's currently supported by the target FHIR server.

Examples:

  • Observation: Filter on Observation for a specific patient.

    1. In the main Observation view, open the Advanced filters panel. In this panel, select Add Row, select Subject (Patient) as the column, and then select one or more patient records from the list.

      A screenshot how to filter on observation for a specific patient.

    2. Select Apply to view the filtered results.

      A screenshot displaying the filter results.

  • Observation: Filter on Observation for one or more codes.

    Following the same steps as explained in the previous section, you can filter on Observation Code. Select a Code and a Code value for filtering.

    A screenshow showing how to filter on observation for a specific code.

    The result would be as follows:

    A screenshot displaying the filter results for the specified code.

  • Observation: Linked entity filtering.

    Only one level of linked entity filtering is supported. Filtering on the linked entity requires that the data being filtered also exists on the FHIR server, such as Patient. The following screenshot displays an example of filtering on Patient with a last name of "James":

    A screenshot displaying an example for linked entity filtering.

    The Azure API for FHIR limits the subquery results to 1,000 records. If the subquery isn't selective enough, the FHIR server will return an error and no results. For example, changing the last name filter to "Smith" in the same example results in the following error:

    A screenshot displaying the error message for linked entity filtering.

    The latest version of the Azure Health Data Services titled Azure Healthcare APIs should address this issue.

What FHIR server versions are supported?

Currently, virtual health data tables support Azure API for FHIR and Azure Health Data Services. Configuration for these two versions has been deployed as part of the baseline solution.

The supported version can be found in the configuration table named FHIR Server Versions using advanced find.

A screenshot displaying the supported FHIR server versions.

Azure API For FHIR is currently available via Azure, while FHIR Server for Azure is currently in preview. For more information, go to Overview of FHIR search and Specific search and sort details.

See also

What is Microsoft Cloud for Healthcare?
Set up and configure Microsoft Cloud for Healthcare
Overview of Data Integration Toolkit
Manage FHIR data using Data Integration Toolkit