Manage FHIR data using Data integration toolkit

Article
06/08/2023

Note

Sync admin for FHIR has now been rebranded as Data integration toolkit.

The Data integration toolkit helps you sync patients' PHI (protected health information) between EHR systems and Dataverse. This integration gives care teams and patients fast and secure access to their data in the Microsoft Cloud for Healthcare environment.

Your FHIR system stays separate, but its data is quickly available via Dataverse. The Dataverse healthcare APIs provide entry points to ingest and retrieve FHIR data into the Microsoft Cloud for Healthcare Dataverse database. Care providers can focus on patient needs and admins can focus on managing people, places, and clinical resources.

Features that simplify working with FHIR data

The following table describes how the Data integration toolkit can simplify working with protected patient data.

Feature	What the feature does	How it helps you manage
Entity maps	Entity maps associate FHIR resources with Dataverse entities. For example, the FHIR resource "Patient" corresponds to the Dataverse entity "Contact". They're both about the same thing - patients. When someone wants to read or change data, the map shows the data's source, and helps the Dataverse healthcare APIs transform data between Dataverse entity records and FHIR resources.	Instead of creating every entity map yourself, you can enable the maps you need. If a map needs a small change, adjust it, or copy it and add a tag. If you don't want to see a map anymore, you can archive it. And for special uses, you can create a new map.
Attribute maps	Attribute maps associate FHIR resource elements with Dataverse attributes. For ingest, these maps contain details on how to locate the data element and its corresponding data type in Dataverse. For retrieve, the maps describe how to transform the Dataverse data element into a FHIR resource node. These maps start off as enabled but they don't affect synchronization unless their parent entity map is enabled.	The default attribute maps provide a baseline set of details driven by the HL7 specification for the parent FHIR resource. Often, you don't need to do anything but enable the parent entity map, but you can configure attribute maps much like entity maps. You can disable the maps if the data isn't relevant for the entity. You can also change from where the data is retrieved if your system differs from the FHIR specification. Attribute maps inherit their parent entity maps' tag to help you tell them apart.
Expansion maps	Expansion maps provide the rules for transforming JSON document based hierarchical data into Dataverse's relational data model.	Expansion maps let admins map complex JSON attributes to multiple, related Dataverse attributes. With expansion maps, you can specify parent link attributes to determine the relationship between parent and child records.
Logs	Data integration toolkit captures details on the Dataverse healthcare API activities and lets you view, sort, and query the transaction logs. Log entries don't contain EMR data; they say what the API calls tried to do and the result. Each entry has a description and additional details to enable troubleshooting for admins and developers.	Logs are how you validate your maps, monitor the Dataverse healthcare API operations, and troubleshoot issues. For example, a log entry with the description "Missing Required Fields" means the data change tried to write to a FHIR resource instance, but didn't include a value for an element that needed one. Maybe the data input form didn't have a field to input the value, or had the field but let you submit the change with that field empty. The log entry includes a list of missing fields, so you can quickly find and fix the problem.
Azure FHIR resources	The Azure FHIR resource record defines the currently supported FHIR resources in the Microsoft Cloud for Healthcare Dataverse solution. This information is also used by the Dataverse healthcare APIs when transforming data between the two systems.	As the Dataverse data model expands, this list is updated to indicate which FHIR resources are available for integration.
Integration settings	Data integration toolkit includes component-specific tools for managing configuration options. These settings tell Dataverse how to recognize FHIR data changes, and where to send FHIR data transactions. They also let you toggle synchronization on or off, and set the logging level for each component.	Allows managing components individually for both new installations and existing running instances.
Healthcare data	When patients give their consent to share their healthcare data, admins can see it in the Data integration toolkit.	Seeing the data helps you manage the data flows, better understand the log items, and lets you validate your data maps.

Entity maps

Entity maps are the highest level in the mapping process. They map FHIR resources to their corresponding Dataverse entities (also known as Dataverse tables). You can map one entity concept to multiple FHIR resources. Entity maps help the Dataverse healthcare APIs and virtual health data tables transform data between the FHIR messages and Dataverse entities. They also help Dataverse post the right FHIR resources back to remote FHIR endpoints.

To see a list of all the entity maps, in the Data integration toolkit application, select the Agent Admin navigation in the Change area section, and then select Entity Maps under Map Setup.

Entity maps included with Data integration toolkit

Data integration toolkit includes many built-in entity maps, attribute maps, and FHIR element maps for standard FHIR resources that you may probably need. Because everyone's system is different, the entity maps aren't enabled during deployment.

To enable the entity maps you need, you just need to change a specific setting.
To modify maps to fit your specific EMR systems, the Data integration toolkit maps are highly configurable.
And if you don't find a specific map to suit your needs, it's easy to create your own map.

For more information on how to configure entity maps, go to Configure entity maps.

The following table lists the entity maps included with Data integration toolkit. This table includes the root level FHIR resources, but not the expansion maps covered in detail in Configure expansion maps.

Azure FHIR Resource	Dataverse Entity	Description
Organization	account	A formally or informally recognized grouping of people or organizations formed for achieving some form of collective action. This grouping includes groups such as companies, institutions, corporations, departments, community groups, and healthcare practice groups.
Patient	contact	Demographics and other administrative information about a person or animal receiving care or other health-related services.
Practitioner	contact	A person who is directly or indirectly involved in the healthcare provisioning.
AllergyIntolerance	msemr_allergyintolerance	Risk of harmful or undesirable physiological response, which is unique to an individual and associated with exposure to a substance.
Appointment	msemr_appointmentemr	A scheduled healthcare event for a patient and/or practitioner(s) where a service may take place at a specific date or time.
CarePlan	msemr_careplan	Describes the intention of how one or more practitioners intend to deliver care for a particular patient for a period of time, possibly limited to care for a specific condition or set of conditions.
CareTeam	msemr_careteam	The care team includes all the people and organizations who plan to participate in the coordination and delivery of care.
Claim	msemr_claim	A provider issued list of professional services and products that have been provided (or are to be provided) to a patient, which is sent to an insurer for reimbursement.
ClaimResponse	msemr_claimresponse	This resource provides the adjudication details from the processing of a claim resource.
Condition	msemr_condition	Used to record detailed information about conditions, problems, or diagnoses recognized by a clinician.
Device	msemr_device	This resource identifies an instance of a manufactured thing that is used in the provision of healthcare without being substantially changed through that activity. The device may be a machine, an insert, a computer, or an application. This includes durable (reusable) medical equipment and disposable equipment used for diagnostic, treatment, and research for healthcare and public health.
DiagnosticReport	msemr_diagnosticreport	The findings and interpretation of diagnostic tests performed on patients, groups of patients, devices, and locations, and the specimens derived from them. The report includes clinical context such as requester and provider information, and some mix of atomic results, images, textual and coded interpretation, and formatted representation of diagnostic reports.
Encounter	msemr_encounter	An interaction between a patient and healthcare provider(s) for providing healthcare service(s) or assessing the health status of a patient.
EpisodeOfCare	msemr_episodeofcare	An association between a patient and an organization or healthcare provider(s) during which time encounters may occur. The managing organization assumes a level of responsibility for the patient during this time.
Goal	msemr_goal	Describes the intended objective(s) of the care.
Group	msemr_group	Represents a defined collection of entities that may be discussed or acted upon collectively, but aren't expected to act collectively, and aren't formally or legally recognized.
Location	msemr_location	Details and position information for a physical place where services are provided and resources and participants may be stored, found, contained, or accommodated.
Medication	msemr_medication	Primarily used for identification and definition of medication, but also covers ingredients and packaging.
MedicationRequest	msemr_medicationrequest	An order or request for both supply of medication and instructions for administration of the medication to a patient. The resource is called "MedicationRequest" rather than "MedicationPrescription" or "MedicationOrder" to generalize the use across inpatient and outpatient settings, including care plans, and to harmonize with workflow patterns.
Observation	msemr_observation	Measurements and simple assertions made about a patient, device, or other subject.
Procedure	msemr_procedure	An action that is performed on a patient. This action can be a physical activity like an operation, or less invasive like counseling or hypnotherapy.
RelatedPerson	msemr_relatedperson	Information about a person who's involved in the care for a patient, but who isn't the target of healthcare, nor has a formal responsibility in the care process.
RiskAssessment	msemr_riskassessment	An assessment of the likely outcome(s) for a patient or other subject and the likelihood of each outcome.
Schedule	msemr_schedule	A container for slot(s) of time that may be available for booking appointments.
Slot	msemr_slot	A slot of time on a schedule that may be available for booking appointments.

Attribute maps

Entity maps have one or more related attribute maps that map the individual elements in the FHIR resource.

To see a list of all the attribute maps, in the Data integration toolkit application, select the Agent Admin navigation in the Change area section, and then select Attribute Maps under Map Setup.

These related maps provide the field-by-field specific detail on which FHIR values map to which Dataverse column values. Each attribute map includes configuration values for data type mapping, and custom JSONPath snippets for selecting the correct value from the inbound FHIR JSON messages.

For more information on how to configure attribute maps, go to Configure attribute maps.

Manage map records

Map records are "solution-aware" components, which means that a component of a solution knows that it's part of that solution. Our maps know that they're part of the Data integration toolkit. Because they're in Dataverse, you can use Application Lifecycle Management (ALM) to safely manage changes you make to your mapping tables. The value it provides is that you can update your maps in a development environment, where you can test and validate your map changes without risking your production EHR/EMR data systems.

In the past, to transfer map changes between environments, you had to copy the data manually. Tools like the Configuration Migration tool could help, but it was possible to make manual errors. Now, after you've made sure your updates are ready, you can safely deploy them to your production environment as a single package. ALM saves you time and helps you protect your business as it evolves and grows. For more information, go to ALM solution concepts.
You can always delete any maps that you create from scratch. However, you can't delete some of the Microsoft Cloud for Healthcare default maps and keep others. You can remove all the Microsoft Cloud for Healthcare default maps by removing the managed solution that gets installed.
Mapping records have a hierarchy with entity maps at the top (Entity maps > Attribute maps > Attribute value maps). Dataverse doesn't allow deleting a record that has child records (related records in lower tiers). To delete an entity map, first find and delete all of its child maps, and then delete the entity map.
If you don't wish to use a default map, you can hide maps in your environment by archiving them. Any entity map can be archived by selecting the record from the grid and then selecting the Archive button in the command bar. When you archive a map, it automatically deactivates and disables the entity map. It also archives the entity's attribute maps and attribute value maps.

You can access archived maps by changing the view to Archived Update Service Entity Maps.

You can restore archived maps using the Restore command. Select the archived entity map from the grid, and then select Restore on the action pane.
You may undo changes to the default maps by removing the active layer of changes on the maps solution. The default maps ship as a managed solution. When you make changes to those maps, they get made in the active layer (unmanaged). You can use the solution layers to review the layers and remove the active layer at any point. Removing the active layer restores the map to the last managed layer for that component. For more information, go to ALM solution layers. For steps to view the solution layers, go to View solution layers.
The latest service release provides entity map tags to help you uniquely identify the maps you add to solutions. You can see the tag at the beginning of the entity map name, and the names of all its attribute maps and attribute value maps. To add a tag, edit the Tag attribute in the entity map header.

Each map may only have one tag assigned at a time.

Logs

Select Logs to view the transactions that are occurring inside Dataverse. Because much of the activity in the Data integration toolkit isn't readily visible, you don't interact with it, and an admin might not see it.

Logs help you understand how the data flows, why something is sent to the Dataverse healthcare API, and why something isn't sent. They provide a view into the Data integration toolkit components' activities.

The logs section contains information about each service, including the custom API.

Azure FHIR resources

The Azure FHIR Resources area defines the universe of FHIR resources. Because you can't see inside FHIR entities directly to see all the resources, the Data integration toolkit provides the list of resources mapped to a Dataverse entity. The FHIR resources are primarily used in entity maps.

Integration settings

The Integration Settings area gives you access to the Dataverse environment variables that define and control the integration of Azure FHIR and Dataverse. You must set these variables to initialize the Data integration toolkit, or make adjustments if something in your Microsoft Cloud for Healthcare environment changes.

For more information about the integration settings, go to Configure integration settings for Dataverse healthcare APIs and Configure integration settings for virtual health data tables.

Healthcare data

Map administrators need to view the healthcare solution data. The Healthcare Data module allows map administrators to view and interact with the user data that's flowing into the system. This feature helps the map administrators understand and troubleshoot the data.

Map administrators can use the healthcare data module to:

View the data.
Understand what is occurring as they build the maps.
Ensure that the data is coming over correctly and into the right field.

For example, to see patient information:

Select Healthcare Data.
Under Administration, select People.
Change the view to Patients FHIR View.

The map administrator can see all the patient data that is flowing into Microsoft Cloud for Healthcare such as the FHIR ID of a patient, the last sync of the patient, if sync is enabled for that record, and the Azure FHIR version of the record.

Contact not synced

This example shows how you might have a Contact entity that doesn't sync when you try to sync a patient record that isn't enabled for sync.

Select Healthcare Data.
In the Patients FHIR View, as an example, find a patient who doesn't have an Azure FHIR ID.
Select the patient record and select Edit. Notice that the Azure FHIR Sync Enabled value is set to No.
Change the phone number for the patient and select Save.
In the Logs section,
- For the Description column for that record, select Write Back Event Handler Successfully processed.
- For the Source, select Write-Back.
- For Entity Type, select contact.
Notice that the record doesn't have an ID for FHIR Resource Id.

Contact missing Azure FHIR details

For another example, try to sync an existing patient who doesn't have a FHIR ID.

Select Healthcare Data.
In the Patients FHIR View, as an example, find a patient who doesn't have an Azure FHIR ID.
Select the patient record and select Edit. Notice that the Azure FHIR Sync Enabled value is set to No.
Update the Azure FHIR Sync Enabled value to set it to Yes.
Change the phone number for the patient and select Save.
In the Logs section,
- For the Description column for that record, select Write Back Process Failed.
- For the Source, select Write-Back.
- For Entity Type, select contact.
The message states that the contact can't be synced to the FHIR endpoint. This behavior is because the Azure FHIR ID or other FHIR version information is missing for an existing record that's being updated. A null Azure FHIR ID indicates the contact didn't originate from the FHIR server, which is an unsupported scenario for the writeback process.

Writeback log entries

When a contact record does have a FHIR ID and a change is made to the record, you can see two log messages. The first message indicates that the writeback process has been initiated. The record titled Write Back Event Handler Successfully processed indicates that the changes have been queued for sending back to the FHIR endpoint.

When the update succeeds in posting to the FHIR endpoint, you can see the record titled Write Back Process Succeeded. If you select this description, you see the message in the log record. The message specifies the attribute that changed and what it sent to the FHIR endpoint.

A final log message titled Write Back FHIR Provenance indicates a final audit message posted to the FHIR endpoint about the recent posted updates.

This combination of messages indicates a successful update to the FHIR endpoint via the writeback processing.

In healthcare data, you might find a patient record with a FHIR ID and the Azure FHIR ID Sync Enabled value set to No. The Dataverse healthcare APIs have a consent flow. The consent flow between FHIR and Dataverse ensures that the whole universe of FHIR doesn't end up in Dataverse. There's a great deal of patient data that resides in FHIR and you only want this data in Dataverse when you're working with that patient.

When the FHIR sync is enabled, the Dataverse healthcare API syncs the universe of patients to Dataverse. However, it doesn't sync the entire patient data to Dataverse unless the Azure FHIR Sync Enabled setting is set to Yes.

This feature is called consent flow because, typically, a patient must consent that they want their information brought into a system. When the patient consents, for example, in Patient access portal, to have their information brought into the system, the Azure FHIR Sync Enabled value is set to Yes, and that patient's data starts flowing into the system.

If the Azure FHIR Sync Enabled value is set to No, and you make a change to the patient's record, you see a log entry with a message that states The data cannot be sent to the Azure FHIR Server, indicating that the Contact entity updates can't be pushed to the FHIR endpoint. Setting Azure FHIR Sync Enabled to No keeps data from flowing to FHIR. It also keeps FHIR from pushing data back to Dataverse for this patient.

The consent flow is unique only to Patients as contacts. No other record entity in Dataverse has this consent flow built into it.

Expansion maps

Expansion maps in Data integration toolkit allow admins to transform FHIR data into Dataverse tables by expanding complex JSON attributes into multiple, related Dataverse records.

For example, you can use expansion maps to bring patient identifiers and patient links into Dataverse. For patient links, expansion maps allow admins to integrate patient merge requests with the following link types:

With expansion maps, you can specify parent link attributes to determine the relationship between parent and child records.

A screenshot displaying a sample parent link attribute.

For more information on expansion maps, go to Configure expansion maps.

Share via