Troubleshoot $convert-data

Article
07/30/2024

Note

In May 2024 we released a stand-alone FHIR® converter API decoupled from the FHIR service and packaged as a container (Docker) image for preview. In addition to enabling you to convert data from the source of record to FHIR R4 bundles, the FHIR converter offers many net new capabilities, such as:

Bidirectional data conversion from source of record to FHIR R4 bundles and back. For example, the FHIR converter can convert data from FHIR R4 format back to HL7v2 format.
Improved experience for customization of default Liquid templates.
Samples that demonstrate how to create an ETL (extract, transform, load) pipeline with Azure Data Factory (ADF).

To implement the FHIR converter container image, see the FHIR converter GitHub project.

In this article, learn how to troubleshoot $convert-data.

Performance

Two main factors come into play that determine how long a $convert-data operation call can take:

The size of the message.
The complexity of the template.

Any loops or iterations in the templates can have large impacts on performance. The $convert-data operation has a post processing step that is run after the template is applied. In particular, the deduping step can mask template issues that cause performance problems. Updating the template so duplicates aren’t generated can greatly increase performance. For more information and details about the post processing step, see Post processing.

Post processing

The $convert-data operation applies post processing logic after the template is applied to the input. This post processing logic can result in the output looking different, or unexpected errors compared to running the default Liquid template directly. Post processing ensures the output is valid JSON and removes any duplicates based on the ID properties generated for resources in the template. To see the post processing logic in more detail, see the FHIR-Converter GitHub repository.

Message size

There isn’t a hard limit on the size of the messages allowed for the $convert-data operation. However, for content with a request size greater than 10 MB, server errors 500 are possible. If you're receiving 500 server errors, ensure your requests are under 10 MB.

Why are my dates being converted when transforming JSON data?

It's possible for dates supplied within JSON data to be returned in a different format than what was supplied. During deserialization of the JSON payload, strings that are identified as dates get converted into .NET DateTime objects. These objects then get converted back to strings before going through the Liquid template engine. This conversion can cause the date value to be reformatted, and represented in the local timezone of the FHIR service.

The coercion of strings to .NET DateTime objects can be disabled using the boolean parameter jsonDeserializationTreatDatesAsStrings. When set to true, the supplied data is treated as a string and won't be modified before being supplied to the Liquid engine.

Default templates and customizations

Default template implementations for many common scenarios can be found on the FHIR-Converter GitHub repository. The default templates can be used as a guide and reference for customizing and creating your own templates. In addition to the default templates, the $convert-data operation supports several customer Liquid filters and tags that help simplify common scenarios.

Debugging and testing

In addition to testing templates on an instance of the service, a Visual Studio Code extension is available. The extension can be used to modify templates and test them with sample data payloads. There are also several existing test scenarios in the FHIR Converter GitHub repository that can be used as a reference.