Create maps to transform data in Azure Logic Apps with Visual Studio Code

Applies to: Azure Logic Apps (Standard)

To exchange messages that have different XML or JSON formats in an Azure Logic Apps workflow, you have to transform the data from one format to another, especially if you have gaps between the source and target schema structures. Data transformation helps you bridge those gaps. For this task, you need to create a map that defines the transformation between data elements in the source and target schemas.

To visually create and edit a map, you can use Visual Studio Code with the Azure Logic Apps (Standard) extension within the context of a Standard logic app project. The Data Mapper tool provides a unified experience for XSLT mapping and transformation using drag and drop gestures, a prebuilt functions library for creating expressions, and a way to manually test the maps that you create and use in your workflows.

After you create your map, you can directly call that map from a workflow in your logic app project or from a workflow in the Azure portal. For this task, you can use the Data Mapper Operations action named Transform using Data Mapper XSLT in your workflow.

This how-to guide shows how to create a blank data map, choose your source and target schemas, select schema elements to start mapping, create various mappings, save and test your map, and then call the map from a workflow in your logic app project.

Limitations and known issues

• Data Mapper currently works only in Visual Studio Code running on Windows operating systems.

• Data Mapper is currently available only in Visual Studio Code, not the Azure portal, and only from within Standard logic app projects, not Consumption logic app projects.

• Data Mapper currently doesn't support comma-separated values (.csv) files.

• The Data Mapper's Code view pane is currently read only.

• The map layout and item position are currently automatic and read only.

• To call maps created with the Data Mapper tool, you can only use the Data Mapper Operations action named Transform using Data Mapper XSLT. For maps created by any other tool, use the XML Operations action named Transform XML.

• To use the maps that you create with the Data Mapper tool but in the Azure portal, you must add them directly to your Standard logic app resource.

Prerequisites

• Visual Studio Code and the Azure Logic Apps (Standard) extension to create Standard logic app workflows.

  Note

  The previously separate Data Mapper extension is now merged with the Azure Logic Apps (Standard) extension. To avoid conflicts, any existing version of the Data Mapper extension is removed when you install or update the Azure Logic Apps (Standard) extension. After extension install or update, please restart Visual Studio Code.

• The source and target schema files that describe the data types to transform. These files can have either the following formats:

  • An XML schema definition file with the .xsd file extension
  • A JavaScript Object Notation file with the .json file extension

• A Standard logic app project that includes a stateful or stateless workflow with at least a trigger. If you don't have a project, follow these steps in Visual Studio Code:

  1. Connect to your Azure account, if you haven't already.

  2. Create a local folder, a local Standard logic app project, and a stateful or stateless workflow. During workflow creation, select Open in current window.

• Sample input data if you want to test the map and check that the transformation works as you expect.

• To use the Run XSLT function, your XSLT snippets must exist in files that use either the .xml or .xslt file name extension. You must put your XSLT snippets in the InlineXslt folder in your local project folder structure: Artifacts > DataMapper > Extensions > InlineXslt. If this folder structure doesn't exist, create the missing folders.

Create a data map

1. On the Visual Studio Code left menu, select the Azure icon.

2. In the Azure pane, under the Data Mapper section, select Create new data map.

   

3. Provide a name for your data map.

4. Specify your source and target schemas by following these steps:

   1. On the map surface, select Add a source schema.

      

   2. On the Configure pane that opens, select Add new > Browse.

   3. Find and select your source schema file, and then select Add.

      If your source schema doesn't appear in the Open window, from the file type list, change XSD File (*.xsd) to All Files (*.*).

      The map surface now shows the data types from the source schema. For the examples in this guide,

   4. On the map surface, select Add a target schema.

   5. On the Configure pane that opens, select Add new > Browse.

   6. Find and select your target schema file, and then select Add.

      If your target schema doesn't appear in the Open window, from the file type list, change XSD File (*.xsd) to All Files (*.*).

      The map surface now shows data types from the target schema.

   Alternatively, you can also add your source and target schema files locally to your logic app project in the Artifacts/Schemas folder, so that they appear in Visual Studio Code. In this case, you can specify your source and target schema in the Data Mapper tool on the Configure pane by selecting Select existing, rather than Add new.

   When you're done, your map looks similar to the following example:

   

The following table describes the possible data types that might appear in a schema:

Symbol	Type	More info
	Array	Contains items or repeating item nodes
	Binary
	Bool	True or false only
	Complex	An XML object with children properties, similar to the Object JSON type
	DateTime
	Decimal
	Integer	Whole numbers only
	Null	Not a data type, but appears when an error or an invalid type exists
	Number	A JSON integer or decimal
	Object	A JSON object with children properties, similar to the Complex XML type
	String

Navigate the map

To move around the map, you have the following options:

• To pan around, drag your pointer around the map surface. Or, press and hold the mouse wheel, while you move the mouse or trackball.

• After you move one level down into the map, in the map's lower left corner, a navigation bar appears where you can select from the following options:

  

  Option	Alternative gesture
  Zoom out	On the map surface, press SHIFT + double select. -or- Scroll down with the mouse wheel.
  Zoom in	On the map surface, double select. -or- Scroll up with the mouse wheel.
  Zoom to fit	None
  Show (Hide) mini-map	None

• To move up one level on the map, on the breadcrumb path at the top of the map, select a previous level.

Select target and source elements to map

1. On the map surface, starting from the right side, in the target schema area, select the target element that you want to map. If the element you want is a child of a parent element, find and expand the parent first.

2. Now, on the left side, from the source schema area, select Select element.

3. In the Source schema window that appears, select one or more source elements to show on the map.

   • To include a parent and direct children, open the parent's shortcut menu, and select Add children.

   • To include a parent and all the children for that parent, including any sub-parents, open the top-level parent's shortcut menu, and select Add children (recursive).

4. When you're done, you can close the source schema window. You can always add more source elements later. On the map, in the upper left corner, select Show source schema ().

Create a direct mapping between elements

For a straightforward transformation between elements with the same type in the source and target schemas, follow these steps:

1. To review what happens in code while you create the mapping, in the map's upper right corner, select Show code.

2. If you haven't already, on the map, select the target elements and then the source elements that you want to map.

3. Move your pointer over the source element so that both a circle and a plus sign (+) appear.

   

4. Drag a line to the target element so that the line connects to the circle that appears.

   

   You've now created a direct mapping between both elements.

   

   The code view window reflects the mapping relationship that you created:

   

Note

If you create a mapping between elements where their data types don't match, a warning appears on the target element, for example:

Create a complex mapping between elements

For a more complex transformation between elements in the source and target schemas, such as elements that you want to combine or that have different data types, you can use one or more functions to perform tasks for that transformation.

The following table lists the available function groups and example functions that you can use:

Group	Example functions
Collection	Average, Count, Direct Access, Distinct values, Filter, Index, Join, Maximum, Minimum, Reverse, Sort, Subsequence, Sum
Conversion	To date, To integer, To number, To string
Date and time	Add days
Logical comparison	Equal, Exists, Greater, Greater or equal, If, If else, Is nil, Is null, Is number, Is string, Less, Less or equal, Logical AND, Logical NOT, Logical OR, Not equal
Math	Absolute, Add, Arctangent, Ceiling, Cosine, Divide, Exponential, Exponential (base 10), Floor, Integer divide, Log, Log (base 10), Module, Multiply, Power, Round, Sine, Square root, Subtract, Tangent
String	Code points to string, Concat, Contains, Ends with, Length, Lowercase, Name, Regular expression matches, Regular expression replace, Replace, Starts with, String to code-points, Substring, Substring after, Substring before, Trim, Trim left, Trim right, Uppercase
Utility	Copy, Error, Execute XPath, Format date-time, Format number, Run XSLT

On the map, the function's label looks like the following example and is color-coded based on the function group. To the function name's left side, a symbol for the function appears. To the function name's right side, a symbol for the function output's data type appears.

Add a function without a mapping relationship

The example in this section transforms the source element type from String type to DateTime type, which matches the target element type. The example uses the To date function, which takes a single input.

1. To review what happens in code while you create the mapping, in the map's upper right corner, select Show code.

2. If you haven't already, on the map, select the target elements and then the source elements that you want to map.

3. In the map's upper left corner, select Show functions ().

   

4. From the functions list that opens, find and select the function that you want to use, which adds the function to the map. If the function doesn't appear visible on the map, try zooming out on the map surface.

   This example selects the To date function. You can also find and select any custom functions in the same way. For more information, see Create a custom function.

   

   Note

   If no mapping line exists or is selected when you add a function to the map, the function appears on the map, but disconnected from any elements or other functions, for example:

   

5. Expand the function shape to display the function's details and connection points. To expand the function shape, select inside the shape.

6. Connect the function to the source and target elements.

   1. Drag and draw a line between the source elements and the function's left edge. You can start either from the source elements or from the function.

      

   2. Drag and draw a line between the function's right edge and the target element. You can start either from the target element or from the function.

      

7. On the function's Properties tab, confirm or edit the input to use.

   

   For some data types, such as arrays, the scope for the transformation might also appear available. This scope is usually the immediate element, such as an array, but in some scenarios, the scope might exist beyond the immediate element.

   The code view window reflects the mapping relationship that you created:

   

For example, to iterate through array items, see Create a loop between arrays. To perform a task when an element's value meets a condition, see Add a condition between elements.

Add a function to an existing mapping relationship

When a mapping relationship already exists between source and target elements, you can add the function by following these steps:

1. On the map, select the line for the mapping that you created.

2. Move your pointer over the selected line, and select the Insert function plus sign (+) that appears, for example:

   

3. From the functions list that opens, find and select the function that you want to use.

   The function appears on the map and is automatically connected between the source and target elements.

Add a function with multiple inputs

The example in this section concatenates multiple source element types so that you can map the results to the target element type. The example uses the Concat function, which takes multiple inputs.

1. To review what happens in code while you create the mapping, in the map's upper right corner, select Show code.

2. If you haven't already, on the map, select the target elements and then the source elements that you want to map.

3. In the map's upper left corner, select Show functions ().

   

4. From the functions list that opens, find and select the function that you want to use, which adds the function to the map. If the function doesn't appear visible on the map, try zooming out on the map surface.

   This example selects the Concat function:

   

   Note

   If no mapping line exists or is selected when you add a function to the map, the function appears on the map, but disconnected from any elements or other functions. If the function requires configuration, a red dot appears in the function's upper right corner, for example:

   

5. Expand the function shape to display the function's details and connection points. To expand the function shape, select inside the shape.

6. In the function information pane, on the Properties tab, under Inputs, select the source data elements to use as the inputs.

   This example selects the FirstName and LastName source elements as the function inputs, which automatically add the respective connections on the map.

   

7. To complete the mapping drag and draw a line between the function's right edge and the target element. You can start either from the target element or from the function.

   

   The code view window reflects the mapping relationship that you created:

   

Create a loop between arrays

If your source and target schemas include arrays, you can create a loop mapping relationship that iterates through the items in those arrays. The example in this section loops through an Employee source array and a Person target array.

1. To review what happens in code while you create the mapping, in the map's upper right corner, select Show code.

2. On the map, in the target schema area, select the target array element and target array item elements that you want to map.

3. On the map, in the target schema area, expand the target array element and array items.

4. In the source schema area, add the source array element and array item elements to the map.

5. Create a direct mapping between the source and target elements.

   

   When you first create a mapping relationship between a matching pair of array items, a mapping relationship is automatically created at the parent array level.

   

   The code view window reflects the mapping relationship that you created:

   

6. Continue mapping the other array elements.

   

Set up a condition and task to perform between elements

To add a mapping relationship that evaluates a condition and performs a task when the condition is met, you can use multiple functions, such as the If function, a comparison function such as Greater, and the task to perform such as Multiply.

The example in this section calculates a discount to apply when the purchase quantity exceeds 20 items by using the following functions:

• Greater: Check whether item quantity is greater than 20.
• If: Check whether the Greater function returns true.
• Multiply: Calculate the discount by multiplying the item price by 10% and the item quantity.

1. To review what happens in code while you create the mapping, in the map's upper right corner, select Show code.

2. If you haven't already, on the map, select the target elements and then the source elements that you want to map.

   This example selects the following elements:

   

3. In the map's upper left corner, select Show functions ().

4. Add the following functions to the map: Greater, If, and Multiply

5. Expand all the function shapes to show the function details and connection points.

6. Connect the source elements, functions, and target elements as follows:

   • The source schema's ItemPrice element to the target schema's ItemPrice element
   • The source schema's ItemQuantity element to the Greater function's Value field
   • The Greater function's output to the If function's Condition field
   • The source schema's ItemPrice element to the Multiply function's Multiplicand 0* field
   • The Multiply function's output to the If function's Value field
   • The If function's output to the target schema's ItemDiscount element

   Note

   In the If function, the word ANY appears to the right of the function name, indicating that you can assign the output value to anything.

7. In the following functions, on the Properties tab, specify the following values:

   Function	Input parameter and value
   Greater	- Value #1: The source element named ItemQuantity - Value #2: 20
   Multiply	- Multiplicand #1: The source element named ItemPrice - Multiplicand #2: .10
   If	- Condition: is-greater-than(ItemQuantity,20) - Value: multiply(ItemPrice, .10)

   The following map shows the finished example:

   

   The code view window reflects the mapping relationship that you created:

   

Save your map

When you're done, on the map toolbar, select Save.

Visual Studio Code saves your map as the following artifacts:

• A <your-map-name>.yml file in the Artifacts > MapDefinitions project folder
• An <your-map-name>.xslt file in the Artifacts > Maps project folder

Generate XSLT file at any time

To generate the <your-map-name>.xslt file at any time, on the map toolbar, select Generate XSLT.

Test your map

To confirm that the transformation works as you expect, you'll need sample input data.

1. Before you test your map, make sure to generate the latest <your-map-name>.xslt file.

2. On your map toolbar, select Test.

3. On the Test map pane, in the Input window, paste your sample input data, and then select Test.

   The test pane switches to the Output tab and shows the test's status code and response body.

Call your map from a workflow in your project

1. On the Visual Studio Code left menu, select Explorer (files icon) to view your logic app project structure.

2. Expand the folder that has your workflow name. From the workflow.json file's shortcut menu, select Open Designer.

3. On the workflow designer, follow these general steps to add the Data Mapper Operations built-in action named Transform using Data Mapper XSLT.

4. On the designer, select the Transform using Data Mapper XSLT action.

5. On the action information pane that appears, specify the Content value, and leave Map Source set to Logic App. From the Map Name list, select the map file (.xslt) that you want to use.

   

   To use the same Transform using Data Mapper XSLT action in the Azure portal, you must add the map to the Standard logic app resource.

Create a custom function

To create your own function that you can use with the Data Mapper tool, follow these steps:

1. Create an XML (.xml) file that has a meaningful name that describes your function's purpose.

   If you have multiple related functions, you can use a single file for these functions. Although you can use any file name, a meaningful file name or category makes your functions easier to identify, find, and discover.

2. In your XML file, you must use the following schema for the function definition:

   <?xml version="1.0" encoding="utf-8"?>
   <xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <xs:element name="customfunctions">
         <xs:complexType>
            <xs:sequence>
               <xs:element maxOccurs="unbounded" name="function">
                  <xs:complexType>
                     <xs:sequence>
                        <xs:element maxOccurs="unbounded" name="param">
                           <xs:complexType>
                               <xs:attribute name="name" type="xs:string" use="required" />
                               <xs:attribute name="as" type="xs:string" use="required" />
                           </xs:complexType>
                        </xs:element>
                        <xs:any minOccurs="0" />
                     </xs:sequence>
                     <xs:attribute name="name" type="xs:string" use="required" />
                     <xs:attribute name="as" type="xs:string" use="required" />
                     <xs:attribute name="description" type="xs:string" use="required" />
                  </xs:complexType>
               </xs:element>
            </xs:sequence>
         </xs:complexType>
      </xs:element>
   </xs:schema>
   

   Each XML element named "function" implements an XSLT3.0 style function with few more attributes. The Data Mapper functions list includes the function name, description, parameter names, and parameter types.

   The following example shows the implementation for a SampleFunctions.xml file:

   <?xml version="1.0" encoding="utf-8" ?>
   <xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <customfunctions>
      <function name="age" as="xs:float" description="Returns the current age.">
         <param name="inputDate" as="xs:date"/> 
         <value-of select="round(days-from-duration(current-date() - xs:date($inputDate)) div 365.25, 1)"/>
      </function> 
      <function name="custom-if-then-else" as="xs:string" description="Evaluates the condition and returns corresponding value.">
         <param name="condition" as="xs:boolean"/>
         <param name="thenResult" as="xs:anyAtomicType"/>
         <param name="elseResult" as="xs:anyAtomicType"/>
         <choose>
            <when test="$condition">
               <value-of select="$thenResult"></value-of>
            </when>
            <otherwise>
               <value-of select="$elseResult"></value-of>
            </otherwise>
         </choose>
      </function>
   </customfunctions>
   

3. On your local computer, open the folder for your Standard logic app project.

4. Open the Artifacts folder, and create the following folder structure, if none exists: DataMapper > Extensions > Functions.

5. In the Functions folder, save your function's XML file.

6. To find your custom function in the Data Mapper tool's functions list, search for the function, or expand the Custom functions collection.

Next steps

• For data transformations using B2B operations in Azure Logic Apps, see Add maps for transformations in workflows with Azure Logic Apps