Čitajte na engleskom Uređivanje

Dijeli putem

Support non-Unicode character encoding in Azure Logic Apps

  • Članak

Applies to: Azure Logic Apps (Consumption + Standard)

When you work with text payloads, Azure Logic Apps infers the text is encoded in a Unicode format, such as UTF-8. You might have problems receiving, sending, or processing characters with different encodings in your workflow. For example, you might get corrupted characters in flat files when working with legacy systems that don't support Unicode.

To work with text that has other character encoding, apply base64ToBinary encoding to the non-Unicode payload. This step prevents Azure Logic Apps from assuming the text is in UTF-8 format. You can then convert any .NET-supported encoding to UTF-8 using Azure Functions.

This solution works with both multitenant and single-tenant workflows. You can also use this solution with the AS2 connector.

Convert payload encoding

First, check that your trigger can correctly identify the content type. This step ensures that Azure Logic Apps no longer assumes the text is UTF-8.

In triggers and actions that have the property Infer Content Type, select No. You can usually find this property in the operation's Advanced parameters list. However, if the operation doesn't include this property, the content type is set by the inbound message.

The following list shows some of the connectors where you can disable automatically inferring the content type:

If you're using the Request trigger for text/plain content, you must set the charset parameter that is in the call's Content-Type header. Otherwise, characters might become corrupted, or the parameter doesn't match the payload's encoding format. For more information, review how to handle the text/plain content type.

For example, the HTTP trigger converts the incoming content to UTF-8 when the Content-Type header is set with the correct charset parameter:

JSON 
{
    "headers": {
        <...>
        "Content-Type": "text/plain; charset=windows-1250"
        },
        "body": "non UTF-8 text content"
}

If you set the Content-Type header to application/octet-stream, you also might receive characters that aren't UTF-8. For more information, see how to handle the application/octet-stream content type.

Base64 encode content

Before you base64 encode content to a string, make sure that you convert the text to UTF-8. Otherwise, characters might return corrupted.

Next, convert any .NET-supported encoding to another .NET-supported encoding. Review the Azure Functions code example or the .NET code example:

Savjet

For single-tenant logic apps, you can improve performance and decrease latency by locally running the conversion function.

Azure Functions version

The following example is for Azure Functions version 2:

C# 
using System;
using System.IO;
using System.Text;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;

public static class ConversionFunctionv2 {
  [FunctionName("ConversionFunctionv2")]
  public static IActionResult Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req, TraceWriter log) {
    log.Info("C# HTTP trigger function processing a request.");

    Encoding inputEncoding = null;

    string requestBody = new StreamReader(req.Body).ReadToEnd();
    dynamic data = JsonConvert.DeserializeObject(requestBody);

    if (data == null || data.text == null || data.encodingInput == null || data.encodingOutput == null) {
      return new BadRequestObjectResult("Please pass text/encodingOutput properties in the input JSON object.");
    }

    Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);

    try {
      string encodingInput = data.encodingInput.Value;
      inputEncoding = Encoding.GetEncoding(name: encodingInput);
    } catch (ArgumentException) {
      return new BadRequestObjectResult($"Input character set value '{data.encodingInput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
    }

    Encoding encodingOutput = null;
    try {
      string outputEncoding = data.encodingOutput.Value;
      encodingOutput = Encoding.GetEncoding(outputEncoding);
    } catch (ArgumentException) {
      return new BadRequestObjectResult($"Output character set value '{data.encodingOutput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
    }

    return (ActionResult) new JsonResult(
      value: new {
        text = Convert.ToBase64String(
          Encoding.Convert(
            srcEncoding: inputEncoding,
            dstEncoding: encodingOutput,
            bytes: Convert.FromBase64String((string) data.text)))
      });
  }
}

.NET version

The following example is for use with .NET standard and Azure Functions version 2:

C# 
    using System;
    using System.IO;
    using System.Text;
    using Microsoft.AspNetCore.Mvc;
    using Microsoft.Azure.WebJobs;
    using Microsoft.Azure.WebJobs.Extensions.Http;
    using Microsoft.AspNetCore.Http;
    using Microsoft.Azure.WebJobs.Host;
    using Newtonsoft.Json;

    public static class ConversionFunctionNET
    {
        [FunctionName("ConversionFunctionNET")]
        public static IActionResult Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)]HttpRequest req, TraceWriter log)
        {
            log.Info("C# HTTP trigger function processing a request.");

            Encoding inputEncoding = null;

            string requestBody = new StreamReader(req.Body).ReadToEnd();
            dynamic data = JsonConvert.DeserializeObject(requestBody);

            if (data == null || data.text == null || data.encodingInput == null || data.encodingOutput == null)
            {
                return new BadRequestObjectResult("Please pass text/encodingOutput properties in the input JSON object.");
            }

            Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);

            try
            {
                string encodingInput = data.encodingInput.Value;
                inputEncoding = Encoding.GetEncoding(name: encodingInput);
            }
            catch (ArgumentException)
            {
                return new BadRequestObjectResult($"Input character set value '{data.encodingInput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
            }

            Encoding encodingOutput = null;
            try
            {
                string outputEncoding = data.encodingOutput.Value;
                encodingOutput = Encoding.GetEncoding(outputEncoding);
            }
            catch (ArgumentException)
            {
                return new BadRequestObjectResult($"Output character set value '{data.encodingOutput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
            }

            return (ActionResult)new JsonResult(
                value: new
                {
                    text = Convert.ToBase64String(
                        Encoding.Convert(
                            srcEncoding: inputEncoding,
                            dstEncoding: encodingOutput,
                            bytes: Convert.FromBase64String((string)data.text)))
                });
        }
    }

Using these same concepts, you can also send a non-Unicode payload from your workflow.

Sample payload conversions

In this example, the base64-encoded sample input string is a personal name that contains accented characters: Héloïse

Example input:

JSON 
{  
    "text": "SMOpbG/Dr3Nl",
    "encodingInput": "utf-8",
    "encodingOutput": "windows-1252"
}

Example output:

JSON 
{
    "text": "U01PcGJHL0RyM05s"
}

Send non-Unicode payload

If you need to send a non-Unicode payload from your workflow, do the steps for converting the payload to UTF-8 in reverse. Keep the text in UTF-8 as long as possible within your system. Next, use the same function to convert the base64-encoded UTF-8 characters to the required encoding. Then, apply base64 decoding to the text, and send your payload.

When you consume the return value from Azure Functions, make sure to use the base64ToBinary function, not the base64ToString function.

Convert payloads for AS2

You can also use this solution with non-Unicode payloads in the AS2 v2 connector. If you don't convert payloads that you pass to AS2 to UTF-8, you might experience problems with the payload interpretation. These problems might result in a mismatch with the MIC hash between the partners because of misinterpreted characters.

Next steps

Encode and decode flat files in Azure Logic Apps by using the Enterprise Integration Pack