Events
Mar 17, 11 PM - Mar 21, 11 PM
Join the meetup series to build scalable AI solutions based on real-world use cases with fellow developers and experts.
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Applies to: Azure Logic Apps (Consumption + Standard)
When you work with text payloads, Azure Logic Apps infers the text is encoded in a Unicode format, such as UTF-8. You might have problems receiving, sending, or processing characters with different encodings in your workflow. For example, you might get corrupted characters in flat files when working with legacy systems that don't support Unicode.
To work with text that has other character encoding, apply base64ToBinary encoding to the non-Unicode payload. This step prevents Azure Logic Apps from assuming the text is in UTF-8 format. You can then convert any .NET-supported encoding to UTF-8 using Azure Functions.
This solution works with both multitenant and single-tenant workflows. You can also use this solution with the AS2 connector.
First, check that your trigger can correctly identify the content type. This step ensures that Azure Logic Apps no longer assumes the text is UTF-8.
In triggers and actions that have the property Infer Content Type, select No. You can usually find this property in the operation's Advanced parameters list. However, if the operation doesn't include this property, the content type is set by the inbound message.
The following list shows some of the connectors where you can disable automatically inferring the content type:
If you're using the Request trigger for text/plain
content, you must set the charset
parameter that is in the call's Content-Type
header. Otherwise, characters might become corrupted, or the parameter doesn't match the payload's encoding format. For more information, review how to handle the text/plain
content type.
For example, the HTTP trigger converts the incoming content to UTF-8 when the Content-Type
header is set with the correct charset
parameter:
{
"headers": {
<...>
"Content-Type": "text/plain; charset=windows-1250"
},
"body": "non UTF-8 text content"
}
If you set the Content-Type
header to application/octet-stream
, you also might receive characters that aren't UTF-8. For more information, see how to handle the application/octet-stream
content type.
Before you base64 encode content to a string, make sure that you convert the text to UTF-8. Otherwise, characters might return corrupted.
Next, convert any .NET-supported encoding to another .NET-supported encoding. Review the Azure Functions code example or the .NET code example:
Tip
For single-tenant logic apps, you can improve performance and decrease latency by locally running the conversion function.
The following example is for Azure Functions version 2:
using System;
using System.IO;
using System.Text;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
public static class ConversionFunctionv2 {
[FunctionName("ConversionFunctionv2")]
public static IActionResult Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req, TraceWriter log) {
log.Info("C# HTTP trigger function processing a request.");
Encoding inputEncoding = null;
string requestBody = new StreamReader(req.Body).ReadToEnd();
dynamic data = JsonConvert.DeserializeObject(requestBody);
if (data == null || data.text == null || data.encodingInput == null || data.encodingOutput == null) {
return new BadRequestObjectResult("Please pass text/encodingOutput properties in the input JSON object.");
}
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
try {
string encodingInput = data.encodingInput.Value;
inputEncoding = Encoding.GetEncoding(name: encodingInput);
} catch (ArgumentException) {
return new BadRequestObjectResult($"Input character set value '{data.encodingInput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
}
Encoding encodingOutput = null;
try {
string outputEncoding = data.encodingOutput.Value;
encodingOutput = Encoding.GetEncoding(outputEncoding);
} catch (ArgumentException) {
return new BadRequestObjectResult($"Output character set value '{data.encodingOutput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
}
return (ActionResult) new JsonResult(
value: new {
text = Convert.ToBase64String(
Encoding.Convert(
srcEncoding: inputEncoding,
dstEncoding: encodingOutput,
bytes: Convert.FromBase64String((string) data.text)))
});
}
}
The following example is for use with .NET standard and Azure Functions version 2:
using System;
using System.IO;
using System.Text;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
public static class ConversionFunctionNET
{
[FunctionName("ConversionFunctionNET")]
public static IActionResult Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)]HttpRequest req, TraceWriter log)
{
log.Info("C# HTTP trigger function processing a request.");
Encoding inputEncoding = null;
string requestBody = new StreamReader(req.Body).ReadToEnd();
dynamic data = JsonConvert.DeserializeObject(requestBody);
if (data == null || data.text == null || data.encodingInput == null || data.encodingOutput == null)
{
return new BadRequestObjectResult("Please pass text/encodingOutput properties in the input JSON object.");
}
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
try
{
string encodingInput = data.encodingInput.Value;
inputEncoding = Encoding.GetEncoding(name: encodingInput);
}
catch (ArgumentException)
{
return new BadRequestObjectResult($"Input character set value '{data.encodingInput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
}
Encoding encodingOutput = null;
try
{
string outputEncoding = data.encodingOutput.Value;
encodingOutput = Encoding.GetEncoding(outputEncoding);
}
catch (ArgumentException)
{
return new BadRequestObjectResult($"Output character set value '{data.encodingOutput.Value}' is not supported. Supported values are listed at https://msdn.microsoft.com/en-us/library/system.text.encoding(v=vs.110).aspx.");
}
return (ActionResult)new JsonResult(
value: new
{
text = Convert.ToBase64String(
Encoding.Convert(
srcEncoding: inputEncoding,
dstEncoding: encodingOutput,
bytes: Convert.FromBase64String((string)data.text)))
});
}
}
Using these same concepts, you can also send a non-Unicode payload from your workflow.
In this example, the base64-encoded sample input string is a personal name that contains accented characters: Héloïse
Example input:
{
"text": "SMOpbG/Dr3Nl",
"encodingInput": "utf-8",
"encodingOutput": "windows-1252"
}
Example output:
{
"text": "U01PcGJHL0RyM05s"
}
If you need to send a non-Unicode payload from your workflow, do the steps for converting the payload to UTF-8 in reverse. Keep the text in UTF-8 as long as possible within your system. Next, use the same function to convert the base64-encoded UTF-8 characters to the required encoding. Then, apply base64 decoding to the text, and send your payload.
When you consume the return value from Azure Functions, make sure to use the base64ToBinary function, not the base64ToString function.
You can also use this solution with non-Unicode payloads in the AS2 v2 connector. If you don't convert payloads that you pass to AS2 to UTF-8, you might experience problems with the payload interpretation. These problems might result in a mismatch with the MIC hash between the partners because of misinterpreted characters.
Events
Mar 17, 11 PM - Mar 21, 11 PM
Join the meetup series to build scalable AI solutions based on real-world use cases with fellow developers and experts.
Register nowTraining
Learning path
Use advance techniques in canvas apps to perform custom updates and optimization - Training
Use advance techniques in canvas apps to perform custom updates and optimization