Azure Logic App sftp create file bad encoding

Question

Azure Logic App sftp create file bad encoding

Carlos Barroso 30

Hello,

Here a very short sample :

I create a variable "Clément", upload it in sftp server, send it by email as attachment.

In the ftp server the file contain "ClÃ©ment".

In the email the file contain "ClÃ©ment".

User's image

NB: For email i found a workaround by encoding "Clément" as base64 string (from windows-1552 to iso-8859-1) but it doesnt work for sftp create file

Ronen Ariely 14,131 Reputation points MVP

2023-02-25T14:54:17.6366667+00:00

In the email the file contain "ClÃ©ment".

Do you man that this is the file name of the file content (string in file text content) ?

Your entire scenario is not clear to me. Please try to elaborate and add the logic app full screenshot + code if posible

Carlos Barroso 30

The content of the file, here the code

{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Create_file": {
                "inputs": {
                    "body": "@variables('text')",
                    "headers": {
                        "ReadFileMetadataFromServer": true
                    },
                    "host": {
                        "connection": {
                            "name": "@parameters('$connections')['sftpwithssh']['connectionId']"
                        }
                    },
                    "method": "post",
                    "path": "/datasets/default/files",
                    "queries": {
                        "folderPath": "/tests",
                        "name": "test.csv",
                        "queryParametersSingleEncoded": true
                    }
                },
                "runAfter": {
                    "Initialize_variable": [
                        "Succeeded"
                    ]
                },
                "runtimeConfiguration": {
                    "contentTransfer": {
                        "transferMode": "Chunked"
                    }
                },
                "type": "ApiConnection"
            },
            "Initialize_variable": {
                "inputs": {
                    "variables": [
                        {
                            "name": "text",
                            "type": "string",
                            "value": "Clément"
                        }
                    ]
                },
                "runAfter": {},
                "type": "InitializeVariable"
            },
            "Send_an_email_(V2)": {
                "inputs": {
                    "body": {
                        "Attachments": [
                            {
                                "ContentBytes": "@{base64(variables('text'))}",
                                "Name": "test.csv"
                            }
                        ],
                        "Body": "<p>@{variables('text')}</p>",
                        "Importance": "Normal",
                        "Subject": "test",
                        "To": "xx"
                    },
                    "host": {
                        "connection": {
                            "name": "@parameters('$connections')['office365']['connectionId']"
                        }
                    },
                    "method": "post",
                    "path": "/v2/Mail"
                },
                "runAfter": {
                    "Create_file": [
                        "Succeeded"
                    ]
                },
                "type": "ApiConnection"
            }
        },
        "contentVersion": "1.0.0.0",
        "outputs": {},
        "parameters": {
            "$connections": {
                "defaultValue": {},
                "type": "Object"
            }
        },
        "triggers": {
            "manual": {
                "inputs": {
                    "schema": {}
                },
                "kind": "Http",
                "type": "Request"
            }
        }
    },
    "parameters": {
        "$connections": {
            "value": {
                "office365": {
                    "connectionId": "/xx",
                    "connectionName": "office365",
                    "id": "/xx"
                },
                "sftpwithssh": {
                    "connectionId": "xx",
                    "connectionName": "xx",
                    "id": "xx"
                }
            }
        }
    }
}

Bruno Lucas 4,126 Reputation points MVP

2023-03-01T10:43:55.6666667+00:00

Hi Carlos

I tried but could not reproduce your scenario. There is this issue: https://learn.microsoft.com/en-us/azure/logic-apps/support-non-unicode-character-encoding but I think it applies from messages sent to a trigger with a certain encoding.
Carlos Barroso 30 Reputation points

2023-03-03T09:22:50.53+00:00

@Bruno , yes my logic app generate his own content (sql query) so this article doesnt apply to my situation :(
What i need is a way to say to 'sendEmailv2' and 'sftp.create' the encoding it should use T-T

Ronen Ariely 14,131 Reputation points MVP

2023-02-25T14:54:17.6366667+00:00

In the email the file contain "ClÃ©ment".

Do you man that this is the file name of the file content (string in file text content) ?

Your entire scenario is not clear to me. Please try to elaborate and add the logic app full screenshot + code if posible
Carlos Barroso 30 Reputation points

2023-02-27T16:58:23.3+00:00

The content of the file, here the code

{ "definition": { "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#", "actions": { "Create_file": { "inputs": { "body": "@variables('text')", "headers": { "ReadFileMetadataFromServer": true }, "host": { "connection": { "name": "@parameters('$connections')['sftpwithssh']['connectionId']" } }, "method": "post", "path": "/datasets/default/files", "queries": { "folderPath": "/tests", "name": "test.csv", "queryParametersSingleEncoded": true } }, "runAfter": { "Initialize_variable": [ "Succeeded" ] }, "runtimeConfiguration": { "contentTransfer": { "transferMode": "Chunked" } }, "type": "ApiConnection" }, "Initialize_variable": { "inputs": { "variables": [ { "name": "text", "type": "string", "value": "Clément" } ] }, "runAfter": {}, "type": "InitializeVariable" }, "Send_an_email_(V2)": { "inputs": { "body": { "Attachments": [ { "ContentBytes": "@{base64(variables('text'))}", "Name": "test.csv" } ], "Body": "<p>@{variables('text')}</p>", "Importance": "Normal", "Subject": "test", "To": "xx" }, "host": { "connection": { "name": "@parameters('$connections')['office365']['connectionId']" } }, "method": "post", "path": "/v2/Mail" }, "runAfter": { "Create_file": [ "Succeeded" ] }, "type": "ApiConnection" } }, "contentVersion": "1.0.0.0", "outputs": {}, "parameters": { "$connections": { "defaultValue": {}, "type": "Object" } }, "triggers": { "manual": { "inputs": { "schema": {} }, "kind": "Http", "type": "Request" } } }, "parameters": { "$connections": { "value": { "office365": { "connectionId": "/xx", "connectionName": "office365", "id": "/xx" }, "sftpwithssh": { "connectionId": "xx", "connectionName": "xx", "id": "xx" } } } } }
Bruno Lucas 4,126 Reputation points MVP

2023-03-01T10:43:55.6666667+00:00

Hi Carlos

I tried but could not reproduce your scenario. There is this issue: https://learn.microsoft.com/en-us/azure/logic-apps/support-non-unicode-character-encoding but I think it applies from messages sent to a trigger with a certain encoding.
Carlos Barroso 30 Reputation points

2023-03-03T09:22:50.53+00:00

@Bruno , yes my logic app generate his own content (sql query) so this article doesnt apply to my situation :(
What i need is a way to say to 'sendEmailv2' and 'sftp.create' the encoding it should use T-T

Answer 1

Sonny Gillissen 656

Hi @Carlos Barroso

Thanks for reaching out on Microsoft Q&A!

Have you tried wrapping your variable in the binary() function, like this:

binary(variables('text'))

Logic Apps assumes all content is UTF-8, however your SFTP-server may not accept UTF-8 characters to be posted in the body. With the binary() function it will wrap your UTF-8 content in a byte array, which may solve your issue.

When you think my answer is helpful, please click "Accept answer" and upvote it. Feel free to drop additional queries in the comments.

Carlos Barroso 30

This work for sftp! but not for email xD

sftp:

"body": "@variables('text')" => ClÃ©ment,
"body": "@binary(variables('text')) => Clément",

Emailv2 Attachments


"body": "@base64(variables('text'))" => ClÃ©ment,
"body": "@base64(binary(variables('text')))" => ClÃ©ment,

Any idea how to make it work in email attachment? :)

Sonny Gillissen 656 Reputation points

2023-03-03T14:55:30.74+00:00

Great! So, 1 down 1 to go ;-) How about just adding it as follows (without base64() wrapped around it)?

binary(variables('text'))

Carlos Barroso 30

@Sonny Gillissen i already tried with

                        "Attachments": [
                            {
                                "ContentBytes": "@{binary(variables('text'))}",
                                "Name": "test.csv"
                            }

But i get an error, sendEmailv2

"status": 400,
  "message": "Unable to read message or attachment content.\r\nclientRequestId: 7f242c3c-4dcc-444f-b9b7-3ba7274bcfcd\r\nserviceRequestId: 5d6aeaf1-f4f6-ffe3-5c8e-4657c9a89e59",
  "error": {
    "message": "Unable to read message or attachment content.",
    "code": "RequestBodyRead",
    "originalMessage": "Cannot convert the literal 'Clément' to the expected type 'Edm.Binary'."
  },
  "source": "office365-fc.azconn-fc-002.p.azurewebsites.net"

Looks like sendEMail attachment only allow base64 content

Sonny Gillissen 656 Reputation points

2023-03-03T17:31:19.18+00:00

Tried the same indeed, got the same error. Got it set up in my test lab, and going to fiddle some. Let you know asap!
Sonny Gillissen 656 Reputation points

2023-03-03T20:55:54.6333333+00:00

I think I found the solution: as told a Logic App handles everything as UTF-8, but CSV does not.

The problem here is not the action creating the email attachment, but the attachment’s content type (CSV). If you put UTF-8 in a “default” CSV file the characters get mixed up, as the default Excel encoding is Windows-1252.

However, there is a solution! If you put a BOM marker in front of the CSV contents you tell it to be handled as UTF-8.

Please refer to the documentation below on how to do this in Logic Apps:

https://techcommunity.microsoft.com/t5/integrations-on-azure-blog/create-csv-files-with-bom-marker-in-logic-app/ba-p/2919113
Carlos Barroso 30 Reputation points

2023-03-03T21:53:00.73+00:00

It works!!! Many thanks 😁
Sonny Gillissen 656 Reputation points

2023-03-03T22:03:56.3133333+00:00

You’re welcome! Glad I could be of help 😊

Azure Logic App sftp create file bad encoding

0 additional answers

Activity