question

SijoJohny-3675 avatar image
0 Votes"
SijoJohny-3675 asked PRADEEPCHEEKATLA-MSFT commented

Non English Data Support in Azure Data Factory - Data Flow

Hi ,

How can I create a data flow/pipeline for a csv file having Non English character in it , While consuming the Non English data column in the data factory, Data is getting formatted to something else.

142115-image.png


azure-data-factory
image.png (3.5 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @SijoJohny-3675,

Welcome to the Microsoft Q&A platform.

As per the repro, I had used a Non English character (Greek Names) as the sources and performed a copy activity using Azure Data Flows and the data format remained same in both source and sink.

142352-image.png

When you say "While consuming the Non English data column in the data factory, Data is getting formatted to something else.", could you please share the more details on your implementation:

  • Sample csv file used.

  • Screenshot of the Source and Sink output.

  • Additional details on the steps which you have performed in the Data Flow.

Much appreciated in advance for your collaboration, I'm looking forward to your reply.

0 Votes 0 ·
image.png (65.1 KiB)

Hi @PRADEEPCHEEKATLA-MSFT

Thank you so much for checking this, I have created a source and sink no other steps in the data flow. If I open the output file directly in the excel it is coming in the formatted version. After opening the output file through notepad and save as with encoding "Unicode" its coming properly.

Is there any way to get the csv file properly without opening it and saving using notepad.

Please find attached screenshots.

142420-nonenglish.png142533-nonenglish-save.png142534-nonenglish3.png




Regards
Sijo Johny

0 Votes 0 ·
nonenglish.png (96.9 KiB)
nonenglish3.png (31.8 KiB)

Hi @SijoJohny-3675,

Following up to see if the below suggestion provided by @nasreen-akter was helpful. And, if you have any further query do let us know.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.

0 Votes 0 ·

1 Answer

nasreen-akter avatar image
0 Votes"
nasreen-akter answered

Hi @SijoJohny-3675,

There is a "Encoding" option in the Dataset. You can try by changing the value e.g., ISO-5589-15. Hope that will help. Thanks!

142525-image.png




image.png (37.9 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.