Unicode chrs in Azure Data Factory

Sridhar V 6

Unicode chrs are not able transfer from csv file to sql table in Dataflow... ex: Local â€“ Field
Let me know if we have any method to resolve this issue

1 answer

ShaikMaheer-MSFT 38,501 Reputation points Microsoft Employee

2022-12-02T09:48:58.173+00:00

Hi @Sridhar V ,

Thank you for posting query in Microsoft Q&A platform.

You should consider using nvarchar() datatype of the column in table.

Please check below link.
https://learn.microsoft.com/en-us/answers/questions/1111839/azure-data-factory-unicode-chrs-import.html

Hope this helps.

--------------

Please consider hitting Accept Answer button. Accepted answers help community as well.
Please sign in to rate this answer.
Sridhar V 6 Reputation points

2022-12-02T13:04:54.683+00:00

Hi ShaikMaheer-MSFT,
Thanks for your response. Already I tried with nvarchar. After seeing ur reply again i tried. But it is not working.
Given below another example

1.CR - ATandT Corp - Global â€“ Field --in csv file
2.CR - ATandT Corp - Global – Field --written in sql table.

In the dataset preview itself I am getting as like 2. string.

ShaikMaheer-MSFT 38,501 Reputation points Microsoft Employee

2022-12-02T13:31:19.6+00:00

Hi @Sridhar V , When I tried it was working perfectly fine. How its showing dataflow preview? If possible could you please share your implementation details along with screenshots to see where exactly the miss is happening.

Sridhar V 6 Reputation points

2022-12-02T14:47:34.007+00:00

Hi Shaik,

Thanks for your reply. I am really happy to see that from some where I am getting help technically. I checked with few people they told like it may be tool limitation. So i left it but just a thought like through another/old tool people able to insert this chr in sql table why not ADF. So tried here to by posting my query.
Not sure is it possible to call you and discuss or not. If u r ok let me know the way.

Coming to implementation part,
I uploaded this csv file in BLOB and through data flow I used Source (as CSV) and access this csv file data (PFA).
For example I have filtered one record where we have those characters. In the image inside red box special chrs are replaced by big hypen.

Let me know if you need any other information.

Sridhar V 6 Reputation points

2022-12-06T10:24:09.167+00:00

Hi ShaikMaheer-MSFT, Kindly reply me with you inputs on this issue. It will be really helpful for me.

ShaikMaheer-MSFT 38,501 Reputation points Microsoft Employee

2022-12-06T13:30:03.617+00:00

Hi @Sridhar-V, Looks strange to me. When i repro its working completly fine nvarchar. You can refer gif image from below thread answer.
https://learn.microsoft.com/en-us/answers/questions/1111839/azure-data-factory-unicode-chrs-import.html
What is the source in your case? Is it csv? If yes, could you please share screenshot of source data(from storage) once to see how it shows there.

ShaikMaheer-MSFT 38,501 Reputation points Microsoft Employee

2022-12-22T09:57:55.66+00:00

Hi @Sridhar V , Just checking if you get chance to share updates? Or found any work around on same? Thankyou.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Unicode chrs in Azure Data Factory

1 answer

Your answer