How do i filter files based on their encodings ?

Question

How do i filter files based on their encodings ?

Charan Duggirala 0

I am currently moving data from source to sink, however, some files at my source are in UTF-16-LE encodings. I need the UTF-16-LE files to be filtered out based on their encodings and then copy them to a new location. How can I use Azure datafactory components to filter the files based on encodings?

phemanth 15,855 Reputation points Microsoft External Staff Moderator

2023-12-01T04:36:07.69+00:00

@Charan Duggirala

Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Your answer

phemanth 15,855 Reputation points Microsoft External Staff Moderator

2023-12-01T04:36:07.69+00:00

@Charan Duggirala

Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

Subashri Vasudevan 11,306 Volunteer Moderator

Charan Duggirala

Hi, thanks for your question and using MS Q&A portal.

Regarding your query to find out the encoding and filter out based on that, it is not directly possible with ADF activities. We need to use azure functions\batch in order to identify the encoding.

You can refer the below SO thread that discusses the similar question.

https://stackoverflow.com/questions/66255548/check-the-csv-file-encoding-in-data-factory

In order to use python to find out encoding, you may find lots of references in this thread.

Hope it helps.

Please let us know if you have more question on this, would be happy to assist.

phemanth 15,855 Reputation points Microsoft External Staff Moderator

2023-12-04T05:06:30.22+00:00

@Charan Duggirala Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

How do i filter files based on their encodings ?

1 answer

Your answer