Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,483 questions
Orc files when downloaded from azure portal has different encodings
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 2%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Akhila
1
Reputation point
I have an observation on the files downloaded from storage account, from the same location but by different means. Could you please shed some light on why this would occur?
when I download a file from the options menu from the file explorer as below I see the UTF-8 encoding and file size increases
I get a different encoding for the same file when I download it from the File details pane as below and the encoding is ANSI the file size is the same no change in it.
{count} votes
-
deherman-MSFT 37,416 Reputation points Microsoft Employee2021-01-21T17:40:15.433+00:00 @Akhila
I would like to work on reproducing your issue. Can you please let me know what the encoding is prior to uploading and how the file is being uploaded? Can you also let me know how exactly you are seeing the difference, what program are you comparing the files with? Also please confirm what the ContentType of the file is in the blob properties. You tagged with this ADLS can you confirm this is an ADLS Gen2 storage account you are working with? -
Akhila 1 Reputation point
2021-01-21T23:00:32.473+00:00 Thanks for your reply! yes it is ADLS gen2 , the files are loaded into the Azure storage account by Nifi pipeline the source is hive table. I compared the files using Notepad++. The content type from blob properties is "CONTENT-TYPE application/octet-stream" . Please let me know if you need more information. Below are the same files which I downloaded from storage account by two means and highlighted the encoding and also the sizes of files. 61kb is the actual size of the ORC file which has ANSI encoding. 105 kb has UTF-8 encoding.
-
deherman-MSFT 37,416 Reputation points Microsoft Employee
2021-01-26T17:01:50.08+00:00 @Akhila
Apologies for the delay following up here. I uploaded and download .orc files from the examples here] from my ADLS Gen2 container, but was unable to reproduce an issue. Can you verify what browser you are using? Are you able to reproduce this with other files or is it only this singular file? It wasn't mentioned but I am assuming the original encoding is ANSI, please let me know if I am incorrect.
Sign in to comment
Sign in to answer