Options to write to Parquet format in Azure Data Flows

Krishnamohan Nadimpalli 406 Reputation points
2022-05-05T05:40:29.73+00:00

What options should I specify if I am writing to Parquet format in Azure Data Flows. Specified name of file in settings of Sink(Pattern --> filename.parquet)

Compression type is Snappy as per documentation

learn.microsoft.com/en-us/azure/data-factory/format-parquet#dataset-properties

but what about compression level, Delimiter (column and row). As far as I know, all these should not be popping up when I choose Snappy and Parquet formats.

199076-image.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,646 questions
{count} votes

1 answer

Sort by: Most helpful
  1. HimanshuSinha-msft 19,491 Reputation points Microsoft Employee Moderator
    2022-05-06T20:02:14.043+00:00

    Hello @Krishnamohan Nadimpalli ,
    Thanks for the question and using MS Q&A platform.

    As we understand the ask here is do we need compression level, Delimiter (column and row) , when dealing the snappy compression , please do let us know if its not accurate.
    I agree with you that we may not be in need all the other options when yiou select snappy . I am reaching out to the internal team on this .

    Please do let me if you have any queries.
    Thanks
    Himanshu


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
      • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.