Partition dimension of parquet files copied in hierarchical folders respect to year, month and date
pmscorca
1,052
Reputation points
Hi,
I've implemented a Synapse mapping data flow, called by a pipeline, to copy source parquet files (NYC green taxi data) into a hierarchical folders, partitioning for year, month and date keys:
At the leaf level of these hierarchical folders I've noticed that some partitions were created, but is it possible to change the dimension or the number of these partitions maintaning their related distribution in the year-->month-->date folders? Thanks
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,385 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,672 questions
Sign in to answer