Data factory COPY activity to Azure synapse

Question

I would like to use the following features to copy data from parquet to Synapse :

Question _1 : It does not support wilcard file ,what if I have multiple parquet files in a folder for a table ?? Now each copy activity can only have one parquet file

Quesion_2 : Using COPY , the command is auto generated in the following :

COPY INTO [dbo].[FACT_sales_from_df_parquet] ([Region] 1,[Country] 2,[Item_Type] 3,[Sales_Channel] 4,[Order_Priority] 5,[Order_Date] 6,[Order_ID] 7,[Ship_Date] 8,[Units_Sold] 9,[Unit_Price] 10,[Unit_Cost] 11,[Total_Revenue] 12,[Total_Cost] 13,[Total_Profit] 14) FROM 'https://****************dfs.core.windows.net:443/ingest/outputtable/day1/sql_dbo_ext_sales_csv_Order_ID_406666631_703333262_1.parquet' WITH (IDENTITY_INSERT='OFF',CREDENTIAL=(IDENTITY='Storage Account Key',SECRET='*'**),FILE_TYPE='PARQUET',COMPRESSION='Snappy') OPTION (LABEL='ADF Activity ID: 3e8b03ad-bd8c-4412-9276-eac79cd90441')

I believe Storage account key can't be used from Synapse to blob storage account , unless the blob storage account has no firewall restriction (allow all network).... unfortunately , most of the customer has their blob storage account enable firewall..... Any idea on how to fix it ?

Answer 1

Hello @sakuraime ,
Thanks for the ask and using the Microsoft Q&A platform .
Please enable the staging option and it should work .

I am sharing my implementation below HTH .

Please do let me know how it goes .
Thanks
Himanshu
Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members