This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 99%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENH%3C/text%3E%3C/svg%3E)
I have several parquet files in Azure storage.
They all have filenames formatted as schemaName.TableName.Parquet
I need to use expressions to load the files to their destination tables.
For ex:
dbo.Client.parquet file IS LOADED to destination table dbo.client
dbo.Table2.parquet file IS LOADED to destination table dbo.Table2
Stage.Table3.parquet file IS LOADED to destination table stage.Table3.
etc...
I have a Get Metadata activity with child items so I can see all the filenames.
I have a linked service to the destination server/database.
I have a generic dataset for the destination sql table.
Do I need a for each with a copy activity?
How do I tell it to use the output of metadata and load to the destination table?
Your help is much appreciated!
Here is the sample output of get metadata with child items
"childItems": [
{
"name": "ArchiveWork.Table1.parquet",
"type": "File"
},
{
"name": "ArchiveWork.Table2.parquet",
"type": "File"
},
{
"name": "ArchiveWork.Table3.parquet",
"type": "File"
},
Sample output from get metadata
"childItems": [
{
"name": "ArchiveWork.Table1.parquet",
"type": "File"
},
{
"name": "ArchiveWork.Table2.parquet",
"type": "File"
},
{
"name": "ArchiveWork.Table3.parquet",
"type": "File"
},
Hey, Please follow the below process 1) use get metadata activity to get filenames 2) pass the output of get metadata activity child items as input to a for each activity 3) within foreach activity use a copy activity with source as blob with filename as item(). Name and sink as sql with table as item(). Name
Nandan,
I am lost.
After the Get Metadata, I have a For Each with settings/Items set to
@activity('Parquet File Listing').output.childItems
as seen below
There is one Activity in the For each loop
where the source file path has item().name
BUT I am stuck and lost with the sink...
Please advise. Thank you for your help!
For sink, I used the split function to get the Schema name and table name as seen below.
That worked well for me. Thanks!
@Nguyen, Hoa Glad to know that your issue has been resolved. And thanks for sharing the solution, which might be beneficial to other community members reading this thread. If you wish, you may re-surveying/rating for the engagement you received on the thread. Your feedback is very important to us.