Azure sql table to Snowflake using ADF - get error on straight mapping

Question

Trying to create a pipline dataflow for tables from Azure SQL to Snowflake using the Snowflake connecter and Copy data Activity? Can this be done?
How? link? without json, functions, etc...

Straight mapping is failing with this error:
Direct copying data to Snowflake is only supported when source dataset is DelimitedText, Parquet, JSON with Azure Blob Storage or Amazon S3 linked service, for other dataset or linked service, please enable staging.

Thanks!

Answer 1

Hello @Sword55 ,
Apologizes for the delay in response . The error which are getting is below

Direct copying data from Snowflake is only supported when sink dataset is DelimitedText, Parquet or JSON with Azure Blob Storage linked service, for other dataset or linked service, please enable staging.

This is due to the fact that at this time the SF connector supports only DelimitedText, Parquet or JSON . This is called out the in the doc here .

*“The sink data format is of Parquet, delimited text, or JSON with the following configurations:
• For Parquet format, the compression codec is None, Snappy, or Lzo.
• For delimited text format:
o rowDelimiter is \r\n, or any single character.
o compression can be no compression, gzip, bzip2, or deflate.
o encodingName is left as default or set to utf-8.
o quoteChar is double quote, single quote, or empty string (no quote char).
• For JSON format, direct copy only supports the case that source Snowflake table or query result only has single column and the data type of this column is VARIANT, OBJECT, or ARRAY.
o compression can be no compression, gzip, bzip2, or deflate.
o encodingName is left as default or set to utf-8.
o filePattern in copy activity sink is left as default or set to setOfObjects.“*

There is a work around you can use the “enabling Staging” option and that should do the trick .

Thanks Himanshu

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members