infer_schema stopped working on Azure Data Factory new Snowflake connector. Error code 2011

Question

I have been using Azure data factory for copying data from Azure to Snowflake with the Snowflake connector linked service.

It has worked well for 2 years.

Now Azure tells me to update to the new version

I am using the SCRIPT activity and I was using the infer_schema command

select * from table(infer_schema(location=>'@xx_parquet_stage/xxx.parquet',file_format=>'parquet_format' ,ignore_case=>true))

Error:

Error code 2011

Failure type User configuration issue

Details Failed to execute the query command during read operation.

With Snowfake (legacy) connector it works. With the new version does not work.

Could someone please help me ? I don't know what to do.

Thanks in advance

Answer 1

Hello kosmos,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are having issue with infer_schema on Azure Data Factory in the new Snowflake connector because it was stopped working.

This recommended solution was based on the provided scenario and your inquiries, with a strong emphasis on addressing the problem statement. Try to do these tasks as a check list to troubleshoot the problem.

• Utilize a pre-copy script in the Snowflake connector to set the correct context, such as specifying the database with the USE DATABASENAME query.
• If infer_schema continues to fail, you can manually define the schema. This approach will ensure that your data pipeline remains functional despite changes in the connector.

CREATE OR REPLACE TABLE your_table_name (
         column1 TYPE1,
         column2 TYPE2,
         ...
     );
     COPY INTO your_table_name
     FROM @xx_parquet_stage/xxx.parquet
     FILE_FORMAT = (TYPE = 'PARQUET');

Accept Answer

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam