Error while Import ADLSGEN2 storage to SQL server using ADF

Dilip Kumar Agnihotra 1 Reputation point Microsoft Vendor
2023-03-02T21:45:13.0366667+00:00

When we import data from ADLS GEN2 to SQL server using ADF . Linked server connecting data source successfully but are getting below error importing schema.

Parquet file contained column 'txn', which is of a non-primitive, unsupported type. Activity ID: e3e32a29-0e50-4adc-ab66-465b8b4a0d51

is there any way we can ignore column 'txn'

Azure SQL Database
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,335 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,478 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. AnnuKumari-MSFT 30,361 Reputation points Microsoft Employee
    2023-03-06T10:27:41.0866667+00:00

    @Dilip Kumar Agnihotra ,

    Welcome to Microsoft Q&A platform and thanks for posting your question here .

    As I understand your issue, you are facing this error: "Parquet file contained column 'txn', which is of a non-primitive, unsupported type." while copying data from sql server to adls gen2.

    The error message you received indicates that the Parquet file you are trying to use contains a column named 'txn' which is of a non-primitive, unsupported type.

    According to the documentation, complex data types such as MAP, LIST, and UNION are not supported in Parquet format. You may need to modify the data in the column 'txn' to a supported type before you can use it. In order to modify the datatype of the file, you can leverage Cast Transformation in Mapping dataflow.

    For more details, kindly check this documentation:

    Data type support

    Data type mapping for Parquet files

    Hope it helps. Kindly accept the answer and mark it as helpful.