External Table in Dedicated Pool using Parquet

Satwik Kashyap 11 Reputation points


  1. Copy Data Pipeline to get data from SAP to Azure Data Lake as parquet files (Files are partitioned)
  2. Reading that file as an external table using Synapse, I am getting 'Failed to detect Schema'


Error details
New external table
Previewing the file data failed. Details: Error: Conversion overflows.
If the issue persists, contact support and provide the following id : 39afb7bf-6726-4bbd-8a49-303a07cb5e40

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,191 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
3,813 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
8,483 questions
{count} vote

2 answers

Sort by: Most helpful
  1. Pratik Somaiya 4,126 Reputation points

    Hello @Satwik Kashyap

    Can you try creating external table from SSMS using the following link: https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/develop-tables-external-tables?tabs=hadoop

    You will have to create following:

    1) Database Scoped Credential
    2) External Data Source
    3) External File Format
    4) External Table Query

    In your case, I think there is a data type issue between data present in file and column data type in external table

    2 people found this answer helpful.

  2. Satwik Kashyap 11 Reputation points

    @Pratik Somaiya

    The folder has parquet partitioned files. I am using the UI option where I am right-clicking on the folder and the below options appear to create an external table. When I click on it then it failes with the above error.