How to define schema when copying Excel file with several sheet with different schema

Mohammad Saber 20 Reputation points
2025-02-14T04:00:13.2633333+00:00

Hi,

I am going to use a Copy activity in ADF to copy excel files from ADLS to Synapse Data Warehouse.

Excel file has several sheets with different column names and schema. A list of excel sheets are passed to the ADF pipeline as a parameter to be processed.

How can I define different Schemas in Dataset object in ADF in this scenario?

Screenshot 2025-02-14 at 2.50.31 PM


Background:

When I trigger Copy activity without defining or mapping Schema, I encounter the below error:

ErrorCode=DelimitedTextColumnNameNotAllowNull,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The name of column index 1 is empty. Make sure column name is properly specified in the header row.,Source=Microsoft.DataTransfer.Common,'

I searched about this error and checked the source file:

  • I have checked all columns in the excel file. All columns have names. There is no empty cell in the 1st row of the excel file. There is no comma, space, or special character in column names.
  • Dataset used as Source in the Copy activity is Excel (not CSV). 
  • "First row as header" option is enabled. 
  • When I use "Preview data" option, I can get a sample of data in ADF, which looks fine. All column names are similar to the source file. 
  • Dataset object allows us to define Schema. When I import schema for every excel sheet, it looks fine. But, the problem is there are several sheets in the excel with different schema. 

As a potential solution, I am going to define schema for the excel sheets given that every excel sheet has a different schema.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.