if column missing then static header

Shambhu Rai 1,411 Reputation points
2024-01-24T16:57:26.2533333+00:00

Hi Expert,

I am loading the data  from excel sheet to databricks and 
create table1 (col1, char(22),col3, char(22),col2, char(22)
 but excel sheet has 2 columns :col1,col2

how we can consider blank value for missing column in excel sheet

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,325 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Adharsh Santhanam 5,790 Reputation points
    2024-01-24T19:27:53.5833333+00:00

    If I understood your question correctly, you're trying to import an Excel containing 2 columns into Azure Databricks which is expecting 3 columns. In this scenario, if your Excel file has fewer columns than you are expecting, you may encounter an error when you try to read the file into Azure Databricks. The error message will depend on the specific method you are using to read the file, but it will likely indicate that the file has fewer columns than expected. One possible way to resolve this issue is to import the Excel containing 2 columns as a dataframe with 2 columns and then using the withColumn() to add the additional column to the dataframe. Your statement would be like this: df = df.withColumn("new_column", lit(None).cast("string")) Alternatively, you can add the additional column in the Excel before importing it.

    If the Answer is helpful, please click "Accept Answer" and upvote it.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.