AnalysisException: Incompatible format detected in Azure Data Bricks

Rohit Kulkarni 676 Reputation points

Hello Team,

I am trying to write the files from salesforce data to blob storage in parquet format. The query is mentioned below :


set the data lake file location to save the delta file out of spark dataframe df:

.option("mergeSchema", "true") #append columns in case of adding new ones from the source and adding data to empty columns already existing in the dataframe

When I included the CURRENT_MONTH_NAME in folder structure then i am getting error :

AnalysisException: Incompatible format detected.

AnalysisException Traceback (most recent call last)
<command-1746602509716500> in <module>
13 print(file_location)
14 (
---> 15 df
16 .write
17 .format("delta")

/databricks/spark/python/pyspark/sql/ in save(self, path, format, mode, partitionBy, **options)
739 else:
--> 740
742 @since(1.4)

/databricks/spark/python/lib/ in call(self, *args)
1303 answer = self.gateway_client.send_command(command)
-> 1304 return_value = get_return_value(
1305 answer, self.gateway_client, self.target_id,

/databricks/spark/python/pyspark/sql/ in deco(*a, **kw)
121 # Hide where the exception came from that shows a non-Pythonic
122 # JVM exception message.
--> 123 raise converted from None
124 else:
125 raise

AnalysisException: Incompatible format detected.

You are trying to write to abfss:// using Databricks Delta, but there is no
transaction log present. Check the upstream job to make sure that it is writing
using format("delta") and that you are trying to write to the table base path.

To disable this check, SET
To learn more about Delta, see

Please advise.


Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,596 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 65,986 Reputation points Microsoft Employee

    Hello @Rohit Kulkarni ,

    Thanks for the question and using MS Q&A platform.

    You are experiencing this error message is because you have files saved into that path already that is not in delta format, so you should choose a new path or delete files in that path.

    For more details, refer to the SO thread addressing similar issue: Trouble when writing the data to Delta Lake in Azure databricks (Incompatible format detected).

    Hope this will help. Please let us know if any further queries.


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is howd
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments No comments