Create a table on serverless database with specified DATA_SOURCE and FILE_FORMAT from PySpark.

Question

Hi Community,

I was trying to create the table with specified DATA_SOURCE and FILE_FORMAT from PySpark. But I was allowed only to create something like this

spark.sql(ddlscript+") USING DELTA partitioned by("\
                                +",".join(partitionby_cols)+") TBLPROPERTIES (delta.enableChangeDataFeed = true) LOCATION '"+silver_path+"'")

what i wanted to create is something like below from pyspark, without using pyodbc:

spark.sql("CREATE EXTERNAL TABLE dbo.table(
	[col1] varchar(400),
	[col2] varchar(400),
	[col3] varchar(400),
	[col4] datetime2(7)
	)
	WITH (
	LOCATION = 'folder/folder1/table',
	DATA_SOURCE = [folder1],
	FILE_FORMAT = [delta]
	)")