how can i solve for this error HTTP header that's mandatory for this request is not specified.", 400 in azure synapse pyspark

Question

hey guys how can solve for this error i am using synapse to read data from a blob with

Hierarchical namespace Enabled but when i try to write back i get

: Operation failed: "An HTTP header that's mandatory for this request is not specified.", 400, PUT, https://bussinessblobtut.blob.core.windows.net/config/config_delta/ADF_CONFIG_TABLES/_delta_log?resource=directory&timeout=90, , ""

file_name = 'config_transport_excel.xlsx'

    file_path =f'abfss://config@bussinessblobtut.blob.core.windows.net/config_excel/{file_name}'

    save_path =f'abfss://config@bussinessblobtut.blob.core.windows.net/config_delta/ADF_CONFIG_TABLES'

    pandas_df = pd.read_excel(file_path)

    delta_table = spark.createDataFrame(pandas_df)

    delta_table.write.format('delta').mode('append').save(save_path)

i want ro read the excel and convert it to delta format and save within the same blob

Answer 1

Hi Adika Stadevant Okelo,

Thanks for reaching out to Microsoft Q&A.

Probably this should help...

https://learn.microsoft.com/en-us/answers/questions/1298911/synapse-notebook-error

this is not for synapse but you can verify this(from storage url perspective) as well

https://learn.microsoft.com/en-us/answers/questions/1353560/an-http-header-thats-mandatory-for-this-request-is

check and let me know.

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.