Ryan Abbey 1,181 Reputation points

I am trying to read multiple parquet files and want to add the source file name to the dataframe using Synapse 2.4 cluster, however when adding the column using "input_file_name", the column is empty*sfile).withColumn("input_file_name", F.input_file_name())

Any known issues with this? Any alternative ways to get the filename added (short of a union loop)?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,669 questions
{count} votes