An Azure relational database service.
Error when creating view on pipeline (problem with BULK path)
Good morning everybody!
Me and my team managed to create part of a pipeline which selects the database and creates a data source named 'files'.
Now we want to create a view in the same Azure Synapse pipeline using a Script activity. However, this error comes up:
{ "errorCode": "2011", "message": "Content of directory on path '01_landing/test/year=*/month=*/day=*/*.parquet' cannot be listed.", "failureType": "UserError", "target": "CREATE VIEW", "details": [] }
This is our query. If we run it manually on a script in the Develop section everything works smoothly:
CREATE OR ALTER VIEW our_view AS SELECT *, res.filepath(1) AS [year], res.filepath(2) AS [month], res.filepath(3) AS [day] FROM OPENROWSET( BULK '01_landing/test/year=*/month=*/day=*/*.parquet', DATA_SOURCE = 'files', FORMAT='PARQUET' ) AS [res]
Is there another way to pass wildcard characters such as '*' to the BULK path for our pipeline to work correctly?
Thanks in advance!
Azure SQL Database
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.