Error when creating view on pipeline (problem with BULK path)
Good morning everybody!
Me and my team managed to create part of a pipeline which selects the database and creates a data source named 'files'.
Now we want to create a view in the same Azure Synapse pipeline using a Script activity. However, this error comes up:
{ "errorCode": "2011", "message": "Content of directory on path '01_landing/test/year=*/month=*/day=*/*.parquet' cannot be listed.", "failureType": "UserError", "target": "CREATE VIEW", "details": [] }
This is our query. If we run it manually on a script in the Develop section everything works smoothly:
CREATE OR ALTER VIEW our_view AS SELECT *, res.filepath(1) AS [year], res.filepath(2) AS [month], res.filepath(3) AS [day] FROM OPENROWSET( BULK '01_landing/test/year=*/month=*/day=*/*.parquet', DATA_SOURCE = 'files', FORMAT='PARQUET' ) AS [res]
Is there another way to pass wildcard characters such as '*' to the BULK path for our pipeline to work correctly?
Thanks in advance!