Trying to use a wildcard file path name, and it does not show error even though there is no file

Question

I am trying to use a wild card file name, and it works when there is a file name that meets the wild card condition.

My expression using a wild card is like this:
@markus.bohland@hotmail.de ('Ma_Check_Detail_Art Hre & Rb Center','*_',formatdatetime(utcnow(),'MMddyyyy'),'.csv')

It works fine when we have a file that that meets the condition of the wildcard file path.

But, when I try to create a case by deleting that file, it still passes the test as if there is a file with a blank data.
I was expecting to see an error message like "specified file does not exist".

Bottom is just a screenshot linked service page (with no specified File name path indicated).

But, when I try to indicate the wildcard path name here as well, it shows an error message as "The specified resource name contains invalid characters.".

Where do I have to modify (to get an error message if there is a case that fails the test - like no file exists)?

Answer 1

Hi @Anonymous

Thanks for the ask and using Microsoft Q&A platform.

As i understand the ask here, you want to make the pipeline fail if the specific file doesn't exist. Please correct my understanding if it is wrong.

In order to make the pipeline fail, you can use a look up activity and just read the first row. If the corresponding file doesn't exist, look up will fail. If the file is available, look up will pass and then you can connect your copy activity.

Below is the error we get if the file doesnt exist while we try to lookup.

"ErrorCode=UserErrorFileNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code 'NotFound'. Account: 'Acctname'. FileSystem: 'Input'. Path: 'Sales1.csv'. ErrorCode: 'PathNotFound'. Message: 'The specified path does not exist.'. RequestId: '82b1f76d-c01f-0004-3b03-e6af62000000'. TimeStamp: 'Sat, 22 Oct 2022 10:44:29 GMT'.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'NotFound',Source=Microsoft.DataTransfer.ClientLibrary,'",

Hope it helps. Let us know for any further clarification.

Answer 2

Hi,

What we mean to say is, you need to use a lookup activity and decide whether to copy or not.

In the lookup settings, you need to mention the appropriate file name in the wildcard file path to make it fail.

See below picture. Im using this expression in the wildcard to match files having todays date. If there is no such file, lookup will fail. I think this case should work for you. If not, let us know what expression you use in the copy activity file name portion

@markus.bohland@hotmail.de ('testfile_',formatDateTime(utcnow(),'yyyyMMdd'),'.csv')

If you use a wildcard like testfile_*.csv, then look up wont fail, as it just doesnt find any matching file of the mentioned criteria. See below pic -

Again, if this doesn't suit your requirement, you can use a get meta data activity and use a if condition like below to decide if you need to copy or not based on filename. If file is there True part of if will run, meaning, you can have your copy activity inside that. If no file, False part of if would work where you can place your web activity inside that.

In my case, it ran false part