How to check if the content of a file is empty or not?

Question

How to check if the content of a file is empty or not?

NishimuraChinatsu-9854 756

If the file is empty, I want to proceed to the delete activity; if not empty, I want to proceed to the next activity. I would like to segregate the empty data files, as they are always created. User's image

If you have any suggestions on how to do this, please let me know.

NishimuraChinatsu-9854 756 Reputation points

2023-04-11T02:08:49.1933333+00:00

I want to take file names dynamically, I can't get it to work if I use the field list as column names.

Please let me know how to solve this problem.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-04-11T22:41:34.4166667+00:00
To handle file names dynamically, you can use ForEach and GetMetadata activities. Please see the below steps:

Use a GetMetadata activity to list all the files in the folder. Set the Field list to childItems.

Add a ForEach activity to iterate through the list of files obtained from the previous step. Set the Items expression to @activity('Get Metadata2').output.childItems.

Inside the ForEach activity, add another GetMetadata activity to get the number of rows for each file. Set the Field list to column Count. Use the dynamic file name from the ForEach activity as the input for the file path.

Add an If Condition activity inside the ForEach loop to check if the number of rows is zero. Set the expression to @equals(activity('Get Metadata3').output.columnCount, 0).

If the count is zero, proceed to the Delete activity.

If the count is not zero, proceed to the next activity.

I hope this helps.
NishimuraChinatsu-9854 756 Reputation points

2023-04-12T00:23:26.17+00:00

Thank you for your reply. I already have this action in foreach,,,, Is there any other way to do this other than using excute? Each time the file to be processed is one file with the same data structure, but the file name changes each time.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-04-12T21:02:14.4533333+00:00
NishimuraChinatsu-9854 If you have only one file to process each time and the file name changes, you can simplify the process by using a single Get Metadata activity and an If Condition activity without the need for a ForEach loop as mentioned in my first reply.

To handle the changing file name, you can use a pipeline parameter for the file name and pass it to the dataset. This way, the pipeline will work with any file name as long as it's provided as a parameter when the pipeline is executed. for this you can follow the below steps:

In your pipeline, create a new parameter called fileName.

In your dataset, create a new parameter called fileName as well. In the dataset's connection properties, use the fileName parameter to dynamically set the file name.

In the Get Metadata activity, set the dataset's fileName parameter to the pipeline's fileName parameter

When executing the pipeline, pass the file name as a parameter:

I hope this helps. Please let me know if you have any further questions.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-04-13T18:35:12.61+00:00

NishimuraChinatsu-9854 I am checking to see if the above solution is helpful. Please let me know if you have any further questions.

1 answer

Your answer

NishimuraChinatsu-9854 756 Reputation points

2023-04-11T02:08:49.1933333+00:00

I want to take file names dynamically, I can't get it to work if I use the field list as column names.

Please let me know how to solve this problem.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-04-11T22:41:34.4166667+00:00

To handle file names dynamically, you can use ForEach and GetMetadata activities. Please see the below steps:

Use a GetMetadata activity to list all the files in the folder. Set the Field list to childItems.

Add a ForEach activity to iterate through the list of files obtained from the previous step. Set the Items expression to @activity('Get Metadata2').output.childItems.

Inside the ForEach activity, add another GetMetadata activity to get the number of rows for each file. Set the Field list to column Count. Use the dynamic file name from the ForEach activity as the input for the file path.

Add an If Condition activity inside the ForEach loop to check if the number of rows is zero. Set the expression to @equals(activity('Get Metadata3').output.columnCount, 0).

If the count is zero, proceed to the Delete activity.

If the count is not zero, proceed to the next activity.

I hope this helps.
NishimuraChinatsu-9854 756 Reputation points

2023-04-12T00:23:26.17+00:00

Thank you for your reply. I already have this action in foreach,,,, Is there any other way to do this other than using excute? Each time the file to be processed is one file with the same data structure, but the file name changes each time.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-04-12T21:02:14.4533333+00:00

NishimuraChinatsu-9854 If you have only one file to process each time and the file name changes, you can simplify the process by using a single Get Metadata activity and an If Condition activity without the need for a ForEach loop as mentioned in my first reply.

To handle the changing file name, you can use a pipeline parameter for the file name and pass it to the dataset. This way, the pipeline will work with any file name as long as it's provided as a parameter when the pipeline is executed. for this you can follow the below steps:

In your pipeline, create a new parameter called fileName.

In your dataset, create a new parameter called fileName as well. In the dataset's connection properties, use the fileName parameter to dynamically set the file name.

In the Get Metadata activity, set the dataset's fileName parameter to the pipeline's fileName parameter

When executing the pipeline, pass the file name as a parameter:

I hope this helps. Please let me know if you have any further questions.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-04-13T18:35:12.61+00:00

NishimuraChinatsu-9854 I am checking to see if the above solution is helpful. Please let me know if you have any further questions.

Answer 1

Hello NishimuraChinatsu-9854, You can achieve this by GetMetadata activity to get the number of rows in the file. If the number of rows is zero, then the file is empty. Please see the below steps:

add Get Metadata activity to get the number of rows in the file.

Field list - Column count

If Condition activity to check if the number of rows is zero.

Expression: @equals(activity('Get Metadata1').output.count, 0)

If the number of rows is zero, proceed to the Delete activity.
If the number of rows is not zero, proceed to the next activity.

To segregate the empty data files, you can use the Filter activity to filter out the files with zero rows and move them to a separate folder.

Get Metadata activity to get the number of rows in the file.
Filter activity to filter out the files with zero rows.
Copy activity to copy the filtered files to a separate folder.
Delete activity to delete the filtered files from the original folder.

User's image

Share via

How to check if the content of a file is empty or not?

1 answer

Your answer