ADF Error logs in Storage are empty

Question

ADF Error logs in Storage are empty

Anshulee Asthana 0

I have enabled Error Mapping in my Postgressql sink to log the error rows in storage. However the files are always empty ( even when the assert HAS failed)

User's image

Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-08T20:04:59.8266667+00:00

Hello Anshulee Asthana,

I am checking to see if you got a chance to look into my above response. Please let me know if you have any further questions.
Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-10T00:20:44.5466667+00:00

Hello Anshulee Asthana,

I am checking to see if you have any further questions here.
Anshulee Asthana 0 Reputation points

2023-08-10T01:34:51.8633333+00:00

Hi , I checked and even when the pipeline succeeds the file is empty (and there are error rows seen in the data studio). Wrt write permissions i am not sure what specific permissions i am looking for...the file did get created so I assume write permissions are there...its just that the file is empty.
Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-10T18:16:29.9866667+00:00

Hello Anshulee Asthana,

When the pipeline succeeds, it won't write anything to the storage. Can you please check if the pipeline is failing before it reaches the PostgresSQL sink? If it fails before then, it may be unable to write the error rows to the storage.
Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-14T16:15:51.3033333+00:00

Hello Anshulee Asthana,

I am checking to see if you have any further questions here.

1 answer

Your answer

Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-08T20:04:59.8266667+00:00

Hello Anshulee Asthana,

I am checking to see if you got a chance to look into my above response. Please let me know if you have any further questions.
Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-10T00:20:44.5466667+00:00

Hello Anshulee Asthana,

I am checking to see if you have any further questions here.
Anshulee Asthana 0 Reputation points

2023-08-10T01:34:51.8633333+00:00

Hi , I checked and even when the pipeline succeeds the file is empty (and there are error rows seen in the data studio). Wrt write permissions i am not sure what specific permissions i am looking for...the file did get created so I assume write permissions are there...its just that the file is empty.
Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-10T18:16:29.9866667+00:00

Hello Anshulee Asthana,

When the pipeline succeeds, it won't write anything to the storage. Can you please check if the pipeline is failing before it reaches the PostgresSQL sink? If it fails before then, it may be unable to write the error rows to the storage.
Bhargava-MSFT 31,246 Reputation points Microsoft Employee

2023-08-14T16:15:51.3033333+00:00

Hello Anshulee Asthana,

I am checking to see if you have any further questions here.

Answer 1

Bhargava-MSFT 31,246 Microsoft Employee

Hello Anshulee Asthana,

Welcome to the Microsoft Q&A forum.

Please check if the pipeline is failing before it reaches the PostgresSQL sink. If it fails before then, it may be unable to write the error rows to the storage.

Also, please check the error output dataset is configured correctly and has the correct permissions to write the data.

I hope this helps. Please let me know if you have any further questions.

Anshulee Asthana 0 Reputation points

2023-09-05T06:57:14.4533333+00:00

The problem got resolved by putting an assert description. With Assert description the error logs are not saved.

Share via

ADF Error logs in Storage are empty

1 answer

Your answer