Azure Synapse Data Factory - Copy Data from Web Table Timeout

Matt Werner 6

I am looking to copy data from an html web table. To do this, I am using a Web Table dataset as my source in the Data Factory copy data step. The issue is that the website I'm pulling from is sometimes slow to respond. This is causing the occasional timeout error in the copy data step of my pipeline. The error message received is:

"Microsoft.Data.Mashup.MashupValueException,Message=The Web.Page function didn't finish within timeout of 100 seconds."

Is there a way to increase the default timeout of 100 seconds on the Web Table source of the copy step? Thanks in advance!

AnnuKumari-MSFT 31,146 Reputation points Microsoft Employee

2022-11-30T06:52:39.06+00:00

Hi @Matt Werner ,

Welcome to Microsoft Q&A platform and thanks for posting your question here.

As I understand your query, you are trying to copy data from web table source. However, it is failing due to timeout issue at the source.

If the web page you are trying to hit is a public web page, could you please provide the URL for the same so that we could replicate your issue? By default copy activity has timeout of 12 hours.

The issue seems to be at the source server side. It might be an intermittent issue, if it doesn't get resolved for you, I would recommend you to create a support ticket with Microsoft so that our product team can directly engage with you and provide solution after deeper investigation. Here is how you can do that: Create an Azure support request
AnnuKumari-MSFT 31,146 Reputation points Microsoft Employee

2022-12-05T05:52:26.343+00:00

Hi @Matt Werner ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, please raise the support ticket as suggested above.
Matt Werner 6 Reputation points

2022-12-06T00:48:20.98+00:00

Apologies on the gap in response. It does, in fact, look like this was a timeout issue on the source side. I will be doing more testing with my pipeline over the next several days. Should I encounter the issue again, I'll follow up here and create a ticket as you suggested.