Azure Data Factory pipelines - Error 5000 - Error encountered during attempt 1 to start job actor

Question

Hello,

Recently we have been encountering a blocking issue with azure data factory pipelines. Until some weeks ago, the same pipeline used to run without issues, but recently, we are receiving this error:

Error encountered during attempt 1 to start job actor. Exception: [System.TimeoutException: This can happen if message is dropped when service is busy or its long running operation and taking more time than configured Operation Timeout.
at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient1.InvokeWithRetryAsyncTResult
at Microsoft.ServiceFabric.Services.Remoting.V2.Client.ServiceRemotingPartitionClient.InvokeAsync(IServiceRemotingRequestMessage remotingRequestMessage, String methodName, CancellationToken cancellationToken)
at Microsoft.ServiceFabric.Services.Remoting.Builder.ProxyBase.InvokeAsyncV2(Int32 interfaceId, Int32 methodId, String methodName, IServiceRemotingRequestMessageBody requestMsgBodyValue, CancellationToken cancellationToken)
at Microsoft.Marlin.JobService.QueueService.SfJobQueue.StartJobActorAsync(JobState state, String traceId, CancellationToken token) in C:\source\JobService\Microsoft.Marlin.JobService.QueueService\Queue\SfJobQueue.cs:line 3172]

Any idea of what the cause might be? Is there some timeout configuration that we should look at?

Thank you.

Best,
Simone

Answer 1

Hello and welcome to Microsoft Q&A @Simone

I see you are getting an unusual error message. By the sound of the error message, it isn't likely caused by something you did. That's my first impression anyway. Are you getting this error every time? Or did this just happen once.
I'd expect this to be a one off where re-try would help.

At what point did this error occur? Did this happen on a specific activity, or did the pipeline not even start?
What Integration Runtime(s) are involved? Is it all AutoResolve or is there SHIR invovled?