ADF Copy Activity - Missing durationInQueue field in copy activity output

Question

The output from a copy activity is missing the durationInQueue field. The copy activity is running concurrently in a For-each activity fetching data from an on-prem SQL server using a self-hosted integration run-time.

The activity with the missing durationInQueue field had no data to transfer - this has happened previously but not caused any problems. The copy activity is shown as succeeded in the monitoring logs.

Should the durationInQueue field be present as it is part of the standard output?

e.g. output of the copy activity with the missing durationInQueue field. Usually durationInQueue would be at the end of the output.

{
	"dataRead": 0,
	"dataWritten": 123,
	"filesWritten": 1,
	"sourcePeakConnections": 1,
	"sinkPeakConnections": 1,
	"rowsRead": 0,
	"rowsCopied": 0,
	"copyDuration": 8,
	"throughput": 0,
	"errors": [],
	"effectiveIntegrationRuntime": "integrationRuntimeLinked(integrationRuntime01)",
	"billingReference": {
		"activityType": "DataMovement",
		"billableDuration": [
			{
				"meterType": "SelfhostedIR",
				"duration": 0.016666666666666666,
				"unit": "Hours"
			}
		]
	},
	"usedParallelCopies": 1,
	"executionDetails": [
		{
			"source": {
				"type": "SqlServer"
			},
			"sink": {
				"type": "AzureBlobStorage",
				"region": "Australia East"
			},
			"status": "Succeeded",
			"start": "8/6/2023, 11:26:43 AM",
			"duration": 8,
			"usedParallelCopies": 1,
			"profile": {
				"queue": {
					"status": "Completed",
					"duration": 6
				},
				"transfer": {
					"status": "Completed",
					"duration": 1,
					"details": {
						"readingFromSource": {
							"type": "SqlServer",
							"workingDuration": 0,
							"timeToFirstByte": 1
						},
						"writingToSink": {
							"type": "AzureBlobStorage",
							"workingDuration": 0
						}
					}
				}
			},
			"detailedDurations": {
				"queuingDuration": 6,
				"timeToFirstByte": 1,
				"transferDuration": 0
			}
		}
	],
	"dataConsistencyVerification": {
		"VerificationResult": "NotVerified"
	}
}

Answer 1

Hello @JoanConnolly-1945

Welcome to QnA forum,

In your output, the queue field has a duration value of 6, which means that the activity spent 6 seconds waiting in the queue before it started running, is that right number?

I see the property is visible in my Pipeline runs (though the integrationRuntimeQueue is Zero), it is strange to see your Copy Output is missing this property!!!

I really hope you should not face such scenarios. But rest assured, it will be fine on the pipeline side.
Please do contact us with new post on this forum and we are happy to help!

Requesting you to please click "Accept Answer" and Click 'Yes' (Short survey). So that we can close this thread.