How to adjust server timeout for Azure DevOps Pipelines PublishToADX Task

Question

I am using the Azure DevOps Pipelines PublishToADX Task to trigger hourly long-running append operations on my ADX cluster. However, the .append command times out after 4 minutes and I cannot find a way to adjust the servertimeout setting.

My current approach is to use .append async with waitForOperation set to true. This works most of the time, but sometimes the .show operation fails, causing the task to fail and the depending tasks to not be triggered.

Is there a way to adjust the servertimeout setting as described in this documentation for the CLI, SDK, etc.: https://learn.microsoft.com/en-us/azure/data-explorer/set-timeout-limits? Or do you have any other ideas for fixing this issue?

Answer 1

Hi Yannik,

Thank you for posting query in Microsoft Q&A Platform.

can you try to adjust the servertimeout setting for the .append command, you can use the .set command to set the timeout value.

Here's an example:

.set servertimeout 10m

This sets the timeout value to 10 minutes. You can adjust the value as per your requirement.

Alternatively, you can try using the .append async command with a longer timeout value and handle the .show operation failure gracefully. You can use a try-except block to catch the exception and retry the .show operation after a certain interval. Here's an example:

let operationId = .append async MyTable <| MyData;
let result = waitForCompletion(operationId, 10m);
if (result == "Completed") {
    .show MyTable
} else {
    let retryCount = 0;
    let maxRetryCount = 3;
    let retryInterval = 1m;
    while (retryCount < maxRetryCount) {
        retryCount += 1;
        try {
            .show MyTable
            break;
        } catch (ex) {
            print ex;
            print "Retrying in " + tostring(retryInterval) + " minutes...";
            sleep(retryInterval);
        }
    }
}

This code retries the .show operation up to 3 times with a 1-minute interval between retries. You can adjust the retry count and interval as per your requirement.

I hope this helps! Please let me know how it goes. Thank you.