How to adjust server timeout for Azure DevOps Pipelines PublishToADX Task

Yannik 0 Reputation points
2023-10-19T09:52:02.1466667+00:00

I am using the Azure DevOps Pipelines PublishToADX Task to trigger hourly long-running append operations on my ADX cluster. However, the .append command times out after 4 minutes and I cannot find a way to adjust the servertimeout setting.

My current approach is to use .append async with waitForOperation set to true. This works most of the time, but sometimes the .show operation fails, causing the task to fail and the depending tasks to not be triggered.

Is there a way to adjust the servertimeout setting as described in this documentation for the CLI, SDK, etc.: https://learn.microsoft.com/en-us/azure/data-explorer/set-timeout-limits? Or do you have any other ideas for fixing this issue?

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
487 questions
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,126 Reputation points Microsoft Employee
    2023-10-20T17:21:38.29+00:00

    Hi Yannik,

    Thank you for posting query in Microsoft Q&A Platform.

    can you try to adjust the servertimeout setting for the .append command, you can use the .set command to set the timeout value.

    Here's an example:

    .set servertimeout 10m
    

    This sets the timeout value to 10 minutes. You can adjust the value as per your requirement.

    Alternatively, you can try using the .append async command with a longer timeout value and handle the .show operation failure gracefully. You can use a try-except block to catch the exception and retry the .show operation after a certain interval. Here's an example:

    let operationId = .append async MyTable <| MyData;
    let result = waitForCompletion(operationId, 10m);
    if (result == "Completed") {
        .show MyTable
    } else {
        let retryCount = 0;
        let maxRetryCount = 3;
        let retryInterval = 1m;
        while (retryCount < maxRetryCount) {
            retryCount += 1;
            try {
                .show MyTable
                break;
            } catch (ex) {
                print ex;
                print "Retrying in " + tostring(retryInterval) + " minutes...";
                sleep(retryInterval);
            }
        }
    }
    

    This code retries the .show operation up to 3 times with a 1-minute interval between retries. You can adjust the retry count and interval as per your requirement.

    I hope this helps! Please let me know how it goes. Thank you.

    0 comments No comments