##[error]Agent failed with exception: Task agent exited with exit code 137 ,##[warning]Received request to deprovision: The request was cancelled by the remote provider.

Question

Pipeline got terminated due to below error from Azure Devops after 48 hrs

##[error]Agent failed with exception: Task agent exited with exit code 137 ,##[warning]Received request to deprovision: The request was cancelled by the remote provider.

Note: timeout already set in the pipeline more than 3 days

Answer 1

The error message you are encountering, specifically the exit code 137, typically indicates that the task agent was terminated due to an out-of-memory (OOM) condition. This can happen if the agent runs out of available memory while executing a task, leading to the operating system killing the process to free up resources.

Since you mentioned that the timeout is set for more than 3 days, it’s important to ensure that the agent has sufficient resources allocated to handle the workload. Here are a few steps you can take to troubleshoot and potentially resolve the issue:

1. Check Resource Allocation: Ensure that your self-hosted agent has enough CPU and memory resources available. If it's running on a virtual machine, consider resizing it to a larger instance type.
2. Enable Verbose Logging: Enable verbose logs for your pipeline to get more detailed information about resource utilization. This can help identify if a specific task is consuming excessive resources.
3. Optimize Tasks: Review the tasks in your pipeline to see if there are any that can be optimized to use less memory or run more efficiently.
4. Monitor Resource Usage: Use monitoring tools to observe the resource usage of your agent during the pipeline execution to identify any spikes or patterns that may lead to the OOM condition.

If the problem persists, consider reaching out to Azure DevOps support for further assistance.

---

References:

• Troubleshoot pipeline runs
• Azure Pipelines agents

Answer 2

Hi @Sivangnanam, Kuraliniyan (AWM, IRL) ,

Thanks for reaching out to Microsoft Q&A.

##[error]Agent failed with exception: Task agent exited with exit code 137,##[warning]Received request to deprovision: The request was cancelled by the remote provider.

Possible Causes of the error above:

1. Exit code 137 usually means the process was killed due to exceeding memory limits. When the agent's available memory gets depleted, especially on Microsoft-hosted agents, it may forcefully terminate processes to reclaim resources.
2. You mentioned setting a timeout of more than three days in your pipeline, but there may be other settings or limits at play that could be causing the pipeline to terminate unexpectedly.
3. If you're using Azure Data Bricks or similar services that use cluster autoscaling, the job could be cancelled due to scaling decisions. Ensure that your cluster settings are optimized for long-running jobs.

Follow below steps to resolve the issue:

1. If your pipeline runs resource-intensive tasks, consider switching to self-hosted agents that can be provisioned with more powerful hardware tailored to your project's requirements. More on self-hosted agents here.
2. To get a clearer picture of the issue, enable verbose logs for your pipeline. This will give you in-depth information about memory and resource usage, helping you identify what might be causing the termination.
3. For long-running jobs, use dedicated clusters to avoid issues with autoscaling that might interrupt your workload.
4. Ensure that no external processes or configurations like autoscaling settings are canceling your job unintentionally.

Hope it helps!

---

Please do not forget to click "Accept the answer” and Yes, this can be beneficial to other community members.

If you have any other questions, let me know in the "comments" and I would be happy to help you.