How to get azure devops pipeline status logs on run time.

Question

I am currently working on a project that involves retrieving logs for each stage of an Azure DevOps pipeline in real time. However, I have discovered that there are no available REST APIs that provide logs during the execution of a stage while it is still in progress. The existing APIs only provide logs once a particular stage, such as "init," "plan," or "apply," has been completed.

My objective is to retrieve and stream the logs in real-time, similar to how they are displayed on the Azure DevOps pipeline status page. I am seeking to determine if Azure offers any services or APIs that can fulfil this requirement by providing logs during the runtime of a pipeline stage.

I would greatly appreciate any guidance or suggestions on how to achieve this capability within the Azure ecosystem.
Thank you for your assistance in this matter.

Answer 1

Hi @Aakash Goswami

The services provide log management and monitoring capabilities that can be integrated with Azure DevOps to achieve your objective. Here's an approach to consider:

Enable Diagnostics: Configure your Azure DevOps pipeline to emit detailed logs and diagnostic information during its execution. This can include console output, diagnostic traces, or custom log messages that capture the relevant information you need.

Send Logs to Azure Monitor: Use Azure Monitor to collect and ingest the logs emitted by your pipeline. You can leverage Azure Monitor's Log Analytics workspace to store and query the log data.

Stream Logs in Real-Time: Utilize Azure Log Analytics Streaming feature to stream the logs in real-time. By setting up a streaming query, you can continuously retrieve and display the logs as they are generated during the pipeline execution.

1. Visualize Logs: Use Azure Monitor Workbooks or other visualization tools to create customized dashboards and visualizations that display the real-time logs in a format similar to the Azure DevOps pipeline status page. This allows you to monitor the progress of the pipeline and view the logs as they are being generated.

if you find this useful kindly accept answer, for any assistance ping here, thanks much