Share via

Looking for Powershell cmdlet that returns information about the DataFactory "Monitor" functionality for DataFactory V2

Weber, Donald A 21 Reputation points
2021-09-07T16:16:59.89+00:00

I am writing a Powershell script to pause Synapse SQL pool after xx time idle. As part of functionality to determine if the Synapse SQL pool can be paused, I would like to determine which DataFactory pipelines are currently running. When specific DataFactory pipelines are running, I do not want to pause the Synapse SQL pool, even if it has been idle for the specified period of time. It appears that the Get-AzDataFactoryActivityWindow cmdlet provided this function for DataFactory V1, but it does not exists for DataFactory V2.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions
Windows Server PowerShell
Windows Server PowerShell
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.PowerShell: A family of Microsoft task automation and configuration management frameworks consisting of a command-line shell and associated scripting language.
5,462 questions
0 comments
Accepted answer
  1. ShaikMaheer-MSFT 38,401 Reputation points Microsoft Employee
    2021-09-08T06:59:13.11+00:00

    Hi @Weber, Donald A ,

    Welcome to Microsoft Q&A platform. Thank you for posting your query here.

    For your case you can leverage Get-AzDataFactoryV2, Get-AzDataFactoryV2Pipeline & Get-AzDataFactoryV2PipelineRun.

    The Get-AzDataFactoryV2 cmdlet gets information about data factories in an Azure resource group. If you specify the name of a data factory, this cmdlet gets information about that data factory. If you do not specify a name, this cmdlet gets information about all of the data factories in an Azure resource group.

    The Get-AzDataFactoryV2Pipeline cmdlet gets information about pipelines in Azure Data Factory. If you specify the name of a pipeline, this cmdlet gets information about that pipeline. If you do not specify a name, this cmdlet gets information about all the pipelines in the data factory.

    The Get-AzDataFactoryV2PipelineRun command returns information about runs for the specified pipeline. If PipelineRunId is specified, it shows details for the run with that ID. If the PipelineRunId is not specified, then it shows information about all runs for the pipelines that happened between the values of LastUpdatedAfter and LastUpdatedBefore.

    By leveraging all above modules you should be able to develop logic of getting pipeline names and looping through each pipeline to see if any running pipeline there.

    Hope this will help. Please let us know if any further queries. Thank you.

    ---------------------

    • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification.

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Limitless Technology 39,511 Reputation points
    2021-09-08T07:40:53.977+00:00

    Hi @Weber, Donald A ,

    Data Factory only stores pipeline run data for 45 days. When you query programmatically for data about Data Factory pipeline runs - for example, with the PowerShell command Get-AzDataFactoryV2PipelineRun - there are no maximum dates for the optional LastUpdatedAfter and LastUpdatedBefore parameters.

    I understand you are in the need of commands for Programmatically monitoring an Azure data factory , if so you can refer this link
    https://learn.microsoft.com/en-us/azure/data-factory/monitor-programmatically

    Hope this Answers all your queries , if not please do repost back .
    If an Answer is helpful, please click "Accept Answer" and upvote it : )

    0 comments