This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
@Andreas Baumgarten In continuation of my prev query : https://learn.microsoft.com/en-us/answers/questions/1077328/extracting-specific-value-within-string-via-powers.html
Regex works properly when I extract the variable $x output in text file and search for the string in accordance with prev query
but when I leverage the variable directly in Regex scan without outputting it in text file I am unable to get the results :
This is what I was asking as to is it even possible to scan in a variable or is it restricted only to file?
If we can scan variables, than why is there a difference in output between the 2 codes
Hi @Nandan Hegde ,
what happens if you modify line 19 to this:
[array]$x=$PipelineDataSet.Activities
This way $x is an array instead of a string variable.
----------
(If the reply was helpful please don't forget to upvote and/or accept as answer, thank you)
Regards
Andreas Baumgarten
Still no output :)
Hi @Nandan Hegde ,
what is the content of $x in your new script?
----------
(If the reply was helpful please don't forget to upvote and/or accept as answer, thank you)
Regards
Andreas Baumgarten
Hey @Andreas Baumgarten , Thank you :)
257931-tempadf.txt
Have attached the output of variable $x.
The file is created by $x| out-file $TempFilePath
Is this is what you are asking about?
the REST API : https://learn.microsoft.com/en-us/powershell/module/az.datafactory/get-azdatafactoryv2pipeline?view=azps-9.1.0 is the actual source of data
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 74%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERM%3C/text%3E%3C/svg%3E)
The get-azdatafactoryv2pipeline cmdlet returns an object having the type "PSPipeline". The PSPipeline object itself has properties (Activities, ETag, Id, Name, and Parameters). The Activities property has the type System.Collections.Generic.IList and that list contains objects having the type Microsoft.Azure.Management.DataFactory.Models.Activity. Each of those Activity objects has a property called "Name".
When you send the Activities to a file (i.e., Out-File), you're converting the data to a series of strings, each of which is written to a distinct record in the file. You can read that file and examine it using regular expressions.
I don't have the means to try this, but I think all you have to do is this:
$x = $PipelineDataSet.Activities.Name
That should create a variable that will have either the value of the only name in the only activity, or an array of all names in all activities. If you must have an array, just cast the variable $x as "[array]"
Hey @Rich Matheisen
unfortunately Activities.name doesnt work with the ADF REST API output which I have confirmed from MSFT ADF team as below :
https://learn.microsoft.com/en-us/answers/questions/993050/get-the-list-of-activity-names-via-api.html
the activities.name only provides the outer names highlighted in yellow and not all the ones which is highlighted by boundary
So just to confirm, it is not possible to use regex aspect directly via the REST API output variable and we have o physically store it within a file to convert the data to a series of strings and use regex ?
Hi @Nandan Hegde ,
here the following code works as expected and it doesn't matter if its [array]$x or just $x:
[array]$x = @'
WaitTimeInSeconds : 1
AdditionalProperties : {[typeProperties, {
"waitTimeInSeconds": 1
}]}
Name : Wait1
Description :
DependsOn : {}
UserProperties : {}
VariableName : dummy
Value : {value, type}
AdditionalProperties : {[typeProperties, {
"variableName": "dummy",
"value": {
"value": "@pipeline().parameters.Test",
"type": "Expression"
}
}]}
Name : Set variable1
Description :
DependsOn : {Microsoft.Azure.Management.DataFactory.Models.ActivityDependency}
UserProperties : {}
IsSequential :
BatchCount :
Items : Microsoft.Azure.Management.DataFactory.Models.Expression
Activities : {Wait2, Wait3, If Condition2}
AdditionalProperties : {[typeProperties, {
"items": {
"value": "2",
"type": "Expression"
},
"activities": [
{
"name": "Wait2",
"type": "Wait",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
},
{
"name": "Wait3",
"type": "Wait",
"dependsOn": [
{
"activity": "Wait2",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
},
{
"name": "If Condition2",
"type": "IfCondition",
"dependsOn": [
{
"activity": "Wait3",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"expression": {
"value": "@equals(pipeline().parameters.Test,'1')",
"type": "Expression"
},
"ifFalseActivities": [
{
"name": "Wait10",
"type": "Wait",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
},
{
"name": "Wait11",
"type": "Wait",
"dependsOn": [
{
"activity": "Wait10",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
}
],
"ifTrueActivities": [
{
"name": "Wait8",
"type": "Wait",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
},
{
"name": "Wait9",
"type": "Wait",
"dependsOn": [
{
"activity": "Wait8",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
}
]
}
}
]
}]}
Name : ForEach1
Description :
DependsOn : {Microsoft.Azure.Management.DataFactory.Models.ActivityDependency}
UserProperties : {}
Expression : Microsoft.Azure.Management.DataFactory.Models.Expression
IfTrueActivities : {Wait4, Wait6}
IfFalseActivities : {Wait5, Wait7}
AdditionalProperties : {[typeProperties, {
"expression": {
"value": "@equals(pipeline().parameters.Test,'1')",
"type": "Expression"
},
"ifFalseActivities": [
{
"name": "Wait5",
"type": "Wait",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
},
{
"name": "Wait7",
"type": "Wait",
"dependsOn": [
{
"activity": "Wait5",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
}
],
"ifTrueActivities": [
{
"name": "Wait4",
"type": "Wait",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
},
{
"name": "Wait6",
"type": "Wait",
"dependsOn": [
{
"activity": "Wait4",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
}
]
}]}
Name : If Condition1
Description :
DependsOn : {Microsoft.Azure.Management.DataFactory.Models.ActivityDependency}
UserProperties : {}
On : Microsoft.Azure.Management.DataFactory.Models.Expression
Cases : {Microsoft.Azure.Management.DataFactory.Models.SwitchCase}
DefaultActivities : {Wait12}
AdditionalProperties : {[typeProperties, {
"on": {
"value": "@pipeline().parameters.Test",
"type": "Expression"
},
"cases": [
{
"value": "Case1",
"activities": [
{
"name": "Wait13",
"type": "Wait",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
}
]
}
],
"defaultActivities": [
{
"name": "Wait12",
"type": "Wait",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"waitTimeInSeconds": 1
}
}
]
}]}
Name : Switch1
Description :
DependsOn : {Microsoft.Azure.Management.DataFactory.Models.ActivityDependency}
UserProperties : {}
'@
$x.GetType()
$RegexPattern='"name": "(.*?)",'
[array]$InnerActivities = ([regex]::Matches(($x),$RegexPattern).groups|Where-Object Name -EQ '1').Value
$InnerActivities
Result looks like this:
Where do you specify the $RegexPattern in your script?
----------
(If the reply was helpful please don't forget to upvote and/or accept as answer, thank you)
Regards
Andreas Baumgarten
The regex was in his question from yesterday: $Pattern='"name": "(.*?)",'
But I believe the regex is a red herring. I think he should be able to get the names directly from the structured data.
hey @Rich Matheisen and @Andreas Baumgarten : thank you both for your inputs but based on my experience and below thread :https://learn.microsoft.com/en-us/answers/questions/993050/get-the-list-of-activity-names-via-api.html it is not possible to get the required result directly from the REST API.
hence wanted to get a final confirmation from you experts before upgrading my powershell module with file extraction.but wanted 2nd thoughts before publishing it back as the original variable and file get content variable outputs are same but still couldnt get regex from the REST API variable.
Where I used Get-Content below, you can use the REST output (which should contain the same JSON you posted:
$x=(Get-Content C:\junk\Something.json -Raw) | ConvertFrom-JSON
$x.properties.activities.name
The output from the JSON you supplied would be:
Wait1
Set variable1
ForEach1
If Condition1
Switch1
Note the different in the number of "name" properties from the results you get from looking at the entire file as if were unorganized glob of strings. Only the names of "activities" is returned, but not the names of "ifFalseActivities" or "ifTrueActivities" which are children of activities.
Hey @Rich Matheisen
$PipelineDataset = Get-AzDataFactoryV2Pipeline -ResourceGroupName $ResourceGroupName -DataFactoryName $DataFactoryName -Name $PipelineName -ErrorAction Stop
$PipelineDataset.Activities|Out-File $TempFilePath
[array]$InnerActivities=([regex]::Matches((Get-Content $TempFilePath),$RegexPattern).groups|Where-Object Name -EQ '1').Value
[array]$OuterActivities=$PipelineDataset.Activities.Name
$ActivitiesName= $InnerActivities + $OuterActivities
I need the list of all activities, wherein I am already following the methodology to get the outer activities list but its the inner activities which are super nested which I am unable to get directly. So I chose the route of regex.
Hope this helps!!