Azure Data Factory - Schedule Trigger recurrence in seconds

Question

Respected,

Need to create a Schedule Trigger that needs to run every 10 seconds.

There is no option for seconds, is there a way to achieve this in ADF or please could you suggest any alternative to achieve this?

Regards

Answer 1

Hello @jigsm ,
Thanks for the ask and using the Microsoft Q&A platform .

Type in 10 in the text box and it should work fine .

Trigger runs

Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

Answer 2

@jigsm

HI,

Before Continuing: Use Powershell Core v7 or above. You can run the same in VS Code

I am adding few things to get started. let me know if that works else, I will update my code and add to my git.

You have 2 files from ADF branch (not the master):

• ARMtemplate.json
• parameters.json

Read the file
$armFile = Get-ChildItem -Path $rootPath -Recurse -Filter "ARMTemplateForFactory.json" | SELECT -First 1

------ > create an empty ARM Template: ARMTemplate_wReplacedValues.json
$armFileWithReplacedValues = $armFile.FullName.Replace($armFile.Name, "ARMTemplate_wReplacedValues.json")

$json = Get-Content $armFile.FullName -raw | ConvertFrom-Json

------ > You need to select the relevant property from $json parameter.

foreach ($n in $json.parameters) {
$n | Add-Member -Type NoteProperty -Name 'updatedValue' -Value @{
type= "String"
defaultValue= $valuefromparamtersfile
}
}

------ > Use this to get the exact field from the json : $n.properties.typeProperties --> can give you exact info about your time. Select and then modify accordingly.

foreach ($n in $json.resources){
$json.update | % { if (<add condition >){
$n.properties.typeProperties = '[parameters(''valuefromparamtersfile'')]'
}
}
}

$json | ConvertTo-Json -depth 100 | Out-file $armFileWithReplacedValues

Try it locally first, before working with build pipeline. Secondly, try to create the logic, if you have problems, let me know.

Let me know If that works.

Regards
Pranay

Answer 3

ADF is currently optimized for big data analysis in batch, is your data processing flow related to streaming processing?