R script taking too long to start execution when triggered thru Devops pipeline

Question

I am running a Rscript which interns calls another Rscript using Azure devops self hosted environment.

Sometimes I notice that the main Rscript is taking too long to start its execution. The main R script is used to call 2 other RScripts based on the input parameters.

In Devops agent, the trigger is as below

"C://Program Files/R../Rscript.exe" Script.R $(parameter)

further my Rscript.R contains the following code

if(parameter=0){
system2(paste0(R.home("bin"), "/R"), args=paste("CMD BATCH", paste0(getwd(),"/01Script.R")))
}
else{
system2(paste0(R.home("bin"), "/R"), args=paste("CMD BATCH", paste0(getwd(),"/02Script.R")))
}

Everything is working fine, and I am able to get the desired output. But the scripts take way too long to get itself started. When I printed a Sys.time() at the beginning of each script I notice that sometimes my Rscript takes 10 minutes to start execution. Same goes to 01 or 02 script. I have more than 100 GB RAM capacity available and more than 60% CPU available.

I checked in task manager and was able to see 2 new Rscript.exe from the Azure devops agent running. Not sure why the script takes 10-15 minutes to start its execution.

There is no other Rscript trigger running from my agent.

Answer 1

Hi ash

The Rterm.exe process consuming 80GB of memory is almost certainly the root cause of your delay.

You're using:

system2(paste0(R.home("bin"), "/R"), args=paste("CMD BATCH", ...))

This is Launching Rterm.exe

Use this instead:

system2(file.path(R.home("bin"), "Rscript"), args = c("--vanilla", file.path(getwd(), "01Script.R")))

Or:

system("Rscript 01Script.R --vanilla")

Hope this helps!

Please Let me know if you have any queries.

Answer 2

After modifying the system call as below I don't see the Rterm.exe anymore in the process and the scripts are running as expected without any delays.

system2(file.path(R.home("bin"), "Rscript"), args=paste0(getwd(),"/01Script.R"),stdout="01script.Rout",stderr="01script.Rerr")