Powershell script is stuck when using workflows and running from a batch execution server

Question

Hi All.

I have noticed that my PowerShell scripts simply gets stuck when called from a batch execution server. I have experienced this issue from autosys and ansible. I am thinking it could be a trivial issue as having to use a flag when calling the script. The issue I have is that because it works on the console without issues, it is only stuck when executed from either Autosys or ansible.

A good example is calling a script file like this

powershell.exe -c "& .\test.ps1 -db_Server 'server1,9999' -db_server_list 'serverx,77','servery,88'"

If the script file has a foreach parallel loop which allows it to run commands in parallel, then it is stuck running. If the workflow is removed, then it works. The PS version on the batch execution server is PowerShell v5.

Answer 1

I have never had a need to use workflows but this site says that there are some restrictions.

https://devblogs.microsoft.com/scripting/powershell-workflows-restrictions/

Have you tried running a very simple script to verify that there isn't some basic problem with workflows when run in that environment?

https://devblogs.microsoft.com/scripting/powershell-workflows-the-basics/

Generate a transcript and see if that detects any error.

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.host/start-transcript?view=powershell-5.1

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.host/stop-transcript?view=powershell-5.1

workflow paralleltest {  
 parallel {  
   Get-CimInstance –ClassName Win32_OperatingSystem  
   Get-Process –Name PowerShell*  
   Get-CimInstance –ClassName Win32_ComputerSystem  
   Get-Service –Name s*  
  }  
}  
  
Start-Transcript -Path c:\temp\ps.log  
paralleltest  
Stop-Transcript   

Answer 2

Thanks for the suggestion, I have got some data from the transcript route, but nothing that points to why the script is failing when executed from a batch scheduling tool, but works fine when executed from standard PowerShell console.

Answer 3

Thanks for the comments, I could not get your script to work. It appears it could be related to the PowerShell version, I am running v5 and i suspect to use the parallel feature requires a newer version of PS.

The below is the workflow format that works outside of batch execution.

param(
[parameter (Mandatory = $true)]
[string []] $servers
)

Workflow Test  {
    param(
        [array[]] $incoming_servers
    )

    foreach -parallel -throttlelimit 4 ($target in $incoming_servers){

        write-output "The target server is $target"
    }
}

Test $servers

When on batch execution server, the issue appears to be this line foreach -parallel -throttlelimit 4 ($target in $incoming_servers). It just doesn't like it, I thought it could be an issue with workflow pop ups, so I disabled this by adding the flag
$global:ProgressPreference='SilentlyContinue'

Are there any alternatives in powershell v5 to using the foreach parallel in a scenario where the list of servers is in an array ?

Answer 4

Your initial question says "it works on the console without issues". I assume that to mean that you are logged on to the batch execution server and running powershell.exe in your RDP session. If so, it should work when called by the scheduling software. Don't silentlyContinue, we need to know what error it encounters.

Let's try to capture stdout and stderr. Schedule cmd.exe.

cmd.exe /c powershell.exe c:\scripts\YourScript.ps1 1>C:\Logs\MyLog.txt 2>&1  

See if that produces an error message.

I am running Powershell 5.1.19041.1023 on Win 10 (10.0.19043.1052). I looked closer at the image that you posted the other day, NT 6.3.9600 is Server 2012 R2 (or Win8.1).

PS version 5.0.10586.117 is obviously older than what I am running.

https://learn.microsoft.com/en-us/powershell/scripting/windows-powershell/install/windows-powershell-system-requirements?view=powershell-5.1

To get from 5.0 to 5.1, this site says that you need to install Management Framework 5.1. Do you have test batch execution server that you can try that on first before you install on your production machine?

Answer 5

Hi,

Yes I meant when logged onto the server via RDP and running powershell.exe

Some very interesting facts.
Running the example you gave above, I can now see the log file, I can see the print statements within the workflow but the script still never ends when executing from the batch server. Its printing output to the log file and no error and it still hangs there for hours for a script that ought to run for seconds. I do not see any errors but it is still running.

Another fact is this, whenever I call a function inside the workflow it hangs just before it gets to the function as I cannot see any print statements after the function.

Lastly, any inline script block also makes the script hang.

As expected, whenever it hangs it is not possible to delete the log file as the PowerShell process is still locking it, however once the process is killed then it is possible to delete the file.

When I remove the following, then the workflow works.

• Removal of the function which tests connectivity to SQL instances.
• Removal of the inline script block.

Once more, when executed from the server directly, no such issues.

The SQL check function can be seen below.

        Function CheckSQLInstance{
            param
            (
                [Parameter(Mandatory=$true,
                            ValueFromPipelineByPropertyName=$true,
                            Position=0)]
                            $sql_server,                    

                            [Parameter(Mandatory=$true,
                            ValueFromPipelineByPropertyName=$true,
                            Position=1)]
                            $db_name         
            )

            try
            {
                #Return extra useful info by using custom objects
                $ping_status = "" | Select-Object -Property log_date, stage, status, error_message             
                $ping_status.log_date = (Get-Date)
                $ping_status.stage = 'Ping SQL instance for $sql_server'

                #test connection for a sql instance
                $connectionstring = "Data Source=$sql_server;Integrated Security =true;Initial Catalog=$db_name;Connect Timeout=5;"
                $sqllconnection = New-Object System.Data.SqlClient.SqlConnection $connectionstring
                $sqllconnection.Open();
                $ping_status.status = $true
                $ping_status.error_message = ''

                return $ping_status

            }
            Catch
            {   
                $ping_status.status = $false
                $ping_status.error_message = $_.Exception.Message
                return $ping_status

            }

            finally{
                $sqllconnection.Close();              

            }
        }