Remote Deskotp Mouse Interactions are not working when TaskManager is in Foreground

Setup Details:
Edition: Windows Server 2016 Datacenter
Version: 1607
OS build: 14393.4704
First of all, we have launched a remote desktop (wfreerdp.exe) as a user with elevated privileges.
Next, we are trying to send mouse inputs to the remote desktop using the SendInput function (winuser.h) from another process launched by the same user (The process is launched from python using the CreateProcessAsUser API with the token we got from WTSQueryUserToken API). And, this process is not elevated.
ut = win32ts.WTSQueryUserToken(int(sessionid))
procHandles = win32process.CreateProcessAsUser(ut, *procArgs)
Initially, it works well in interacting with the applications like Notepad, File Explorer, etc. As soon as we open TaskManager and bring it to the foreground, interactions will stop working until the TaskManager is taken out of the foreground. After that, interactions work again as usual.
Surprisingly, the SendInput function is not giving any error codes when interactions are not working.
Please help!