Okay I figured it out! I’m very curious if this is something you guys did not know, since y’all made this app, I mean.
The arrows goes away when you press the keyboard button, one of the three in the top middle of the screen while using the app.
I had not been using that button to remove the keyboard however. I used the button on the keyboard itself that is dedicated to remove it. It’s the button you use in every other part of iOS and iPadOS to remove the on screen keyboard.
But apparently the Remote Desktop app does not register that ad having removed the keyboard so it keeps that circle with the arrows on screen until you press the keyboard button on the top middle of the screen.
I hope this is helpful and that you can help others with this problem in the future now. :)