Hello
Thank you for your question and reaching out.
This is because When you try to login to first time it loads users profile in to Memory and cache. it will be fast during second time as RDP server has already information about user profile and settings or you may reconnecting a disconnected session.
--If the reply is helpful, please Upvote and Accept as answer--