@Žygimantas Jackūnas Thanks for posting in our Q&A.
To clarify this issue, we appreciate your help to collect some information:
1.Please tell us how did you set up to unlock with just password. If possible, please share us the screen shot of your configuration.
2.Is there any app protection policy deployed to Teams app?
If there is anything update, feel free to let us know.
If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.