Hi @Matt
I suppose the cause may be due to Teams cache.
Per my test, once the Teams cache is cleared manually or by other methods, this setting would be reverted to Teams.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.