Hi @Matt Fitzpatrick ,
Have you created other rules in your mailbox?
According to your description, your rules only work when you run it mannually, it seems that there may be other reasons that affect the rule, please first check if there are any other rules that has similar conditions in your mailbox. Then, to avoid it further, we could try to turn off all rules temporarily in your owa and Outlook client, and create a new rule to check if the issue has any differences after that.
Also, due to the issue could be reproduced on your owa, it may not be related to your Outlook client, in order to avoid any interference of the rules or other reasons in your mailbox, it's recommended to contact your mailbox administartor to check your mailbox.
If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.