Outlook Rules - At Least When You Tell It To
[This is now documented here: https://msdn.microsoft.com/en-us/library/ff960239.aspx ]
We just had a customer whose Wrapped PST based store wasn't getting rules to fire in Outlook 2007, when everything was working in 2003. In the course of figuring their issue out, we found a few flags that we should have documented before.
Suppose you built a message store based on the Wrapped PST provider. When your new messages are delivered to your backend database and you create matching messages in the PST, how do you inform Outlook that a new mail has arrived? In Outlook 2003, the way you solve this is to use the IMAPISupport object that was passed to you during logon. From there, you would call Notify, passing fnewNewMail and the details of the new message. When Outlook gets this new message notification, it passes the Entry ID over to the rules engine and everything is hunky dory.
Unfortunately, this doesn't work in Outlook 2007. You fire your notification, Outlook sees it, then Outlook decides not to run rules on the message. Why? Because when Outlook 2007 looked at PR_STORE_SUPPORT_MASK on the wrapped PST, it found the STORE_ITEMPROC flag was set. This means the wrapped PST is an "Item Proc" store, and handles rules differently. Without getting into too many details about "Item Proc", the general idea is that such stores will feed new items into a pipeline where rules, junk mail, and spam processing can happen before listening clients will get a notification. This concept was introduced in Outlook 2007 as a way to streamline the processing.
When an Item Proc store knows it has a new message, it calls a special callback function in Outlook to run it through the rules engine. In the PST, the concept of "newly delivered message" is handled through internal flags. These flags may be set or not set depending on how the message arrives in the PST. For instance, if the PST is being used as an OST and new mail is synchronized from an Exchange server, the flags will be set. Similarly, when the PST is being used as a back end POP3/IMAP, the flags will be set when a new mail is delivered.
For a wrapped PST, however, the way you set these flags is to pass ITEMPROC_FORCE in the SaveChanges call when the newly delivered message is committed to the store. This tells the PST to set its internal flags and mark the item as being eligible for the callback.
There's a twist though. If the mail being delivered to the PST came from Exchange, then rules would already have run on the server and the message has to be handled differently. And the default assumption is that the mail came from Exchange. So you have to indicate that Exchange was not the source of the mail by also passing the NON_EMS_XP_SAVE flag. When both flags are passed, the item will go through rules/junk/spam processing as soon as you call SaveChanges if Outlook is running, and shortly after Outlook starts if it wasn't.
Definitions of the new flags:
| ITEMPROC_FORCE | 0x00000800 | A client can call IMAPIProp::SaveChanges setting the ITEMPROC_FORCE flag in the ulFlags parameter to indicate that a message is being delivered to a PST and should be eligible for rules processing. This flag is only available if the message is being saved to a PST. A wrapped PST store would use this flag to indicate new mail is being delivered through IMAPIFolder::CreateMessage and IMAPIProp::SaveChanges. If this flag is not passed in combination with NON_EMS_XP_SAVE when a message is delivered in this fashion rules will not fire. |
| NON_EMS_XP_SAVE | 0x00001000 | A client can call IMAPIProp::SaveChanges setting the NON_EMS_XP_SAVE flag in the ulFlags parameter to indicate that a message is being delivered to a PST and should be eligible for rules processing, and that the mail was not delivered from Exchange. This flag is only available if the message is being saved to a PST. A wrapped PST store would use this flag to indicate new mail is being delivered through IMAPIFolder::CreateMessage and IMAPIProp::SaveChanges. If this flag is not passed in combination with ITEMPROC_FORCE when a message is delivered in this fashion rules will not fire. |
| STORE_ITEMPROC | 0x00200000 | The message store implements Item Proc logic. Outlook 2007 will not fire rules on messages in this store when it receives an fnevNewMessage notification. |
Comments
Anonymous
January 13, 2008
If I'm offline in OST mode, create a new message and set the above flags before the first SaveChanges, should I expect to see rules kick in?Anonymous
January 13, 2008
To clarify, those flags are for SaveChanges, so you would set them in, not before, the first SaveChanges call. That said, I don't know about the OST case. Give it a test and let us know what you find.Anonymous
January 13, 2008
Works like a charm with OST. ( tested with OL 07 ) Thanks! P.S. Is there any way from the client side to trigger a new-mail notification as well? ( create a message, populate appropriately, call SaveChanges & make Outlook think that it's a new mail )... I don't keep my hopes up here, but it certainly would be nice.Anonymous
January 13, 2008
Well, as noted, if you're a message store, like a wrapped PST, you got a support object during Logon that you can use to send notifications. If you're not a mesage store, then I think you're out of luck.Anonymous
January 13, 2008
To avoid crisis after rolling out an update, is setting NON_EMS_XP_SAVE to indicate that "the mail was not delivered from Exchange" going to affect anything other than rule processing? Thanks.Anonymous
January 13, 2008
The comment has been removedAnonymous
January 14, 2008
Lev - as far as I can tell, right now NON_EMS_XP_SAVE only affects rules. Mahesh - You should look at SPAMFILTER_ONSAVE: http://msdn2.microsoft.com/en-us/library/bb820947.aspxAnonymous
January 16, 2008
What does one need to do to trigger rules on the sending side? ( as opposite to the receiving side above ). Thanks.Anonymous
January 17, 2008
Hrmm - I didn't investigate sender rules.Anonymous
January 18, 2008
Griffin, is there anything else that needs to be done for this to work for a normal PST in Outlook 2007? I tried to set both flags when calling SaveChanges() the first time for a new message added to a PST. The Client Side Rule does not fire on it. Outlook 2007 shows the new mail desktop alert notification window, but the rule does not fire after that. The only difference is that I am using a normal PST instead of the Wrapped PST. However, using the normal PST in Outlook 2003 does fire the Client Side Rule properly... Should I include these flags each time calling SaveChanges(), or just for the very first call? Also does calling SaveChanges() with these flags multiple times has any side-effects? Thanks.Anonymous
January 18, 2008
Following up for my previous comment, actually the Client Side Rule did fire :-) Because I was testing a rule to set a category, and there is a default rule in Outlook 2007 to remove category... However, it did only work partially so far. After the rule is fire, the category is added for the message. Then it looks like the SetProps() + SaveChanges() are not called on the message to commit the change to PST. So, the really next question is: does the Client Side Rule makes the change directly from within PST, or should the rule fire through the wrapping message store interface?Anonymous
January 20, 2008
After setting these flags if I call getcontents table on the Inbox, sometimes the new message saved is not being listed in the contents table. Any idea how to overcome this ? Thanks,Anonymous
January 23, 2008
"When an Item Proc store knows it has a new message, it calls a special callback function in Outlook to run it through the rules engine." Does this means that the PST gives out a message interface to Outlook, and Outlook makes changes to the message directly through that interface (or callback)? If this is the case, then it also means that the wrapping store provider has no way to detect that some changes are made under the hood because of a Client-Side Rule. Is there a way that the wrapping store provider can detect that some changes will be/has been made because of Client-Side Rule? Or, it there a way to restore the Outlook 2003 behavior for Client-Side Rules? Thanks, HaiAnonymous
February 18, 2008
OL 03/SP 3 no longer supports NON_EMS_XP_SAVE | ITEMPROC_FORCE - I get 0x80040106 ( MAPI_E_UNKNOWN_FLAGS )...Anonymous
February 19, 2008
They only ever worked in Outlook 2007. They were never available in 2003. But you don't need to special case your code - check for STORE_ITEMPROC - if it's there, the store supports the flags, if not, it doesn't.Anonymous
March 16, 2008
I'm just a normal guy whose Outlook 2007 rules don't work. The same rules work in Thunderbird. I use Junkmail and Spamkiller. Even when I disarm both the rules don't work. Is there away to fix the problem? Thanks.Anonymous
March 17, 2008
Given that Outlook and Thunderbird can't use the same set of rules, stating that the rules work in Thunderbird doesn't help. It just means you set up rules in Thunderbird to do work and they appear to work, but you set up rules in Outlook and they don't appear to work. Since I don't think you're writing a store provider, this isn't the appropriate forum to work out why your rules don't function for you. However, general troubleshooting techniques would apply - first try deleting all the rules and creating a simple rule to see if it fires. See if firing the rules manually works. Try a new profile. Etc.Anonymous
November 03, 2008
I have made the changes you describe and mail rules now run in Outlook 2007. But now read receipts are not generated when a user first reads a message, even though the message does have PR_READ_RECEIPT_REQUESTED set to true. I'm wondering if it has something to do with how/where I made changes to get rules working -- in our IXPLogon::StartMessage implementation. Passing the new flags to the SaveChanges call on the message passed into StartMessage didn't work, so I created a new message (as in comments for ITEMPROC_FORCE) and passed the new flags to its SaveChanges (and return from StartMessage without calling SaveChanges on the passed-in message). BTW - Thanks for your recent help with _CRT_INIT issue.Anonymous
December 01, 2008
The comment has been removedAnonymous
March 09, 2009
I’m repeating myself – I did use a variation on this title before. In this case, suppose you’re using