Too Many Notifications

We had a customer recently who’s application iterated through mailboxes on the Exchange server, advising for notifications on a handful of folders in each mailbox. They were fine with a few thousand mailboxes, but as they tried to scale up, they ran into a couple limitations.

The first limitation was already well known: MAPI uses shared memory to track a number of details between multiple processes, including notification registrations. This memory is allocated by the first MAPI client to log on to a given profile. By default, this memory will be 1 MB. When this customer ran their test with 1 MB of shared memory, they were able to call Advise 7828 times. The total size of the memory needed to track the 7828 notifications, plus all the other data also kept in this shared memory, used up the entire megabyte, so the next attempt at calling Advise failed. The reason the article gives a range of values (7800-7900) is that the practical limit will vary depending on whether other MAPI applications are running, what they’re doing, what needs various providers have for notifications, etc.

Increasing the size of the shared memory got them past this first limitation, but also makes it possible to hit the next limitation. MAPI tracks the notifications which have been registered in a structure similar to an array. This structure, which lives in shared memory, will grow and shrink depending on how many notifications have been registered. However, the largest this array can be is 65528 bytes, as that is the largest block the shared memory routines can allocate. This is a fixed limitation and cannot be altered through any registry keys. Since each notification takes up 4 bytes in the array, we have a theoretical maximum number of notifications of 16382. Again, practical concerns (MAPI’s own use of notifications, providers which may use notifications, and some fixed overhead in the structure) will limit how many notifications a client will actually be able to register. In the customer’s test, they were able to get 16296 before Advise failed with MAPI_E_NOT_ENOUGH_MEMORY. Your own mileage may vary – expect to top out just north of 16000 calls to Advise no matter how large you make the shared memory.

Note that due to their common lineage, both Outlook and Exchange’s implementation of MAPI have the same behavior. Both of them implement the SharedMemMaxSize reg key and both have the same practical limitations on the number of times Advise can be called.

Now – since this limitation is derived from how shared memory is used, running more MAPI clients that use the same shared memory will not work around the problem. So running another process as the same user will just spread the limit across two processes. If you want to work around the problem with a second process, the second process will need to run as a different user altogether.

Alternatively, if the goal is to monitor as many mailboxes as possible, you can try reducing the number of times you call Advise. The article on ignoring notifications may come in handy here.

Finally, this limitation might be enough to get you looking at EWS, in which case you’ll probably want to read Vikas’ overview of notifications in EWS.

Comments

• Anonymous
  February 19, 2009

1.  Do you think EWS is more beneficial here vs ICS?
2.  Registered-notifications structure ( 65528 limit ).  My understanding is that it's just an array of cookies ( that MAPI hands out to each advise ).  In this case, this limitation is per process, right?   So between splitting among n processes & adjusting SharedMemMaxSize, one could deal with the above situation, right?  ( not saying we should do it wastefully, just trying to see if this would work ). Thanks.

• Anonymous
  February 19, 2009
  1 - I haven't compared EWS to ICS. 2 - No - I covered this. The limit is per shared memory block, not per process. And that shared memory is just that - shared. So more processes running as the same user (using the same shared memory) won't help.

• Anonymous
  February 19, 2009
  Thanks.  Was not immediately clear that the array holding Advise handles also lives in the shared memory.

• Anonymous
  February 19, 2009
  I see - I added a few clarifying notes.