Share via

Managing Entry IDs in the Wrapped PST

  • Article

Update – this is now part of the Outlook MAPI Code Samples.

[This is now documented here: https://msdn.microsoft.com/en-us/library/bb905271.aspx, https://msdn.microsoft.com/en-us/library/bb821132.aspx]

Sam Khavari of Zimbra asked me if I had noticed that the Wrapped PST sample crashes when you turn on the preview pane. After a bit of wrangling over the repro, we found that it will crash if you preview one of the first handful of messages created in the store. Since I had been focused on the replication API, and my sample implementation of the replication API created many items in many different folders, every time I had tested it, those first few, dangerous items were in folders such as Drafts, which don't have a preview pane. So no, I hadn't noticed it, but since they pointed it out I've got to fix it. :)

A bit of debugging showed that Outlook had somehow gotten a hold of a folder object instead of a message and was trying to preview it, leading to much hilarity. Further debugging showed that the reason it had the folder object was because the CompareEntryIDs function was insisting that the entry ID for the folder and the entry ID for the message were trying to open referred to the same object. At this point I threw up my hands and went to the guys that wrote the PST to begin with.

They pointed out that my handling of the PR_OST_OLFI property in SetOLFIInOST was botched. They agreed to let me document more about the OLFI structure so we can handle this property correctly. Here goes:

Original SetOLFIInOST:

Let's review what we used to do in this function. We built an OLFI structure with dwAlloc set to 0x7FFFFFFF and everything else nulled out, then we write this to PR_OST_OLFI. We do this every time SetOLFInOST is called, regardless of what may have already been in the property. To understand why this almost worked and why it's such a bad idea, we have to discuss what the OLFI structure is for.

What is OLFI?:

The Offline File Info (OLFI) structure is used by the PST provider for allocating entry IDs in offline mode. Interestingly, it is not used by the PST when it is not being wrapped, so only a PST wrapper needs to worry about this. Here's the revised OLFI structure:

 // OffLineFileInfo
typedef struct {
 ULONG    ulVersion;     // Structure version
    MAPIUID  muidReserved;
  ULONG    ulReserved;
    DWORD    dwAlloc;       // Number of primary source keys
    DWORD    dwNextAlloc;
   LTID     ltidAlloc;
 LTID     ltidNextAlloc;
} OLFI, *POLFI;

Every time the PST needs an entry ID for a new object (message or folder) it needs two pieces of information from the wrapper, a GUID and an index. These are combined to make the ID. The OLFI structure is the mechanism for tracking GUIDs and indexes which have been handed out. The PST reserves entry IDs in blocks. To reserve a block of entry IDs, it consults the ltidAlloc structure to get the current GUID and index. It decrements dwAlloc by the number of entries being allocated and writes the next index back into ltidAlloc. It will continue to do this as long as dwAlloc is larger than the block size it's trying to reserve.

When dwAlloc is smaller than the block size, the PST copies ltidNextAlloc and dwNextAlloc over to ltidAlloc and dwAlloc respectively. It then nulls out ltidNextAlloc and dwNextAlloc. The wrapper is expected to periodically check ltidNextAlloc to see if it's NULL and if it is populate it with a new guid and reset dwNextAlloc.

If dwAlloc is still too small, we fail. Otherwise, we've updated dwAlloc and ltidAlloc, so we write the structure back to PR_OST_OLFI so we can check it next time.

What Almost Worked:

By initializing dwAlloc to 0x7fffffff, we guaranteed that many reservations of EntryIDs should work. We didn't set a guid, but that's not what killed us. Remember that we used to rewrite the property whenever SetOLFInOST is called? Turns out we're calling it from Logon, which can be called multiple times! So when the PST makes its second batch of reservations, it gets the same block as the first batch. The net effect is that some messages end up getting entry IDs that compare as equivalent to some of the default folders. As long as we never try to CompareEntryIDs two of these conflicting entry IDs we won't see a problem. But as soon as we do, we're toast.

Better SetOLFIInOST:

First, we need to read the current PR_OLFI_OST and see what's in it before we go writing anything. Blindly restamping dwAlloc is a recipe for failure. Second, we need to write guids into ltidAlloc and ltidAllocNext so our PST has a chance of working after dwAlloc is eventually exhausted. Finally, we need to periodically call SetOLFIInOST to ensure that we restamp ltidNextAlloc before it's too late.

New code:

I've shared the updated code out here:

https://stephengriffin.members.winisp.net/wrappst/wrappst.zip

Update: 10/05/06 - 10:44AM Sam let me know it's OK to give his name

Comments

  • Anonymous
    November 07, 2006
    Hi Stephen: I have tried to add the wrapped pst store to one of my profiles and it works great. Outlook loads the store with no problem and I can perform basic operations in the store. However, I encountered some problem when I try to load the store manually by myself. What happens is in the following sequence: In my testing application, I create a profile (CreateProfile()) on the fly, add a wrapped pst file (CreateMsgService) to the profile, when prompted for an NST file I supply a location and file name, then after logon to the profile (MAPILogonEx()) I try to open the store manually (OpenMsgStore()) through the session. This call returned error 0x8004011C. From the wrapped pst log file, I can see that this is returned from the delegated m_pPSTMS->Logon() call inside CMSProvider::Logon(). Stephen, can you offer any insight of what might be wrong? I have tried different flags for MAPILogonEx() and OpenMsgStore() without success. Another question that I can find no answer from anywhere is: Is it safe to use the wrapped pst in non-Outlook process, such as sending mail using Simple MAPI from a non-Outlook application? I am asking this because it seems Outlook can load and use wrapped pst happily but OpenMsgStore manually in my code fails.

  • Anonymous
    November 05, 2008
    Hi Stephen, Does this apply for both NSTs and PSTs? Andy.

  • Anonymous
    November 06, 2008
    This article is about wrapped PSTs - aka NST.