Roadmap for Outlook Personal Folders (.pst) Documentation
By Paul Lorimer, Group Manager, Microsoft Office Interoperability
[UPDATE: 05/24/2010, Two open source projects to facilitate interoperability with Outlook .pst data files]
[UPDATE: 02/20/2010, New Office Documentation Now Publicly Available ]
Data portability has become an increasing need for our customers and partners as more information is stored and shared in digital formats. One scenario that has come up recently is how to further improve platform-independent access to email, calendar, contacts, and other data generated by Microsoft Outlook.
On desktops, this data is stored in Outlook Personal Folders, in a format called a .pst file. Developers can already access the data stored in the .pst file, using Messaging API (MAPI) and the Outlook Object Model—a rich set of connections to all of the data stored by Outlook and Exchange Server—but only if Outlook is installed on the desktop.
In order to facilitate interoperability and enable customers and vendors to access the data in .pst files on a variety of platforms, we will be releasing documentation for the .pst file format. This will allow developers to read, create, and interoperate with the data in .pst files in server and client scenarios using the programming language and platform of their choice. The technical documentation will detail how the data is stored, along with guidance for accessing that data from other software applications. It also will highlight the structure of the .pst file, provide details like how to navigate the folder hierarchy, and explain how to access the individual data objects and properties.
This documentation is still in its early stages and work is ongoing. We are engaging directly with industry experts and interested customers to gather feedback on the quality of the technical documentation to ensure that it is clear and useful. When it is complete, it will be released under our Open Specification Promise, which will allow anyone to implement the .pst file format on any platform and in any tool, without concerns about patents, and without the need to contact Microsoft in any way.
Designing our high volume products to enable such data portability is a key commitment under our Interoperability Principles, which we announced in early 2008. We support this commitment through our product features, documented formats, and implementation of standards. The move to open up the portability of data in .pst files is another step in putting these principles in action.
Over the past year, Microsoft Office has taken several steps toward increased openness and document interoperability. We’re proud of the work we’ve done around document interoperability, offering customers a choice of file formats and embracing a comprehensive approach that includes transparency into our engineering methods, collaboration with industry stakeholders, and shared stewardship of industry standards.
We’re excited about the possibilities created for our customers and partners by this kind of effort, and we look forward to continued collaboration with the industry in our pursuit of improved interoperability with Microsoft Office. Stay tuned.
Paul Lorimer, Group Manager, Microsoft Office Interoperability.
Related Items:
- Feature Stories: Microsoft Publishes Implementation Notes for File Formats in Office 2007 – Dec. 16, 2008
- Press Releases: Update: Microsoft Releases OpenXML Implementation Notes – Jan. 19, 2009
- Microsoft Resources: Microsoft Interoperability Web site
- External Resources: Document Interoperability Initiative (DII) Web site
Comments
Anonymous
October 26, 2009
Please make OneNote 2007 format open as well.Anonymous
October 26, 2009
Are there any plans to also document the .MSG file format? This is another useful format to have for importing into document management systems.Anonymous
October 26, 2009
Hi Paul Is it possible to get early access to .pst format documentation? I'm working with many of MS formats, and we're planning to implement support for them all in our products. P.S. My company has signed agreement with MS about support for different office formatsAnonymous
October 26, 2009
@Leitg Bade: the .MSG File Format Specification is actually already available at http://msdn.microsoft.com/en-us/library/cc463912.aspxAnonymous
October 26, 2009
how can I help. I can't wait to having alternatives in a central manner. My productivity could greatly improve with alternatives to traditional email. a next generation integrated email...maybe called eknowledge is desperatedly needed. EMAIL HAS TO GET SMART. IT IS HOLDING US BACK...I am tired of all the notifiations from social network sites requiring horrendous time for organization and filtering...AY CARAMBA! Help@Anonymous
October 26, 2009
I'm an "interested customer" (and Microsoft Partner) - can I sign up to get an early look at the documentation? Is there a way for me to be notified when it is available?Anonymous
October 26, 2009
The comment has been removedAnonymous
October 26, 2009
The PST file is one of the biggest problems with Outlook. Why would anyone else want to implement this on other platforms?Anonymous
October 26, 2009
Brilliant! Any timeframe on this?Anonymous
October 26, 2009
What about the .OST file format too? Is that documented somewhere as yet or will it be covered in the .PST format documentation set?Anonymous
October 26, 2009
Interesting. Any comments on whether the format is documented in the context of MAPI calls (requiring the implementation of underlying COM as well as MAPI APIs for use), or simply as a byte stream oriented structure? The latter would be preferable from a multi-platform development standpoint.Anonymous
October 26, 2009
Why not rather work on a format which removes the limitations inherent in the .pst format ( 2gb file size, 65535 items ) such as storing in a local MS SQL DB, or heaven forbid an open format such as sqlite?Anonymous
October 26, 2009
The comment has been removedAnonymous
October 26, 2009
The comment has been removedAnonymous
October 26, 2009
Interop is good, but please first: -give us .NET Developers and MS partners easy .NET APIs rather than specs -put formats in XML (e.g MSG, EML) like you did for the Windows Event Log since Vista
- Outlook (PST) storage could be extended/replaced by SQL Server (Filestream) or Sharepoint for better centralization/backup/retention/ .. you name it
Anonymous
October 26, 2009
Please release as soon as possible, ie beta documentation. Incomplete documentation is better than no documentation.Anonymous
October 26, 2009
Any timeline for the release of this documentation ? Is there any way to join the beta testing ?Anonymous
October 26, 2009
What is the timeframe for releasing said documentation?Anonymous
October 26, 2009
The comment has been removedAnonymous
October 27, 2009
That's a good idea to open the specification to the public, so we can develop 3rd party tools more easily. We are looking forward to the release. Best regards Wolfgang ZerzawyAnonymous
October 27, 2009
PST file format API already exist. Check PST .NET: http://www.independentsoft.de Regards, DavidAnonymous
October 27, 2009
As the underlying database engine is both the same for pst and ost files, will that documentation allow 3rd party tools to access and read ost files as well ? It's not the question how the usage is designed. The general question is: Would this documentation allow a hacker to read my confidential data from an ost file ? VolkerAnonymous
October 27, 2009
Is there any plans to also release a specification for winmail.dat stuff?Anonymous
October 27, 2009
The comment has been removedAnonymous
October 27, 2009
@OneNote question: A draft of the OneNote file format specification has already been posted: http://msdn.microsoft.com/en-us/library/dd924743.aspx. It covers OneNote 2010.Anonymous
October 27, 2009
Thanks everybody for your participation and interest. Here are a few more answers/comments from the team :) @Robertb The 2GB file size limit has been removed with the release of Outlook 2003. Please refer to this article http://support.microsoft.com/kb/830336 for more information. We are committed to investing in our storage technology to meet the increasing demands of our users. @Hai The .pst file format specification will cover wrapped .pst stores as wrapped pst stores are just a type of .pst files. It is important to note that MAPI and the Outlook Object Model provide a runtime service to serialize requests from multiple processes that are trying the access the same .pst file. If you want to implement a solution that directly accesses a .pst file without going through the runtime service provided by MAPI, you will need to make sure no other processes are using the same file. @Volker Dilg-Gruschinski In theory software developers can use information provided in the .pst file format specification to access ost files as well. Regarding the security concern, we don’t recommend users to rely on the rudimentary encoding in our file formats to protect confidential data. We recommend users to take advantage of advanced encryption features provided by the operation system or 3rd party tools to secure confidential data. @David We welcome developers to take advantage of the .pst file format specification to create tools that interoperate with Outlook. As of today we have not released a draft or final version of the .pst file format specification to the public. @David winmail.dat contains data encoded in TNEF format, which is documented in http://msdn.microsoft.com/en-us/library/cc425498.aspxAnonymous
October 27, 2009
you have to open all formats or adopt odf.Anonymous
October 29, 2009
The format was hacked long ago. It's about time MS recognized it.Anonymous
October 29, 2009
The best way to participate is to send an email message to pstinfo@microsoft.com . Daniel Ko from the Outlook development team has just posted this information: http://blogs.msdn.com/outlook/archive/2009/10/29/outlook-pst-file-format-and-interoperability.aspxAnonymous
October 29, 2009
I am eager to see the documentation and how it can be used better than the MAPI?Anonymous
October 30, 2009
I fear the documentation for this format will 5000 pages and just show that this is yet another very complex, binary format. A format so complex that no-one will be able to implement it correctly and robustly, except someone who was been working in the Office Division of Microsoft for the past ten years, and knows two dozen people who have been there too. Or am I too pessimistic? It would be better for Office to move to some clean new format, like an relational data store format or a (ZIP'ed) XML format so that people can use SQL or XPath to manipulate it. And preferably standardize such a format in an open standards body. Is anything like that on the agenda? I regret the day I started to use Outlook, as my email of the past 15 years (> 10 GB) is locked in a proprietary, undocumented file format. It's amazing to me that PST has never been in the spotlight as much other Office file formats. Those formats, unlike PST, have been supported by other products (like OpenOffice) for years, so opening up PST should IMO have been done years before and with higher priority that developing Office Open XML. But in any case, document PST is a good step! PimAnonymous
November 03, 2009
The comment has been removedAnonymous
November 06, 2009
It will be interesting to see the final documentation.Anonymous
December 03, 2009
To: Pim van der Eijk I would read all 5000 pages, nine times if necessary. Consider the following: The PST file format has been with us since the Microsoft Exchange Client was released with Exchange 4.0. since at least 1996. Every version of Outlook uses the PST file format. I would be willing to bet there are far more PST files backing Outlook installations, than Outlook to Exchange Server (which uses Exchange Database). This makes the PST file wildly popular. I have been waiting for this documentation for many years. Kudos to Microsoft. Russell Mangel Las Vegas, NVAnonymous
December 04, 2009
This is a great strategic move. I was considering how I need to long term archive email and the fact that pst is Not publicly documented, and that there are really no 3rd party tools, really made me question it as an option. I may end up stayingwith pst if there was signifcant 3rd party development and integration. Can't wait.Anonymous
December 31, 2009
Hu, Angelo, We use a 3rd party tool for few months now, that is already able to backup and syncronize PST files incrementally, even if Outlook is open. EdgeSafe PST2PST Backup. http://www.datamills.com. Very fast and also able to retain email history.Anonymous
January 07, 2010
I'm interested in the format for another reason: I'm using different mail-clients on different systems - of which Windows Outlook is just one. The problem I face is taht I need to exchange information from one environnment to the other - and each system has it's more or less own format for storing mail, contacts and so on. This makes any of these products a closed to other systems. For most of these systems, the format is described so it's easy to write a program to store the information centrally - in Outlook's .PST files, for instance. But I do NOT want to be restricted to .NET; moving .PST files to other systems for proicessing ought to be an option.Anonymous
March 08, 2010
It will be interesting to see the final documentation.Anonymous
March 18, 2010
I thought a roadmap usually included some form of timeframe?Anonymous
March 22, 2010
Hi, I have also did reverse engineering on PST file and ready with a recovery utility for free. If anyone want help email me jm2251@gmail.comAnonymous
April 14, 2010
I am a Microsoft Outlook user. I use outlook 2007, I usally get an error in outlook. I read your post, it was really nice and I got lot of knowledge. Do you suggest me to use Outlook 2010, Is that better then 2007 and does it has inbuild pst repair tool. Please suggest.Anonymous
May 11, 2010
"I thought a roadmap usually included some form of time fram" Not necessarilyAnonymous
May 11, 2010
The final documentation has been posted. See: http://blogs.msdn.com/interoperability/archive/2010/02/19/New-Office-Documentation-Now-Publicly-Available.aspxAnonymous
July 27, 2012
Perfect Data Solutions is gives you wonderful solution for extract OST file to PST. PDS Outlook OST PST converter program can convert OST file to PST file instantly. Use this link and make OST to PST file exactly. www.transferosttopst.net http://www.ost-to-pst.org/