If you Export message as msg, then text information from categories should stay, but Color is a RDO repository and works with Categories but that is a deferent mechanizm.
I build Addin to Categories to that account saved as in POP so I make test and I found deference's. between Exchange and IMAP categories.
I do export two mails from 2 kind of account type
And I see that in Exchange color in outside msg is not save, but in IMAP (as on POP) works as well.
But in object mail (as you can use in code) you can not use Color parameter. Then you should take it as a rule that there are no colors when exporting.
Regards