Ugh, this situation is excruciatingly painful and makes it virtually impossible to work with custom policies. I've been trying for two days now to figure out how to add an email attribute to my SAML responses. I've found several suggestions on Stack Overflow describing how to do this, but (unsurprisingly) they're not always clearly explained and often give somewhat contradictory advice.
Anyway, I've been trying all sorts of tweaks to my custom policy XML files, uploading the updated files, and testing it with the MS SAML test application. But none of my changes ever seemed to have any effect at all. Finally, after much frustration I realized that when I upload policy files those changes don't immediately take effect. As reported here it can take 5-15 minutes before the changes take effect. The problem is I have no way of knowing when they actually take effect, so I have no way of knowing whether or not the change I made simply was incorrect and therefore didn't work or if the change just hasn't gone into effect.
There has to be a better way! As others already suggested, either this lag time between policy updates taking effect needs to be dramatically shortened, or we need someway to manually flush the cache and know when our changes have gone into effect.