I found the issue, and parts of the solution.
When the email appears in outlook it looks perfectly fine.
However there seems to be some form of background job that tries its best to inline all styles.
in my case I had what I assumed to be pretty harmless style
<style>
.my-class td {
padding-left: 15px;
padding-right: 15px
}
</style>
However the job that removes my style does not seem to understand that type of nesting and simply ignores it.
The 2 curious parts are why this job is being run to begin with (in the time between the mail arriving and this job being run, the email looks fine), and why the email viewer (double clicking the email and getting a new window) doesn't show the same version of the email without a style tag