Same issue here. The ANC adds a distinct hiss that is distracting. I would love for MS to add an option "Off" between "Low" and "Amplified" that just shuts off ANC and does not do anything with outside noise.
Simply turning off ANC and ignoring outside noise (in theory) should be easy to implement in a firmware update. If there is a solid reason why such an option was not included, I would love to hear it.