Adding this as an answer even though it's more of a workaround: It looks like the ONLY header that it lets you forward currently is 'Ocp-Apim-Subscription-Key'. So what I did is on my MCP server policy I take my auth header and put it into that key header, and then in my rest API policy I take that token and put it back into the auth header. In my actual policies I have it doing a token exchange in an OBO flow but below is a simplified version for just passing it straightup to the backend.
Frontend MCP server policy (in the inbound section):
<set-header name="Ocp-Apim-Subscription-Key" exists-action="override">
<value>@((string)context.Request.Headers.GetValueOrDefault("Authorization",""))
)</value>
</set-header>
API policy at the 'all operations' level (also in the inbound section):
<set-header name="Authorization" exists-action="override">
<value>@((string)context.Request.Headers.GetValueOrDefault("Ocp-Apim-Subscription-Key",""))</value>
</set-header>
Hope this helps you or whoever else runs into this!