@paul libbrecht We understand that you are trying to configure HTTP2 for your Azure Web Apps for Container.
By default, yes http2 is turned off. It sounds like you were able to locate the setting to turn it on.
The request will only be taken to the front end as http2 by default. If you wish the front end to pass the request onto the web workers as https2, you will need to enable the below setting called "HTTP 2.0 proxy." Otherwise, testing will indeed show the request coming across as http 1.2.
Can you please attempt to use this setting to see if it helps complete the chain for HTTP 2.0? We look forward to your reply.