Blazor Webassembly: Swagger endpoint does not work as expected when PWA is enabled and is publised to Azure appservice

Question

I created a simple Blazor WebAssembly project with Asp.Net hosted and PWA feature on.
Added <PackageReference Include="Swashbuckle.AspNetCore" Version="6.1.1" /> to Server project.

Ran it locally and I can hit the swagger page when I type this in URL:
https://localhost:44339/swagger/

I published the same app to Azure appservice, and hit the swagger endpoint,
The application did not display swagger UI, it seems it reloaded the web app.

When I press Control+F5, it showed the swagger UI

It seems the problem is caused due to PWA because when I published without PWA, the swagger endpoint worked as expected.