With Application Gateway backends and SSL, you can configure it with TLS Termination(Application Gateway terminates the SSL connection and establishes an unencrypted session with the backend) or End to End SSL (Application Gateway terminates the SSL connection and establishes a second encrypted session with the backend).
As long as you can get the certificate that caddy uses and whitelist it with Application Gateway (there are differences between the Application Gateway SKUs) this should be possible. This will most likely be a self-signed certificate scenario.