Hi Zubair Siddiqui,
thank you for posting your question, it's exactly the same issue we had.
It's a shame how "husnian ch" was answering your question - he/she (?) didn't even picked up your context.
Anyway, after some research we found out that Windows Server (which runs the Azure Infrastructure if not linux is used) can't understand the p12/pfx content because of algorithm issues.
We re-encoded the p12/pfx file as described here: https://kb.globalscape.com/Knowledgebase/11040/Converting-an-Incompatible-PKCS12-Format-File-to-a-Compatible-PKCS12, found by SO: https://stackoverflow.com/a/73969568/660428
openssl pkcs12 -in prod.p12 -inkey prod.pem -out new_prod.pem
openssl pkcs12 -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -export -in new_prod.pem -out new_prod.p12
Depending on your setup you might use passwords in the new export.
In addition to that we had a follow-up issue that the certificate, even if the initialization works, can't be used for http calls with the message "Can't establish SSL connection".
The reason to that was that the certificate is initialized with the default certificate storage, which is the User. Because the process is executed in a non-user context (system context) the store is not available.
Changing this to the machine storage solved that problem as well.
new X509Certificate2(Path, (string?)null, X509KeyStorageFlags.MachineKeySet);
I hope this might help you - at least this answer address your context :)