Troubleshooting "Certificate validation failed. UntrustedRoot" Error in Azure IoT Operations with OPC UA Connector

Question

Troubleshooting "Certificate validation failed. UntrustedRoot" Error in Azure IoT Operations with OPC UA Connector

MohdFhG 55

Hi everyone,

I'm experiencing an issue when establishing a secure connection using the Azure IoT Operations connector for OPC UA to an OPC UA server. The error message I received after adding an asset is:

{
{"errors":[{"code":400,"message":{"title":"Disconnected","detail":"Session creation failure","lastTransitionTime":"2025-03-13T12:54:19.6823734Z","reasons":{"StatusCode":"BadNoCommunication","SimbolicId":"BadNoCommunication","Message":"Error establishing a connection: Error received from remote host: Certificate validation failed. UntrustedRoot: Eine Zertifikatkette wurde zwar verarbeitet, endete jedoch mit einem Stammzertifikat, das beim Vertrauensanbieter nicht als vertrauensw\u00FCrdig gilt.\r\n"}}}]}

Error establishing a connection: Error received from remote host: Certificate validation failed. UntrustedRoot: A certificate chain was processed, but ended with a root certificate that is not trusted by the trust provider.

I have configured the environment according to the Microsoft documentation, including the setup of a self-signed application instance certificate with cert-manager and the trusted certificates list in Kubernetes.

The OPC UA Server has security settings set to none and the authentication setting is anonymous.

I have added an asset endpoint using this command:

az iot ops asset endpoint create opcua --name -asset-endpoint -g  --instance  --target-address   --security-mode none --security-policy none

I can reach the OPC UA server using UaExpert.

I keep notice that the OPC UA server puts the connector of OPC UA certificates aio-opc-opcuabroker [...].der is in the rejected PKI folder.

Despite these checks, the error persists. Could anyone advise on potential causes for this error and suggest troubleshooting steps?

Thank you in advance

Dominic 1,631 Reputation points Microsoft Employee

2025-03-13T14:05:37.9766667+00:00

Does your OPC UA server give any reason as to why it's putting the certificate in the rejected PKI folder?
MohdFhG 55 Reputation points

2025-03-13T14:18:04.5166667+00:00

No, it does not give a reason, the error message (as shown above) that I managed to retrieve is from the MQTT broker using MQTT explorer.
Manas Mohanty 5,700 Reputation points Microsoft External Staff Moderator

2025-03-13T14:35:21.96+00:00
Hi MohdFhG

Thank you for sharing the details on Server setting.

It might not be meeting server security standard. Ideally, we should be putting a security policy to avoid possible DDOS attacks.

Please test below suggestion and let us know.

You can move rejected certs to trusted certs manually from rejected folder to trust folder location and restart the OPA server

You can skip parts of mentioning "security policy" and "security mode" to use anonymous authentication.
az iot ops asset endpoint create opcua --name myprofile -g myresourcegroup --instance myinstance --target-address opc.tcp://opcplc-000000:50000

Reference

Please let us know if it resolves the issue.

Thank you.
MohdFhG 55 Reputation points

2025-03-13T15:06:39.4166667+00:00

Hi Manas Mohanty,

I have tried the suggestions, but unfortunately the same error as above.

It just creates another certs in the rejected folder.

Thank you.
Manas Mohanty 5,700 Reputation points Microsoft External Staff Moderator

2025-03-13T16:29:27.0433333+00:00
Hi MohdFhG

Thank you for following up

Error suggests Root certificate is not trusted by trust provider.

Could you confirm that

You have enabled secure settings on Cluster.

You have added OPC UA server's application instance certificate to the trusted certificates list. This list is implemented as a Kubernetes native secret named aio-opc-ua-broker-trust-list that's created when you deploy Azure IoT Operations. Reference on adding certificates to trusted certificate list

Thank you
MohdFhG 55 Reputation points

2025-03-13T19:20:49.1533333+00:00
Hi Manas Mohanty,

Yes, I followed your suggestion. However, I’m encountering some confusion with two different use cases:

First Use Case: I connected to a custom-built OPC UA server using C# that requires a username and password. In this case, Azure IoT Operations worked successfully following the documentation.

Second Use Case: I installed the Prosys OPC UA server simulation with anonymous authentication enabled. Here too, Azure IoT Operations worked as expected.

However, the issue arises with a third OPC UA server, which was developed by a colleague who is no longer reachable, and I don’t have access to its source code. UaExpert, being a client application, is able to connect to this OPC UA server with no security enabled (anonymous authentication). However, Azure IoT Operations is not.

Could you provide any insights or possible solutions for this issue?

Thank you!
Manas Mohanty 5,700 Reputation points Microsoft External Staff Moderator

2025-03-14T01:27:13.4433333+00:00

Hi MohdFhG

I have asked my Team lead to look into this case.

It might be server specific setting issue, please check internally and OPC UA server people for supportability of your third OPC UA server security standard with IOT operations.

For now, I can only suggest you switch to username and password authentication as currently supported feature for OPC UA user authentication is username and password.

Reference on OPC UA user authentication

Thank you.
Sander van de Velde | MVP 36,766 Reputation points MVP Volunteer Moderator

2025-03-14T08:07:09.8566667+00:00

Hi MohdFhG
welcome to this moderated Azure community forum.

Can you check out this blog post with an actual device and custom OPC-UA server.

Does it work for you, connecting to that OPC-UA server example?
MohdFhG 55 Reputation points

2025-03-14T09:33:40.62+00:00
Hi Sander van de Velde | MVP,

I recently came across your blog, and I found it extremely helpful. I followed your steps to set up the custom-built OPC UA server using C# and successfully got it working. However, I wanted to share some feedback on an issue I encountered during the process.

Issue:

While following your guidelines step by step, I ensured that:

The certificate was uploaded correctly.

Secrets were stored properly on the edge and in Key Vault.

Both the edge and the OPC UA server were reachable and on the same local network.

Despite this, the session between the connector for OPC UA and the OPC UA server was not established. Interestingly, when I switched to the anonymous authentication method (by commenting out the username and password lines in the code), I was able to connect to the OPC UA server.

After further investigation, I was able to narrow down the issue to something between the edge and Key Vault. It seems that the edge was unable to retrieve the secrets, despite being assigned the Key Vault Secrets Officer role.

Solution:

After multiple attempts to troubleshoot, I eventually deleted the Azure IoT Operations deployment and the cluster, then redeployed everything from scratch. This resolved the issue. However, I had to repeat this process multiple times before achieving a successful connection, despite strictly following the steps outlined in your blog.

I wanted to share this experience in case it helps others facing a similar challenge. Thank you again for your insightful blog—it has been a great resource!

Best regards and Thank you
Sander van de Velde | MVP 36,766 Reputation points MVP Volunteer Moderator

2025-03-14T17:58:06.0933333+00:00

Hello MohdFhG,

You're welcome, thank you for your kind words. Nice to hear the blog post added value.
TheironneV 5 Reputation points

2025-04-30T15:19:39.99+00:00
Hi @MohdFhG ,

I ran into the same issue when trying to import a self-signed certificate from the aio-opc-opcuabroker-default-application-cert into a OPC UA server.

After following the documentation to connect my aio-cluster tto the OPC UA server, the connection failed. Theopcua-connector logs, retrieved via:

yaml kubectl logs aio-opc-opc.tcp-1-<> -n <namespace>

produced the following error:

azdeveloper :Error while establishing connection to the OPC UA server... failed because of: Error establishing a connection: BadNotConnected

The issue was with the encoding of the self-signed certificate. When retrieving it using:

bash kubectl -n azure-iot-operations get secret aio-opc-opcuabroker-default-application-cert -o jsonpath='{.data.tls\.crt}' | base64 -d > opcuabroker.crt

The resulting .crt file was in UTF-16, not UTF-8, which caused OpenSSL to fail silently.

I noticed this when trying to convert the .crt into a .der file. Running:

bash cat -vet

revealed null bytes (^@), and the file appeared to have zero valid content.

The fix:
Convert the certificate to UTF-8 with:

bash iconv -f UTF-16 -t UTF-8 opcuabroker.crt -o clean.crt openssl x509 -in clean.crt -text -noout

After this conversion, the certificate became readable and usable. You can now convert it to a .der file if needed.

Let me know if this works for you!

MohdFhG 55

Hello TheironneV,

thank you for your effort and time.

I have tried your fix, but unfortunately, I have faced the following error while converting the certificate to UTF-8.

Any recommendations?

~/Desktop/test$ openssl x509 -in clean.crt -text -noout
Could not read certificate from clean.crt
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:../crypto/store/store_result.c:151:
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:../crypto/store/store_result.c:151:
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
4007E11563700000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input structure: Certificate
Unable to load certificate

TheironneV 5 Reputation points

2025-05-01T11:00:04.08+00:00
HI @MohdFhG ,
It looks like the 'clean.crt' file is still not converted right, could you check what the clean.crt file look like if you use:

cat clean.crt

It should look like this:

-----BEGIN CERTIFICATE----- -----END CERTIFICATE-----

Instead of this:

??-----BEGIN CERTIFICATE----- -----END CERTIFICATE-----

Also what device are you working on, when I tried it on a Mac I had to use this command:

iconv -f UTF-16 -t UTF-8 opcuabroker.crt > clean.crt

1 answer

Your answer

Dominic 1,631 Reputation points Microsoft Employee

2025-03-13T14:05:37.9766667+00:00

Does your OPC UA server give any reason as to why it's putting the certificate in the rejected PKI folder?
MohdFhG 55 Reputation points

2025-03-13T14:18:04.5166667+00:00

No, it does not give a reason, the error message (as shown above) that I managed to retrieve is from the MQTT broker using MQTT explorer.
Manas Mohanty 5,700 Reputation points Microsoft External Staff Moderator

2025-03-13T14:35:21.96+00:00

Hi MohdFhG

Thank you for sharing the details on Server setting.

It might not be meeting server security standard. Ideally, we should be putting a security policy to avoid possible DDOS attacks.

Please test below suggestion and let us know.

You can move rejected certs to trusted certs manually from rejected folder to trust folder location and restart the OPA server

You can skip parts of mentioning "security policy" and "security mode" to use anonymous authentication.
az iot ops asset endpoint create opcua --name myprofile -g myresourcegroup --instance myinstance --target-address opc.tcp://opcplc-000000:50000

Reference

Please let us know if it resolves the issue.

Thank you.
MohdFhG 55 Reputation points

2025-03-13T15:06:39.4166667+00:00

Hi Manas Mohanty,

I have tried the suggestions, but unfortunately the same error as above.

It just creates another certs in the rejected folder.

Thank you.
Manas Mohanty 5,700 Reputation points Microsoft External Staff Moderator

2025-03-13T16:29:27.0433333+00:00

Hi MohdFhG

Thank you for following up

Error suggests Root certificate is not trusted by trust provider.

Could you confirm that

You have enabled secure settings on Cluster.

You have added OPC UA server's application instance certificate to the trusted certificates list. This list is implemented as a Kubernetes native secret named aio-opc-ua-broker-trust-list that's created when you deploy Azure IoT Operations. Reference on adding certificates to trusted certificate list

Thank you
MohdFhG 55 Reputation points

2025-03-13T19:20:49.1533333+00:00

Hi Manas Mohanty,

Yes, I followed your suggestion. However, I’m encountering some confusion with two different use cases:

First Use Case: I connected to a custom-built OPC UA server using C# that requires a username and password. In this case, Azure IoT Operations worked successfully following the documentation.

Second Use Case: I installed the Prosys OPC UA server simulation with anonymous authentication enabled. Here too, Azure IoT Operations worked as expected.

However, the issue arises with a third OPC UA server, which was developed by a colleague who is no longer reachable, and I don’t have access to its source code. UaExpert, being a client application, is able to connect to this OPC UA server with no security enabled (anonymous authentication). However, Azure IoT Operations is not.

Could you provide any insights or possible solutions for this issue?

Thank you!
Manas Mohanty 5,700 Reputation points Microsoft External Staff Moderator

2025-03-14T01:27:13.4433333+00:00

Hi MohdFhG

I have asked my Team lead to look into this case.

It might be server specific setting issue, please check internally and OPC UA server people for supportability of your third OPC UA server security standard with IOT operations.

For now, I can only suggest you switch to username and password authentication as currently supported feature for OPC UA user authentication is username and password.

Reference on OPC UA user authentication

Thank you.
Sander van de Velde | MVP 36,766 Reputation points MVP Volunteer Moderator

2025-03-14T08:07:09.8566667+00:00

Hi MohdFhG
welcome to this moderated Azure community forum.

Can you check out this blog post with an actual device and custom OPC-UA server.

Does it work for you, connecting to that OPC-UA server example?
MohdFhG 55 Reputation points

2025-03-14T09:33:40.62+00:00

Hi Sander van de Velde | MVP,

I recently came across your blog, and I found it extremely helpful. I followed your steps to set up the custom-built OPC UA server using C# and successfully got it working. However, I wanted to share some feedback on an issue I encountered during the process.

Issue:

While following your guidelines step by step, I ensured that:

The certificate was uploaded correctly.

Secrets were stored properly on the edge and in Key Vault.

Both the edge and the OPC UA server were reachable and on the same local network.

Despite this, the session between the connector for OPC UA and the OPC UA server was not established. Interestingly, when I switched to the anonymous authentication method (by commenting out the username and password lines in the code), I was able to connect to the OPC UA server.

After further investigation, I was able to narrow down the issue to something between the edge and Key Vault. It seems that the edge was unable to retrieve the secrets, despite being assigned the Key Vault Secrets Officer role.

Solution:

After multiple attempts to troubleshoot, I eventually deleted the Azure IoT Operations deployment and the cluster, then redeployed everything from scratch. This resolved the issue. However, I had to repeat this process multiple times before achieving a successful connection, despite strictly following the steps outlined in your blog.

I wanted to share this experience in case it helps others facing a similar challenge. Thank you again for your insightful blog—it has been a great resource!

Best regards and Thank you
Sander van de Velde | MVP 36,766 Reputation points MVP Volunteer Moderator

2025-03-14T17:58:06.0933333+00:00

Hello MohdFhG,

You're welcome, thank you for your kind words. Nice to hear the blog post added value.
TheironneV 5 Reputation points

2025-04-30T15:19:39.99+00:00

Hi @MohdFhG ,

I ran into the same issue when trying to import a self-signed certificate from the aio-opc-opcuabroker-default-application-cert into a OPC UA server.

After following the documentation to connect my aio-cluster tto the OPC UA server, the connection failed. Theopcua-connector logs, retrieved via:

yaml kubectl logs aio-opc-opc.tcp-1-<> -n <namespace>

produced the following error:

azdeveloper :Error while establishing connection to the OPC UA server... failed because of: Error establishing a connection: BadNotConnected

The issue was with the encoding of the self-signed certificate. When retrieving it using:

bash kubectl -n azure-iot-operations get secret aio-opc-opcuabroker-default-application-cert -o jsonpath='{.data.tls\.crt}' | base64 -d > opcuabroker.crt

The resulting .crt file was in UTF-16, not UTF-8, which caused OpenSSL to fail silently.

I noticed this when trying to convert the .crt into a .der file. Running:

bash cat -vet

revealed null bytes (^@), and the file appeared to have zero valid content.

The fix:
Convert the certificate to UTF-8 with:

bash iconv -f UTF-16 -t UTF-8 opcuabroker.crt -o clean.crt openssl x509 -in clean.crt -text -noout

After this conversion, the certificate became readable and usable. You can now convert it to a .der file if needed.

Let me know if this works for you!
TheironneV 5 Reputation points

2025-05-01T11:00:04.08+00:00

HI @MohdFhG ,
It looks like the 'clean.crt' file is still not converted right, could you check what the clean.crt file look like if you use:

cat clean.crt

It should look like this:

-----BEGIN CERTIFICATE----- -----END CERTIFICATE-----

Instead of this:

??-----BEGIN CERTIFICATE----- -----END CERTIFICATE-----

Also what device are you working on, when I tried it on a Mac I had to use this command:

iconv -f UTF-16 -t UTF-8 opcuabroker.crt > clean.crt

Answer 1

Hello MohdFhG,

welcome to this moderated Azure community forum.

This blog post explaining how to connect an actual device and custom OPC-UA server describes the steps to create a connect asset.

Yes, managing the key is cumbersome and easy to do it wrong.

That is why I introduce that dummy asset endpoint because it is not possible to add credentials via code and have it synchronized to the edge as a 'Kubernetes secret'.

Keep an eye that the name of the secret (like opcuassetpassword) is only used within the Azure Keyvault. in the CLI commands the generic 'password' must be used. This can be seen in the properties of the dummy endpoint using the k9s tool.

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.

Share via

Troubleshooting "Certificate validation failed. UntrustedRoot" Error in Azure IoT Operations with OPC UA Connector

1 answer

Your answer