[Azure IoT Edge]could not read config from /etc/aziot/edged/config.d

Question

aziot-edge can not run on my device(ubuntu 18.04 arm64), how to fix the following error?

# iotedge  list  
A module runtime error occurred
 caused by: Could not list modules
 caused by: connection error: Connection reset by peer (os error 104)

# iotedge system status
System services:
    aziot-edged             Down - failed 
    aziot-identityd         Ready
    aziot-keyd              Ready
    aziot-certd             Ready
    aziot-tpmd              Ready

aziot-edged is in a bad state because:
aziot-edged.service: Down - failed : Printing the last 10 log lines.
-- Logs begin at Wed 2021-04-07 06:11:18 UTC, end at Wed 2021-04-07 06:20:35 UTC. --
Apr 07 06:20:35 qcs610-odk-64 aziot-edged[3619]: 2021-04-07T06:20:35Z [INFO] - Starting Azure IoT Edge Module Runtime
Apr 07 06:20:35 qcs610-odk-64 aziot-edged[3619]: 2021-04-07T06:20:35Z [INFO] - Version - 1.2.1
Apr 07 06:20:35 qcs610-odk-64 aziot-edged[3619]: 2021-04-07T06:20:35Z [ERR!] - The daemon could not start up successfully: Could not load settings
Apr 07 06:20:35 qcs610-odk-64 aziot-edged[3619]: 2021-04-07T06:20:35Z [ERR!] -         caused by: Could not load settings
Apr 07 06:20:35 qcs610-odk-64 aziot-edged[3619]: 2021-04-07T06:20:35Z [ERR!] -         caused by: could not read config from /etc/aziot/edged/config.d
Apr 07 06:20:35 qcs610-odk-64 systemd[1]: aziot-edged.service: Main process exited, code=exited, status=153/n/a
Apr 07 06:20:35 qcs610-odk-64 systemd[1]: aziot-edged.service: Failed with result 'exit-code'.
Apr 07 06:20:35 qcs610-odk-64 systemd[1]: aziot-edged.service: Start request repeated too quickly.
Apr 07 06:20:35 qcs610-odk-64 systemd[1]: aziot-edged.service: Failed with result 'exit-code'.
Apr 07 06:20:35 qcs610-odk-64 systemd[1]: Failed to start Azure IoT Edge daemon.

aziot-edged.mgmt.socket: Down - failed : Printing the last 10 log lines.
-- Logs begin at Wed 2021-04-07 06:11:18 UTC, end at Wed 2021-04-07 06:20:35 UTC. --
Apr 07 06:11:24 qcs610-odk-64 systemd[1]: Starting Azure IoT Edge daemon management socket.
Apr 07 06:11:24 qcs610-odk-64 systemd[1]: Listening on Azure IoT Edge daemon management socket.
Apr 07 06:20:24 qcs610-odk-64 systemd[1]: Closed Azure IoT Edge daemon management socket.
Apr 07 06:20:24 qcs610-odk-64 systemd[1]: Starting Azure IoT Edge daemon management socket.
Apr 07 06:20:24 qcs610-odk-64 systemd[1]: Listening on Azure IoT Edge daemon management socket.
Apr 07 06:20:35 qcs610-odk-64 systemd[1]: aziot-edged.mgmt.socket: Failed with result 'service-start-limit-hit'.

aziot-edged.workload.socket: Down - failed : Printing the last 10 log lines.
-- Logs begin at Wed 2021-04-07 06:11:18 UTC, end at Wed 2021-04-07 06:20:35 UTC. --
Apr 07 06:11:24 qcs610-odk-64 systemd[1]: Starting Azure IoT Edge daemon workload socket.
Apr 07 06:11:24 qcs610-odk-64 systemd[1]: Listening on Azure IoT Edge daemon workload socket.
Apr 07 06:20:24 qcs610-odk-64 systemd[1]: Closed Azure IoT Edge daemon workload socket.
Apr 07 06:20:24 qcs610-odk-64 systemd[1]: Starting Azure IoT Edge daemon workload socket.
Apr 07 06:20:24 qcs610-odk-64 systemd[1]: Listening on Azure IoT Edge daemon workload socket.
Apr 07 06:20:35 qcs610-odk-64 systemd[1]: aziot-edged.workload.socket: Failed with result 'service-start-limit-hit'.

# iotedge check

Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
‼ daemon configurations up-to-date with config.toml - Warning
    /etc/aziot/config.toml was modified after keyd's config
    You must run 'aziotctl config apply' to update keyd's config with the latest config.toml
√ identityd config toml file specifies a valid hostname - OK
× aziot-identity-service package is up-to-date - Error
    could not query https://aka.ms/latest-aziot-identity-service for latest available version
‼ host time is close to reference time - Warning
    Could not query NTP server
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
× read all preloaded certificates from the Certificates Service - Error
    could not load cert with ID "aziot-edged-trust-bundle"

    Caused by:
        internal error
√ read all preloaded key pairs from the Keys Service - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK

Connectivity checks (aziot-identity-service)
--------------------------------------------
× host can connect to and perform TLS handshake with iothub AMQP port - Error
    Could not connect to myEdgeHub1.azure-devices.net : could not complete TLS handshake
× host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - Error
    Could not connect to myEdgeHub1.azure-devices.net : could not complete TLS handshake
× host can connect to and perform TLS handshake with iothub MQTT port - Error
    Could not connect to myEdgeHub1.azure-devices.net : could not complete TLS handshake

Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
‼ configuration up-to-date with config.toml - Warning
    /etc/aziot/config.toml was modified after edged's config
    You must run 'iotedge config apply' to update edged's config with the latest config.toml
√ container engine is installed and functional - OK
× configuration has correct URIs for daemon mgmt endpoint - Error
    Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.2.1' locally
    docker: Error response from daemon: Get https://mcr.microsoft.com/v2/: dial tcp: lookup mcr.microsoft.com: Temporary failure in name resolution.
    See 'docker run --help'.
‼ aziot-edge package is up-to-date - Warning
    Error while fetching latest versions of edge components: could not send HTTP request
× container time is close to host time - Error
    Could not query local time inside container
‼ DNS server - Warning
    Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
    Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
    You can ignore this warning if you are setting DNS server per module in the Edge deployment.
√ production readiness: container engine - OK
‼ production readiness: logs policy - Warning
    Container engine is not configured to rotate module logs which may cause it run out of disk space.
    Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
    You can ignore this warning if you are setting log policy per module in the Edge deployment.
× production readiness: Edge Agent's storage directory is persisted on the host filesystem - Error
    Could not check current state of edgeAgent container
× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error
    Could not check current state of edgeHub container
× Agent image is valid and can be pulled from upstream - Error
    Failed to get edge Agent image

Connectivity checks
-------------------
× container on the default network can connect to upstream  AMQP port - Error
    Container on the default network could not connect to myEdgeHub1.azure-devices.net:5671
× container on the default network can connect to upstream HTTPS / WebSockets port - Error
    Container on the default network could not connect to myEdgeHub1.azure-devices.net:443
× container on the default network can connect to upstream MQTT port - Error
    Container on the default network could not connect to myEdgeHub1.azure-devices.net:8883
× container on the IoT Edge module network can connect to upstream AMQP port - Error
    Container on the azure-iot-edge network could not connect to myEdgeHub1.azure-devices.net:5671
× container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error
    Container on the azure-iot-edge network could not connect to myEdgeHub1.azure-devices.net:443
× container on the IoT Edge module network can connect to upstream MQTT port - Error
    Container on the azure-iot-edge network could not connect to myEdgeHub1.azure-devices.net:8883
14 check(s) succeeded.
6 check(s) raised warnings. Re-run with --verbose for more details.
16 check(s) raised errors. Re-run with --verbose for more details.

# ls -l /etc/aziot/edged/config.d
total 4
-rw------- 1 iotedge iotedge 836 Apr  7 06:20 00-super.toml

# ls -l /etc/aziot/edged/
total 8
drwx------ 1 root root 4096 Apr  7 06:13 config.d

# ls -l /etc/aziot/              
total 72
drwxr-xr-x 1 root root  4096 Jun  4  2021 certd
-rw-rw-rw- 1 root root 11098 May 26  2021 config.toml
-rw------- 1 root root 11091 Dec  1 15:48 config.toml.edge.template
-rw------- 1 root root  5873 Sep 16  2020 config.toml.template
drwxr-xr-x 1 root root  4096 Jun  4  2021 edged
drwxr-xr-x 1 root root  4096 Jun  4  2021 identityd
drwxr-xr-x 1 root root  4096 Jun  4  2021 keyd
drwxr-xr-x 1 root root  4096 Jun  4  2021 tpmd

Answer 1

Sorry, this issue happened when I pre-install aziot-edge in our yocto project.

The reason is the owner of config.d was incorrect.

To fix this issue we need to chown for each file/folder manually when build yocto project.

Answer 2

Hello @罗宇恒 罗宇恒 ,

All network protocols are giving an error.

Do you have an open internet connection to the cloud? eg. Can you ping 8.8.8.8?

Have you followed the documentation regarding deploying Azure IoT on Linux?

It also says:

You must run 'aziotctl config apply' to update keyd's config with the latest config.toml  

Please check all the steps as seen in that documentation:

You should have run the following steps for version 1.2 (please note this is just a reference, use the original document for the actual steps!):

curl https://packages.microsoft.com/config/ubuntu/18.04/multiarch/prod.list > ./microsoft-prod.list   
sudo cp ./microsoft-prod.list /etc/apt/sources.list.d/  

for Ubuntu 18.04. Do not run the equivalent steps for other Operating systems. Make sure curl is delivering that file.

Then, public key stuff:

curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg  
sudo cp ./microsoft.gpg /etc/apt/trusted.gpg.d/  

Install the moby engine:

sudo apt-get update  
sudo apt-get install moby-engine  
curl -sSL https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh -o check-config.sh  
chmod +x check-config.sh  
./check-config.sh  

Install the latest Azure IoT Edge runtime:

sudo apt-get update
sudo apt-get install aziot-edge

Finally, you need to provide the security (eg. symmetric key) to the configuration:

sudo cp /etc/aziot/config.toml.edge.template /etc/aziot/config.toml  
sudo nano /etc/aziot/config.toml  

Add the key here:

# Manual provisioning with connection string  
[provisioning]  
source = "manual"  
connection_string = "<ADD DEVICE CONNECTION STRING HERE>"  

Save the changes. Apply the changes:

sudo iotedge config apply  

Check out the connection again.

Bonus question: is the local time on the device close to the actual time in your timezone?

Answer 3

@Sander van de Velde | MVP , Facing an issue while trying to provision the device using X.509 certificate.

So I am referring to Microsoft link:https://learn.microsoft.com/en-us/azure/iot-edge/how-to-provision-single-device-linux-x509?view=iotedge-1.4&tabs=azure-portal%2Cubuntu

I am facing the below issue which is faced by other guys as well:

https://github.com/Azure/iotedge/issues/6170