Connect to web services

Article
08/08/2023

The Azure Sphere SDK includes the libcurl library, which high-level applications can use to connect and authenticate with HTTP and HTTPS web services. Both server and client authentication are supported, so that applications can verify that they are communicating with the expected server and can prove to the server that their device and Azure Sphere catalog are legitimate. Mutual authentication combines the two.

The Azure Sphere samples repo on GitHub includes the following curl samples:

HTTPS_Curl_Easy uses a synchronous (blocking) API for server authentication.
HTTPS_Curl_Multi sample uses an asynchronous (non-blocking) API for server authentication.

Although the synchronous approach to server authentication in HTTPS_Curl_Easy is quite simple, Azure Sphere applications should generally use the more complex asynchronous technique shown in the HTTPS_Curl_Multi sample, along with an epoll-based, single-threaded event-driven pattern.

The libcurl website provides thorough documentation of the libcurl C API and many examples. Differences between the cURL Library and the Azure Sphere SDK runtime library are as follows:

Constant Name (definition)	cURL range limits	Azure Sphere range limits
CURLOPT_BUFFERSIZE (buffer size)	Default: 16 KB	Default: 1536 KB
CURLOPT_UPLOAD_BUFFERSIZE (upload buffer size)	Default: 64 KB Maximum: 2MB Minimum: 16 KB	Default: 1536 KB Maximum: 64 KB Minimum: 1536 KB
CURLOPT_HEADERFUNCTION (complete HTTP header passed to this function)	Maximum: 100 KB	Maximum: 16 KB
CURLOPT_DNS_CACHE_TIMEOUT	Default: cache results for 60 seconds Maximum: cache results forever Minimum: 0 (do not cache results)	All values are overridden to 0, and results are not cached.

Requirements for applications that use curl

Applications that use the curl library must include the appropriate header files, and provide Azure Sphere (Legacy) tenant UUID and internet host information in the application manifest.

Header files

To use curl, include these header files in your application:

#include <applibs/storage.h>  // required only if you supply a certificate in the image package
#include <tlsutils/deviceauth_curl.h> // required only for mutual authentication
#include <curl/curl.h>
#include <applibs/networking_curl.h>  // required only if using proxy to connect to the internet

The storage.h header file is required only if you supply one or more certificates in the application image package. The deviceauth_curl.h header is required for performing mutual authentication. The networking_curl.h header is required if the application is using a proxy to connect to the internet.

Application manifest

The AllowedConnections field of the application manifest must specify the hosts to which the application connects. It must also contain the name of each domain that the connection may encounter if redirected. For example, both microsoft.com and www.microsoft.com are required for an application that connects to the Microsoft home page.

If the application uses mutual authentication, the DeviceAuthentication field of the manifest must include the Azure Sphere (Legacy) tenant UUID. Device authentication certificates are issued only if the device's catalog is linked to an Azure Sphere (Legacy) tenant UUID that matches the tenant UUID in the application manifest. This restriction provides defense in depth: an application running on a device in a different catalog (say, that of a different customer or a rogue entity) cannot authenticate to the server.

During development, you can find the legacy tenant UUID by using the az sphere catalog show command and extracting the MigratedCatalogId value from the tags object.

If the application uses a proxy, the ReadNetworkProxyConfig field indicates whether the application has permission to retrieve the proxy configuration.

In the following example, the AllowedConnections field specifies that the application connects only to www.example.com, the DeviceAuthentication field specifies the Azure Sphere (Legacy) tenant UUID, enabling the application to use the device certificate for mutual authentication, and the ReadNetworkProxyConfig field specifies that the application can retreive proxy configuration information.

  "Capabilities": {
    "AllowedConnections": [ "www.example.com" ],
    "Gpio": [],
    "Uart": [],
    "WifiConfig": false,
    "DeviceAuthentication": "00000000-0000-0000-0000-000000000000",
    "ReadNetworkProxyConfig": true
  }

Supported functionality

Libcurl for Azure Sphere supports only the HTTP and HTTPS protocols. In addition, the Azure Sphere OS does not support some functionality, such as writable files (cookies) or UNIX sockets. Features that will not be supported in future libcurl releases, such as the mprintf() family, are not available.

Libcurl for Azure Sphere supports TLS 1.2 and TLS 1.3, and has retired TLS 1.0 and TLS 1.1 in alignment with the broader Microsoft TLS security strategy.

The following are the supported cipher suites:

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
TLS_DHE_RSA_WITH_AES_128_CBC_SHA256

Attempts to use an unsupported version of TLS return the error CyaSSL does not support <version>.

Server authentication

Azure Sphere supports server authentication through libcurl. The server’s certificate must be signed by a Certificate Authority (CA) that the device trusts. For libcurl to authenticate a server, the application must provide the path to the CA file.

Add CA certificates to the image package

To use one or more CAs, you must add the certificates to your image package. Each certificate must be base-64 encoded. The simplest approach is to create a single file that contains all the additional certificates. The file must have the .pem filename extension. To add certificates:

Create a certs folder in the project folder for your application. The project folder contains the CMakeLists file for your application.
In the certs folder, create a text file with the .pem extension, copy each certificate into it, and save the file.
In the CMakeLists.txt file, add the certificate file to the image package as a resource file. For example:

azsphere_target_add_image_package(${PROJECT_NAME} RESOURCE_FILES "certs/DigiCertGlobalRootCA.pem")

The certificate file should now appear in the certs folder in the image package.

Set certificate locations

In your application, use the CURLOPT_CAPATH and CURLOPT_CAINFO options to set the locations of the certificates. Call Storage_GetAbsolutePathInImagePackage to retrieve the absolute path to the certificates in the image package and then call curl_easy_setopt.

CURLOPT_CAPATH sets a default folder for the certificates. For example, the following code tells curl to look for certificates in the certs folder in the image:

char *path = Storage_GetAbsolutePathInImagePackage("certs");
curl_easy_setopt(curl_handle, CURLOPT_CAPATH, path);

CURLOPT_CAINFO sets a path to a file that contains one or more certificates. Curl searches this file in addition to the default folder set in CURLOPT_CAPATH.For example:

char *path = Storage_GetAbsolutePathInImagePackage("CAs/mycertificates.pem");
curl_easy_setopt(curl_handle, CURLOPT_CAINFO, path);

This code tells curl to trust any CAs that are defined in the mycertificates.pem file, in addition to CAs that are defined in the directory set in CURLOPT_CAPATH.

Mutual authentication

Mutual authentication verifies that both the server and the client device are legitimate. It's a multi-step process:

The application authenticates the server using a CA certificate, as described in Server authentication.
The application presents an x509 client authentication certificate to the server so that the server can authenticate the device.
The server uses the Azure Sphere catalog's certificate chain to verify that the device belongs to the catalog.

An application can set up the device-authentication side of mutual authentication in either of two ways:

Configure the Azure Sphere DeviceAuth_CurlSslFunc function as the SSL function that performs authentication.
Create a custom SSL function that calls the Azure Sphere DeviceAuth_SslCtxFunc function for authentication.

Note

Azure Sphere does not support SSL/TLS renegotiation.

Before you use either function, you must update the CMakeLists.txt file for your application to add curl and tlsutils to TARGET_LINK_LIBRARIES:

TARGET_LINK_LIBRARIES(${PROJECT_NAME} applibs pthread gcc_s c curl tlsutils)

Use DeviceAuth_CurlSslFunc

The simplest way to perform device authentication is to configure DeviceAuth_CurlSslFunc as the callback function for curl SSL authentication:

// Set DeviceAuth_CurlSslFunc to perform authentication
CURLcode err = curl_easy_setopt(_curl, CURLOPT_SSL_CTX_FUNCTION, DeviceAuth_CurlSslFunc);
if (err) {
	// Set ssl function failed
	return err;
}

The DeviceAuth_CurlSslFunc function retrieves the certificate chain for the current Azure Sphere catalog and sets up the curl connection to perform mutual authentication. If authentication fails, the function returns CURLE_SSL_CERTPROBLEM.

Use DeviceAuth_SslCtxFunc

An application can also use a custom SSL callback function that calls the Azure Sphere DeviceAuth_SslCtxFunc function for authentication.

Your custom SSL function must call DeviceAuth_SslCtxFunc to perform the authentication, but may also do other tasks related to authentication. DeviceAuth_SslCtxFunc returns a value of the DeviceAuthSslResult enumeration, which provides detailed information about the failure. For example:

static CURLcode MyCallback(CURL *curl, void *sslctx, void *userCtx)
{
    int err = DeviceAuth_SslCtxFunc(sslctx);
    Log_Debug("ssl func callback error %d\n", err);
    if (err) {
        // detailed error handling code goes here
    }
    return CURLE_OK;
}
...

err = curl_easy_setopt(curl, CURLOPT_SSL_CTX_FUNCTION, MyCallback);
    if (err) {
        goto cleanupLabel;
    }

Use the catalog certificate chain on your server

To perform mutual authentication, the server must be able to verify that the device belongs to your Azure Sphere catalog and that the catalog itself is legitimate. To perform this authentication, the server requires the Azure Sphere catalog's certificate chain, which signs all your Azure Sphere devices:

To get the certificate chain for your catalog, download it to a .p7b file, as in the following example:

az sphere ca-certificate download-chain --destination CA-cert-chain.p7b

You can then use the .p7b file on your server.

Additional tips for using curl

Here are some additional tips for using curl in an Azure Sphere application.

If you plan to store page content in RAM or flash, keep in mind that storage on the Azure Sphere device is limited.
To ensure that curl follows redirects, add the following to your code:
```
curl_easy_setopt(curl_handle, CURLOPT_FOLLOWLOCATION, 1L);
```
To add verbose information about curl operations that might be helpful during debugging:
```
curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L);
```
Some servers return errors if a request does not contain a user agent. To set a user agent:
```
curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");
```

When handling curl_multi timer callbacks, avoid recursive calls when the reported timeout is 0ms, as this can lead to unpredictable behaviour. Instead, treat 0ms as 1ms by triggering an EventLoopTimer (0ms EventLoopTimers are also recursive and should be avoided).

static int CurlTimerCallback(CURLM *multi, long timeoutMillis, void *unused)
{
     // A value of -1 means the timer does not need to be started.
     if (timeoutMillis != -1) {

         if (timeoutMillis == 0) {
             // We cannot queue an event for 0ms in the future (the timer never fires)
             // So defer it very slightly (see https://curl.se/libcurl/c/multi-event.html)
             timeoutMillis = 1;
         }

         // Start a single shot timer with the period as provided by cURL.
         // The timer handler will invoke cURL to process the web transfers.
         const struct timespec timeout = {.tv_sec = timeoutMillis / 1000,
                                          .tv_nsec = (timeoutMillis % 1000) * 1000000};
         SetEventLoopTimerOneShot(curlTimer, &timeout);
     }

     return 0;
}