Edit

Share via

Initialize execution providers with Windows ML

This page discusses more advanced ways your app can gracefully handle downloading and registering execution providers (EPs) using Windows ML. Even if an EP is already downloaded on the device, you must register the EPs every time your app runs so they will appear in ONNX Runtime.

Download and register in one call

For initial development, it can be nice to simply call EnsureAndRegisterCertifiedAsync(), which will ensure EPs compatible with your device are present (and will download the EPs if they're not present), and then it registers all present EPs with the ONNX Runtime. Note that on first run, this method can take multiple seconds or even minutes depending on your network speed and EPs that need to be downloaded.

// Get the default ExecutionProviderCatalog
var catalog = ExecutionProviderCatalog.GetDefault();

// Ensure execution providers compatible with device are present (downloads if necessary)
// and then registers all present execution providers with ONNX Runtime
await catalog.EnsureAndRegisterCertifiedAsync();

Tip

In production applications, wrap the EnsureAndRegisterCertifiedAsync() call in a try-catch block to handle potential network or download failures gracefully.

Register existing providers only

If you want to avoid downloading and only register execution providers that are already present on the machine:

var catalog = ExecutionProviderCatalog.GetDefault();

// Register only providers already present on the machine
// This avoids potentially long download times
await catalog.RegisterCertifiedAsync();

Discover if there are EPs (without downloading)

If you want to see if there are not-present EPs compatible with your device and drivers, but don't want to start the download, you can use the FindAllProviders() method and see if any providers have a ReadyState of NotPresent. You can then decide to handle this however you would like (launching your users into an "Installing screen", asking them if they want to install, etc). You can choose to continue using any already-downloaded EPs (by calling RegisterCertifiedAsync() as shown above) if you don't want to make your users wait right now.

var catalog = ExecutionProviderCatalog.GetDefault();

// Check if there are new EPs that need to be downloaded
if (catalog.FindAllProviders().Any(provider => provider.ReadyState == ExecutionProviderReadyState.NotPresent))
{
    // TODO: There are new EPs, decide how your app wants to handle that
}
else
{
    // All EPs are already present, just register them
    await catalog.RegisterCertifiedAsync();
}

Download and register a specific EP

If there is a specific execution provider your app wants to use, you can download and register a particular execution provider without downloading all compatible EPs.

You'll first use FindAllProviders() to get all compatible EPs, and then you can call EnsureReadyAsync() on a particular ExecutionProvider to download the specific execution provider, and call TryRegister() to register the specific execution provider.

var catalog = ExecutionProviderCatalog.GetDefault();

// Get the QNN provider, if present
var qnnProvider = catalog.FindAllProviders()
    .FirstOrDefault(i => i.Name == "QNNExecutionProvider");

if (qnnProvider != null)
{
    // Download it
    var result = await qnnProvider.EnsureReadyAsync();

    // If download succeeded
    if (result != null && result.Status == ExecutionProviderReadyResultState.Success)
    {
        // Register it
        bool registered = qnnProvider.TryRegister();
    }
}

Production app example

For production applications, here's an example of what your app might want to do to give yourself and your users users control over when downloads occur. You can check if new execution providers are available and conditionally download them:

using Microsoft.Windows.AI.MachineLearning;

var catalog = ExecutionProviderCatalog.GetDefault();

// Filter to the EPs our app supports/uses
var providers = catalog.FindAllProviders().Where(p =>
    p.Name == "MIGraphXExecutionProvider" ||
    p.Name == "VitisAIExecutionProvider" ||
    p.Name == "OpenVINOExecutionProvider" ||
    p.Name == "QNNExecutionProvider" ||
    p.Name == "NvTensorRtRtxExecutionProvider"
);

if (providers.Any(p => p.ReadyState == ExecutionProviderReadyState.NotPresent))
{
    // Show UI to user asking if they want to download new execution providers
    bool userWantsToDownload = await ShowDownloadDialogAsync();

    if (userWantsToDownload)
    {
        // Download all EPs
        foreach (var p in providers)
        {
            if (p.ReadyState == ExecutionProviderReadyState.NotPresent)
            {
                // Ignore result handling here; production code could inspect status
                await p.EnsureReadyAsync();
            }
        }

        // And register all EPs
        await catalog.RegisterCertifiedAsync();
    }
    else
    {
        // Register only already-present EPs
        await catalog.RegisterCertifiedAsync();
    }
}

See also

  • Last updated on