IntraopNumThreads interface

Provides an ability to change the number of threads used in the threadpool for Intra Operator Execution for CPU operators through LearningModelSessionOptions. By default, WinML sets the value as the maximum number of threads, which is the same number of logical cores on the user's CPU. Setting this value higher than the number of logical cores on the CPU may result in an inefficient threadpool and a slower evaluation.

Sample code

void SetIntraOpNumThreads(LearningModel model) {
    // Create LearningModelSessionOptions
    auto options = LearningModelSessionOptions();
    auto nativeOptions = options.as<ILearningModelSessionOptionsNative>();
 
    // Set the number of intra op threads to half of logical cores.
    uint32_t desiredThreads = std::thread::hardware_concurrency() / 2;
    nativeOptions->SetIntraOpNumThreadsOverride(desiredThreads);
 
    // Create session
    LearningModelSession session = nullptr;
    WINML_EXPECT_NO_THROW(session = LearningModelSession(model, LearningModelDeviceKind::Cpu, options));
}

Requirements

	Requirement
Minimum supported client	Windows 10, build 17763
Minimum supported server	Windows Server 2019 with Desktop Experience
Header	windows.ai.machinelearning.native.h

Note

Use the following resources for help with Windows ML:

• To ask or answer technical questions about Windows ML, please use the windows-machine-learning tag on Stack Overflow.
• To report a bug, please file an issue on our GitHub.