Microsoft Foundry Local SDK ignores CUDAExecutionProvider, forces CPUExecutionProvider on Windows

Question

I am developing a local RAG application using foundry-local-sdk (v1.2.3) on Windows 11 with an NVIDIA RTX 5070 Laptop GPU. I am unable to get the SDK to utilize my GPU for inference, as it consistently falls back to CPUExecutionProvider despite my explicit configuration.

Environment:

OS: Windows 11

• Hardware: NVIDIA RTX 5070 Laptop GPU
• SDK Package: foundry-local-sdk / foundry-local-sdk-winml (installed via pip)

Runtime: Python 3.14 (venv)

The Issue: When initializing the FoundryLocalManager with CUDAExecutionProvider settings, the model fails to load with the following error:

Plaintext

System.InvalidOperationException: Cannot load model 'Phi-3.5-mini-instruct-cuda-gpu:2': 
it requires the 'CUDAExecutionProvider' execution provider, which is not available. 
Available EPs: [CPUExecutionProvider].

Steps I have taken:

I have installed onnxruntime-genai-cuda and onnxruntime-genai-directml.

I configured the Configuration object as follows:

Python

config = Configuration(
    app_name=

I verified that the model variant Phi-3.5-mini-instruct-cuda-gpu:2 exists and is cached.

I suspect that the foundry-local-sdk-winml package is bundled with a winml specific core runtime that is hardcoded to use CPUExecutionProvider, effectively ignoring custom provider settings in Configuration.

My Question: Is there a way to force foundry-local-sdk to use CUDAExecutionProvider when using the winml variant, or is this SDK package strictly limited to CPU-only execution? If the latter, what is the recommended way to utilize GPU acceleration for inference with models managed by this SDK (e.g., extracting the model path and using onnxruntime-genai directly)?