The "correct" way would be to scope the role to Administrative unit, which in turn should be populated with the devices in question. Unfortunately, the UI doesn't always expose the correct "target" for scoping, but you can try using the Graph API directly, as detailed here: https://learn.microsoft.com/en-us/entra/identity/role-based-access-control/custom-create
Another thing to have in mind that officially, only application-related custom roles are supported. In other words, scoping such role might not work.