Hi everyone,
I got an update from the product team that a fix has been pushed, but it may take up to two weeks for the changes to be applied in production.
This is an issue only when the What-If tool is run on a Conditional Access policy (CAP) where there is a group assigned. Therefore, the workaround for now in this limited testing capacity is to assign users directly to the CAP instead of specifying a group.
A recommended approach to test Conditional Access Policies and understand how a policy acts is to use the Conditional Access Report-Only mode functionality. The results are logged to the Conditional Access and Report-only tabs in the Sign-in log details. The Conditional Access Insights workbook in Activity Monitor can be used to visualize queries and the impact of multiple report-only policies for a given time-range, set of apps and users. This is a good option if you are currently testing policy assignments.