Posting answer as per solution provided to @Jörg Walter by Azure Support:
The service principal of the cluster was missing for some reason. After recreating it the scaling works fine.
@Jörg Walter Please "Accept as Answer" if it helped so it can help others in community looking for help on similar topics.