The problem is the Azure DNS documentation is correct only from a pure DNS perspective: yes, a CNAME entry to alias "www" works in an IP-resolution sense, but no, it requires additional steps to work with HTTPS.
i.e. the Azure Static Web App documentation doesn't make clear that only domains listed in the custom list will have SSL certificates created for them and just talks about CNAME entries.
Hence the IP mapping is identifying the correct target but then the TLS on top fails because the original URL (www.example.com) is not in the SSL certificate of the final target IP (example.com).
So to make https://www.example.com work, add the CNAME alias and also add it as a Web App Custom Domain.
Finally for additional http-only mappings (e.g. http://www.example.co.uk) a work-around is to add a Classic CDN end point (to a dummy storage account with static web pages enabled), add these domain(s) to it, add the CNAME(s), and then set a Global rule to Url Redirect to https://example.com.