So for anybody else: No, azure does not manage that automatically. You need to change dns entries, or at least, the A record.
In the portal, there is a region for "app service domains", that lists your domains "managed" by azure. go there. It has an "advanced" button which goes to a totally different site that has nothing to do with azure, not even branding. that site has settings for DNS and you can also change the A record there.
WM_FYI
-thomas woelfer