Hello @Dominik Hattrup
Unfortunately you cannot use DNS A record to create custom domain for Static Web App.
Possible DNS records to set up a custom domain are TXT, CNAME, ALIAS
https://learn.microsoft.com/en-us/azure/static-web-apps/custom-domain?tabs=other-dns