Dear Maya,
I'm afraid that the Page Numbering feature in Word for the web is different from the Word desktop client. It can't be formatted like the Word desktop client.
Based on the article Page numbering in Word - Microsoft Support, the best way to work with page numbers is in Word on a laptop or desktop computer, rather than in a web browser or mobile app.

I suggest you format the page numbering in the Word desktop client and the changes may be synced to Word for the web like the following.

As our product developers are willing to hear users' requirements, I suggest you submit your feedback and convey your suggestions to our new Feedback platform mentioned in the following article for the product developers to know your suggestions and opinions for product improvements.
Introducing Feedback for Microsoft 365 - Microsoft Community Hub
Please note the platform page will be loaded after some time based on the network environments.
You may access the page as follows to submit the feedback as well.

You may use a personal account or a Microsoft 365 business work or school account to submit the feedback.
Thanks for your understanding.
Have a nice day!
Sincerely
Cliff | Microsoft Community Moderator