A family of Microsoft word processing software products for creating web, email, and print documents.
By way of explanation: The "small icon" you are looking for is called a dialog launcher, but it is available only in Windows versions of the Office programs. They are needed because the Windows programs have done away with Menus. Dialog launchers never have been used in any release of Mac Office because the Main Menu Bar still is used.
To change the formatting of the endnotes, go to Insert> Footnote in the Main Menu Bar as indicated in the screen shot provided by Neo Wu [odd as it may seem]. Both footnotes & endnotes are managed in that dialog window depending on which option you select when you get there. To finish, be sure to click Apply rather than Insert.