Hi Cindy,
Thanks for using Microsoft products and posting in the community, I'm glad to offer help.
According to your description, I'd like to suggest you create a new list based on your workbook instead of creating links in the document library, it will import all the items form Excel to the list at one time, here are the steps:
1. Make sure you have formatted the data as a table in Excel.
2. Go to SharePoint Online site > Site contents > click "+New" > select "List" > From Excel > upload your Excel workbook and import the data.
However, there is a small flaw that this method can save a lot of time and effort, but the column type of the "URL" cannot be set or modified to "Hyperlink type", which will result in the links cannot be opened directly, you may need to manually copy the link to the address bar to open it.
Hope the above suggestion can be helpful, please feel free to let me know if you have any concerns, I'll continue to help you.
Best Regards,
Arck