Most of the react documentation and tools are geared for creating an SPA. You can build a MPA (multi page app) with react but you will need to create a custom webpack or use typescript to build the project (typescript has native support of jsx files).
You might find vue.js a better option for building MPAs.
Note: I have created MPA apps with react, but use the SPA template for new apps. One of the benefits of using react is all the UI code is in the react project, you are not using server for some html and JavaScript for other.