Hi CHynes_RCSD,
Thanks for posting in the community.
According to your description, I assume you toke the screenshot from https://www.onenote.com/edu/classnotebook. Please click Manage notebooks, are you able to see the missing notebooks? If so, it means these notebooks are created in Teams. If you want to add/remove students from the Teams class notebook, you can just add/remove students from that team.
If these problematic notebooks are not Teams class notebooks, please change another browser or open the private window of the browser, then accessing https://www.onenote.com/edu/classnotebook. If the issue still exists, I would like to know whether your colleagues have the same issue.
If you have any updates, please feel free and come back to share with us.
Regards,
George