Hi Michiyuki,
There is an article that describes the whole process to only allowed access for specific Sharepoint sites.
Please refer article below:
Hope this helps.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".