Hi Jackson Teo,
Thanks for reaching out. You can append the '?$filter=folder+ne+null' filter after the Graph query URL to filter Document Library by folders in that library. You can find more information on listing items here https://learn.microsoft.com/en-us/graph/api/driveitem-list-children?view=graph-rest-1.0&tabs=http. These queries will not recurse folders and list only items in the path mentioned in the query URL.
If the reply is helpful, please click Accept Answer and kindly upvote it. If you have additional questions about this answer, please click Comment.
Thanks.