Hi Rasmus,
Thanks for posting back.
After searching, some members mentioned exporting the SharePoint lists into a CSV file and then using Power BI to get data will save the time. Please check if the following threads help.
Power BI - Get Data From SharePoint List / Library
SharePoint Online slow refresh
Also, if you have more concerns about how to improve getting data from SharePoint list, since we have a specific channel Microsoft Power BI Community coping with such Power BI questions, I would like to suggest you post a new thread there for expert help.
Sorry for that our category may have limited resources on checking on the issue further.
Thanks for your understanding.
George