I have exactly the same issue. But I think my problem is that I want to use me office 365 Family account. As far as I know those accounts don't have an AAD integration. So I can't use MS Graph to query me todo list. When you use your office 365 account to create a new azure tenant you'll get a new account. It seems it's the same account but it's another. And this account hasn't sign up for "to do".
Could anyone confirm my thoughts or have another idea? And is there an solution to use ms graph to implement custom stuff with "to do" by using an office 365 family account.
BR Sebastian