Hi Deon,
My attitude when I switched to Word from WP in 1991 was about the same. I didn't know much about styles and who needs them!
I was wrong.
This is about much more than whether to tie your numbering to styles.
Please look at The Importance of Styles in Microsoft Word and perhaps Yet Another "Use Styles" Verbal Beating! by Dian Chapman, MVP.
Here is my recounting of that experience:
Trying to use Word without understanding and using styles is like pushing on a string. I resisted learning and using styles for years and now regret every day of those years because although that string was still very hard to push, it kept getting longer and longer, and had some very important projects tied to it!
Once you understand styles and the Word concept of organizing things into nested Russian dolls everything falls into place and instead of pushing a string, you can push a button that turns on the very powerful text processing machine known as Microsoft Word and it will start doing your work for you instead of running around behind you trying to undo what you thought you just did.
See also:

Back to numbering
The advice to tie MultiLevel Lists to paragraph styles has been good advice since Word was introduced. The key analysis I've seen on this is Word Numbering Explained by John McGhie, MVP Shauna Kelly's article How to create numbered headings or outline numbering in Ribbon Versions of Word by Shauna Kelly is sort of the "Cliff's Notes" McGhie's article and provides instructions on how to create stable numbering.
For your speaker's notes you may not need stable. Doing it your way is likely easier initially. However, once you learn to tie your numbering to styles it goes very quickly. Properly used, it does not break.
Sorry for all the pontificating. I am just a fellow user trying to spare you some pain.