Thanks for the follow up. I appreciate it :)
Regarding the Python reference code, yes, the ref code just downloads the files in the first level. But I have a recursive implementation.
On the overall issue, I still don't have a clean solution. It would be great if Microsoft can expose an API to download folder as a zip. But, for now, we are just planning to solve it by zipping the folders and then upload to SP. The downside here is, even if need a single file, I might end up downloading the entire zip.
I am planning to request my users to zip the folders and upload them in SharePoint, and place the frequently accessed files separately (for now, it is just one file that I have to access often). And when in need, through the SP Automation code, download the entire zip and unzip it programmatically after download. We decided to temporarily go ahead with this workflow.
Thanks!