Thank you @Prrudram-MSFT ,
I already opened a support ticket. In fact, I opened 2 as I was anxiously waiting for support to contact.
After further investigation and testing today, I was able to replicate the issue in our DevTest environment.
So if I push our script manually to the VM, still using Invoke-azvmruncommand, and restore it using the snapshot taken prior, the recovery is ok. I will not run the script again and reapply the LGPO and registry changes.
It is a different case if I use the runbook to push our script. After swapping OS disk to restore the VM, the script will run upon start up again, thus re applying the LGPO and registry changes.
Our script, downloads the LGPO file from a storage account from inside the machine. So, to break the cycle, we removed the LGPO files from the storage account, preventing the script to apply it. A little progress so we can safely proceed to restore our prod VM.
But I still do not know what and how our script is being triggered automatically.
I know that when I using Invoke-azvmruncommand, it creates a plugin in the vm, and on the path C:/Packages/Plugins/Microsoft.CPlat.Core.RunCommandWindows/1.1.9/Downloads, it stores the script.
What puzzles me is, when we were restoring the VM using the snapshot taken prior the activity, our script is not yet in this path C:/Packages/Plugins/Microsoft.CPlat.Core.RunCommandWindows/1.1.9/Downloads. So how was it triggered upon start up.
Was able to talk to someone from the support but we were on the rdp issue for now. We will continue next week and will further update here as soon as I get their explanation and recommendation.