As you have an Azure VM active, one of the simplest ways to see this is via the Network Interface of the VM, there is an option called "Effective Routes". When this loads, you should the address ranges configured for your local network gateways with a next hop of Virtual Network Gateway.
As you can get from Azure to on-prem, but not the reverse - 99/100 this an issue with the on-prem configuration.
A second note, your NSG rule actually isn't required, it has a default rule allowing "vnet" traffic. When you connect on-prem via VPN, those ranges become part of the system table and are therefore viewed as "vnet".