As far as I'm aware there is no direct replica of Session Manager.
If you want to use a browser session and keep all connectivity private you will need to use Bastion. You could create SSH keys within Azure and inject those into you're VM and call it from Bastion or store keys within a Key Vault. But yeah, some key management will be required.
Bastion now supports ssh & gui access to linux vm's once xfmc & xrdp are enabled
Alternatively you could create a secure jump box but you'll need to up some sort of external access. You can restrict access to your VM's from the private IP of said jump box.