Hi Matt,
You need to most likely get the relevant team to created a Network Security Group rule which will allow your local office IP inbound access to the VM on port 1433. There's more on best practices around SQL access and how to do this here: https://learn.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/security-considerations-best-practices?view=azuresql#access-control
There are some other options laid our from the docs here which may be of interest: https://learn.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/ways-to-connect-to-sql?view=azuresql
Ideally, your office would have a S2S VPN to the Azure hub network so you can access the VM via port 1433 just like you would do for any other server on your local corp network only so it's private. If that is not the case, review the above two options and see which one would be acceptable for your current network.
Hope this helps, if so, please mark as accepted answer.