If you remove SMS as an option, a user who previously got a phone call with the option to change to another option will no longer be able to change options.
Users who have phone or app as their defaults will not be affected, but users who set text message as their default will be required to re-register. In the registration they will be told that their organization needs further information and that they can only MFA using "call me." Users who previously had only the deselected options registered will be required to register again.
If you have SSPR enabled, changing the authentication methods available may prevent users from being able to use SSPR. From the Changing authentication methods documentation,
Changing the available authentication methods may also cause problems for users. If you change the types of authentication methods that a user can use, you might inadvertently stop users from being able to use SSPR if they don't have the minimum amount of data available.
In that scenario as long as the user has enough registered options (which you can enforce if you want by requiring two or more methods for SSPR) , then you shouldn't have an issue.
The main repercussion is that some users will need to re-register, but that shouldn't be that big of a deal.
Interestingly enough there's a really good blog post on this topic by Microsoft MVP Brian Reid. It's also covered a bit in the first document I linked but not as explicitly.