@Paul Nerie Apologies for not getting back sooner here. From what I heard from our Product group, restricting access by IP address is currently not supported/planned for deploying using FTP/S.
If placing this restriction is a necessary requirement at your end, switching over to other deployment methods like Zip deploy is suggested, perhaps disabling the FTP endpoint altogether. Then in that case, you will be able to secure publishing inbound traffic for your web app via access restrictions on your SCM site, or using one of the patterns described in this blog post.
Hope this helps. Do let us know if you have further questions.
If an answer is helpful, please "Accept answer" and/or "Up-Vote" which might help other community members reading this thread.