These older versions are no longer being updated or maintained. Any issues you may run across will not be resolved. The only thing that MS may remotely do is fix a severe security hole if one is found but since the bulk of the functionality is provided by the core library they would simply fix it as part of the framework update.
The migration path from MVC 4 is to MVC 5 and from Web API to Web API 2. Depending upon your application this may just involve updating some references, fixing compiler errors and recompiling your code but could go so far as to requiring you to rewrite or replace whole parts of your code. It really depends upon what you're doing. In my experience the upgrade isn't that bad.
The official migration process is documented here.
Note that MVC 5/Web API 2 is actually in support mode as well. While no future development will occur on it (or .NET Framework) there will still be security fixes as needed. All effort is now on .NET 5/6. Unfortunately this is a big upgrade and in most cases the recommendation is to rewrite your ASP.NET front end instead as there are simply too many differences. Short term you could upgrade to MVC 5 as it isn't going away anytime soon. Long term you need to come up with a migration plan to move to .NET 5/6 which is not compatible with MVC 5.