Thanks for the suggestions but I found the answer.
The Web Pool for the application was changed from Classic to Integrated.
Why would this happen?
This literally stops the Response.Redirect() from redirecting.
Switch back to Classic it redirects again. Switch to integrated and it stops working again.