This being the case, the other option would be to delete the public channel, and re-create it as private. However it doesn't let you re-create a channel with the same name even though you have deleted the previous version. You get an error message that says "Channel name is already taken".
Very frustrating.