Solved my own problem, and now I really do feel stupid.
It turned out to be a mistake on my part. My environment variable for the "__ams_storage_connection_string" was 1-character different in Azure than it is in my development environment. Fixed that and it finally works in both local and Azure. When I copied it from my dev environment to paste it into the Azure environment, I missed copying the very last character.