The problem has been resolved. The issue is that currently Azure handles only one latest session ID for one SAML app. It does not consider multiple Assertion Consumer URLs. Whenever we log in with a different AC URL, it generates a new session ID. The latest session ID will be registered in Azure, but when we try to log out with the old app, we get an error saying the session ID does not match.
This was resolved by using the global logout link provided by Azure.