@Anindya Kumar Banerjee
Thank you for posting this in Microsoft Q&A.
Blocking outlook alone on a device is not possible. Because there are some other services also that depends on outlook, like teams etc.
To block them via conditional access policy you can configure Azure to block any request which is not coming from approved client apps or app protection policy.
Below are the list of approved client apps,
Outlook is also part of approved client apps.
And regarding blocking clients on accessing exchange online from IMAP clients you can configure "other clients" option under conditions in conditional access policy.
"Other clients" include all clients which uses POP, IMAP, SMTP etc protocols.
Let me know if you have any further questions. Please "Accept the answer" if the information helped you. This will help us and others in the community as well.