@D reddy Yes, you're right. Retire action only can be performed when the device is managed by intune.
Currently, there is no method to make the Azure AD account sign out from the app when the device is unmanaged. I'll share with you two alternatives, maybe one of them will meet your requirements.
Method 1: App protection policy can wipe account and data.
If the device is unmanaged, please try to deploy an app protection policy to the target user. Please set "Device types" to "unmanaged" and set "offline grace period" to 1 wipe day.
When the app run offline more than one day, it will perform a selective wipe of the users' account and data. For more details, please read the content about "Offline grace period" in the following link:
https://learn.microsoft.com/en-us/mem/intune/apps/app-protection-policy-settings-ios#conditional-launch
Method 2: Conditional access policy will make the end user to re-enroll the device and then we can do retire action.
It is suggested to try to deploy a conditional access policy to the target user, add the target app in the setting "cloud apps or actions", select "Require device to be marked as compliant" in Grant. For more details about creating conditional access policy, we can refer to the following article:
https://learn.microsoft.com/en-us/mem/intune/protect/app-based-conditional-access-intune-create
When you use the taget user to sign in the target app on the unmanaged device, it may ask you to enroll the device. Then we can try to do the retire action as I said before.
Hope it will give you some ideas.
If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".