Hi Omkaar Salodkar,
Thank you for choosing Microsoft forum.
Based on your description, I tried to reproduce your scenario from my end. However, I fail to find the "edit button" mentioned in your question. May I know what kind of "edit button" you're referring to? Is it the edit button for message editing? See Edit or delete a sent message in Teams. You may control if the edit button appears both in Team level and organization level.
Team level
Org level
Hope this helps! Thank you.
Stay safe & Sincerely,
Qian