@Yi Ning Shi - This is a known issue. If the bot is sideloaded, then the icon is taken from the bot registration. As a workaround for now you can use the same icon in both places. Please follow any one of below mentioned ways depending on where you did registration to change your icon.
- https://dev.botframework.com/bots --> Select your bot --> Settings --> Upload custom Icon
- Azure portal --> Bot services --> Select your bot --> bot profile --> Upload custom Icon
After changing the icon please logout and login again to see the changes.
This issue is only for side loaded apps, it won't be there is app is published to App store.
If the response is helpful, please click "Accept Answer" and upvote it. You can share your feedback via Microsoft Teams Developer Feedback link. Click here to escalate.