For those that are frustrated in trying to get rid of the colored dots in Office 2016 Contacts/Email, you can't. Here is the answer from Microsoft:
I understand however the dots are actually a system design so we may not able to modify them . In this case, we can send a feedback to our Engineer for this to be changed on the next updates.