Limit the Default sensitivity labels shown to users

ComputerHabit
651
Reputation points
I have enabled Information Protection for Sharepoint. All the sensitivity labels are showing. I have not deployed a labeling policy to these sites. I would think I would need to deploy to them first.
How do I restrict the list? Is it by deploying a policy?