In Bing Maps you can use the math library to calculate the centroid of the polygon and use a pushpin to add a label. Here is a sample: https://samples.bingmapsportal.com/?search=polygon&sample=polygon-with-label
In Azure Maps there are a couple of ways to do this. The symbol layer supports polygons, so you can simply attached a data source of polygons to that layer and create labels, however the position may not be ideal. Alternatively, you can calculate a label position similar to the Bing Maps sample and create a data source of points for each label. Here are some samples:
https://samples.azuremaps.com/?sample=polygon-labels---symbol-layer
https://samples.azuremaps.com/?sample=polygon-labels---calculated