Is there a way to "change all" when using a label template or do I have to copy & paste on each label?

Doug has reasonably assumed that this is a mail merge template. However it may not be a mail merge. The update labels function is part of the mail merge process and it will only work if you set the document type to Labels from the mailings tab of the ribbon. It will also add the Next Record field to each label except the first, which may not be what is required, and by the time you have removed the fields it would have been quicker to copy and paste.

Copy and paste works fine (copy one label paste to all) unless graphics are involved. In that case a macro will be required. See the macro at the end of - http://www.gmayor.com/graphics_on_labels.htm  

This macro can also be used instead of cut and paste for the text content.

Use the Update Labels facility to set up all of the labels in the same fashion as the first one on the sheet.

If you already have content in the other labels, you could use the Replace facility, though the Update Labels should also do the trick.

You can also copy one label, then select all of them and use paste.