I have this working now - it was the way I was creating the hover layer. I was creating a new hover layer for each of the 4 datasources. To fix the issue I added the mouse event to the symbol layer that was created for the different marker colors (those in turn were part of an image sprite).
So final arrangement is:
4 datasources for 4 types of pins
4 symbol layers to be for the different datasources and pop up events
4 image sprites for the 4 different colors.