Hello Heinz57Sriracha,
The USB port on the charger should charge any USB device you have.
Have you tried different outlets to see that isn't the case?
Some USB devices can demand more power than the charger can provide while connected to the Surface Pro 4 and as such will stop charging the Surface Pro 4 as a result.
The charger itself might be faulty and if this is the case, then I would suggest contacting Surface Support for a charger replacement.
Best Regards