Which pen do you have? The pens that were sold when the Surface Pro 4 launched did not have rechargeable batteries. Identify your Surface Pen and features - Microsoft Support - if your pen has replaceable batteries, see Change Surface Pen batteries - Microsoft Support
The newer Slim Pen and Slim Pen 2 are rechargeable but the batteries are not replaceable. You'd have to be using the separate USB charging pen holder if you are using one of these with a Surface Pro 4.