We got the response from the Product Team stating:
As a part of the pgbouncer docker image being used, it won't automatically update the certificate file. Customers will need to use the updated the new certificate that will help in connections going forward. The process to update the cert file is available here : Cert File Update
We are still checking with the team if there is a plan to update the Azure OSS pgbouncer sidecar docker image with the new cert or customers can update their pgbouncer.ini files themselves.
Hope this helps. We will keep you posted as we get more information.