Hello Tony,
I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer.
Issue: Entra Private Access policy rules not removing
Solution: Resolved by Tony by following the below steps
After deleting the app yesterday, it took overnight to wait, and the GSA policy rules cleaned up. Everything is as it should be now. Just had to wait Microsoft time is all.
If you have any other questions or are still running into more issues, please let me know. Thank you again for your time and patience throughout this issue.
Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.