This has been an issue for almost 6 months now for me and just wanted to chip in and say it's also happening on my Lenovo Legion with i7-13700HX and RTX 4060 - a completely different system to the ones mentioned above. If you repeatedly close and reopen File Explorer, the memory usage goes up by 20 MB each time. You can restart the task in task manager to reset the RAM usage, but you shouldn't have to do this.
There is 100% a memory leak in file explorer but Microsoft, being one of the most lazy and unprofessional companies on the planet, still hasn't noticed it.