Hi, MatthewCumpton. I'm Robinson, and I’m happy to help you today.
It is not a problem, but a limitation of the port, despite both ports looking equal, they are different one port is USB 3.2 Gen2 Type-C with DisplayPort, and the other port is USB 3.2 Gen2 Type-C with Thunderbolt 4.
You can use both ports to connect to your monitor, but you will need a different adapter for each port, this is why your adapter worked in one port and did not work in the other.