There's probably a few things you may need to do to narrow it down:
Try with audio downloaded on the computer so you know its not a steaming/network/internet related issue.
Try removing one of the devices and then repairing it.
Try without Bluetooth which I don't see if you said that you tried that.
Check for possible interfering devices and move them away (eg cell phone).
Check other Bluetooth devices not audio related.
However, it could be possible that the BT hardware isn't fully compatible with Windows 11 or some other issue. The above stuff should help you to understand what is happening. The more you can do the better you can know if its just audio or all BT stuff on that hardware, etc. Right now, you believe its only audio related which would throw things off if its not.
Just tested myself on Windows 11 and BT audio works fine so I would think its a greater chance to be a compatibility issue for your hardware. You have to do the above tests to be sure.