It seems like you are having trouble opening a custom port in your virtual machine. You mentioned that you have tried configuring inbound rules from the portal, configuring the rule into your Windows Server 2019 Datacenter, and using PowerShell, but the port checker still says that the port is closed.
One thing you can try is to check if the port is allowed in the network security group associated with your virtual machine. You can navigate to the Networking settings of your virtual machine in the Azure portal and check the inbound security rules for the network security group. Make sure that there is a rule that allows traffic on the custom port you are trying to open.
If the port is allowed in the network security group, you can also try checking if the application or service that is listening on the custom port is running and configured correctly.
I hope this helps you troubleshoot the issue.
References: