@tyctor Thank you for reaching out to Microsoft Q&A, apologize for any inconvenience caused on this.
Based on the shared information, we have understood that you application is hosted on app service using the custom containers, running on two instances (scaled to two instances) and you are seeking for some help on how to SSH into those instances.
To connect via SSH to a specific container running on a custom Docker container, you must add SSH support to your Docker image by adding an 'sshd_config' file to your repository, which configures OpenSSH and must include the items that were explained here.
Once the SSH is enabled, you can SSH to specific container that is running in your app service by pulling their instance names, SSH links using either Azure CLI cmdlets or through the Azure portal as well.
Method 1: Azure CLI Method
To pull the list of container instances running under your app service with their respective SSH links.
az webapp list-instances --name <webappName> --resource-group <resourceGroupName> --query '[].{sshurl:consoleUrl,Name:name}' --output table
Here is the sample output for reference:

select the ssh url of the specific container from the above and browse it in your browser to connect to that specific container or you can SSH to that container using the below CLI cmdlet.
az webapp ssh --name <app-name> --resource-group <rg-name> --instance <instance-name>
Method 2: Azure Portal
To list all instances without Azure CLI, you can use Azure Resource Manager (ARM). Specifically, this resource:
[...]/providers/Microsoft.Web/sites/<app-name>/instances?api-version=2018-02-01
Azure Resource Explorer is quite helpful when navigating ARM:

go to https://<app-name>.scm.azurewebsites.net/webssh/host/instance?=<instancename>
directly to SSH to that specific container.
Feel free to reach out to us if you have any further questions on this.