Setting up TMG 2010 Where EMS is a Domain Member and Array Servers are in a Workgroup

 

Introduction

I have seen a number of cases where customers were installing TMG 2010 in a “hybrid” scenario. What I mean by this is that the EMS was part of the Domain but the Array Servers were in a workgroup. There are a couple of “gotchas” that I wanted to talk about today.

Assumptions

I am going to make a few of assumptions before I get started. First I am going to assume that you have already installed the TMG Enterprise Management Server (EMS) on a server that is a domain member. I am also going to assume that you have installed the TMG Array Member on a server that is in a workgroup. I recommend getting both of them to the latest Service Pack, Updates, and hotfixes before proceeding. You definitely want both the EMS and TMG Array server to be the same code level. Please refer here for version number information on TMG.

Certificates and Accounts

The first thing you want to do is request a Server Authentication certificate for your EMS. It needs to be issued to the Fully Qualified Domain Name (FQDN) of the EMS with the Private Key Exportable option checked. In my lab the EMS is called ems.fabrikam.com so I requested a Server Authentication certificate from my Certification Authority and installed it into the Certificate Store. Make sure you also export the .PFX file for the certificate, with the private key, and put the file somewhere handy on the EMS machine.

Next you will want to make sure that both the EMS and the TMG Array Server trust the CA that issued the Server Authentication certificate to the EMS. You can do this by importing the certificate for the CA into the Trusted Root Certificate Authorities branch in the Computer Store on each of those machines.

Another thing that is sometimes overlooked in the scenario is that mirrored accounts are needed on both the EMS and the TMG Array Server. For example, I just used the fabrikam/administrator account on the EMS and the local Administrator account on the TMG Array server. They both have the same password.

Keep in mind that if you have any firewalls that reside between the EMS and the TMG Array Servers you will initially want to allow ALL traffic between them. You can tweak this down later but it can cause you a lot of heartache with communications in TMG.

Create the New Array

On your EMS, open the TMG MMC, highlight the Arrays branch, and then on the far right-hand Tasks pane choose to Create New Array

Give your new array a name.

Type in the DNS name of the array.

Choose the Default Policy.

Click Next at the Array Policy Rule Types.

Complete the New Array Wizard.

TMG will create the new array and you should see that it was a success.

Apply this on the EMS.

Wait for the configuration changes to be saved.

In the TMG MMC on your EMS, there should now be a branch called Arrays. Below it should be the array that you just created.

 

Joining the New Array

Back on your TMG Array Server go into the TMG MMC and highlight the branch that says Forefront TMG (servername). On the far right-hand pane under the Tasks Tab, click Join Array.

You will see a welcome screen for the Join Array Wizard.

Under the Array Membership Type choose to “Join an array managed by an EMS Server”.

Give it the Fully Qualified Domain Name of your EMS. (Note: you will want to make sure name resolution is working properly on the TMG Array Server before you do this step).

The newly created array should come up as a choice.

Click Finish on the Completing the Join Array Wizard.

You should get a message that you successfully joined the array.

Give it a few minutes but you will probably notice that the configuration is not synching and you will get an error and a red X under the Configuration Status. The error reads “Forefront TMG Management cannot establish a connection with the Forefront TMG Computer.”

The Problem

So why isn’t the TMG Array Server able to communicate with the EMS? It seems like everything was set up correctly. TMG in a workgroup scenario relies on Authentication over SSL encrypted channel (LDAPS). That is the reason we requested the Server Authentication certificate for the EMS Server.

You can verify this by going into the MMC on your EMS, right-clicking on the top level branch of the array that you just created and choosing Properties.

Under the Configuration Storage branch the authentication type near the bottom should be set to “Authentication over SSL encrypted channel”

The problem is that the Server Authentication certificate was never bound to the ISASTGCTRL service.

You can verify this by creating a Certificates Snap-in MMC.

Choose to manage snap-in for a Service Account.

Select the Local Computer.

Select the ISASTGCTRL service and finish.

You should now see that there is not a certificate under the Personal branch.

Keep this open, you will refresh it in a few minutes.

To correct the certificate issue, download the TMG Cert Tool Pack from here

Install the tool on the EMS but then move the ISACertTool.exe to the same directory where TMG is installed. Open an administrative command prompt and navigate to that directory. Run the command as explained below against the .PFX file you have for your EMS Server Authentication Certificate.

The syntax is listed in the DOC file that comes with it and is:

• ISACertTool /st file_name [ /pswd password] [ /keepcerts]

Where:

• /st file_name installs the exported certificate on the Configuration Storage server. File_name specifies the path and name of the exported .pfx certificate file.

• /pswd password specifies the password that may be required when installing the server certificate. It is only required if a password was specified during export of the certificate file.

• /keepcerts specifies that existing certificates should not be deleted. By default when you run ISACertTool.exe, all certificates in the ADAM_ISSTGCTRL local store are erased. To specify that existing certificates should not be deleted, specify the /keepcerts parameter.

After running this you should see a message that the Storage certificate installation was successful.

Go back to the Certificate Snap-in for the ISASTGCTRL Service and refresh the Personal Branch. You should now see the Server Authentication Certificate.

Now go back to Monitoring on your EMS and you should see that the TMG Server is successfully synching.

Conclusion

Setting TMG up where the EMS is in a domain and the TMG Array Servers are in a workgroup can be tricky. TMG in a workgroup relies on an SSL Encrypted channel and sometimes getting that to work correctly is not always straight-forward. In this article I have shown you a couple of the common pitfalls and how to correct them.