Joe Calev's WebLog

Debugging Lync Delegation

jscalev — Mon, 16 Jun 2014 16:20:43 GMT

Why is my admin not able to act as a delegate? How can I check whether delegation is even setup? These are great questions and it occurred to me after I spent some time searching that no one has ever bothered to explain exactly how delegation works. With that in mind I present the following – which some may find useful when debugging while others simply find it interesting to know.

While this may not be completely obvious, delegation is a presence feature whose configuration is well documented in MS-PRES. This covers in gory details how to add, change, and remove a delegate. What it does not cover at all is how the delegate and delegator learn of the relationship and how this information is stored.

No delegation configured

So let’s start at the very beginning with two users – Boss and Admin – who do not have a delegation relationship setup. We are about to configure this, but before we do we need to examine sign in for both users. During sign in both users will self subscribe using a message similar to the following (note that some headers have been removed for brevity).

SUBSCRIBE sip:admin@vdomain.com SIP/2.0
Via: SIP/2.0/TLS 192.168.0.240:52386
From: ;tag=bd94be2ff5;epid=fc44a406d8
To:
Call-ID: 0423f26f9e6c40db8dda82276cbddd9d
CSeq: 1 SUBSCRIBE
Contact:
Event: vnd-microsoft-roaming-self
Accept: application/vnd-microsoft-roaming-self+xml
Supported: com.microsoft.autoextend
Supported: ms-benotify
Supported: ms-piggyback-first-notify
Content-Type: application/vnd-microsoft-roaming-self+xml
-




                    type="delegates"/>

The two that we care about are the subscribers and delegates categories. As we will see shortly, these will be key to understanding delegation. Before we setup delegation, however, let’s have boss add admin as a contact and see what happens. We’ll skip the details of adding a contact though and instead concentrate on the prompted subscriber – that prompt that tells admin that boss added her as a contact. How does she receive this notification – through the subscribers category subscription of course.

NOTIFY sip:192.168.0.240:52807;transport=tls;ms-opaque=a589590cd5;ms-received-cid=1ABE500;grid SIP/2.0
To: ;tag=de5be0b9f5;epid=fc44a406d8
Content-Length: 636
From: ;tag=79520080
Call-ID: 904acbdc96704c16afe7206e92e7fd09
CSeq: 4 NOTIFY
Require: eventlist
Content-Type: application/vnd-microsoft-roaming-self+xml
Event: vnd-microsoft-roaming-self
Supported: ms-dialog-route-set-update

-
                sub="https://schemas.microsoft.com/2006/09/sip/presence-subscribers">
    -
        +
                        displayName="MrBoss Bosso"
                        acknowledged="false"
                        type="sameEnterprise">
    -

Let’s not acknowledge it just yet, but instead take a look at what we have in the database. You’ll need to look at the database for admin’s FE, which if you do not know where it is located you can find with

Get-CsUserPoolInfo admin@vdomain.com | select-object PrimaryPoolPrimaryRegistrar

Now at the database the following will provide the prompted subscriber list.

select r1.UserAtHost as Publisher, r2.UserAtHost as Subscriber, p.Permanent, p.AuthMask, p.Ackd, convert(varchar(4000), convert(varbinary(4000), p.Context)) as Context

from rtc.dbo.PromptedSubscriber as p

inner join dbo.Resource as r1

on r1.ResourceId = p.PublisherId

inner join rtc.dbo.Resource as r2

on r2.ResourceId = p.SubscriberId

where r1.UserAtHost = 'admin@vdomain.com'

Notice that Ack’d is 0, but what is more interesting is the Context.

This is not overly interesting, but play along for now. No ack the prompted subscriber message and query the table again. Since the prompted subscriber is ack’d, we have removed the row from the database.

Delegation configured

Now let’s add admin as a delegate for boss. The following message from boss sets up the delegation.

06/16/2014|15:44:25.561 F30:64C INFO ::
SERVICE sip:boss@vdomain.com SIP/2.0
From: ;tag=d24448c9c5;epid=18dfefe32c
To:
Call-ID: bb1ff151b5364659a7c0dad4dcdc85c8
CSeq: 1 SERVICE
Contact:
Content-Type: application/msrtc-setdelegate+xml
-
                version="7">

                    action="add"/>

Boss will then see the following NOTIFY, telling her that the delegation has been configured. Note that this time it was sent back in the 200 OK response instead of in a separate NOTIFY. This happened because we indicated support for best effort NOTIFY (BENOTIFY) in the supported: ms-benotify header when we setup the subscription.

SIP/2.0 200 OK
From: "MrBoss Bosso";tag=d24448c9c5;epid=18dfefe32c
To: ;tag=91831FC8D1A4176B2C748B9E0BB2DA10
Call-ID: bb1ff151b5364659a7c0dad4dcdc85c8
CSeq: 1 SERVICE
Content-Type: application/vnd-microsoft-roaming-self+xml

-
                del="https://schemas.microsoft.com/2007/09/sip/delegates">
    -
                    version="8">

                        publish="false"
                        redelegate="false"/>
    -

We can see that the delegation has been configured by examining the Delegate table on admin’s front end.

select r1.UserAtHost as Delegator, r2.UserAtHost as Delegate, d.AuthMask from rtc.dbo.Delegate as d

inner join dbo.Resource as r1

on r1.ResourceId = d.DelegatorId

inner join dbo.Resource as r2

on r2.ResourceId = d.DelegateId

where r1.UserAtHost = 'boss@vdomain.com'

What is important to note here about delegation is this is the final source of information about delegation. In various cases when the server needs to know if a delegate relation exists, it will send a message to boss’s front end, which will in turn lookup the relationship in this table. Only the delegate table on the boss’s front end matters – not the table on the admin’s front end.

Now comes the disclaimer – you may actually see these rows in other front ends because they are automatically replicated to the backup front ends. Therefore you may see the rows on admin’s FE if that FE is a backup for boss’s.

Now the question will naturally arise – how does admin learn that she is a delegate for boss? A quick examination of the delegate table on her FE will bring up nothing – so how is this stored and sent. The answer is related to the prompted subscriber notification sent earlier. A prompted subscriber message is sent to admin with one key difference.

NOTIFY sip:192.168.0.240:52807;transport=tls;ms-opaque=a589590cd5;ms-received-cid=1ABE500;grid SIP/2.0
To: ;tag=de5be0b9f5;epid=fc44a406d8
From: ;tag=79520080
Call-ID: 904acbdc96704c16afe7206e92e7fd09
CSeq: 8 NOTIFY
Require: eventlist
Content-Type: application/vnd-microsoft-roaming-self+xml
Event: vnd-microsoft-roaming-self
Supported: ms-dialog-route-set-update

-
                sub="https://schemas.microsoft.com/2006/09/sip/presence-subscribers">
    -
        -
                        displayName="MrBoss Bosso"
                        acknowledged="false"
                        type="sameEnterprise">
            -
                -
                                majorVersion="1"
                                minorVersion="0">
                    -

                    -

                -
            -
        -
    -

That little node called “delegation” is the key. You can see this context also if querying the PromptedSubscriber table. This element informs the client that this is no normal prompted subscriber but instead a delegation configuration. Admin now knows that she is a delegate for boss.

Unlike an actual prompted subscriber, however, the client never acknowledges it and the entry remains in admin’s prompted subscriber table until the relationship is removed. From the server’s perspective, this is simply an unacknowledged prompted subscriber and each time admin signs and self subscribes to her subscribers the server will respond with this one.

So, now both admin and boss are aware of the delegation relationship. Boss will continue to see the relationship returned in the delegates category subscription each time she signs in and admin will continue to see the relationship in the subscribers category subscription with a context that contains the delegation element each time she signs in. The Delegate table on boss’s front end will remain the master arbiter of delegate decisions until boss is moved to another machine or a failover occurs.

Debugging address book issues with Debug-CsAddressBookReplication

jscalev — Wed, 10 Oct 2012 15:51:04 GMT

As I explained in a previous post, the address book component of Lync underwent major changes in Lync Server 2013, but the basic functionality remains the same. Address Book is a deceivingly simple component. While how it works is easy to explain, there are a great number of things that may go wrong. Debugging issues can therefore become very difficult.

In Lync 2010, we introduced Powershell cmdlets such as Test-CsAddressBookWebQuery and Test-CsAddressBookService. The former is used to test the web query while the latter is used for file download. Documentation may be found within the management console (get-help cmdlet-name) and elsewhere online. These cmdlets are useful for determining whether a problem exists but are not as useful for determining why.

For this reason we have Debug-CsAddressBookReplication. This cmdlet provides a wealth of information about the replication process itself and provides the ability to process an individual user. There are several ways to use it.

This is the simplest use.

PS C:\> Debug-CsAddressBookReplication

When run without parameters, the cmdlet uses the default DC and whatever pool the machine is in. It provides the following general info.

Pool Fqdn – This is the pool that was checked.

Replication State – I will get to this one later.

Task Owner Fqdn – If this is an enterprise pool, this states which front end owns the address book task. This is the machine that should be checked for event logs for address book replication.

Backup Fqdn – Unfortunately due to a bug this should be ignored.

Indexed Object Count – This is the number of users searchable from this pool.

Objects not indexed that should be – This should always be 0. If it is not 0, then there is a problem indexing some users. This is a good time to call support.

Abandoned objects – This should also always be 0, but there are a few scenarios where non-zero numbers of acceptable. Proper understanding of this value and the previous one will require a future blog post.

Normalization Failure Count – This is the number of users who had a number that failed to normalize. In Lync 2010 this information was provided by a text file on the file share. It is now available from Powershell, but I will explain more shortly.

Normalization Failures – When requested, this will provide the users and fields that failed to normalize.

Server Heartbeats – This provides information on the current status of all front ends in the pool. If a front end has not heartbeated for some time, the backend will consider it down and will not assign tasks to it. To view more information, do something similar to the following.

PS C:\> $a = Debug-CsAddressBookReplication
PS C:\> $a.ServerHearbeats

Server                         Last Hearbeat          Last Register          Last Unregister
------                         -------------          -------------          ---------------
SERVER.vdomain.com             10/10/2012 10:10:48 PM 10/3/2012 5:31:21 PM   1/1/0001 12:00:00 AM

So what can you determine from this information? This will verify that address book replication is up and running and where to look for event logs. The objects not indexed value is perhaps the most important. A non-zero number there indicates trouble. It means there are users that should be searchable but are not.

The output differs significantly with the –User parameter. This parameter accepts one of the following

SIP address
Active Directory object guid
Mail
SamAccountName

It will search for the user in Active Directory and return results similar to the following.

PS C:\> Debug-CsAddressBookReplication -user sip:iggy@vdomain.com

Pool Fqdn                          : pool0.vdomain.com
Replication State                  : NotTested
Task Owner Fqdn                    : SERVER.vdomain.com
Backup Fqdn                        : fakepool.vdomain.com
Server Heartbeats                  : {Microsoft.Rtc.SyntheticTransactions.Activities.Database.AbServerHeartbe
                                     at}
Indexed Object Count               : 122
Objects not indexed that should be : 0
Abandoned objects                  : 0
User Guid                          : 98fea012-915c-4545-9ccf-6d981fc1dcd1
Distinguished Name                 : CN=iggy Flipopovich,CN=Users,DC=vdomain,DC=com
Sip Address                        : sip:iggy@vdomain.com
Attribute Values                   : {entryid=98fea012-915c-4545-9ccf-6d981fc1dcd1, sn=flipopovich,
                                     givenName=iggy, msRTCSIP-PrimaryUserAddress=iggy...}
Is Indexed                         : True
Should Be Indexed                  : True
Is Processed                       : True
Normalization Succeeded            : True
Manager                            :
Normalization Failure Count        : Not checked
Normalization Failures             :

With the –User parameter the cmdlet provides the following additional information.

User Guid – This is the guid of the object in Active Directory.

Distinguished Name – The DN of the object in Active Directory.

Sip Address – Should be obvious

Is Indexed – If true, then the user is searchable. If false, other users will not be able to search for this user. Note that there are some users (those will attributes set specifying Address Book should ignore them) where false is the valid value.

Should be Indexed – This indicates whether the user should be searchable. If this value is true and is indexed is false, then there is a problem. If this value is false but you expect the user to be searchable, then you have likely set some attribute for the user causing address book to ignore it or the user does not have a Sip address or phone number.

Is Processed – Whether address book processed this user at all. If the user has a value indicating address book should ignore it, this will be false.

Normalization succeeded – Whether the user has any phone numbers that failed to normalize.

Manager – The distinguished name of the user’s manager, if set.

Attribute Values - Key value pairs of the information we have for the user in the database. This is what is available for search for the user. To see them better, do something similar to the following.

PS C:\> $a = Debug-CsAddressBookReplication -User sip:iggy@vdomain.com
PS C:\> $a.AttributeValues

Name                           Value
----                           -----
entryid                        98fea012-915c-4545-9ccf-6d981fc1dcd1
sn                             flipopovich
givenName                      iggy
msRTCSIP-PrimaryUserAddress    iggy
displayName                    iggy flipopovich
msRTCSIP-PrimaryUserAddress    iggy@vdomain.com
msRTCSIP-PrimaryUserAddress    sip:iggy@vdomain.com

The following other parameters are useful.

DomainController – Specifies the DC to use instead of the default.

PoolFqdn – Specifies the pool to examine instead of the one the current machine belongs to.

VerifyNormalization – This will provide information about all users in the database that failed to normalize. It is a potentially expensive operation because it must pull apart binary blobs in the database to provide the information. Therefore it should not be run at mission critical times. The following is an example of its use.

PS C:\> Debug-CsAddressBookReplication -VerifyNormalization

Normalization failures
Return information on users that failed to normalize.
[Y] Yes [N] No [S] Suspend [?] Help (default is "Y"): Y

Pool Fqdn                          : pool0.vdomain.com
Replication State                  : NotTested
Task Owner Fqdn                    : SERVER.vdomain.com
Backup Fqdn                        : fakepool.vdomain.com
Server Heartbeats                  : {Microsoft.Rtc.SyntheticTransactions.Activities.Database.AbServerHeartbe
                                     at}
Indexed Object Count               : 122
Objects not indexed that should be : 0
Abandoned objects                  : 0
Normalization Failure Count        : 40
Normalization Failures             : {orgSearchU3@vdomain.com(942e3b3d-e0f0-4dc2-b836-0193e075e49c)
                                     otherTelephone=288-2062 x8865,
                                     orgSearchU3@vdomain.com(942e3b3d-e0f0-4dc2-b836-0193e075e49c)
                                     ipPhone=616-3862 x4841,
                                     InetOUser@vdomain.com(5c49c430-df18-4e2a-8371-088c12c5a44a)
                                     telephoneNumber=708-1283 x7773,
                                     InetOUser@vdomain.com(5c49c430-df18-4e2a-8371-088c12c5a44a) ipPhone=+1
                                     (441) 475-3351 EXT3825...}

To better view the normalization failures do something similar to the following.

PS C:\> $a = Debug-CsAddressBookReplication -VerifyNormalization

Normalization failures
Return information on users that failed to normalize.
[Y] Yes [N] No [S] Suspend [?] Help (default is "Y"): y
PS C:\> $a.NormalizationFailures

AttributeName               AttributeValue              SipAddress                 UserGuid
-------------               --------------              ----------                 --------
otherTelephone              288-2062 x8865              orgSearchU3@vdomain.com    942e3b3d-e0f0-4dc2-b836...
ipPhone                     616-3862 x4841              orgSearchU3@vdomain.com    942e3b3d-e0f0-4dc2-b836...
telephoneNumber             708-1283 x7773              InetOUser@vdomain.com      5c49c430-df18-4e2a-8371...
ipPhone                     +1 (441) 475-3351 EXT3825   InetOUser@vdomain.com      5c49c430-df18-4e2a-8371...
telephoneNumber             700-8800 x0360              Grp_manager2@vdomain.com   f0ed5c70-c279-4955-956b...
ipPhone                     706-7007 x2837              Grp_manager2@vdomain.com   f0ed5c70-c279-4955-956b...
telephoneNumber             707-8044 x0088              MoveUserTest@vdomain.com   f800e93d-945c-48f1-89e1...
otherTelephone              704-4374 x5370              MoveUserTest@vdomain.com   f800e93d-945c-48f1-89e1...
telephoneNumber             170-6181 x7880              orgSearchU1@vdomain.com    590e071a-8e4a-41d9-8522...
ipPhone                     +1 (022) 152-4752 EXT5172   orgSearchU1@vdomain.com    590e071a-8e4a-41d9-8522...
otherTelephone              318-1125 x1133              orgSearchU16@vdomain.com   9ef32825-2a4b-4d97-b17b...
otherTelephone              463-4133 x2783              abwsTenantTester@user      77b2b3f3-302c-44b8-a12a...
ipPhone                     705-4041 x6023              abwsTenantTester@user      77b2b3f3-302c-44b8-a12a...
otherTelephone              +1 (520) 125-4267 EXT5563   orgSearchU15@vdomain.com   5dbb9fbe-c026-40ae-bbcb...
telephoneNumber             157-1563 x4363              vt1_user66NT@vdomain.com   872d2a5b-48ea-4159-b016...
ipPhone                     705-5734 x5503              vt1_user66NT@vdomain.com   872d2a5b-48ea-4159-b016...
otherTelephone              708-5235 x8175              orgSearchU6@vdomain.com    4d545448-8059-4ec4-8415...
ipPhone                     704-8184 x3017              orgSearchU6@vdomain.com    4d545448-8059-4ec4-8415...
telephoneNumber             +1 (445) 577-7031 EXT0542   orgSearchU5@vdomain.com    1fcc7136-9715-4dab-b811...

As you can see, this will provide the specific attribute, value, Sip address and user guid in Active Directory for the problem accounts.

VerifyReplication – This will do the following for a user.

Append a value to the sn in Active Directory
Wait for replication to occur
Verify the new sn appears in the database
Changes the sn back to the previous value

For most of you, this is most useful for obtaining logs to send to support as this forces replication for one user – making the logs far easier to read. For most other purposes you should use Update-CsAddressBook.

-Verbose – Similar to all other cmdlets, this provides more information in the console. Some may prefer the output here to that above.

While certainly our hope is you will never need to use this cmdlet, it is there to provide useful information that can help you better understand what is happening when a user fails to appear within search results.

Address Book changes for Lync 2013

jscalev — Tue, 02 Oct 2012 15:06:23 GMT

For those of you familiar with the address book from previous versions of Lync, you will find the address book feature in Lync 2013 very familiar. On the outside we have made very few changes. The Powershell cmdlets used to manage the address book are essentially the same (with one change that I will mention soon) and the Soap interfaces to call it remain unchanged. Nevertheless address book has undergone dramatic changes in this wave. Before I began discussing the changes, let's rehash how ABS worked in Lync 2010.

In Lync 2010, address book required two databases - named rtcab and rtcab1. At any time one of these databases is active and one is being rebuilt. The active database is used to satisfy queries. Every day (usually at night) a sync process makes the inactive partition active and rebuilds the now inactive partition. This may lead to delays from the time a user is present in Active Directory and can sign in to when the user is searchable in the address book.

Let us now turn our attention to where this sync process (running as ABServer.exe on one of the front ends) obtains its data. It reads its data from the rtc database, which is populated by user replicator (running within RtcSrv.exe). Whenever a change is detected in Active Directory, user replicator writes the change to the AbUserEntry table in the rtc database and once a day ABServer.exe reads the entire contents of this table and rebuilds one of the rtcab databases.

As you can see, this process is not overly efficient. From the time a user is replicated to Active Directory to when the user is searchable could be as long as two days (if the user is entered right after a sync process begins). Therefore our primary goal for this release was to fix this.

The improvement is quite simple in description but far more complicated in implementation. User replicator now writes the data directly to rtcab. Address book data is no longer stored in rtc in the interim and rtcab1 is no longer used. Instead rtcab is updated with changes from Active Directory at the same time it responds to queries. This required rewriting most of address book to fit into user replicator, which is native code as compared to ABServer.exe which is managed.

From an administration perspective there is little change. We still handle normalization files exactly the same and the Powershell cmdlets work very similarly. The only change lies in the cmdlets Update-CsUserDatabase and Update-CsAddressBook.

In Lync 2010, it was necessary to first run Update-CsUserDatabase to force user replicator to completely populated the rtc database and then run Update-CsAddressBook to trigger a sync run. In Lync 2013 it is no longer necessary to call Update-CsUserDatabase for this purpose. In fact for address book this cmdlet will be a noop. We have subtly changed Update-CsAddressBook to trigger user replicator to update address book data.

Although feature wise we have not improved address book, this is really a major update.

Partitioning Lync Address Book using msRTCSIP-GroupingID

jscalev — Thu, 07 Jun 2012 16:06:14 GMT

In certain large organizations, the need exists to not allow certain users to search for others. One simple example of this is a school district. Perhaps the school district wants to only allow users within schools to search for each other. So if I belong to Cougar Hill Elementary I can search for my fellow teachers there, but I cannot search for users at Tiger Hill Elementary.

Partitioning users in this way is supported using a little known AD attribute called msRTCSIP-GroupingID. Before I explain this further, however, I would like to mention that this does not prevent users from communicating with each other, it simply prevents them from finding each other using the address book. Therefore, while I cannot look up another teacher at Tiger Hill Elementary if I am not in the same grouping, I can call or IM the teacher if I know the Sip address or phone number (though I will not be able to see the user’s photo).

The basic premise is you can define groups of users in your organizations and assign them different groupingIds. A groupingId is simply a guid and only users who have the same groupingId may search for each other. To set the groupingId for a user, you must set the value of msRTCSIP-GroupingId in Active Directory.

Unfortunately this is slightly tricky as we do not expose this attribute anywhere in our UI, nor do we expose it in Powershell. The best way to set this, therefore, is to make use of Active Directory’s Powershell module.

import-module ActiveDirectory

To select your users, you may use either Get-CsUser or Get-AdUser. If you choose to use Get-CsUser, there is a simple way to convert between them.

$myuser = Get-CsUser myuser@mydomain.com

Get-AdUser $myuser.Identity.DistinguishedName –Properties * | get-member

This will show all of the properties in Active Directory for the user myuser@mydomain.com. To work on multiple users at a time you may pipe the results.

Get-CsUser | foreach {Get-AdUser $_.Identity.DistinguishedName}

Though I certainly would not run the above command on most servers due to the result set size. To set the value of groupingId, the following will work.

$guid = [guid].NewGuid()

$myuser = Get-CsUser myuser@mydomain.com

Set-AdUser $myuser.Identity.DistinguishedName –Replace @{“msRTCSIP-GroupingID”=$guid}

Again, you can either create a script to do this for all the necessary users, or you can pipe the output. Just make sure all users you want to be able to see each other in the address book have the same groupingId.

How to enable last name, first searches in Lync Address Book

jscalev — Wed, 06 Jun 2012 14:35:00 GMT

Over time, a few people have asked me why, given someone with a first name of Ziggy and a last name of Zooga, you will see the following results when searching using the Lync Address Book.

Ziggy – success

Zooga – success

Ziggy Zooga – success

Zooga Ziggy – failure

Shouldn’t we be able to find this when searching? Alas, this is not the way things work. However, there is a way to fix this.

Now before I go into details I must first provide a disclaimer. This workaround uses a completely undocumented and unsupported feature.

That being said, let’s get to the fun part. First, a bit of explanation on how we process attributes is necessary. When the address book processes an entry, it iterates through each attribute that it is interested in (which itself is configurable using a resource kit utility) and reads the value. It then may choose to break up the value. Some values, such as displayName, are just passed as is to the next step. A few attributes, however, are handled specially. One example of this is msRTCSIP-PrimaryUserAddress. Given a Sip address, we expect the following searches to be possible.

sip:ziggy@mydomain.com

ziggy@mydomain.com

ziggy

To do this, we use regular expressions to pull the value apart into the above pieces.

The finally piece involves using normalization rules to apply formatting to phone numbers. This is detailed in a number of other blogs. As you can guess, if there is a way to pull apart names, we may be able to use it to our advantage.

The way to do this is via a file called RtcAbAttributeIndexRules.xml. This file exists in the server\core directory on each front end and you will need to restart RtcSrv if you change it. The main reason for this is we never intended for you to need to change this file.

If you open this file, you will see an attributes node at the top and a charDigitMaps node towards the bottom. In this post, I will only cover the attributes node. Each of the attributes under the attributes node tells us how to pull apart a particular attribute. Most of these are “as-is” or in other words we change nothing.

                  Index sn as-is
        ]]>

You can see that the regular expression for valuePattern will match anything passed in. The index value, in turn, says to use the entire match (group 0) so the result is the string is unchanged. Now compare this to what we do for the Sip address.

                    Index msRTCSIP-PrimaryUserAddress as-is
        ]]>












                  "sip:user@domain.com" =>
          "user"
          "user@domain.com"
        ]]>

Here we index the value of msRTCSIP-PrimaryUserAddress as is, but we have another index rule that has a regular expression to remove the sip: (or whatever precedes the colon) from the address. We then reference the regular expression groups (the sip: is group 1) to add values without the protocol and without the domain.

If you think this may work for last name, first name searches, you would be partially correct. This will work, but is a limited solution. First I will explain the solution, then I will discuss its limitations. The following is one way to handle displayName.

                  Index displayName as-is
        ]]>












                Handle last name first name searches
      ]]>

Here, you can see that we are simply looking for one or more white space characters between the names and reversing it. There is no need to add an index for [$1 $3] because the first index rule will cover that. If you try this out, you will see that “Zooga Ziggy” is now searchable.

Of course, there are limitations to this approach. Besides the pain of setting this on every front end and restarting the service there, the solution is limited by the following.

1) This will obviously only work for two names. You can, of course, create additional rules for three and four names, but at some point a user will simply have too many names.

2) This approach will not work for languages that do not use spaces. In particular Chinese and Thai have this issue and the solution will not work there.

Nevertheless, this solution can be used to improve searches in your organization, even if it does not cover every name.

How to create a new group and add contacts to it in UCMA

jscalev — Mon, 04 Jun 2012 17:07:38 GMT

It is quite easy to add groups, add contacts, and enumerate both contacts and groups using UCMA’s ContactGroupServices extension.

To enumerate groups, you can use the following.

ICollection groups = endpoint.ContactGroupServices.EndGetAllGroups(endpoint.ContactGroupServices.BeginGetAllGroups(null, null));

To add a group, the following will work.

endpoint.ContactGroupServices.EndAddGroup(endpoint.ContactGroupServices.BeginAddGroup(“mygroupname”, null, null, null));

And to add a contact in this new group you may use the following.

ContactAddOptions contactOptions = new ContactAddOptions();
contactOptions.GroupIds.Add(groupId);

// Add the contact
endpoint.ContactGroupServices.EndAddContact(endpoint.ContactGroupServices.BeginAddContact(contactUri, contactOptions, null, null));

Where it becomes slightly confusing is, how do you obtain the groupId? At first glance, you may think the answer is to add the group, call GetAllGroups to retrieve the groups, find your new group, and use its groupId. Unfortunately, that is not the way to do it.

The problem is you really do not know when the group will actually be added, and until it is actually added the group will not have a groupId – so GetAllGroups will not return it.

The answer is therefore to subscribe for notifications and wait for the group to be created. Here is some sample code demonstrating this.

int groupId = 0;

AutoResetEvent foundGroupIdEvent = new AutoResetEvent(false);
EventHandler contactNotificationHandler =
    delegate(object sender, ContactGroupNotificationEventArgs e)
    {
        foreach (NotificationItem groupNotification in e.Groups)
        {
            if (String.Equals(“mygroup”, groupNotification.Item.Name, StringComparison.InvariantCultureIgnoreCase))
            {
                groupId = groupNotification.Item.GroupId;
                foundGroupIdEvent.Set();
            }
        }
    };

endpoint.ContactGroupServices.NotificationReceived += contactNotificationHandler;
endpoint.ContactGroupServices.EndAddGroup(endpoint.ContactGroupServices.BeginAddGroup("mygroup", null, null, null));
foundGroupIdEvent.WaitOne(WaitTimeForGroupIdInSeconds * 1000, true);
endpoint.ContactGroupServices.NotificationReceived -= contactNotificationHandler;

Sending a SUBSCRIBE message with headers

jscalev — Fri, 01 Jun 2012 10:42:02 GMT

UCMA makes it quite easy to send a subscribe message and then add, remove, or retrieve contacts and groups.

endpoint.ContactGroupServices.EndSubscribe(endpoint.ContactGroupServices.BeginSubscribe(null, null));

In general the ContactGroupServices object, accessible from the Microsoft.Rtc.Collaboration.ContactsGroups namespace, provides a number of useful methods that make it easy to work with contacts.

Recently, however, I came across the challenge that I needed to add a header to my SUBSCRIBE message. Granted, this is not an extremely common scenario but there are a number of headers that have special meanings to Lync and I needed to add one of them.

Unfortunately, this is where ContactGroupServices loses its usefulness as it does not allow you to change the headers. You will therefore need to move one level down to the Sip signaling level. Fortunately, there is an object here that will help – SipSubscription.

This is a slightly confusing object to use, so I thought I would provide some clarification here. The following is how you will send the SUBSCRIBE message.

SipSubscription subscription = new SipSubscription(

endpoint.InnerEndpoint,

target,

MicrosoftRoamingContactEvent,

contactSubcriber);

try

{

subscription.EndSubscribe(subscription.BeginSubscribe(null, null));

}

catch (Exception ex)

{

// Do something slightly more advanced than crying

}

Well, that isn’t so difficult. But what are these parameters you are passing? Well, first I am assuming the endpoint object you have is of type UserEndpoint. Therefore, the first object is of typeRealTimeEndpoint, which is what the lower level Sip signaling layer uses for your endpoint . This is easily obtainable using the InnerEndpoint property of UserEndpoint.

The second parameter is also not rocket science. It is a RealTimeEndpoint indicating the target address. If you are subscribing to yourself, the easiest way to obtain this is the following.

RealTimeAddress target = new RealTimeAddress(endpoint.OwnerUri);

The third parameter is the event package name. There are different event packages possible, but for simplicity sake I will just tell you that the one ContactGroupServicesuses is called “vnd-microsoft-roaming-contacts”. So just pass this string for this case. A Bing search will provide more information on this document as well as other event packages available.

The last parameter is where the fun begins. This must be an instance of an object that implements the ISipSubscriptionProcessor interface. This interface has the following methods.

GetExtensionHeaders– If you guessed that this is where you need to add your headers, you would be correct.

GetMessageBody –Typically you do not need to do much here for contacts.

ProcessErrorResponse– If something bad happens, this is how UCMA will notify you.

ProcessNotification– When contacts are returned, UCMA will call you here. Unfortunately you will need to parse the xml returned to understand what happened – something that ContactGroupServicesnormally handles for you.

SubscriptionStateChanges– Called each time your subscription changes.

This is not a difficult interface to implement, but the entire process is not overly straightforward. The following is my simplified implementation of it.

public class AddHeaderSubscriber : ISipSubscriptionProcessor

{

private Exception _exception;

public Exception Exception

{

get { return _exception; }

}

public void GetExtensionHeaders(SipSubscription.RequestType requestType, out System.Collections.Generic.IEnumerable<SignalingHeader> extensionHeaders)

{

// Add our header

List<SignalingHeader> headers = new List<SignalingHeader>();

headers.Add(new SignalingHeader("SUPPORTED", "levitation"));

extensionHeaders = headers;

}

public void GetMessageBody(SipSubscription.RequestType requestType, out System.Net.Mime.ContentType contentType, out byte[] messageBody)

{

messageBody = new byte[0];

contentType = null;

}

public void ProcessErrorResponse(SipResponseData message)

{

_exception = new Exception(message.ResponseText);

}

public void ProcessNotification(SipMessageData message)

{

// Process a contact if we dare

}

public void SubscriptionStateChanged(SubscriptionStateChangedEventArgs eventArg)

{

switch (eventArg.NewState)

{

case SubscriptionSignalingState.Subscribed:

// We subscribed!

break;

case SubscriptionSignalingState.Subscribing:

// We are thinking about it

break;

case SubscriptionSignalingState.Terminated:

// I'll be back!

break;

case SubscriptionSignalingState.WaitingForRetry:

// Doo bee doo bee doo bee doo

break;

default:

// ignore other states

break;

}

Note that it is entirely possible to use bothContactGroupServices and SipSubscription together, though I recommend unsubscribing from one first before using the other. When possible it is always best to allowContactGroupServices to handle the bulk of the work. To unsubscribe with SipSubscription, useBeginTerminate and EndTerminate.

Why is UCMA telling me my session is not established when I add a contact?

jscalev — Thu, 24 May 2012 16:22:29 GMT

Recently I found myself in the situation where I need to add a contact to a user using UCMA but did not care about what contacts currently existed. I did the standard plumbing to create a CollaborationPlatform and register the user, then added the contact with the following code.

userEndpoint.ContactGroupServices.EndAddContact(this.Subscriber.ContactGroupServices.BeginAddContact(contactUri, null, null));

To my surprise, this failed with the following error.

Failed to add contact: The operation can be performed only when the state of the session is established.

I spent some time scratching my head over this one, double checking that my session indeed was established, until I gave up and used my privileges at MSFT to debug into UCMA itself to see what was going on.

The issue is we do not support adding a contact without a subscription. At first you may wonder why this is the case. “But I don’t care what the current contacts are!” you may say. That may be true, but in order to publish a contact add request, we need a delta number so the server can sort out multiple requests. In order to receive this delta number you must subscribe. Fortunately, subscribing is quite easy – just use the BeginSubscribe and EndSubscribe methods on ContactGroupServices.

So if you happen to see this message, rest assured that there is nothing wrong with the session for your endpoint; it is the subscription session we are complaining about.

Running UCMA 2.0 applications on Lync Server 2010

jscalev — Thu, 23 Sep 2010 11:47:00 GMT

While the most ideal scenario is to only run UCMA 3.0 applications with Lync Server 2010, for many of you this is simply not possible. You may own an application that was written for UCMA 2.0 and you want to upgrade to Lync Server 2010, but the application has not yet been updated to UCMA 3.0. Likewise, you may want to continue running your UCMA 2.0 application on an Office Communications Server 2007 R2 front end but still want it to be accessible by users homed on a Lync Server 2010 front end.

These scenarios are all possible, but unfortunately the steps to configure them are rather confusing. In some respects, this is one area where you really need to follow the directions to the exact letter. Several steps may look rather peculiar but unfortunately must be run exactly as specified.

A note about the UCMA 3.0 runtime

Unfortunately, UCMA 3.0 and UCMA 2.0 cannot be setup on the same machine. This is a limitation caused by their dependencies and therefore when you plan a topology that contains both UCMA 2.0 and UCMA 3.0 applications, you must take this into account. If you have applications of both types, they must exist on separate machines.

UCMA 2.0 on OCS 2007 coexisting with Lync Server 2010

This is the most common scenario, and from here I will cover the typical lifetime of such an application. I will cover directly installing a UCMA 2.0 application on Lync Server 2010 at the end of this blog.

In this scenario, you have at least two pools. One pool has Lync Server 2010 and the other pool has OCS 2007 R2. If your application is not already installed on the OCS 2007 R2 pool, you can do so by running the ApplicationProvisioner tool that is used with OCS 2007 R2 or making the WMI changes yourself. This is the same setup procedure for any OCS 2007 R2 application. After you do this, there is one more step to take to enable it to coexist with Lync Server 2010.

You will need to run the Merge-CsLegacyTopology cmdlet. To use it, run the following

Merge-CsLegacyTopology -TopologyXmlFileName newtopology.xml
Publish-CsTopology -FileName newtopology.xml
Enable-CsTopology

Merge-CsLegacyTopology reads information from WMI and the current Central Management Server topology and creates new entries for the OCS 2007 R2 servers not already in the topology. You then must publish this new topology document for the changes to take effect. The UCMA 2.0 application residing on the OCS 2007 R2 server will now interoperate with Lync Server 2010.

Migrating a UCMA 2.0 application running on OCS 2007 R2 to Lync Server 2010

So your application is happily coexisting now, when your boss informs you that you should retire the OCS 2007 R2 pool and only use Lync Server 2010. This means you will now need to migrate your application to Lync Server 2010. What is the best (only) way to do this?

When you ran Merge-CsLegacyTopology, it created a new trusted application pool for your application's server. This pool is similar to a Lync Server 2010 pool but has a lower version number. However, what really determines what version your application is running under is the version of the registrar that the trusted application pool depends on. Therefore, the first part of migration will be changing the registrar from the OCS 2007 R2 registrar to a Lync Server 2010 registrar.

To view the current registrar, run Get-CsTrustedApplicationPool {pool fqdn}. You will see the Registrar dependency listed.

On your system, though, the version will be set to 4 instead of 5. OCS 2007 R2 = Version 4, while Lync Server 2010 = Version 5. To see the registrar, run Get-CsService {registrar identity}. Alternatively you can use

$myserver = Get-CsTrustedApplicationPool {fqdn}
Get-CsService -Identity $myserver.Registrar

Of course, your registrar will show a version of 4. You can view the available registrars using Get-CsService -Registrar. Once you have selected the registrar, you will need to run the following to change the registrar of the OCS 2007 R2 trusted application pool to a Lync Server 2010 registrar.

Set-CsTrustedApplicationPool {fqdn} -Registrar {identity of Lync Server 2010 registrar}

The interesting thing is normally Set-CsTrustedApplicationPool does not allow you to make any changes to an OCS 2007 R2 pool. To make changes to an OCS 2007 R2 pool, you must make the appropriate changes in WMI and then rerun Merge-CsLegacyTopology as shown above. However, this is the one exception to the rule and Lync Server will allow you to use this cmdlet for an OCS 2007 R2 pool if the only thing being changes is the registrar and the registrar is a Lync Server 2010 registrar. Note that this is a one way operation! It is not possible to change the registrar back to an OCS 2007 R2 registrar.

Once this step is complete, you are still not finished. You will need to change all contact objects so they work in Lync Server 2010. To do this, you will need to migrate them. The syntax for doing this is.

Move-CsApplicationEndpoint {endpoint identity} -TargetApplicationPool {pool fqdn} -Force

Yes, you need to move each endpoint to the pool where it is already homed. In addition, the Force flag is extremely important. Without it the move will not work and you will notice a DCOM error. Once you have 'moved' all of your contacts, your application is now running on Lync Server 2010 and you can safely retire the OCS 2007 R2 pool.

Also note that whenever you create a new contact, you will again need to 'move' it using this cmdlet.

Migrating to a UCMA 3.0 application on Lync Server 2010

Your application is happily chugging away on Lync Server when a few months later your boss comes in and mentions that a UCMA 3.0 version of your application has now been released! You now need to configurate the UCMA 3.0 application on Lync Server while making sure all contacts are preserved. Provisioning the UCMA 3.0 application on Lync Server is the easy part and I will leave that to the product documentation. Where things get tricky is the fact that only one application with the same applicationId can depend on a given registrar at one time.

What you will therefore have to do is set the legacy application's registrar to null first, then configure your UCMA 3.0 application on Lync Server. This means that a service outage in this scenario is unavoidable. To set the registrar to null, simply run.

Set-CsTrustedApplicationPool {pool fqdn} -Registrar $null

Once your UCMA 3.0 application is configured and the pool depends on the same registrar, you will need to run the following to again 'move' your contacts.

Move-CsApplicationEndpoint {identity] -TargetApplicationPool {fqdn} -Force

Again, the pool is the fully qualified domain name of the trusted application pool where the contacts reside. Again this is the exact same pool where they already reside and again the force flag is required.

Luckily, from here the ApplicationProvisioner tool or WMI changes are no longer required. You can now use New-CsTrustedApplicationEndpoint to create new endpoints for your application as it uses UCMA 3.0.

Installing a UCMA 2.0 application on Lync Server 2010

Now that we have gone through the above steps, you may be able to guess how one installs a UCMA 2.0 application on Lync Server. To do this, you need to first use the OCS 2007 R2 ApplicationProvisioner tool or WMI to provision your application exactly as you did in OCS 2007 R2. For the registrar though, you will use the Lync Server 2010 registrar.

The next step is to run Merge-CsTopology

Merge-CsLegacyTopology -TopologyXmlFileName newtopology.xml
Publish-CsTopology -FileName newtopology.xml
Enable-CsTopology

This will create the OCS 2007 R2 version of the pool in your Lync Server topology. Even though you do not have an OCS 2007 R2 front end, as far as Lync Server is concerned this is still an OCS 2007 R2 pool.

Finally, you will need to 'move' your contacts.

Move-CsApplicationEndpoint {identity] -TargetApplicationPool {fqdn} -Force

In order to create new contacts for your application, you will need to continue using the ApplicationProvisioner tool and then run Move-CsApplicationEndpoint with the above parameters. You will not be able to use the *-TrustedApplicationEndpoint cmdlets because they will only work with UCMA 3.0 applications running on Lync Server.

The Complexity of Simple Urls

jscalev — Wed, 22 Sep 2010 17:28:00 GMT

One of the many new features in Lync Server 2010 is a change to how meeting Urls are formatted. The name of this feature is "Simple Url", but in truth this this elicits a smirk from anyone on our team who has dealt with them, as simple Urls are anything but simple. The goal of this blog will be to shed some light on this rather confusing topic.

What is a Simple Url?

In Offices Communications Server 2007, when you send someone a meeting invite, the Url is formatted similar to the following.

https://cwa.contoso.com/join?uri=sip%3Aadon%40microsoft.com%3Bgruu%3Bopaque%3Dapp%3Aconf%3Afocus%3Aid%3A3814A82809A34ED0958CFDB60957BDF&key=923089

This isn't exactly easy to remember and if you needed to transcribe it to someone it would take a few minutes. One of the goals in Lync Server 2010 was to make these Urls much easier - hence the name "simple" Url. Now, Url's look similar to the following.

https://meet.contoso.com/071200

This is a lot easier, right? The only drawback here is these Urls must now be configured. To configure these Urls, you must create simple Urls.

So how can I create a simple Url?

There are two options for creating simple Urls. You can either use the topology builder or you can use Powershell to create simple Urls. It is also possible to view the existing simple Urls in the Lync Server Control Panel, but simple Urls cannot be changed there. This blog will cover managing simple Urls in Powershell.

The primary cmdlets used for creating simpleUrls are.

Get-CsSimpleUrlConfiguration

New-CsSimpleUrlEntry

New-CsSimpleUrl

The following is an example on creating a new Simple Url.

$newUrlEntry = New-CsSimpleUrlEntry -Url "https://meet.vdomain.com"

$newUrl = New-CsSimpleUrl -Component "meet" -Domain "vdomain.com" -SimpleUrl $newUrlEntry -ActiveUrl "https://meet.vdomain.com"

Set-CsSimpleUrlConfiguration Global -SimpleUrl @{Add=$newUrl}

Each simple Url can contain multiple entries, but only one can be active. The ActiveUrl property determines which Url is active. In order to properly understand the other two parameters, I need to explain a bit more.

Sipdomains and Simple Urls

Within the topology document multiple Sip domains can be specified. Sip domains specify which domain names are valid for users, contacts, and other endpoints. All users must end with a valid Sip domain. For instance, if you have a user, john@contoso.com, a SipDomain for contoso.com must exist in the topology. If you have another user jack@marketing.contoso.com, the Sip domain marketing.contoso.com must also exist in the topology.

To view the existing Sip domains, use Get-CsSipDomain. To create a new Sip domain use New-CsSipDomain. Of all Sip domains, one domain is marked as the default and is used for creating new endpoints.

The domain parameter passed to New-CsSimpleUrl must be an existing SipDomain. Why is this necessary? As it happens, Sip domains and simple Urls are partners in crime. Neither can exist without the other.

When a user creates a conference, Lync Server examines the domain for that user and finds the corresponding simple Url. This is the simple Url where domain = {domain of the user}. The ActiveUrl of this simple Url is then used to create the meeting. Therefore, in order for the user to schedule online meetings a corresponding simple Url must exist.

Types of Simple Urls

There are actually three types of simple Urls. The one we have been discussing here is the "meet" type - specified by the Component parameter in New-CsSimpleUrl. The meet type is used for online meetings. There are also two other types.

Dialin - This is the Url to use for all dialin users. Note that the domain for a dialin simple Url must always be "*".

Admin - This allows one to create an easier Url for the Lync Server Control Panel.

After you have created your simple Urls

After you have created your simple Urls, you are still not finished. You must still create DNS A records for each of these simple Urls and you must run Enable-CsComputer on each machine that runs the web services role.

When Enable-CsComputer is run on each computer, it modifies the IIS service on each machine to contain the appropriate mappings for each Url. This is how simple Urls work.

Simple Urls and wildcards

In Office Communications Server 2007, it was possible to create allowed domains (the equivalent to Lync Server Sip domains) that contained a wildcard or simply specify that all domains are allowed. For instance, instead of creating entries for marketing.contoso.com and sales.contoso.com, one could create a single allowed domain *.contoso.com.

Unfortunately, due to the interdependence of simple Urls and Sip domains in Lync Server 2010 and the fact that simple Urls do not work with these schemes, we had to remove these features in Lync Server. Therefore, you must now specify all domains that are allowed and wildcards will not work. In cases where you have a large number of domains, it is possible to create many simple Urls and Sip domains using Powershell scripting.

SimpleUrlConfiguration and scope

As already stated, Get-CsSimpleUrlConfiguration returns a Central Management Server document that contains all configuration info for simple Urls. This document can be of global or site scope. For the vast majority of cases, you should only use the global scope. This means that this document will apply regardless where the user is homed.

However, there are some cases where users in a wide geographical area may share the same Sip domain, but you would prefer to minimize traffic that goes across. For instance, imagine that you have two sites - one in North Carolina and the other in Latvia. Users in both sites have the same Sip domain - contoso.com. In this case you could create two documents and scope them to each site.

New-CsSimpleUrlConfiguration Site:"North Carolina" -other params

New-CsSimpleUrlConfiguration Site:Latvia -other params

When determining the meeting Url, Lync Server will read the document for the site on which the user is homed. Note that Set-CsSimpleUrlConfiguration Site:Latvia will not work unless New-CsSimpleUrlConfiguration Site:Latvia has already been run.

Requirements of a simple Url

The following are requirements for all simple Urls.

- Left prefix simple Urls are not allowed. For instance the following two simple Urls would not be allowed because one is a subset of the other.

https://meet.contoso.com/meet

https://meet.contoso.com/meet/me

- Must begin with https:// (http is not allowed)
- Must have a corresponding Sip domain (see above)
- Cannot be empty
- Cannot be the same as a computer or pool Fqdn in the topology
- Cannot contain a query string parameter. A Url of the format https://meet.contoso.com/meet.htm?type=foo is not allowed.
- Must be a valid fully qualified domain name

Also note that we will automatically add a trailing / to the Url. Therefore if you create a Url https://meet.contoso.com/default.htm we will change this to https://meet.contoso.com/default.htm/.

Interaction between Powershell and the Topology Builder

As already mentioned, it is possible to create simple Urls using either Powershell or the topology builder. However, it is generally unwise to mix these together when creating a single batch of simple Urls. The behavior between the two is slightly different.

When using Powershell, if one creates a Sip domain, Lync Server will not create the corresponding simple Url. This must be accomplished separately. After all, how would the cmdlet know which Url to use?

The Topology Builder, however, will create a simple Url for each Sip domain. These Sip domains can then be edited using the Topology Builder and when you publish the topology, both the simple Urls and Sip domains will take effect at the same time.

One strange side effect though is if one creates the Sip domain using Powershell, then opens the Topology Builder and loads the topology, it will appear that the simple Urls were automatically created. This is sort of the case. The topology builder will notice that Sip domains exist for which no corresponding simple Urls exist and will create new simple Url entries that can be changed in the Topology Builder, but these changes will not take effect until the topology is published.

Conclusion

Simple Urls in practice may be simple for users, but they are far from simple for administrators. Most important is the relationship between simple Urls and Sip domains. Once these concepts are understood managing them is much easier.