In an on-premises Lync/Skype for Business deployment, the clients can be internal or external depending on whether they connect via the Edge or not. The location is written to the quality report sent by the clients and can be used when querying QoEMetrics and CQD. However in an Skype for Business Online deployments all clients are by definition external, since they all connect via the Edge. That has the implication that all clients by default are seen as Outside in CQD Online, no matter if the client was actually connected on an internal corporate network.

When you work with call quality you are interested in knowing the location of a client and whether it was using a network you can manage or a network you can't manage. The assumption being that you can only improve networks you can manage. By uploading network and building information to CQD Online you can enable CQD to determine, if the client was connect to an internal corporate/managed network or to an external/unmanaged network.

CQD use the Network and the NetworkRange to correlate the Lync/Skype for Business clients IP address to a network. When the clients sends it quality report (VQReport) to the server, it contains the actual IP address and subnet mask in use. CQD takes these two, calculates the subnet and correlates it to networks by using Network and the NetworkRange. The NetworkRange is expressed using CIDR. For example if the clients IP address is 10.1.1.15, CQD will be able to correlate that to the Lyngby Office network given the sample attached. You would also be able to see that this would fall in the Inside bucket, since the Lyngby Office network has InsideCorp = 1. Please note that if you use supernetting, you will have to specify Network and NetworkRange as your clients are configured, i.e. if you use 10.1.0.0/16 as a supernet, but your clients see 10.1.1.0/24 you need to have a record for 10.1.1.0/24 in your building information.

The file format is a tab separated text file (TSV) as described in the link above. The easiest way to generate the file is to use Excel. You can add a header line with the fields from the schema and then just add rows with the information. When you're done, save the information as "Text (tab delimited)" and using the extension *.tsv from within Excel. Remember to remove the header line from the resulting file before you upload it to CQD.

When you upload the network information you specify a date range. You can look at the date range as the validity period of the network information.

CQD will start processing the information as soon as the upload happens. You can monitor the progress by using the Tenant Health option on the gear icon. This will show you how far CQD has come in the processing. An example is shown below

To validate that CQD has used the uploaded information, look at the Server-Client Summary Report. You should now see information in the Wired Inside and/or Wifi Inside categories, as shown in the example below:

Thanks to Sasa and the CQD team for background information.

Building_and_Network_Sample.tsv

As implied by the name, the screen sharing flows via video streams, but how is it possible to distinguish these streams from "normal" video streams, when analyzing quality in QoE? It's easy. In Skype for Business Server 2015 we have introduced the MediaLineLabel = 16 to specify that this is a VbSS stream. In Lync Server 2013 these streams will have MediaLineLabel = 3 and be found in AppSharingStreams.

An example of looking at these streams using an SQL query against Skype for Business Server 2015 QoEMetrics database is below:

USE QoEMetrics;

DECLARE @beginTime AS DateTime = '1/1/2015';
DECLARE @endTime   AS DateTime = '12/31/2015';
                          

 SELECT
  s.StartTime as StartTime
  ,CallerUser.URI as Caller
  ,CalleeUser.URI as Callee
  ,vs.PacketUtilization
  ,vs.VideoAllocateBWAvg
  ,vs.InboundVideoFrameRateAvg
  ,vs.OutboundVideoFrameRateAvg
  ,vs.SendBitRateAverage
  ,vs.SendBitRateMaximum
  ,vs.RecvBitRateAverage
  ,vs.RecvBitRateMaximum
  ,vs.VideoPacketLossRate
  ,vs.SenderIsCallerPAI
  ,m.ClassifiedPoorCall

 FROM [Session] s WITH (NOLOCK)
  INNER JOIN [MediaLine] AS m WITH (NOLOCK) ON
   m.ConferenceDateTime = s.ConferenceDateTime
   AND m.SessionSeq = s.SessionSeq   
  INNER JOIN [VideoStream] AS vs WITH (NOLOCK) ON
   vs.MediaLineLabel = m.MediaLineLabel   
   and vs.ConferenceDateTime = m.ConferenceDateTime
   and vs.SessionSeq = m.SessionSeq
  INNER JOIN [User] AS CallerUser WITH (NOLOCK) ON
   CallerUser.UserKey = s.CallerURI
  INNER JOIN [User] AS CalleeUser WITH (NOLOCK) ON
   CalleeUser.UserKey = s.CalleeURI 
 WHERE
  s.StartTime >= (@beginTime) and s.StartTime < (@endTime)
  and m.MediaLineLabel = '16'

You use the New-SettingOverride cmdlet and specify one for IMServerName and one for IMCertificateThumbprint. When you have multiple Exchange servers using different certificates to talk to Skype for Business/Lync you need a pair for each server. Below you'll find an example for 2 Exchange servers called mbx1 and mbx2. They both connect to the Skype for Business 2015 pool sfbpool1.contoso.dk.

New-SettingOverride SetIMServerNameMbx1 -Server mbx1 -Component OwaServer -Section IMSettings -Parameters @('IMServerName=sfbpool1.contoso.dk') -Reason "OWA IM config"
New-SettingOverride SetIMCertificateThumbprintMbx1 -Server mbx1 -Component OwaServer -Section IMSettings -Parameters @('IMCertificateThumbprint=F61D1F3922260B58C2AD779B7C6DDCAA69ECEC0F') -Reason "OWA IM Config"
New-SettingOverride SetIMServerNameMbx2 -Component OwaServer -Section IMSettings -Parameters @('IMServerName=sfbpool1.contoso.dk') -Reason "OWA IM config"
New-SettingOverride SetIMCertificateThumbprintMbx2 -Component OwaServer -Section IMSettings -Parameters @('IMCertificateThumbprint=A3B4AD7A6E5A21D05EFB1710A2788312C5BD2223') -Reason "OWA IM config"

Please note that it is only the use of OWA web.config that has changed in Exchange 2016. You still need to configure OWAMailboxPolicy and OWAVirtualDirectory as in Exchange 2013.

We have updated both the Call Quality Methodology and the Scorecard. The updated methodology and the documentation for the Scorecard have been combined into a single Word document (Microsoft Call Quality Methodology.docx in the Documentation folder) and the complete package of PS scripts, SQL queries, Microsoft Excel Macro-Enabled Workbook and Documentation is shipping as single ZIP file.

The methodology has been updated:

• New poor stream condition
• New suggested targets

In the CQM Scorecard we have fixed a few bugs and added a number of new features including:

• Added support for Skype for Business Server 2015
  • We support the Skype for Business Server LcsCDR and QoEMetrics schemas
• Added dial-tone and completed session Reliability charts and Scorecard elements
  • This enables customers not only to look at audio quality, but also to see how reliable their Lync/Skype for Business deployment is
  • The data is coming from call detail records in LcsCDR
• Changed the default poor stream condition to be ClassifiedPoorCall
  • This is the same definition used by Lync 2013/Skype for Business 2015 and Call Quality Dashboard
  • Using the same default definition allows for better comparison between the different tools and limits potential confusion over different results
  • We continue to offer the option to use the CQM V1.4 poor stream definition and to use a complete custom definition
• Displaying the Rate My Call chart for a given scenario next to the poor stream ratio chart
  • This allows the administrator to correlate the poor stream ratio numbers to the subjective user feedback
  • It enables the administrator to take the user feedback into consideration, when deciding on which areas to focus on for troubleshooting
• Decreased the Scorecard suggested targets to reflect what Microsoft has been able to achieve via the MSIT Get 2 Green project
• Added the possibility to filter out streams generated by Synthetic Transaction
• Added a number of Reliability and Audio diagnostics SQL queries for common scenarios & questions
  • This provide the administrator an additional level of detail for a given scenario
  • We have added a diagnostic query to surface the new detailed Wifi information being sent by Skype for Business 2015 clients

Samples of the new Rate My Call charts and Reliability info in the Scorecard are shown below. 

Please download the ZIP file above and open the Microsoft Call Quality Methodology Word document. It has more details about the changes and new features and detailed information about how to use the PS scripts and the CQM Scorecard itself.

We are currently finishing the documentation of the Web API and it will be available soon. I'll update this post with the link, when it becomes available.

The sample is shown below.

Installation of the sample

The sample is provided in the attached ZIP file. Unzip the file and copy the CQMScorecardSample.html file to a web server in your deployment. I have it located on my CQD Portal server in the c:\inetpub\wwwroot directory.

People, who are already able to access the CQD Portal and see data, should also be able to use the sample.

Configuration of the sample

You need to edit the HTML file and replace the two references to CQDSRV1 with the hostname or FQDN of your own CQD Portal server.

Implementation Details

It is a sample, so I've have not done much to make the code pretty or smart. The HTML file both contains the mark-up to define the CQM Scorecard table and text and the JavaScript code to access the Web API. In production code you would probably separate these components into separate files.

Target

The target for the different elements are hard-coded in the definition of the corresponding query.

Query

The query is comprised of Dimensions, Filters, Measurements and Trend. The easiest way of writing the query you want is to use the CQD Portal Query Editor. After you have specified your query and seen that it return the results you expect, you can use Fiddler or F12 in IE to see the query the Portal sends via the Web API. Look for url's ending in QoEDataService/RunQuery and then for the Request Body. That contains the query and you can use that directly in your JavaScript.

Trending

In the sample I'm using the API to get data return per day for a number of months. The var numMonthsToShow specifies how many months are wanted.

In order to get the Web API to return data from a date range you need to do the following:

• You need to have Month added as both a Dimension and a Filter - [StartDate].[Month]
• Have a valid month defined in the filter. If you leave it out, you'll get all data returned
• Have enabled Trend in the query. SpanCount will control how many months of data to return

In my sample I want to get the data returned by date instead of month and I've therefore added the Dimension [StartDate].[Date] to achieve that. An example query, where it asks for 6 months of data, looks like this:

{"Dimensions":[

{"DataModelName":"[StartDate].[Month]"},

{"DataModelName":"[StartDate].[Date]"}

],

"Filters":[

{"DataModelName":"[First User Agent].[User Agent Type]","Caption":"MediationServer","Value":"[MediationServer]","Operand":0,"UnionGroup":""},

{"DataModelName":"[Second User Agent].[User Agent Type]","Caption":"AV-MCU","Value":"[AV-MCU]","Operand":0,"UnionGroup":""},

{"DataModelName":"[StreamType].[StreamType]","Caption":"Valid","Value":"[False]","Operand":0,"UnionGroup":""},

{"DataModelName":"[StartDate].[Month]","Caption":"Valid","Value":"2015-4","Operand":0,"UnionGroup":""}

],

"Measurements":[

{"DataModelName":"[Measures].[AudioOnePercentPacketLossPercentage]"}

],

"Trend":{"EnableTrend":true,"SpanCount":6,"TrendDate":"2015-4","Type":0}}

Referencing query results

You need to be able to reference the actual result returned by the query. You do that by looping over the result set. The actual result can be obtained by referencing into the DataResult as an array with base 0. DataResult is constructed as a combination of the dimensions and the measurements you ask for in the query. In the sampel query above I have [StartDate].[Month], [StartDate].[Date], [Measures].[AudioOnePercentPacketLossPercentage]. That is reflected in the DataResult reponse which has month, date and AudioOnePercentPacketLossPercentage:

{"ExecutionTime":"00:00:00.0508488","DataResult":[["December 2014","Monday, December 15 2014",0.0]],"ResultIsFromCache":false,"ErrorType":0}

    I'm only interested in the AudioOnePercentPacketLossPercentage value, so I reference the 3rd element by using index 2 into the array:

for (var m = 0; m < data.DataResult.length; m++) {

sumValue = sumValue + parseFloat(data.DataResult[m][2]);

};

Support

This is a sample and comes with no support what so ever J

http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-60-41/CQMScorecardSample.zip

• [First Network Name].[Network Name]
• [First Network Name].[Network Type]
• [First Building].[Building Key]
• [First Building].[Building Name]
• [First Building].[Building Office Type]
• [First Building].[Building Type]
• [First Building].[City Name]
• [First Building].[Country]
• [First Building].[Inside Corp]
• [First Building].[Ownership Type]
• [First Building].[Region]
• [First Building].[State]
• [First Building].[Zip Code]
• [Second Network Name].[Network Name]
• [Second Network Name].[Network Type]
• [Second Building].[Building Key]
• [Second Building].[Building Name]
• [Second Building].[Building Office Type]
• [Second Building].[Building Type]
• [Second Building].[City Name]
• [Second Building].[Country]
• [Second Building].[Inside Corp]
• [Second Building].[Ownership Type]
• [Second Building].[Region]
• [Second Building].[State]
• [Second Building].[Zip Code]

In my lab I've used the SQL statements below to create network and building information. I used the SQL Management Studio to execute them against the QoEArchive database. The information will show up for calls made after the network and building information has been inserted into the QoEArchive.

NOTE: CQD use the Network and the Network Range to correlate the Lync/Skype for Business clients IP address to a network. When the clients sends it quality report (VQReport) to the server it contains the actual IP address in use. CQD takes this IP address and correlates it to networks by using Network and the NetworkRange. For example if the clients IP address is 10.1.1.15, CQD will be able to correlate that to the TR22-Network-1 network given the example below.

USE [QoEArchive]
GO

-- Building Type
DECLARE @nextBuildingId AS INT;
SELECT @nextBuildingId = MAX(BuildingTypeId) FROM [dbo].[CqdBuildingType];

INSERT INTO [dbo].[CqdBuildingType]
 ([BuildingTypeId],[BuildingTypeDesc])
VALUES
 (@nextBuildingId+1,'TR22 Hang-Out');

SELECT [BuildingTypeId]
      ,[BuildingTypeDesc]
FROM [QoEArchive].[dbo].[CqdBuildingType]
WHERE BuildingTypeDesc = 'TR22 Hang-Out';

-- Building Ownership Type
DECLARE @nextOwnerShipId AS INT;
SELECT @nextOwnerShipId = MAX(OwnershipTypeId) FROM [dbo].[CqdBuildingOwnershipType];

INSERT INTO [dbo].[CqdBuildingOwnershipType]
 ([OwnershipTypeId],[OwnershipTypeDesc],[LeaseInd],[Owner])
VALUES
 (@nextOwnerShipId+1,'State Owned',2,'Washington State');

SELECT * FROM [QoEArchive].[dbo].[CqdBuildingOwnershipType]
WHERE [Owner] = 'Washington State';

-- Network Name and Type
INSERT INTO [dbo].[CqdNetworkName]
 ([NetworkName],[NetworkType])
VALUES
 ('TR22-Network-1','EventNet');

SELECT * FROM [QoEArchive].[dbo].[CqdNetworkName]
WHERE [NetworkName] = 'TR22-Network-1';

-- Building
DECLARE @ownerShipTypeId AS INT;
DECLARE @buildingTypeId AS INT;

SELECT @nextBuildingId = MAX(BuildingKey) FROM [dbo].[CqdBuilding];
SELECT @ownerShipTypeId = OwnershipTypeId FROM [dbo].[CqdBuildingOwnershipType]
WHERE [Owner] = 'Washington State';
SELECT @buildingTypeId = BuildingTypeId FROM [QoEArchive].[dbo].[CqdBuildingType]
WHERE BuildingTypeDesc = 'TR22 Hang-Out';

INSERT INTO [dbo].[CqdBuilding]
 ([BuildingKey],[BuildingName],[BuildingShortName],[OwnershipTypeId],[BuildingTypeId]
  ,[CityName],[ZipCode],[CountryShortCode],[StateProvinceCode],[InsideCorp]
  ,[BuildingOfficeType],[Region])
VALUES
 (@nextBuildingId+1,'Washington State Convention Center','WSCC',@ownerShipTypeId,@buildingTypeId
  ,'Seattle','98101','US','WA',0
  ,'Events','PUGET SOUND');

SELECT * FROM [dbo].[CqdBuilding] WHERE BuildingShortName = 'WSCC';

-- Subnet
DECLARE @buildingKey AS INT;
DECLARE @networkNameId AS INT;

SELECT @buildingKey = BuildingKey FROM [dbo].[CqdBuilding] WHERE BuildingShortName = 'WSCC';
SELECT @networkNameId = NetworkNameId FROM [dbo].[CqdNetworkName] WHERE NetworkName = 'TR22-Network-1';

INSERT INTO [dbo].[CqdNetwork]
 ([Network],[NetworkRange],[NetworkNameID],[BuildingKey],[UpdatedDate])
VALUES
 ('10.1.1.0','24',@networkNameId,@buildingKey,'2016-02-1');

SELECT * FROM [dbo].[CqdNetwork] WHERE Network = '10.1.1.0';

First versus Second

CQD has introduced the concepts of First and Second and you can use these in Dimensions and Filters. First will always be the server endpoint of a stream and Second will be the client endpoint. If a stream involves two servers or two client endpoints CQD will make the endpoint with the lowest User Agent Type First. If both endpoints are of the same type, i.e. two desktop clients, CQD will make the Caller in the corresponding QoEMetric.dbo.MediaLine table entry First and the Callee will become Second.

The User Agent Types are defined in the table QoEMetrics.dbo.UserAgentDef by the column UAType.

Let me give you some examples to help you understand the concept.

Is Server Agent

CQD has a filter and a dimension called [First Is Server].[Agent] and [Second Is Server].[Agent]. The filter/dimension can be used to differentiate the User Agent at a broad server/client level, i.e. is this a server side component like the AVMCU or is it a client like Skype for Business 2015.

CQD defines these User Agent Types as Servers: 1, 2, 256, 512, 1024, 1032, 1040, 16393, 16397, 16410, 32769 and 32770. They correspond to MediationServer, AVMCU, CAS, CAA, RGS, CPS, RGS AS, ExUM, ASMCU, XMPPGateway, Gateway and GatewayMediationServerPair.

CQD defines all other known User Agent Types as Client.

QoEMetrics to QoEArchive copy process

CQD uses a SQL Agent job to copy data from the QoEMetrics database into the QoEArchive database. The job runs on a 15-minute schedule. When it run it will copy all new data from QoEMetrics, since last time it ran.

Due to implementation technicalities it is not copying data corresponding to the last MediaLine row from QoEMetrics. In a production environment it is not an issue, since that row will just be picked up on the next run. However, in a test environment it can be an issue, since you might perform a test call, see the data in QoEMetrics and then not see the data in CQD. So if you're in a test environment remember always to make two test calls to ensure that CQD picks up the data.

Cube update process

Due to implementation technicalities the Cube will not pick up data corresponding to the last MediaLine row from QoEArchive. As above it's not an issue in a production environment, but could be in a test environment. Just make sure to do some extra test calls.

Measurements

When you add Measurements in the QueryEditor you should think about these as being logically AND'ed together. For instance, if you ask for say 3 measurements and one or more of these measurements does not produce any results, you won't see any results at all. As an example in my lab I only had audio streams and app sharing streams for May 21, 2015, because I hadn't done any tests with video yet. As you can see from the screenshot below I got 8 Audio Streams and 4 App Sharing Streams shown, when I have just those 2 measurements.

I then added the Video Streams Count measurement, but since I don't have any video streams for that day, I don't get any results returned at all.

When is the form shown

• The user needs to be using Skype for Business 2015 (version 15.0.4711.1002 or later)
• The user needs to be homed on a Skype for Business Server 2015 pool or homed in Skype for Business Online
• The user has been granted a Client Policy, where the parameter RateMyCallDisplayPercentage is larger than 0
• The form will be shown in both Lync and Skype for Business UI

How and what can it be controlled

• RateMyCallAllowCustomUserFeedback
• RateMyCallDisplayPercentage

How is the data sent to server

How the data is stored

• dbo.CallQualityFeedback
• dbo.CallQualityFeedbackToken
• dbo.CallQualityFeedbackTokenDef

How to use the data

The root cause of this appears to be related to problems mentioned in KB3025036 and here and performing the steps in the Resolution 1 section of the KB has fixed the issue..