• Breadth of listeners and formatters for free with access to source code
• Config file driven design allowing you to turn on/turn off logging without changing a line of code
• Proven and tested components helping you build on top of them with a high sense of reliability

The logging component of our framework provides a simple design-time experience to developers, abstracting away the generic APIs that Enterprise Library provides:

This is essentially a façade to Enterprise Library (currently version 3.1) where implementation of Debug looks similar to:

public static void Debug(string log)

{

try

{

Logger.Write(log, CATEGORY_DEBUG); // CATEGORY_DEBUG is a string for category "Debug"

}

catch (System.Configuration.ConfigurationException configErr)

{

System.Diagnostics.Trace.Write(string.Format("Config info missing for Logger. Details: {0}. Exception is: {1}" , GetDebugInfo() , configErr.ToString()));

}

catch (Exception e)

{

System.Diagnostics.Trace.Write(string.Format("Error occured while logging the debug statement: {0}:. Exception is: {1}", log,e.ToString()));

}

}

Overloaded methods of Debug, Info, Warn and Error provide the consuming components to pass in context ids, severity for error messages among multiple others.

Consuming .NET applications (exe's, web site, web services, windows services) would:

1. Reference essentially only the framework assembly (don't need to add all the enterprise library assemblies)
2. Ensure appdomain's config file (app.config/web.config/.exe.config) has the requisite entries (In my next blogpost I will talk as how we went about automating this for SharePoint Web Applications)

Now all the developer would need to do is use the lightweight Log.Debug("Just log me")and the operations/engineering team can use tail/sql query/DebugView/trace listeners/event viewer/inbox to troubleshoot and/or analyze system issues

1. Design, save or share reports
2. View reports available on the platform
3. Schedule often viewed report

Requirements 1 and 2 could conflict and contradict each other given the global nature of our platform. An example would be a marketer who builds a report a certain way and saves it to the platform. Another marketer views this report and there is no way for us, as a platform, to guarantee the validity of the report. Scenarios like this could certainly erode the trust and our vision to be the single source of truth.

To overcome this challenge while still satisfying the overall goal of an open platform which allows users to design, view, save and schedule reports, we introduced a concept of report approval workflow.

• Use cases of scenario 1 would primarily design and save reports to a draft reports list.
• Use cases of scenario 2 would primarily view published reports list which are certified reports

What happens between a) and b) is where the report approval workflow triggers and that's what makes the reports in published reports list certified.

1. Users from use case 1 (let's call him contributor) above design and save a report to the drafts list. After going through multiple iterations, once the user is comfortable with the design of the report, he chooses "Publish" from the SPS context menu. The contributor is prompted to choose a destination report folder before he can submit the report for publishing (the report folder drop down is populated from a Report Approvers SharePoint list which contains the report folders and their corresponding approvers alias)
2. At this point the workflow kicks in. Workflow looks up the name of the approvers for the report folder chosen by the user. It creates a SharePoint task for each approver and sends an email with content asking for report approval
3. The approver(s) can choose to approve, reject or ask the user to make modifications to the report.
   1. If the report is approved, it is moved from the draft to the corresponding folder in the published reports list. If the destination folder doesn't exist, the workflow creates a new folder assigns full permissions only to the approvers/admins and RO to all other authenticated users.
   2. If the report is rejected or medications have been requested, the report stays in the drafts list

      An email is sent to the report contributor letting him know the outcome of the process

This workflow was developed using the Approval Workflow Sample from the SharePoint Server 2007 SDK as the base. Some of the features of this modified workflow that I would like to call out are:

• Meta-data inside a SharePoint list: The workflow as well as certain pieces of UI needs the list of approvers for a given report folder which was stored in a SharePoint list. The nice thing of doing it this way is that a) you can re-use the list info in other places within SharePoint like DIP (document information panel), SharePoint web parts among other places b) You get both a persistence medium as well as nice UI for your site admins to manage the meta-data.
• Config info inside a SharePoint list: Report approval workflow is heavily config driven where you can customize the same workflow to publish to a different list, modify subject/content of email to approvers, contributors, and enable/disable sending email among multiple other ones.
• Logging: Centralized logging using the Logging component described here.

Some of the common activities that were fairly used across the workflow are:

• Create Folder: Creating a folder on the destination Published Reports List requires that all List<Contact> admins have full permissions:

  SPFolder spFld = list.RootFolder.SubFolders.Add(folder);

  spFld.Item.BreakRoleInheritance(true);

  foreach (Contact admin in admins)

  {

  SPRoleAssignment userRoleAssignment = new SPRoleAssignment(admin.GetPrincipal(list.ParentWeb));

  userRoleAssignment.RoleDefinitionBindings.Add(list.ParentWeb.RoleDefinitions.GetByType(SPRoleType.Administrator));

  spFld.Item.RoleAssignments.Add(userRoleAssignment);

  }

  spFld.Item.Update();

• File Exists: Check if a fileName exists in the destination Published Reports SPList list in the folder

  SPFolder subFolder = list.RootFolder.SubFolders[folder];

  SPQuery query = new SPQuery();

  query.Folder = subFolder;

  SPListItemCollection folderFiles = list.GetItems(query);

  for (int i = 0; i < folderFiles.Count; i++)

  {

  if (fileName.Equals(folderFiles[i].Name, StringComparison.CurrentCultureIgnoreCase))

  {

      ret = true;

  break;

  }

  }

• Get Approvers: We stored the report folder name and its approvers in a SharePoint list. When the workflow needed to know the names of the approvers for a given report folder, we discovered that SharePoint doesn't exactly return what we store inside it where the pattern seemed to be: 1;#Joe;#23;#Henry;#19;#Andrew. So we wrote our little method to retrieve the List of approvers where Constants.APPROVERSDELIMITCHAR = ';':

  List<string> ret = new List<string>();

  string[] splitApprovers = approverItemVal.Split(new char[] { Constants.APPROVERSDELIMITCHAR }, StringSplitOptions.RemoveEmptyEntries);

  for(int i = 0; i < splitApprovers.Length; i++)

  {

  if(0 != i % 2) //The pattern needs us to split where every second one is our value

  ret.Add(splitApprovers[i].Split(new char[] { Constants.METAINFOPARSERCHAR })[1]);

  }

One of them first pre-requisites for customer facing component has got be Logging or Tracing as you would like to call it. Logging is highly useful to troubleshoot or debug issues, capturing errors, auditing code paths or events as they occur. The analysis of these logs can lead to increased code quality, improved business scenarios, quick turnaround on issues as they arise and high predictability in getting to the root cause of an issue among other things.

A few choices for the logging component we could use were:

• Write our own customizable listeners, writers, formatters as part of our logging library
• Write custom TraceProvider utilizing the underlying MOSS framework to emit logs into ULS
• Enterprise Library

In the end our choice was to use Enterprise library as our core component for logging functionality since it encompasses and provides beyond what 1 and 2 above offers. In order to maintain consistency and to standardize the logging functionality while ensuring backward compatibility, we provided a wrapper class around the Enterprise Library's Logger functionality. To the consuming component or application we abstract all the base functionality and provide four simple methods Debug, Info, Warn, and Error.

Config File: Once the consuming application or component integrates the library for logging, they use a configuration file to drive Logger to emit the tracing statements to a common destination. This common destination could be windows trace, flat file, event viewer, database or a remote service. This file needs to be part of the main application's config file (app.config/web.config/[MAIN APPLICATION NAME].exe.config).

We provided a SharePoint solution using SPWebConfigModification which allowed us to add Enterprise Library related config entries to any web application in our farm. The same solution was added to the central admin, SSP Admin as well as our main site.

For viewing the logs as they are being written, tenants would use tail/sql query/DebugView/trace listeners/event viewer/e-mail inbox/custom destination depending on how the destination(s) is configured.

• Central Administration -> Operations -> Services on Server -> Office SharePoint Server Search Service Settings

• Central Administration -> Application Management -> Search Service

In case you wanted to reconfigure your Farm's Search settings and/or wanted to use the above pages for other administrative tasks, follow the steps below on the Index server to work around the problem:

1. Stop Windows SharePoint Services Timer service (windows service) on Index server
2. Change the application pool identity of the root web site Office Server Web Services, which is typically OfficeServerApplicationPool, to run under the SSP's domain account domain\sspacct
3. Set your farm settings by navigating to the Farm's Search settings pages listed above

In order to revert steps (1) and (2) above, start the Windows SharePoint Services Timer service on the Index server.

As we had mentioned in our Enterprise Search blog post, Office Server Web services web site contains ECS as well as Search component for each of the SSPs created in the farm. Setting up SPN's for Ids 1, 2 and 4 (Configuring Kerberos Delegation (without infrastructure update)) would force both the search services (Central Administration and SSP administration) to use Kerberos protocol.

If Index and ECS roles are assigned to separate servers during the farm setup, and as a pre-requisite for Kerberos authentication, let us say you would have created a SPN in KDC for Ids 1, 2 and 4 from Step 1 and completed Step 2 and 3. Now if you try to access the search settings page under the SSP (/ssp/admin/_layouts/searchsspsettings.aspx) you would encounter authentication failure to access SSP search administration pages. The error message on the UI would be similar to:

"The search service is currently offline. Visit the Services on Server page in SharePoint Central Administration to verify whether the service is enabled. This might also be because an indexer move is in progress."

while the more descriptive errors in the ULS logs would be:

Exception caught in Search Admin web-service proxy (client). System.Net.WebException: The request failed with HTTP status 401: Unauthorized.

at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)

at Microsoft.Office.Server.Search.Administration.SearchWebServiceProxy.RunWithSoapExceptionHandling[T](String methodName, Object[] parameters)

To resolve this issue ensure an SPN for the index server is setup using the SSP account as Id 3 prescribes in the blog post Configuring Kerberos Delegation (without infrastructure update).

Configuring Kerberos delegation in a SharePoint farm for Excel Services involves the following steps:

1. Setting up SPN's:

As mentioned in the Kerberos Protocol blog post, setting up SPN's is an essential prerequisite for Kerberos delegation. Following is an example of the SPN's that will need to be setup where the data source is a SSAS 2005 cube:

Id	Service Class	Host	Service account
1	HTTP	All servers which have WFE configured	Web app's app pool account
2	HTTP	All servers which have ECS configured	SSP Account
3	HTTP	MOSS Server configured for Index server role	SSP Account
4	MSOLAPSvc.3	SSAS Server	SQL Server Analysis Server Account

Note: Setup SPN's for both netbios name as well as FQDN names.

In addition, constrained delegation will need to be setup for the following SPN's:

• Between the set of SPN's in 1 and the set of SPN's in 2
• Between the set of SPN's in 2 and the set of SPN's in 4


2. Enabling Kerberos for Shared Services Provider:

The web front end (step 1) rendering the excel report calls the ECS component (step 2) to refresh the data (refer section for more details). To make ECS delegate user credentials to back end data sources, Kerberos authentication will have to be enabled for the web site hosting the ECS component.

Running the following stsadm command will change the authentication settings of the "Office Server Web Services" (refer to the following blog posts for more details: SharePoint Server 2007 and Enterprise Search) web site to use only Kerberos protocol:

STSADM –o setsharedwebserviceauthn –negotiate

STSADM –o execadmsvcjobs

3. Enabling delegation for ECS

Enabling Kerberos authentication protocol would provide the ability for the ECS sitting under that web site to delegate the end user credentials to the back end sources. But to specifically let ECS to do the delegation following stsadm admin command has to be executed:

STSADM –o set-ecssecurity –ssp <SSPTitle> -accessmodel delegation

STSADM –o execadmsvcjobs

Execution of the above commands would result in ECS successfully forwarding the Kerberos token to the data source (step 3).

After completion of the above steps, you should be able to refresh your excel based report in a browser with user credentials successfully getting forwarded to your data source.

    Administration of Enterprise Search in MOSS 2007 is twofold:

• Central Administration: Allows the farm administrator to specify farm level search settings
• Shared Services Provider Administration: Allows the SSP Administrator to administer search settings for their SSP

For more details regarding the MOSS 2007 Search architecture, refer to https://msdn.microsoft.com/en-us/library/ms570748.aspx

    SharePoint introduces the concept of Shared Services Provider (SSP) which are composed of a set of services and functionality that can be applied to multiple SharePoint sites at the same time. These services include Excel Services, Enterprise Search among others. During a complete install of MOSS 2007 on a server, "Office Server Web Services" Web site is created, which provides the functionality required by Shared Service Providers. For each SSP created in farm, a virtual directory with the same name as the SSP title is created under this web site. Each of the virtual directories created for a SSP in farm will have the following directories under it: ExcelCalculationServer, Search

Excel/Excel Services

Microsoft Office Excel 2007 provides a rich set of features allowing BI users to connect to data sources, prepare reports, perform real time analysis thereby simplifying the decision making process. In order to provide a seamless experience for Microsoft Office Excel users on the web, MOSS 2007 includes Excel services that enable some of the powerful Excel features to be available on the browser.

Reports built using Excel client can be published to document libraries in MOSS 2007 sites. Excel services can load and render these reports as a web page providing interactive capabilities on the browser. Excel services require the location of these reports to be configured as a Trusted Location which have to be configured as part of SSP administration.

Excel can connect to external data using data connections created and persisted as Office data connection (.ODC) files which include authentication settings for Excel Services. In a SharePoint farm running ECS, these data connection files are secured in central library called Data Connection libraries (DCL's) by site administrators. In a typical Excel Services configuration, these DCL locations will need to be configured as a Trusted Data Connection Library as part of SSP administration. Refer to this technet article for more details: Plan external data connections for Excel Services

The following are the core Excel Service components:

    Microsoft's implementation of the Kerberos protocol involves three parties – client, server and the Key Distribution Center (KDC). The client and server referred above are also known as Service Principals. An essential pre-requisite for Kerberos to work is that target services in the conversation must have had their Service Principal Names (SPN) registered. An SPN comprises of the following format: [service class]/[host]:[port]/[service name]. Service class represents the type of service (well known examples being HOST, HTTP), host is the fully-qualified DNS name or the NetBIOS name of the host. Port and service name are optional fields. An example of a SPN entry would be: HTTP/3rdRock where the service class is HTTP and the host is 3rdRock.

A few points to note about SPN's:

• A SPN must be registered only for one service account. We cannot have the same SPN to be registered under different service account.

E.g.: Following registration will result in the second SPN to be active.

http/server1 redmond\account1

http/server1 redmond\account2

• For a service behind a load balancer, services in all the servers should be configured to use same account.
• It is generally a good idea to create the SPN for both the FQDN as well as the NetBIOS name

http/server1 redmond\account1

http/server1.redmond.corp.microsoft.com redmond\account1

• Currently some Kerberos clients do not construct the SPN correctly if the service is configured to be listening on a non-default port. https://support.microsoft.com/kb/908209/en-us

Kerberos Domain Controller (KDC), running on a Windows Domain Controller, serves as the trusted third party, and performs two service functions: Authentication Service (AS) and Ticket Granting Service (TGS).

    The following is a simplified view of the authentication process:

Handy tools to help you with Kerberos:

A few other good resources to learn about Kerberos Protocol:

1. Kerberos Authentication in Windows Server 2003: One stop source to learn, plan and deploy Kerberos authentication in a Windows Server 2003 environment.
2. Designing Secure Web-Based applications for Microsoft Windows 2000, Michael Howard: Even though the title of the book suggests it is for Windows 2000, chapter 14: An Introduction to Kerberos Authentication in Windows 2000, is one of the best primer for anyone who wants to know the basics of Kerberos in a Windows environment.
3. How to troubleshoot Kerberos-related issues in IIS: A good start point to get help in troubleshooting Kerberos-related issues

Last year this time, a major overhaul was planned for our suite of reporting tools. Until then reporting was done in silos where each application would provide its own set of reports. Listening to our customers, we could see there was huge ask for our team to build a reporting platform which can transcend application data and have the ability to report and analyze it all in one place.

In this blog series, I wanted to share how we went ahead in putting the platform together, the technologies chosen; some of the architectural decisions made and share our experiences along the way. The content for this blog series is out of a white paper jointly authored by your host and a colleague of mine, Subbu. I would like to take this opportunity and acknowledge the time and effort, Derek Bevan, Senior Architect of our group, had put in to help us publish this white paper. We are also grateful to the contributions of Mark Grossbard, Ira Levin, Pej Javaheri, John Campbell and last but not the least Praba Manoharan.

Without further ado ...

Scenario:

Most enterprise reporting and analytics applications (referred to as Business Intelligence (BI)), require knowing the user who is requesting the data at the data layer. Ours was no different as our platform would be required to host secured data wherein authorizing at each layer became one of the top requirements from a security stand point.

Technologies:

Microsoft Office SharePoint (MOSS) 2007 offers a complete suite of components and services to build rich business intelligence (BI) portals. Excel is the most widely used tool for performing data analysis and reporting. Excel Services, offered as part of MOSS 2007, enhances this capability by enabling these features to be available on the browser through a server-based excel calculation service (ECS) and interactive web-based user interface.

A typical BI portal using SharePoint farm can be represented by the figure below:

In the above scenario, if the Excel report, hosted in a SharePoint site, connects to external data sources which require the end user credentials, then client identity (1) has to flow from the SharePoint farm (2) to the data source server (3) as illustrated in the figure (x).

In this scenario, following three authentication types are supported by Excel Services

• Windows Authentication
• SSO
• None

Once the choice to go with Windows authentication was made we had to decide what the underlying protocol would be, choices being - NTLM or Kerberos. NTLM is plain easy in a Windows domain and doesn't require any additional configuration as long as you dont cross a physical machine boundary. In our scenario above NTLM would have lead us to the well-known "double hop" issue and would result in authentication failure at Excel Services layer (ECS) given the multiple hops it would take for us to reach the data source. Kerberos protocol can address this issue as it supports delegating credentials across services/physical machines.

    In order to help readers of this blog series understand the technicalities of enabling Kerberos in a SharePoint farm, I plan to provide an overview of the technologies involved in the process:

• Kerberos Protocol
• SharePoint Server 2007 and Excel Services
• Enterprise Search

and then move to:

• Configuring Kerberos Delegation (without infrastructure update)

and a few follow ups once SSP is Kerberos enabled:

• SPN for Index Server
• Accessing CA Search Admin with SPN set for Index Server