However, I worked up other material on HNSC some time ago that’s not found in Kirk’s article.  Mine is targeted more to the SharePoint Architect who is considering the best way to design and use HNSC in a SharePoint farm.

One small discrepancy with Kirk’s article—he relates a commonly accepted fact of HNSC: “if you are going to implement host named site collections, you should not use a host header for the containing web application that houses the host named site collections or it will not work properly.”  The most unfortunate thing about this is that it requires you to use different IP  addresses or ports if you want more than one web application to host HNSC.  Fortunately, I was able to find that Microsoft does indeed support Host Headers and HNSC on the same web application; I give the procedure below and tips for verifying support through Microsoft Premier.  This provides an additional degree of flexibility for architecting your SharePoint farm.

Host Named Site Collections

The mechanism for consolidating existing web applications into individual site collections, yet retain their existing URLs, is to use Host-Named Site Collections (HNSC). This is a feature of SharePoint that allows individual site collections to have their own top-level URL. This section explores pertinent points from TechNet article Plan for host-named site collections (SharePoint Server 2010).

Benefits

• Much more scalable for vanity URLs because there can be thousands of site collections per web application, but only 20 web applications per farm.
• Managed paths may still be used on an HNSC web application, so that the more familiar “path-based” site collections may be created.
• An explicit or wildcard managed path may be created in the “normal” way, so that non-HNSC site collections may be created on them, as usual. This is done using Central Admin.
• Also, explicit and wildcard managed paths may be created on the web app that will apply to each HNSC site collection created. That is, each HNSC site collection will have the same managed paths that are created this way. This is done using PowerShell scripts; an example below shows adding these kind of managed paths..

Drawbacks

• HNSC may only be created using PowerShell scripts (or programmatically), not via the SharePoint UI.
• This introduces additional effort in creating them, similar to the effort of creating an additional web application, which requires a farm administrator, but with the added requirement of having to execute a PowerShell script on one of the farm servers. This also means that a higher level of access—server administrator—is required to create new HNSC. It would be possible to develop a simple extension to SharePoint that could be executed from Central Administration (or other administration site) to call the appropriate PowerShell script, so that this operation could be delegated back to a farm administrator.
• HNSC cannot use alternate access mappings. Host-named site collections are automatically considered to be in the Default zone, and the URL of the request must not be modified between the end user and the server.

Special considerations

For using SSL and SSL with encryption offloading, some special considerations apply.

• Because SharePoint Server 2010 uses the public URL in the Default zone of the Web application to determine whether host-named site collections will be rendered as HTTP or SSL, you can now use host-named site collections with off-box SSL termination. There are 3 requirements to use SSL termination with host-named site collections:
• The public URL in the Default zone of the Web application must be an HTTPS-based URL.
• The SSL terminator or reverse proxy must preserve the original HTTP host header from the client.
• If the client SSL request is sent to the default SSL port (443), then the SSL terminator or reverse proxy must forward the decrypted HTTP request to the front-end Web server on the default HTTP port (80). If the client SSL request is sent to a non-default SSL port, then the SSL terminator or reverse proxy must forward the decrypted HTTP request to the front-end Web server on the same non-default port.
• To use host-named site collections with off-box SSL termination, configure your Web application as you normally would for SSL termination and ensure that it meets the requirements described above. In this scenario, SharePoint Server 2010 will render links of its host-named site collections in that Web application using HTTPS instead HTTP.
• SharePoint Server 2010 does not support a host-named site collection using both HTTP- and SSL-based URLs simultaneously. If some host-named site collections need to be available over HTTP while other host-named site collections need to be available over SSL, separate the host-named site collections into two different Web applications dedicated for that type of access. In this scenario, HTTP host-named site collections should be in a Web application dedicated for HTTP access and SSL host-named site collections should be in a Web application dedicated for SSL access.

Both types of site collections may be mixed on the same web application. Comparison of features:

* Managed paths for HNSC and the site collections created on them must both be created using PowerShell scripts. The example later in this section shows both of these techniques.

** Offloading encryption for SSL sites is possible with HNSC, but has slightly more stringent requirements for configuration. Specifically: the public URL in the Default zone of the Web application must be an HTTPS-based URL, and the SSL terminator or reverse proxy must preserve the original HTTP host header from the client. Also, best to use standard ports 80 and 443.

Kerberos authentication works with HNSC as it does with an HH web application—must register a Service Principal Name (SPN) for the external URL on the identity of the application pool that hosts the web app.

The strategy recommended is:

• Create a Host-Named Site Collection for every request for a site that requires its own “vanity” URL. That these URLs can be either of the format https://newsite.customer.com, or even https://www.newsite.com, as far as SharePoint is concerned.
• DNS must be configured to serve the appropriate addresses.  If there are more than one web application in the farm that contain Host-Named Site Collections, then it will probably not be possible to use a wildcard DNS “A” record such as *.customer.com to direct all traffic correctly, but each HNSC will need a separate DNS “A” record. See analysis of options for creating multiple web apps, below.
• This option is for a less frequent, more “long-term” type of configuration request, justifying the additional effort.
• Create a Path-Based Site Collection for every request for a site that falls in the category of an existing web application. These use URLs such as https://teams.customer.com/sites/TeamOne, where TeamOne is the path-based site collection that may be created through the SharePoint UI. Explicit managed paths may also be used, such as https://teams.customer.com/TeamTwo.
• This option is for a more frequent type, “immediate” type of configuration request for new teams, communities, projects, etc., and can be accomplished entirely within SharePoint without server or external dependencies.

From an organization standpoint, it may be desirable to use a few individual web applications to organize sites by certain criteria, to better manage different types of sites from a standpoint of security, reliability, performance, scalability or SLA.  For the examples below, we’ll consider two web applications labeled “Standard” and “Sensitive”, then deploy HNSC to each web application.

Basic instructions for creating HNSC assume there is only one web application that supports these on the farm. When deploying multiple web applications that each will host HNSC, some additional complexity is required to create the web applications correctly. The following table compares the options for accomplishing this.

Options for creating multiple web apps for Host-Named Site Collections (HNSC)

When creating multiple web applications in SharePoint, they are often configured to use different Host Header (HH) names to ensure uniqueness, while having the same IP address and port (80 or 443). Host Header names on web applications are largely incompatible with Host-Named Site Collections, but can be made to work in a supported way.  The following table examines each option for creating multiple web apps, and their implications for HNSC:

Option 1 – Identify by Port

This is the simplest approach, described in the reference article Plan for host-named site collections (SharePoint Server 2010). 

The problem with this approach is that if you want to configure more than one web application for HNSC, then only one of them may use default port 80 or 443, while the others must use different ports; many organizations don’t wish to require their users to enter ports for URLs to the various sites they plan for their intranet.

Option 2 – Identify by IP Address

This approach allows the creation of multiple web applications that are uniquely identified by IP address.  This means that “clean” URLs may be presented to the end user, without ports.  The drawbacks here are not necessarily terrible, but worth considering:

• Multiple IP addresses are required on the farm WFE servers; one per HNSC web application per WFE server.  This isn’t hard to configure, but some companies have policies against it that must be waived with a business justification.
• Creating a new HNSC web application means a new IP address must be provisioned onto each of the WFE servers, and into the load balancer, with a new Virtual IP (VIP) address as well.

IP Addresses. The first web application configured to host HNSC could use the “default” IP address of the server, but each additional web application to host HNSC must have a unique IP address. To accomplish this, the Network Interface Card (NIC) providing user access on each WFE server must be configured with multiple IP addresses. Naturally IP addresses will be different across all servers.

Load Balancer. The load balancer that distributes requests across all web front end servers must be configured with multiple Virtual IP (VIP) addresses. Each VIP will load balance out to the IP addresses associated with a separate web application across the WFE servers.

DNS. Finally, DNS must be configured with the root URLs of the HNSC, each targeted to the appropriate load-balanced VIP. A wildcard “A” record like *.customer.com cannot be used, unless DNS can be configured to guarantee that it will first honor all explicit “A” records such as teams.customer.com.

The following diagram shows conceptually the relationship among these aspects, also illustrating how non-HNSC web applications are configured:

In this diagram, each WFE has a “default” IP address; this is used for all requests to the server that are not host-named site collections, including Host Header web applications (e.g. mysite.customer.com, app.customer.com) and web applications distinguished with a separate port. Also, one web app hosting HNSC may also use this IP address. Then, additional IP addresses are shown, corresponding to each additional web application serving HNSC. Note, this arrangement minimizes the number of IP addresses required on each server. If that is not of great concern, it will be clearer to simply use a unique IP address for each HNSC web app, not configuring one of these on the default IP.

There must be a load-balanced VIP corresponding to each of the separate IP addresses across the WFE servers. VIP.1 is the “typical” VIP that distributes inbound requests to each server’s default IP address. VIP.2 distributes inbound requests to each server’s next additional IP address, and so on.

Note: for DNS entry *.customer.com, this will only work if DNS can be configured to examine inbound requests against all other explicit A records prior to following this wildcard entry.

How to create web applications with different IP addresses for HNSC

The following PowerShell scripts illustrate steps for configuring different IP addresses per web app, per server. This script creates two web apps, configures them to different IP addresses, then creates managed paths and a series of host-named site collections on the web apps. This approach follows KB article 927376, which gives the supported method for configuring web applications on different IP addresses. Written for MOSS 2007, KB article 927376 also applies to SharePoint 2010.

Full example scenario for option 2

1. Create web applications on the farm using host header names. Use the SharePoint UI to do this, or a PowerShell script such as the following.

2. Change the bindings on the web applications just created in order to use IP addresses. It’s recommended to do this using PowerShell because the scripts can be created ahead of time and reviewed for correctness over multiple runs, instead of relying on a manual configuration via IIS manager each time. Note that the IP addresses will be different for each WFE server.

3. Create Host-Named Site Collections on the web applications. PowerShell must be used to do this because the SharePoint UI does not provide this specific configuration option.

Option 3 – Identify by Host Header

The TechNet article we’ve been referencing says several things against this option:

“In most cases, applying a host header binding at the IIS Web site level makes it impossible to access host-named site collections through the IIS Web site. This is because IIS will not respond to requests for host names that differ from the host header binding.”

“To ensure that both types of site collections are accessible to users, do not put host header bindings on the IIS Web site assigned to the Default zone of your Web application, if you have host-named site collections in that Web application.”

“You can manually modify host header bindings on the IIS Web site from the IIS Manager, but this is not recommended. Any changes you make using the IIS Manager will not be recorded in SharePoint Server 2010. If SharePoint Server 2010 tries to provision an IIS Web site on another computer in the farm for the same Web application and zone, the original host header binding is used instead of the modified binding.”

However, these can be overcome by following a process to maintain host header bindings on the IIS web site. A proposal to use this approach has been approved by Microsoft Premier as supportable; reference Microsoft Premier case “[REG:111022261144490] prem/SharePoint Server 2010 AL/Requesting RCS for support ability scenario”

The main implications of this option are the operational procedures for creating a new HNSC. Instead of using the SharePoint Central Administration console, or the feature for self-service site creation, PowerShell commands must be run, in the following specific order:

1. On each WFE server in the farm, must add a host header binding to the IIS web site hosting the web application/zone. This could be done via IIS manager, but it’s recommended to use a PowerShell script for accuracy and repeatability. Example of adding the HNSC URL binding for “sitename.customer.com” to the web application whose host header name is “Standard”:

(This requires the IIS Administration Module to be imported for PowerShell)

2. Once for the whole farm, must create the HNSC using a PowerShell command; it can’t be done via the SharePoint UI. Example of creating the HNSC “sitename.customer.com”

The steps must be performed in this order so that IIS is configured to respond correctly to the desired HNSC URL before SharePoint is asked to create it.

Note: if the -Template argument is omitted from the New-SPSite command, then the first access of the site collection will prompt for a choice of site template to be instantiated.

Note: this additional operational procedure would be necessary for option 2 also, because that is the recommended technique for creating web applications on different IP addresses.

Full example scenario for option 3

The PowerShell scripts below are an example showing how a set of web applications and site collections can be created using host headers to uniquely identify the web applications, thus avoiding the drawbacks of the “port” and “ip address” approaches.

Sample steps only

• Web app “Standard” will host intranet.customer.com and communities.customer.com
• Web app “Sensitive” will host teams.customer.com and projects.customer.com. In addition, managed paths are added so that teams will also contain teams.customer.com/sec and teams.customer.com/it.

1. Create web applications on the farm using host header names. Use the SharePoint UI to do this, or a PowerShell script such as the following.  Scripting is recommended for repeatability (dev, test and prod!)

2. Add new bindings on web applications so IIS recognizes Host-Named Site Collection names

3. Create Host-Named Site Collections on the web applications.

Note: Wildcard managed paths for HNSC can also be added, by omitting the -Explicit parameter from New_SPManagedPath.

This type of configuration is supported, and illustrates a way to partially automate this configuration on each WFE in the farm.

Measuring concurrent users on your web site over time is important to understand characteristics of the site’s performance—its overall throughput—based on its load.  In fact, planning worksheets on TechNet tell us to find a number for concurrent users, but they don’t tell us how to get that number.

However, the idea of concurrent users is hard to pin down.  What exactly does that mean?  Conceptually, you might think of it as the number of users who are hitting the site right now.  Ok, but what does right now mean?  We really must define it as a specific duration of time, but how long?  I’ve seen many calculations count the number of unique users every hour.  Is that meaningful?  I think in many cases, it’s not.

Let’s assume for the moment that your web site doesn’t use session state, it just gets requests and returns results.  For this, it’s got to do some work to assemble the result.  SharePoint especially has a lot of construction to do, gathering data from the database, the user profile, the file system, applying master pages, page layouts, maybe executing page controls and web parts, phew!  So this is going to take some time on the server, and if another user sends a request, then those tasks will add to the overall load.  But if that doesn’t happen until later in the same hour, then those tasks don’t really add to the overall load.  So I think the duration should be less than an hour.  Note, if your site does use session state, that changes things slightly… more later.

How short should the duration be?

Let’s look at the impact of measuring logs using different durations.  I used LogParser to query through some logs from a small SharePoint 2010 farm; 8 million requests over a 2 week period, collated and quantized into behavior per hour of day.  This graph counts the number of distinct (unique) IP addresses that were found in four different-sized “buckets of time” which I’m calling quanta.  It does not show the number of requests.  For 5 minute quanta, I count the total number of unique IP addresses in each 5 minute period throughout the day, then average those results over each hour.  For 60 minute quanta, I count the total number of unique IP addresses in that whole hour.

Wow, that’s quite a difference.  If we quantize users into 5 minute buckets versus 60-minute buckets, we find a wide range of results.  Designing a production infrastructure for SharePoint, I’m certainly going to want to know if it has to support 20 or 120 concurrent users during those busy parts of the day.

Abstractly, I think the size of the quanta into which we count distinct users hitting the farm should be somewhat longer than it would take the server to “come to rest” after a heavy stream of incoming requests suddenly stopped, allowing all requests to be finished and results returned.  It seems to me that after this time, since the server is done processing those requests, it can take more without additional load.  Am I going to try and compute this time? Erm, no. I’m going to WAG it at about 5 minutes in general, based on some undocumented experiences over many years. But maybe the server could only finish all of its work in that time because nothing else was coming in. Ok, so let’s double it: 10 minutes.

Of course there’s a body of knowledge around load testing that might help us here.  One site that has a lot of great detailed technical information about load testing is the blog at webperformance.com.  For load testing, ramp-up is more important than the ramp-down abstraction above.  They like to ramp-up over a long period—many minutes to several hours—in order to understand the behavior of the system under different loads.  But ramp-down is more about leaving the system in a stable state, allowing all user requests to complete normally, instead of just killing all virtual user threads in-request.  In this case, the ramp-down is dependent mostly on maximum request processing time, which is pretty much page latency.  Now, we may want our page latency to be just a few seconds in most cases, allowing more on pages that do “complex” things like generate reports.  But even with relatively fast pages, ramp-down takes a while longer.  Following this line of thinking, the 5 minute and 10 minute durations for concurrency continue to seem more reasonable than larger buckets.

For another sanity check.  How do people typically use SharePoint and other publishing/collaboration intranet sites?  Again from personal experience, I tend to use individual sites in relatively short bursts.  If I’m searching for information, I’ll want some results in just a few minutes, or I’m going to give up and try something different.  Or if I’m collaborating, I’ll be uploading and downloading documents one at a time or a few at a time, then going off to work with those documents.  So, I might have multiple SharePoint sites open all day long, but my direct interactions with them are bunched together, separated by other work.  For me, the idea of 10 minutes well encompasses each of my individual direct interactions with collaborative sites such as SharePoint.

I’m really talking myself into this figure of 10 minutes as a guideline for measuring concurrency.  And I welcome any debate on the topic… please post comments!

So, what if you do use session state on your site?  For SharePoint, the typical session state available for developers is disabled by default (see my Session State blog posting for more information).  But if you enable it, or are using another web site host with session state, then whenever a user sends an authenticated request to the server, it will set aside some resources to maintain that session, for some period of time.  Way back in my ASP days, I recall 20 minutes being a common default, and it’s always configurable.  If session state is maintained in the database, that’s fewer resources that the WFE servers must maintain, but it’s still something.  In any event, if you’re using session state, you might want to use the session duration as your guideline for quantizing users to measure concurrency.

The script

Here’s the batch file with LogParser commands that I used to compute concurrent users by hour throughout the day.  I know, I should have used PowerShell, but I only started with a couple lines and it just grew from there.  Typical.

SET inputCSV=%1
if '%inputCSV%'=='' GOTO USAGE

REM outputCSV has no extension, it's added later
SET outputCSV=UserConcurrency

SET concurrencyPeriod=%2
if '%concurrencyPeriod%'=='' GOTO USAGE

SET concurrencyField=%3
if '%concurrencyField%'=='' SET concurrencyField=c-ip

REM Set a filter to match requests that should be excluded
REM %%Service%% matches our service accounts (like search), exclude them
REM ...if you don't want a filter, use SET filter=0 IS NULL
SET filter=(cs-username LIKE '%%Service%%')

echo.
echo Counting Concurrent Users
echo inputCSV         : %inputCSV%
echo outputCSV        : %outputCSV%.csv
echo concurrencyField : %concurrencyField%
echo concurrencyPeriod: %concurrencyPeriod% seconds

echo.
echo First stage, quantizing to %concurrencyPeriod% seconds...

logparser -i:CSV -o:CSV "SELECT DISTINCT %concurrencyField%, date, QUANTIZE(TO_TIMESTAMP(time,'hx:mx:sx'), %concurrencyPeriod%) AS Period, COUNT(*) as Requests INTO temp1-%outputCSV%.csv FROM %inputCSV% WHERE %concurrencyField% IS NOT NULL AND NOT %filter% Group By  Date, Period, %concurrencyField%"

echo.
echo Second stage, grouping...

logparser -i:CSV -o:CSV "SELECT date, to_string(to_timestamp(Period,'hx:mx:sx'),'hh') as Hour, Period, count(%concurrencyField%) as UserCount, sum(Requests) as RequestCount INTO temp2-%outputCSV%.csv From temp1-%outputCSV%.csv Group by date, Period"

logparser -i:CSV -o:CSV "SELECT Hour, avg(UserCount) as Concurrent-Users(q%concurrencyPeriod%), sum(RequestCount) as Total-Requests(q%concurrencyPeriod%) INTO %outputCSV%-%concurrencyPeriod%.csv From temp2-%outputCSV%.csv GROUP BY Hour ORDER BY Hour"

GOTO DONE

:USAGE

echo.
echo # Usage:
echo #
echo # ConcurrentUsers inputCSV seconds [fieldname]
echo #
echo # inputCSV : csv file (or *.csv) of log entries with fields: date, time, c-ip or [fieldname], other
echo # seconds  : concurrency quantization level.  Typical values 300 and 600 seconds
echo # fieldname: field to evaluate for concurrency.  Typical values c-ip (default) and cs-username
echo #
echo # Example  : ConcurrentUsers BigO.csv 300
echo.

:DONE

Several things about this script. 

The input is a CSV file, it must have fields named “date” and “time” at the very least, then another field named “c-ip”  This field could have a different name, which you pass in as a parameter.  This allows you to specify a different way to count users, such as cs-username, or maybe you want to count some other information quantized by time, maybe cs(User-Agent).  I just default this to c-ip because I was working with a site that had anonymous access, and most of the cs-username values were null.

The file may have many other fields that come from the IIS logs. 

The way I got my BigO.csv file was by following Mike Wise’s excellent whitepaper Analyzing SharePoint Usage: https://www.microsoft.com/download/en/details.aspx?id=4616.  This will help you get started with LogParser in about 10 minutes (there’s that number again… an omen!).  Some of his first steps show how to run LogParser on a whole set of raw IIS logs and collate them into a CSV file that has some extra info added, in order to do more analyses on that. 

My script doesn’t require a CSV made in this way, this is just a good example of how to get one pretty easily from your logs.  And of course, LogParser could just as easily be run directly against the logs, by using the option –i:IISW3C instead of –i:CSV.  Mike’s paper talks about this too.

So, you can run this a few times and get results for different intervals that define your idea of concurrent use.

ConcurrentUsers BigO.csv 300

ConcurrentUsers BigO.csv 1200

ConcurrentUsers BigO.csv 3600

Running these in succession will output 3 files, UserConcurrency-300.csv, UserConcurrency-1200.csv and UserConcurrency-3600.csv, so you can easily compare for your analysis.  It also creates some intermediate temp csv files that are useful to look into for sanity checking and understanding what the final output is telling you.

Oh, what’s that “SET filter=0 IS NULL” for?  I just wanted an easy way to shut off the filter.  Since the filter  appears in the WHERE clause as AND NOT %filter%, I need something that evaluates to FALSE, meaning “don’t filter,” and as everyone knows, 0 is not null.  Stupid geeks and their jokes.

Well, best of luck in this grey area.  Let me know if any of this looks wrong please!!!

SharePoint 2010’s iFilter excludes content inside of a <div class="noindex"> tag.  Note that each div nested in that one is re-evaluated for the noindex class, so nested divs may not work as you expect.  The product team is considering changing this so that all nested divs are excluded.

<div class="companyFooterStyle noindex">
      Copyright 2010 © Company Name | <a href="/sitepages/PrivacyPolicy.aspx">Privacy Policy</a>
</div>

Now, when you search the portal for “Company Name” or “Privacy” you won’t get every page in the portal showing up in your results.

Note the ability to use other css classes along with noindex, to get visual styles as well.  I haven’t tested the noindex class on tags other than div.

1. ASP.NET session state may be used on SharePoint 2010 pages. This service is automatically disabled in normal installations of SharePoint 2010; it may be enabled using the instructions below. Once enabled, it appears on the Service Applications page as “SharePoint Server ASP.NET Session State Service.”
2. The State Service service application is designed for and only available to Office internal components such as InfoPath Forms Services and Visio Services. This service is automatically enabled in typical installations of SharePoint 2010 and is not related to ASP.NET session state. It appears on the Service Applications page as “State Service.”

To enable ASP.NET session state:

1. Enter the following PowerShell command in the SharePoint 2010 Management Shell window:

   Enable-SPSessionStateService –DefaultProvision

2. On each web application for which you want to use session state, edit the web.config file and set the enableSessionState property of the pages element as follows:

   <pages enableSessionState="true"

Guidelines for using SharePoint Server ASP.NET Session State:

1. This session state may be used by custom code in web parts and other page-hosted controls.
2. Session state information is stored in a configurable SQL database; this implies the following:
   1. Load-balancer affinity is not required.
   2. Heavy use of session state must be included in database/capacity planning. By default, the session state database is created on the same database server hosting the SharePoint 2010 farm configuration database. This can be changed to place it on another server; this can be done at a later date if necessary. To move it to another SQL server, it should first be disabled, then enabled again with different configuration that points to the other server.
3. When enabled, Session state is available to the whole 2010 farm. Each web application can choose to enable it for use on ASP.NET pages (enableSessionState="true").
4. Session state is partitioned using the root URL. This means that different web applications will have different session state objects; a web part can only access the session state object available to it. If host header site collections are used, then different session objects will be used for each site collection (the browser will have multiple session ids in different cookies).

Additional information about enabling session state:

Syntax 1	Syntax 2
Enable-SPSessionStateService -DefaultProvision	Enable-SPSessionStateService -DatabaseName <String> [-DatabaseServer <String>] [-DatabaseCredentials <PSCredential>]

Additional parameters are not discussed here; use Get-Help Enable-SPSessionStateService –full for more information.

The Enable-SPSessionStateService cmdlet creates a session state database, installs the ASP.NET session state schema, and updates the Web.config files on the farm to turn on the session state service.

If the DefaultProvision form of the command is used, all default settings are used. These are:

• DatabaseName = “SessionStateService_<GUID>”
• DatabaseServer = the same database server as the SharePoint 2010 configuration database
• DatabaseCredentials = Integrated Windows Authentication

If the DatabaseName form of the command is used, then these parameters may be set explicitly. If some of the parameters are not included, they default as above. To set specific credentials for accessing the session state database, note that this allows you to specify a SQL Authentication credential only, not a Windows credential. Then, use the Get-Credential command to create a PSCredential object from a username and password, then use that object as the argument of the DatabaseCredentials parameter.

After this service is enabled, “SharePoint Server ASP.NET Session State Service” will appear on the Service Applications management page.

Using DPM to Backup/Restore SharePoint 2010

Microsoft System Center Data Protection Manager 2010 (DPM) is a product designed to manage ongoing, comprehensive backups of Microsoft enterprise products, including SharePoint 2010.

• Protect at farm level – recover and track at granular level.
• Automatic discovery & protection of new databases.
• Seamlessly supports all SQL high-availability solutions.
• Easy to configure and manage short-term and long-term backups
• Optimized for large farms.
  • Faster backups than SharePoint-only backup solutions.
  • Requires significantly less storage than SQL-only backup solutions.
• Search for SharePoint item through all recovery points of the farm.
• Retains metadata/security settings for the recovered item.
• Granular recovery without a recovery farm for SharePoint 2010.

More information on DPM: https://technet.microsoft.com/en-us/systemcenter/dm.

Backup/Restore for SharePoint using its own tools

SharePoint provides many of its own tools for backup and restore. This section discusses these options.

The Central Administration Backup and Restore page organizes SharePoint’s monitoring tools:

Central Administration provides tools for performing individual backups of the entire farm, individual site collections, sites, libraries or lists. A full farm backup can be restored from Central Administration, but tools are not provided here for scheduling recurring backups, or for restoring individual site collections, site, libraries or lists. These operations can be managed using PowerShell scripts:

Cmdlet	Synopsis
Backup-SPFarm (Restore-SPFarm)	Creates a backup of an individual database, Web application, or the entire farm. Can be full or differential. Use Backup-SPFarm -ShowTree to display a list of what would be backed up; this does not perform the backup.
Backup-SPSite (Restore-SPSite)	Performs a backup of a site collection.
Backup-SPConfigurationDatabase (Restore-SPFarm)	Performs a farm-level configuration-only backup. This is not a typical backup operation, see Note* below.

To schedule automated recurring backups, use these PowerShell cmdlets in scripts that are scheduled with the Windows Task Scheduler.  This recommendation is from the TechNet article Backup and recovery overview (SharePoint Server 2010).

*Note that Central admin gives an option for backing up the config database, but you normally never restore the config database, even when restoring the farm. In DR scenarios, you create a new farm and restore the site collection DBs. An example use for a configuration database backup is to create a farm configuration template that can then be applied to create other new SharePoint farms that have identical physical topologies, by using Restore-SPFarm.  This could be very useful in hosting scenarios with baseline virtual images.

SQL backup tools will process the data faster. DPM will process even faster and repeated backups require much less space.

So if you're planning daily backups of 500 GB, CA farm backup may be too slow. SQL backup will get it faster but require 2.5 TB to store 5 backups. DPM will be fastest and probably require less than 1 TB for 5 backups of 500 GB (depending on content churn).

This is an overview of major tools and techniques available to monitor the health of your SharePoint 2010 farm.

Using SCOM to monitor SharePoint 2010

Microsoft System Center Operations Manager 2007 (SCOM) is a product designed for enterprise health monitoring. It integrates closely with SharePoint 2010 and provides the most comprehensive and flexible solution for monitoring the health of SharePoint farms. With SCOM, the level of reporting and alerting is more granular and easily managed than SharePoint’s standard health monitoring.

• SharePoint 2010 ships a management pack for System Center Ops Manager
• Improved Knowledge Articles
• More relevant events and monitors
• Surfaces SharePoint Health Analyzer (SPHA) rules
• Integrated with Unified Logging System (ULS)

The SCOM 2007 Management Pack for SharePoint 2010 contains more documentation for using these products together, in a Management Pack Guide for SharePoint Foundation 2010 and one for SharePoint Server 2010. Be sure to reference both guides for managing the server product. Download from: https://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=c8a9d749-b7a8-412a-b2db-f3e464ed3fcf. These management pack guides are also included as a deliverable of this engagement, for convenience.

More information on SCOM: https://technet.microsoft.com/en-us/systemcenter/om.

Monitoring SharePoint with its own tools

With or without SCOM, SharePoint 2010 provides many of its own tools for health monitoring. This section discusses all of these options.

The Central Administration > Monitoring page organizes SharePoint’s monitoring tools:

Reports available from this page:

• Health analysis problems and solutions
  • Administrative configuration
  • Performance issues
  • Availability issues and
  • Potential Security issues
  • …results can be consumed by SCOM
• Administrative reports
  • Search administration (crawl rates, query latency, etc)
• IRM Policy Usage reports
• Health reports
  • Slowest Pages
  • Top Active Users
• Web Analytics reports
  • Traffic (Page Views, Visitors, Referrers)
  • Search usage (query totals, trends)
  • Inventory (Number of sites)

Custom reporting

Custom reports can be built and deployed to Central Admin Monitoring using Excel or SQL Reporting Services.

Custom health rules can also be created. Each custom health rule requires an assembly that will perform the desired health check. See How to: Create a Health Rule.

ULS logging service

SharePoint’s logging service (which uses the Windows Unified Logging Service or ULS) has been improved in several ways.

1. ULS Log Viewer
   1. A SharePoint product team member discusses this tool on their blog: “In SharePoint 2010 we can use correlation id to trace a series of event inside SharePoint. For example, in this screenshot, my machine is trying to flush usage log. The different entries may be buried in the big ULS log file, but with correlation id this viewer can easily filter and find them all.”
   2. Download ULS Log Viewer from: https://code.msdn.microsoft.com/ULSViewer
2. PowerShell cmdlets for managing the logs
   1. Diagnostic Configuration
      1. Get-SPDiagnosticConfig retrieves Diagnostic Configuration values
      2. Set-SPDiagnosticConfig allows setting Diagnostic Configuration values
   2. Trace Log and Event Log Throttling
      1. Get-SPLogLevel displays a list of diagnostic levels
      2. Set-SPLogLevel allows setting of trace/event level on categories
      3. Clear-SPLogLevel resets the trace/event levels to default
   3. Log File control
      1. New-SPLogFile ends the current log and starts a new one
      2. Merge-SPLogFile combines trace log files from all farm servers into a single file
   4. Trace Log Querying/Filtering
      1. Get-SPLogEvent reads and queries ULS trace logs
3. Correlation ID tracing
   1. Every request has a unique correlation ID associated with all calls triggered by that request, even in SQL Profiler. Correlation ID appears in error message and can be used to find all associated actions immediately.
4. Manageability improvements
   1. The ULS log file from each WFE server can now be directed to accumulate into a single SQL Server database. This makes it easier to find all relevant messages, and provides integration with SCOM monitoring. To enable this, click Configure usage and health data collection under Monitoring in Central Administration.
   2. All events that appear in the Windows Application log also appear in the ULS logs, so no longer necessary to correlate from these two different sources.
   3. Event flood protection. Heavily recurring events can be given a threshold so that they are only reported once every several minutes.

Developer Dashboard for Page Performance

This tool is useful for measuring the behavior and performance of individual pages.  To analyze performance of a page, turn on SharePoint's "Developer Dashboard" that reports on details of page performance.  Using stsadm commands for this:

stsadm –o setproperty –pn developer-dashboard –pv on

After this is executed, each page will show a report at the bottom that gives information about the performance of all the page calls and database queries.

Article: https://blogs.technet.com/speschka/archive/2009/10/28/using-the-developer-dashboard-in-sharepoint-2010.aspx

To cancel:

stsadm –o setproperty –pn developer-dashboard –pv off

To display an icon that allows you to toggle it on/off as you are working:

stsadm –o setproperty –pn developer-dashboard –pv ondemand

To set this parameter using PowerShell, use the following code snippet:

$DevDashboardSettings = [Microsoft.SharePoint.Administration.SPWebService]::ContentService.DeveloperDashboardSettings;
$DevDashboardSettings.DisplayLevel = ‘OnDemand’;
$DevDashboardSettings.RequiredPermissions =’EmptyMask’;
$DevDashboardSettings.TraceEnabled = $true;
$DevDashboardSettings.Update()

SPMonitoredScope

This class gives developers the ability to instrument their code so that it can be monitored using SharePoint tools, such as the Developer Dashboard. Example of use:

using (new SPMonitoredScope(
	“Some Scope Name”,
	TraceSeverity.Verbose, 1000,
	new SPRequestUsageCounter(3),
	new SPSqlQueryCounter())
	)
	{
		callExternalCode();
	}

So in SharePoint 2010, there is no longer an option to create a site with this template, and the portal site template does not include a Site Directory.

Now, if you upgrade a MOSS 2007 farm to SharePoint 2010, any Site Directory sites that you have will be upgraded and remain fully functional. So existing site directories are supported, but not the creation of new ones.  This is why Central Administration still has an area to configure a Site Directory (under General Application Settings); so you may continue to manage any upgraded site directories correctly.

Bill Baer’s excellent blog gives some additional information on this topic, including the new features of 2010 that are envisioned to be better replacements, such as Tagging, Notes, Managed Metadata and Metadata-based Navigation.

Finally, for those who want the specific functionality of the Site Directory in SharePoint 2010, a CodePlex solution is available: https://spsitedirectory2010.codeplex.com/. This solution “follows the same basic principle of the Site Directory in SharePoint 2007 but adds some commonly requested features”

Download the SDK now.

Tons of new “How-To” topics and code samples!

• Code samples for SharePoint Foundation
  • Implementation of IBackupRestore
• Code samples for Business Connectivity Services
  • AdventureWorks .NET Assembly
  • AdventureWorks ASP.NET Web Service
  • AdventureWorks WCF Service
  • AdventureWorks 2008 .Net Assembly and Web Services
  • AdventureWorks 2008 Declarative Outlook Solution Reusable Components
  • AdventureWorks 2008 Declarative Outlook Solution Sample
  • Authenticating and Consuming Data from NetFlix (Web 2.0 Application)
  • BDC Model for XML Snippets
  • Custom Web Part and Word Add-in Sample
  • Sample Orders .Net Assembly and Web Services
  • Sandboxed BCS Workflow Sample
• Code samples for Enterprise Content Management
  • Content Type-Based Metadata
  • Creating a Routing Rule
  • Custom Document ID Provider
  • Managed Metadata Dependencies
  • Pausable Custom Timer Jobs
  • Publishing Cache Services
  • Silverlight Controls and Silverlight Thumbnail Extraction Sample
  • Retention Events
• Code samples for User Profiles and Social Data
  • Combine information from my public profile on Linkedin (or Facebook) with my User Profile
  • Gold star or thumbs to a colleague
  • Insert an event/document change event in feed
  • Share a link with my colleagues
  • Social data statistics
• Code samples for Visio Services
  • Annotations Sample
  • Custom Error Messages Sample
  • Mouse Interaction Sample

For IT Pros: https://MSSharePointITPro.com.  This TechNet resource center is the starting place for understanding how to plan the physical architecture, infrastructure and configuration of SharePoint deployments.

• Hands-On Lab: https://technet.microsoft.com/en-us/sharepoint/ff420396.aspx

For Developers: https://MSSharePointDeveloper.com.  This MSDN resource center is the starting place for understanding how to plan development and implementation of custom SharePoint deployments.

• Hands-On Lab: https://msdn.microsoft.com/en-us/sharepoint/ee513147.aspx

These labs let you get hands-on with an instance of SharePoint in the cloud, so no installation is necessary; you can get started right away.

SharePoint 2010 IW Demos: https://go.microsoft.com/?linkid=9728417.  This is a Hyper-V virtual machine that has SharePoint 2010, Office 2010, Visual Studio 2010, SharePoint Designer 2010, SharePoint Workspace, and more.  Plus, a ton of content is installed: sites, pages, users, documents, videos, workflows, libraries, etc., and the scripts that take you through how to use and demonstrate it all.  This is an incredible resource.  It’s also huge: 18 GB.  But there is an optional link on the page to use Akami download manager to speed the download with pause/resume functionality. Requires Windows Server 2008 R2 with Hyper-V.

Corresponding documents:

• SharePoint 2010 Overview Evaluation Guide https://go.microsoft.com/?linkid=9727161
• SharePoint 2010 Walkthrough Guide https://go.microsoft.com/?linkid=9727427
• Microsoft SharePoint Server 2010 Evaluation Guide For IT Professionals https://go.microsoft.com/?linkid=9727162
• Microsoft SharePoint 2010 Professional Developer’s Evaluation Guide https://go.microsoft.com/?linkid=9727163

Max recommended SharePoint 2010 list size is 30 million items or documents!

Plus, the maximum recommended number of items or documents in a view is now 5,000.  Specific guidelines are given in SharePoint Server 2010 Capacity Management- Software Boundaries and Limits.

We still highly recommend nesting documents in a folder structure when you want to store millions of documents in the library.  There are two new features to help you do this easily: the Content Organizer and Document IDs.

Content Organizer

This is a configuration that you can enable on a document library, to automatically organize documents into folder structures, even creating new folders when necessary.  Steve Peschka describes it best in his detailed article on the Content Organizer:

“…it can be used for mundane housekeeping type activities such as making sure that no folder within a document library contains more than 5000 items, for example. When item number 5001 is added to the library, the Content Organizer can automatically create a new folder and put the document in that folder. All of this routing and folder creation can be handled automatically without further user intervention. After a document is uploaded and the required metadata applied, the upload form displays a Url to a document that has been routed so the user knows where to find it in the future.”

Document IDs

SharePoint now provides the ability to automatically assign a unique ID to documents as they are created or uploaded that can be used to locate them no matter where they are stored or moved in the farm.  It works for single documents and the new Document Set.  Blair Biggens discusses this in more detail.

Also very nice: it’s not a GUID but a bit more friendlier, shorter ID code!