“Configuration Failed.”

“Failed to create the configuration database”

“An exception was thrown: This SQL Server instance does not have the required “maxdegree of parallelism” setting of 1”

This can be resolved as follows:

1. Open SQL Server Management Studio
2. Right Click on your SQL instance and select Properties
3. Select “Advanced” and change “Max Degree of Parallelism” from 0 to 1

• System & Log Files

This is the direct attached storage (DAS) required on each server for its operating system, 14 hive, IIS Logs, SharePoint Logs and any other temporary storage. The common values for these are present in my blog article at https://blogs.msdn.com/b/mohamed_el_ashmawi/archive/2011/10/18/sharepoint-2010-capacity-management-confirm-farm-size-after-modelling.aspx 

• SharePoint databases

This is the space required for the various SharePoint databases to be used by the SharePoint farm. There are core databases that are present in any deployment such as the configuration database, central administration database and content databases. Moreover, many service applications have their own databases (one or more). An overview of the difference databases and their characteristics is present in https://technet.microsoft.com/en-us/library/cc678868.aspx#Section1

This space is usually present on SAN and is exposed to SQL 2008 clustered instances.

•  Search Index

The search index is stored on the query servers and its size depends on the number of items in an index partition. We will go through how to estimate the storage for the index in the article on estimating search requirements.

SharePoint Databases

The SharePoint 2010 configuration database and the central administration database are small in size and each is less than 1 GB.

Estimating storage for content databases can be done using the formula in https://technet.microsoft.com/en-us/library/cc298801.aspx

It is basically: Database size = ((D × V) × S) + (10 KB × (L + (V × D)))

Where

D: Expected number of documents

S: The average size of the documents that you will be storing

L: The number of list items in the environment

V: The approximate number of versions

Another way to estimate the content databases size is to extrapolate from the current SharePoint 2007 (if you are migrating from SharePoint 2007 to SharePoint 2010). To be able to extrapolate correctly, you need to get the size of the current content databases, assume a growth factor for 3 to 5 years; then build the new content database estimate.

Note that it is recommended limiting the size of content databases to 200 GB to ensure system performance.

I will detail the estimates for the service applications databases in the next articles along with capacity estimates.

Below are some decisions that have implications on the farm logical architecture:

1. Should content be served from one SharePoint farm or multiple SharePoint farms?

A SharePoint farm can publish many different vanity URLs each having its own web applications, so different URLs and even purpose shouldn’t be the driving force for having multiple farms. However, in some scenarios, the business requirements are totally different between different web sites and it is expected to have different  administration teams for the different farms. Many corporations currently have different SharePoint farms for different purposes, while others have multiple SharePoint farms that are geographically dispersed to cater for regional requirements.

Another example of cases where is more than one SharePoint farm is when you have an authoring farm and a publishing farm.

The selection of the number of farms and their roles is a blend of a business decision and a technical decision and need to be confirmed in case of large farms.

2. What are the required service applications?

The following list includes the service applications for SharePoint 2010:

• Access Services
• Business Data Connectivity Services
• Excel Service Application
• Managed Metadata Service
• PerformancePoint Service Application
• Search service
• Secure Store Service
• State service
• Usage and Health Data Collection service
• User Profile service
• Visio Graphics Service
• Web Analytics service
• Word Automation Services
• Microsoft SharePoint Foundation Subscription Settings Service
• Office Web Apps services
• Microsoft Project Server 2010

Depending on the business needs, a farm can have a subset of these service applications. For example, Access service, Visio service, Office Web App or project server might not be needed on a specific farm as there is no need for them.

Thus, it is important to identify the required service applications and plan for them.

        3. How are the service applications deployed? What is the service applications architecture?

Technet has an article on services architecture planning at https://technet.microsoft.com/en-us/library/cc560988.aspx. This article discusses service application design considerations andoptions. 

The most important decisions to make at the moment are the follows:

• Whether to share some service applications across farms or not?

The following service applications can be shared across farms:

         User Profiles

         Managed Metadata

Business Connectivity Services (BCS)

Search

Secure Store

Web Analytics

Thus, it is an important decision to decide on whether to share them across farms. In this case, you can have these services reside on one farm (enterprise services farm) and publish them so that other farms can consume them. Moreover, you can decide on having dedicated farms for specific purpose (ex. Search farm that serves all search requests).

Note:  It is most common to share User Profile, Managed Metadata, BCS and Search across farms.

Note: Not all service applications are recommended for use in WAN environments. The recommended service applications in WAN are search and managed metadata. 

• How to group service applications within one farm?

After deciding on the farm that a service application will run on, we can decide to group them in service application groups. The criteria for grouping are their use by the various web applications that you have in your farm

The first action to apply when designing the environment is to verify the expected size of the SharePoint farm.

SharePoint farms are generally categorized in 3 different sizes. Each has its characteristics and expected workloads and datasets. The details of the different sizes are present in the TechNet article at https://technet.microsoft.com/en-us/library/ff758647.aspx.

Below is a summary of the farm sizes and their characteristics (This part is extracted from the TechNet article mentioned above):

1. Small Farm:

It consists of a single database server or cluster and one or two SharePoint Server 2010 computers.

Characteristics:

• Limited deployments with a minimal set of service applications enabled
• Small user base
•  Low usage load (Very few requests per second)
• Small volume of data (10 or more gigabytes).

Note: A smaller option for a SharePoint farm is to have a single server deployment where one server has SharePoint and SQL. This topology is not recommended for a production environment and should be used for evaluation or development purposes only.

 2. Medium farm:

The Topology is broken down into three distinct tiers: dedicated Web servers, dedicated application servers, and one or more database servers or clusters.

Characteristics:

• Service applications are distributed across multiple servers.
• Tens of thousands of users and a load of 10 to 50 requests per second.
• A data store of one or two terabytes.

3. Large Farm:

In large farms, there are Web servers, multiple application servers, depending on the usage characteristic of each of the local (non-shared) services, and multiple SQL Server–based servers or SQL Server clusters, depending on the content size and the application services databases that are enabled on the farm. Moreover, services and solutions are broken down and published across multiple farms.

Characteristics:

• A user base in the range of hundreds of thousands of users.
• A usage load in the range of hundreds of requests per second.
• A dataset in the range of ten or more terabytes.
• Several service applications federated and consumed from dedicated services farm, typically the User Profile Service, Search, Managed Metadata service, and Web Analytics.
• Most other service applications are enabled locally.

Based on the results of the modeling, you should be able to identify the size of the farm you should design. This is a very important decision as it will define the next steps that you will apply.

If you have a small farm, then the design is clear as per the topology above and there are not much of decisions that need to be made. If you have a medium farm, then the topology is clear and the expected number of servers would be in the range above.

If you have a large farm, then there are many decisions to be made as follows:

• Is there a need to have a separate farm for enterprise services?
• Should My Sites be present in another farm?
• Should there be one SharePoint content farm or are there reasons to have more than one farm serving content?
• Which service applications would you run locally?
• How are service applications grouped on application servers?

In case of small or medium sizes farm, options are rather clear so the rigor of the next steps will not be needed; however, large size farms require more effort to come up with a concrete design.

 Minimum hardware requirements for SharePoint farms servers are present at https://technet.microsoft.com/en-us/library/cc262485.aspx

In general, the following server configurations are common for large farm server deployments:

• Web Servers

                     Processor:           2x Quad Core

                     Memory:             8-16 GB

                    Hard Disk:            OS: 80 GB

                                              Logs: 140 GB

• Application Servers

                     Processor:          2x Quad Core

                      Memory:             8-16 GB

                     Hard Disk:            OS: 80 GB

                                             Logs:  140 GB

• Database Servers

                      Processor:           4x Quad Core

                      Memory:             32 GB

                      Hard Disk:            OS: 80 GB

                                               Logs: 140 GB

Note:  the values mentioned above for the hardware requirements are tentative and is a generalization. The exact values will differ from one implementation to the other, based on capacity calculations. For example, total content size has an impact on the memory needed for the database servers, thus, memory needed
reaches 64 GB in some cases. Search Index has implications on the Memory and the disk size for the index servers, so they will impact the values mentioned above.

Note: When building estimates for the new environment, you need to take future growth into consideration. One common consideration is to extrapolate for a 3 years period; thus, you will put the estimate for 3 years in the future. We generally set it this way so that we can have 3 years of operation without need for servers scale up/scale out. Based on customer’s need you might decide to build the estimate for more years as 5 years.

Following are the main extrapolations you will need to make:

1. What is the expected number of users for the new environment?

If this is an intranet portal that includes all company employees, then you could estimate the current number of employees and add to it based on the 3 year growth expectation. Otherwise, you can use the current value from the previous analysis and estimate the growth for the next period.

 2. What is the expected peak RPS (Requests per Second)?

Without prior knowledge of the current environment it is hard to build an estimate for the RPS. However, it is relatively easy to extrapolate it from the results received earlier with the LogParser exercise. One option is to assume that the requests will be the same or will grow by a percentage. An option is to assume that the requests will double for the new environment.

Note that you will need to build RPS for the different service applications that we will have in the new environment. It’ll be convenient to have this breakup from the current system; otherwise, you can estimate it by breaking up the RPS from the full system to its difference components.

3.  What is the expected Data Size?

There are formulas to estimate the data sizes for the SharePoint content databases; however, a clearer estimate can be done if sizes of current content databases are present.

Note: Based on your comfort with the values estimated above, you might decide to add 10% or 20% as safety margins.

In addition to the items above, you can get more information regarding SharePoint usage in the current environment to use in building the architecture of the new farm. One such important value is the number of items in the search index. This value has a profound impact on the design of the search architecture as we will see later, so having a basis for its estimate helps a lot when architecture is being built.

After building the estimates for the new environment, you should use them to fill the workload tables presented earlier.

This will be the end of the modeling step in the capacity management cycle.

The next step is to start on the design of the new environment. This will be handled in the next article.

Since you will get the results for a specific day, then the choice of that day is critical. Weekend days usually have lower use than the weekdays. Moreover, sometimes there are low months due to higher vacation rates or business reasons. A good sample for this is summer months where the number of users using the system is lower than other months.

A good strategy is to apply the scripts for different days to get clearer view on the current system usage. Keep in mind when designing the new environment that you need it to be able to handle the peak load at the peak period; moreover, you need to keep room for future growth. Thus, it is important to identify peak period and apply usage analysis on it.

The output of logparser queries created in the last blog is in tabular format, but you can use them to build graph that would highlight the required values. Samples of the results of the pervious queries in graph format are as follows:

Peak Requests Per Second (Peak RPS)

Average Requests Per Second (Average RPS)

Distinct Users Per Hour:

Using these graphs and the accompanying tables, it should be clear how to build the modeling tables mentioned earlier. They should be enough for identifying the current load on the system and be the basis for estimating the future load for the new system.

LogPraser can be downloaded at https://www.microsoft.com/download/en/details.aspx?displaylang=en&id=24659

Log Parser is a command-line tool that takes an input and a command to execute. It returns an output to the screen, an output file, SQL database or an image.

It is made up of three components:

1. Input Formats: Log Parser supports many input formats such as Event Logs, XML Files, and Windows Registry. The one we will need is IIS Log files.
2. SQL-Like Engine Core: This component processes data from the input source.  The syntax is SQL based, so it uses keywords like “select”, “where”, “group by”, etc.
3. Output Formats: Log Parser supports many output formats such as text file formats or SQL databases. The most important for our purpose is screen output and CSV format.

 More information on logparser formats and the syntax for logparser queries is present in the log parser documentation that is downloaded with the setup file mentioned above.

One of the best documents on how to use log parser for analyzing SharePoint usage is present at https://www.microsoft.com/download/en/details.aspx?displaylang=en&=tm&id=4616

 For easier use of logparser, it is better to add it in the path environment variable. This will enable you to use the logparser command from any folder that you created. The default path is C:\Program Files (x86)\Log Parser 2.2

To start using LogParser on your environment, create a folder and put in it the log files for a specific day. Copy to that folder the log files from the different front end servers too if you have multiple load balanced front ends.

Below are the most important queries that will help you identify the usage patterns for your environment:

1. To get different users and the number of times they are present in the logs, apply the following command. (Note that the output is present in LogFileUsers.csv file)

             logparser -i:IISW3C -o:CSV "select count(*) as ct,cs-username,logfilename from *.log group by cs-username,logfilename" >LogFileUsers.csv

            To get the total number of users per log file, run the following command:

            logparser -i:CSV "select sum(ct) as sum,count(*) as users,logfilename from LogFileUsers.csv group by logfilename"

       2.  Get the top 20 users of the environment by the following query:

logparser -i:IISW3C "select top 20 count(*) as ct,cs-username as user from *.log group by user order by ct desc"

       3. Get the Requests Per Second:

           Requests per second and their distribution over time is one of the most important measures to the environment usage. Thus, it is important to determine sizing for the new environment.

           Generally, when a user requests a page, the browser sends an anonymous request to the server. In case of SharePoint with Windows Authentication, the server will return HTTP 401 response, so the client browser will return authentication details. This 401 response produces no load on the server, but it is logged in the IIS log files. This is why the previous queries results contained requests with empty username.

           In next queries, you will find a “where” clause that will exclude these 401 responses as they have no load on the server and shouldn’t be considered within the Request Per Second (RPS).

         The queries to get the Requests per Second are:

          (Note: These queries use a script file “load.txt” as input. The script of the file is present at the end of this blog)

           logparser -i:IISW3C file:load.txt -o:csv -q >RequestsRes.csv

           -- distribution by Seconds

           logparser -i:CSV –o:CSV "select count(*) as ct,secs,max(ss) as ss,max(mi) as mi,max(hh) as hh from RequestsRes.csv group by secs order by secs"  -q >secsdist.csv

           -- distribution by minutes

           logparser -i:CSV -o:CSV "select count(*) as ct,div(secs,60) as minu,max(ss) as ss,max(mi) as mi,max(hh) as hh from RequestsRes.csv group by minu order by minu"  -q >mindist.csv

           --Hourly average Requests Per Minute

          logparser –i:CSV –o:CSV "select hh as Hour,avg(ct) as AvergeCT from mindist.csv group by hh order by hh" >HourAvgRPS.csv

           -- Hourly Peak Requests Per Minute

          logparser -i:CSV –o:CSV "select hh,max(ct) from mindist.csv group by hh order by hh"  >MinPeakRPS.csv

           -- Hourly Average Requests Per Second

           logparser -i:CSV –o:CSV "select hh as Hour,avg(ct) as SecPeak from secsdist.csv group by hh order by hh"  >SecAvgRPS.csv

           --  Hourly Peak Requests Per Second

           logparser -i:CSV –o:CSV "select hh as Hour,max(ct) as SecPeak from secsdist.csv group by hh order by hh"  >SecPeakRPS.csv

           The requests include requests for images orstatic files. Usually, user operations are 1/3 of the total requests.

         4. Get Distinct Users Over Time:

             Apply the following commands to get the number of distinct users in an hour

            logparser -i:CSV -o:CSV "selet count(*) as ct,cs-username,secs,max(ss) as ss,max(mi) as mi,max(hh) as hh from RequestsRes.csv group by secs,cs-username order by secs,cs-username"  -q >userdist.csv

           -- Distinct Users By Hour

           logparser -i:CSV -o:CSV "select hh,cs-username,sum(ct) as req from userdist.csv group by hh,cs-username order by hh,cs-username" >userhhdist.csv

            logparser -i:CSV -o:CSV "select hh,count(*) from userhhdist.csv group by hh" –q >userhhdist_PerHour.csv

           -- Distinct Users By Minute

           logparser -i:CSV -o:CSV "select div(secs,60) as minu,cs-username,sum(ct) as req from userdist.csv group by minu,cs-username order by minu,cs-username" >usermidst.csv

          logparser -i:CSV -o:CSV "select minu,count(*) from usermidst.csv group by minu" >usermidst_PerMin.csv

The script of the load.txt file is as follows: 

select
EXTRACT_FILENAME(LogFilename),LogRow,

 date, time, cs-method, cs-uri-stem,
cs-username, c-ip, cs(User-Agent), cs-host, sc-status, sc-substatus, sc-bytes,
cs-bytes, time-taken,

add( 
add(
mul(3600,to_int(to_string(to_localtime(to_timestamp(date,time)),'hh'))),
mul(60,to_int(to_string(to_localtime(to_timestamp(date,time)),'mm')))

   ),

  
to_int(to_string(to_localtime(to_timestamp(date,time)),'ss'))

) as secs,

to_int(to_string(to_localtime(to_timestamp(date,time)),'yy'))
as yy,

to_int(to_string(to_localtime(to_timestamp(date,time)),'MM'))
as mo,

to_int(to_string(to_localtime(to_timestamp(date,time)),'dd'))
as dd,

to_int(to_string(to_localtime(to_timestamp(date,time)),'hh'))
as hh,

to_int(to_string(to_localtime(to_timestamp(date,time)),'mm'))
as mi,

to_int(to_string(to_localtime(to_timestamp(date,time)),'ss'))
as ss,

to_lowercase(EXTRACT_PATH(cs-uri-stem)) as
fpath, 

to_lowercase(EXTRACT_FILENAME(cs-uri-stem))
as fname, 

to_lowercase(EXTRACT_EXTENSION(cs-uri-stem))
as fext

from *.log

where sc-status<>401

Additional information on using LogParser for SharePoint 2010 analysis will be included in my next blog.

LogPraser can be downloaded at https://www.microsoft.com/download/en/details.aspx?displaylang=en&id=24659

Log Parser is a command-line tool that takes an input and a command to execute. It returns an output to the screen, an output file, SQL database or an image.

It is made up of three components:

1. Input Formats: Log Parser supports many input formats such as Event Logs, XML Files, and Windows Registry. The one we will need is IIS Log files.
2. SQL-Like Engine Core: This component processes data from the input source.  The syntax is SQL based, so it uses keywords like “select”, “where”, “group by”, etc.
3. Output Formats: Log Parser supports many output formats such as text file formats or SQL databases. The most important for our purpose is screen output and CSV format.

 More information on logparser formats and the syntax for logparser queries is present in the log parser documentation that is downloaded with the setup file mentioned above.

One of the best documents on how to use log parser for analyzing SharePoint usage is present at https://www.microsoft.com/download/en/details.aspx?displaylang=en&=tm&id=4616

 For easier use of logparser, it is better to add it in the path environment variable. This will enable you to use the logparser command from any folder that you created. The default path is C:\Program Files (x86)\Log Parser 2.2

To start using LogParser on your environment, create a folder and put in it the log files for a specific day. Copy to that folder the log files from the different front end servers too if you have multiple load balanced front ends.

Below are the most important queries that will help you identify the usage patterns for your environment:

1. To get different users and the number of times they are present in the logs, apply the following command. (Note that the output is present in LogFileUsers.csv file)

             logparser -i:IISW3C -o:CSV "select count(*) as ct,cs-username,logfilename from *.log group by cs-username,logfilename" >LogFileUsers.csv

            To get the total number of users per log file, run the following command:

            logparser -i:CSV "select sum(ct) as sum,count(*) as users,logfilename from LogFileUsers.csv group by logfilename"

       2.  Get the top 20 users of the environment by the following query:

logparser -i:IISW3C "select top 20 count(*) as ct,cs-username as user from *.log group by user order by ct desc"

       3. Get the Requests Per Second:

           Requests per second and their distribution over time is one of the most important measures to the environment usage. Thus, it is important to determine sizing for the new environment.

           Generally, when a user requests a page, the browser sends an anonymous request to the server. In case of SharePoint with Windows Authentication, the server will return HTTP 401 response, so the client browser will return authentication details. This 401 response produces no load on the server, but it is logged in the IIS log files. This is why the previous queries results contained requests with empty username.

           In next queries, you will find a “where” clause that will exclude these 401 responses as they have no load on the server and shouldn’t be considered within the Request Per Second (RPS).

         The queries to get the Requests per Second are:

          (Note: These queries use a script file “load.txt” as input. The script of the file is present at the end of this blog)

           logparser -i:IISW3C file:load.txt -o:csv -q >RequestsRes.csv

           -- distribution by Seconds

           logparser -i:CSV –o:CSV "select count(*) as ct,secs,max(ss) as ss,max(mi) as mi,max(hh) as hh from RequestsRes.csv group by secs order by secs"  -q >secsdist.csv

           -- distribution by minutes

           logparser -i:CSV -o:CSV "select count(*) as ct,div(secs,60) as minu,max(ss) as ss,max(mi) as mi,max(hh) as hh from RequestsRes.csv group by minu order by minu"  -q >mindist.csv

           --Hourly average Requests Per Minute

          logparser –i:CSV –o:CSV "select hh as Hour,avg(ct) as AvergeCT from mindist.csv group by hh order by hh" >HourAvgRPS.csv

           -- Hourly Peak Requests Per Minute

          logparser -i:CSV –o:CSV "select hh,max(ct) from mindist.csv group by hh order by hh"  >MinPeakRPS.csv

           -- Hourly Average Requests Per Second

           logparser -i:CSV –o:CSV "select hh as Hour,avg(ct) as SecPeak from secsdist.csv group by hh order by hh"  >SecAvgRPS.csv

           --  Hourly Peak Requests Per Second

           logparser -i:CSV –o:CSV "select hh as Hour,max(ct) as SecPeak from secsdist.csv group by hh order by hh"  >SecPeakRPS.csv

           The requests include requests for images orstatic files. Usually, user operations are 1/3 of the total requests.

         4. Get Distinct Users Over Time:

             Apply the following commands to get the number of distinct users in an hour

            logparser -i:CSV -o:CSV "selet count(*) as ct,cs-username,secs,max(ss) as ss,max(mi) as mi,max(hh) as hh from RequestsRes.csv group by secs,cs-username order by secs,cs-username"  -q >userdist.csv

           -- Distinct Users By Hour

           logparser -i:CSV -o:CSV "select hh,cs-username,sum(ct) as req from userdist.csv group by hh,cs-username order by hh,cs-username" >userhhdist.csv

            logparser -i:CSV -o:CSV "select hh,count(*) from userhhdist.csv group by hh" –q >userhhdist_PerHour.csv

           -- Distinct Users By Minute

           logparser -i:CSV -o:CSV "select div(secs,60) as minu,cs-username,sum(ct) as req from userdist.csv group by minu,cs-username order by minu,cs-username" >usermidst.csv

          logparser -i:CSV -o:CSV "select minu,count(*) from usermidst.csv group by minu" >usermidst_PerMin.csv

The script of the load.txt file is as follows: 

select
EXTRACT_FILENAME(LogFilename),LogRow,

 date, time, cs-method, cs-uri-stem,
cs-username, c-ip, cs(User-Agent), cs-host, sc-status, sc-substatus, sc-bytes,
cs-bytes, time-taken,

add( 
add(
mul(3600,to_int(to_string(to_localtime(to_timestamp(date,time)),'hh'))),
mul(60,to_int(to_string(to_localtime(to_timestamp(date,time)),'mm')))

   ),

  
to_int(to_string(to_localtime(to_timestamp(date,time)),'ss'))

) as secs,

to_int(to_string(to_localtime(to_timestamp(date,time)),'yy'))
as yy,

to_int(to_string(to_localtime(to_timestamp(date,time)),'MM'))
as mo,

to_int(to_string(to_localtime(to_timestamp(date,time)),'dd'))
as dd,

to_int(to_string(to_localtime(to_timestamp(date,time)),'hh'))
as hh,

to_int(to_string(to_localtime(to_timestamp(date,time)),'mm'))
as mi,

to_int(to_string(to_localtime(to_timestamp(date,time)),'ss'))
as ss,

to_lowercase(EXTRACT_PATH(cs-uri-stem)) as
fpath, 

to_lowercase(EXTRACT_FILENAME(cs-uri-stem))
as fname, 

to_lowercase(EXTRACT_EXTENSION(cs-uri-stem))
as fext

from *.log

where sc-status<>401

Additional information on using LogParser for SharePoint 2010 analysis will be included in my next blog.

Modeling the environment is basically analyzing and estimating the usage of the environment in terms of workload and dataset. Workload describes the usage of the environment in terms of total number of users, concurrent users, requests per second, usage distribution, etc. Dataset describes the volume of content stored in the system and how it can be distributed.

Workload information that is required to have a model of the new environment is present at https://technet.microsoft.com/en-us/library/ff758645.aspx and they are as follows

Dataset information required to have a model for the data sizes needed by the system are present at https://technet.microsoft.com/en-us/library/ff758645.aspx and they are as follows

Generally, the hard task is to come up with values to put in these tables. Usually, business users are the ones best equipped to fill these tables; however, they don’t have the technical knowledge to do so. Thus, the technical capacity planner must work with the business users to build up a model of the environment based on the speculated needs.

If SharePoint 2010 is the first version of SharePoint that the customer is using, then filling the tables above will be like shooting in the dark as there is very little historical data to use for estimation. Thus, it is important to come up with logical estimates for usage of the new environment and monitor usage patterns for the environment and consider scaling up/out when needed.

Note: Underestimating the usage of the new environment can result in a smaller farm size and reduced performance. On the other hand, overestimating will result in a larger farm that will not be fully utilized.
Thus, it is important to choose an approach either to overestimate or underestimate based on expected usage of the environment.

Estimating the environment usage is easier if there is historical data to use as a basis for further estimation. Historical data could come from a SharePoint 2007 environment that will be upgrade or another present SharePoint 2010 environment with comparable usage patterns.

Thus, extracting usage data from a current system is crucial to building a correct model for the new environment. One preferred method to do that is to use LogParser for retrieving this information. The most important advantage of LogParser is its flexibility as it allows you to simply write queries against the IIS logs and get the results in the format you prefer.

The next blog post will be about LogParser. It will include the URL to download it, the query language options, and the most common queries that you will need to use to extract the tables needed above.

It is important to plan for your SharePoint environment carefully before starting implementation. Many companies opt for a small or medium size farms (3 to 6 servers in the farm), but they need to make this decision based on a capacity planning exercise (Use the farm size guidelines at https://technet.microsoft.com/en-us/library/ff758647.aspx to determine the size farm appropriate to your organization).

Larger enterprises with large user base would generally opt for large size farms, so they need to do a more rigorous capacity planning exercise to select their initial deployment configuration.

It is important to note that Capacity Planning for SharePoint is part of a larger process for capacity management. Due to the nature of SharePoint, the data sizes and usage can get much higher than initially anticipated. For example, departments may decide to move more content into SharePoint than initially estimated, users may use My Sites heavily and demand even more quota for their sites, or custom built applications for some line of business applications may increase the usage of the SharePoint environment.

Thus, the SharePoint environment needs to be monitored carefully to identify any bottlenecks and remedy them quickly. Such issues are generally resolved by scaling-up or scaling-out. Scaling-up means adding more processing power, RAM, or disk to the current servers to cater for the load. Scaling-out means adding more servers to the SharePoint farm.  

The general approach for capacity management in SharePoin is detailed at https://technet.microsoft.com/en-us/library/ff758647.aspx and it includes the following steps:

1. Model
2. Design
3. Pilot, test & Optimize
4. Deploy
5. Monitor and Maintain

In the next posts, I’ll draft the approach for sizing of SharePoint 2010 and its different service applications. The approach will include the different decisions made and the reasoning behind them.