# Begin Example Script

Add-PSSnapin Microsoft.SharePoint.PowerShell

$oldestEntryDate = Get-Date "January 1, 2014"
$sitesToClean = get-spsite -ContentDatabase WSS_Content

$sitesToClean | ForEach-Object { $_.Audit.DeleteEntries($oldestEntryDate) }

# End Example Script

First, lets start with what continues to be good guidance from that earlier blog entry:

• The intranet root should be a publishing site. This continues to be true, as does the recommendation to use abstractions rather than organization names and the recommendation to focus on topically related information that is understandable from a content and navigation perspective and not organization names structure. No one cared then about the fact that the Travel department was under Finance, and no one does today.
• The recommendation from "SharePointea SiteCollectionFearophobia" (aside from the fact that I now realize that including fear and phobia in the same name is redundant) also continues to be true. Beneath the intranet there should be at least 4 wildcard managed paths titled "teams", "projects", "communities", and "programs" or "services". In some cases there should be more... for example, a law firm may require "matters" because that's the primary focus of collaboration for them. Self-Service Site Creation should be enabled, and Site Use Confirmation and Deletion should also be enabled.
• There should still be only one central Enterprise Wiki. For better or worse there may be other wiki sites out there with highly specialized knowledge or information... but when it comes to that "one view of the truth" perspective, there should be one, it should be at the top, and it should follow the rules of wikis as described in "Do You Weally Want a Wiki?". Everyone can edit, everyone can see everything.

There have also been numerous changes from the 2007 product:

• The Site Directory has been deprecated in 2010 and no longer exists at all in 2013. While I personally consider this extremely unfortunate, I'm fairly certain that the SharePoint product group had metrics indicating that the directory wasn't be used or wasn't being used correctly. I agree with this perspective but I also think that the reasons why it wasn't being used were of greater concern and if we had fixed those issues (people didn't know how to use it, people didn't create site collections, etc.) it would have been a valuable feature and template to keep.
• Search is now fully Enterprise Class. It has gone through two major revisions and the direct integration/merging of many of the features of FAST for SharePoint. Aside from the fact that web analytics is now a Search feature, the Search service now has the ability to scale to a relatively insane degree and can truly index an entire company's worth of information. It is no longer about "SharePoint Search"... it is truly Enterprise Search.
• The "Collaboration Portal" no longer exists (it was removed in SharePoint 2010). Instead we have the Publishing Portal template which offers much of the same functionality and is the intended replacement.
• The 2007 My Sites have gone through extensive revision and although the fundamental architecture remains the same, the purpose of them has evolved. At minimum we have seen at least 1 name change from "My Site" to "SkyDrive Pro" and soon will be renamed to "OneDrive for Business". SharePoint also continues to refocus attention to the personal site, most visibly with the introduction of the social features of the platform.

Finally, the introduction of completely new features potentially changes the landscape for the platform:

• An emphasis on host-named site collections as the preferred method of creating new sites. This is most valuable when there is not already a well defined structure for SharePoint site collections.
• A completely new way to do branding.
• Metadata based navigation.
• Cross-site publishing.
• Increased functionality for mobile

I'm not going to go into the details of all of this... but suffice it to say a lot has changed since SharePoint 2007. However, the goal of the model I outlined in that blog entry so many years ago was for it to be resilient to change. You shouldn't have to re-architect with every product release and upgrade or whim of the business. The real questions are whether or not the model outlined continues to be of value, if it has served the purpose it was originally designed for, and what changes need to be integrated to continue to let it serve its primary purpose.

I am happy to report that the model still works and I still strongly recommend it to my customers. As I mentioned, so far I've received nothing but positive feedback on how it has impacted my customers. There are some slight enhancements/modifications I would like to make though...

• Do NOT align your site/web application naming strategy with our product names. We just change things too often to make that a good decision. If I never see http://mysite or http://my or http://sharepoint again it will still be too soon (and I know I'll see them again). Instead, come up with an ACTUAL name... something that represents the ideals of the business. For example, http://collaborate or http://share or http://us (as in, all of us... not united states) are fantastic demonstrations of what the solution is all about. Use those names... not our product names.
• DO enable Self-Service Site Creation (SSSC) AND Site Use Confirmation and Deletion. These two features are intended to work together. The first increases the number of sites that will be created. The second forces people to acknowledge that the site is still relevant and has value to them and should be retained. One creates content, the other cleans it up. Together they ensure that your SharePoint environment doesn't grow out of control. Yes, there will be a spike as sites are created... but over time they will be cleaned up if they're not used. This will also have the added benefit of cleaning up sites that are no longer used at all... even if they weren't created with self-service site creation. Many people would be concerned about the idea of "deleting sites"... but remember: We have recycle bins for this kind of thing, and you have backups if absolutely necessary. Deleting a site is not a permanent loss... it simply removes the site from availability to the end user so that they MUST tell you it is required for their business activity at which time you can identify them as the new owner. This is a good thing!
• DO use real business users for your site collection administrators... and in fact, demand both a primary and a secondary (this can be automatically enforced by SSSC). This is reinforced by SharePoint's implicit separation of the Farm Administrators from the Content Owners. It is the responsibility of the Site Collection Administrators to manage the content, not the Farm Admins. If they don't know how or you're concerned they may do the wrong things, that's a training issue, not a reason to violate a policy recommendation or put the content decisions on the support teams.
• DO use Quotas. They're there for a reason - to keep your end users from consuming an entire farm's worth of storage. Too many times I hear of customers say that they don't want to use quotas and in the same breath discuss how their users never clean up or archive content. Quotas are they way that we encourage people to clean up their content. We have to provide a choice: Clean up what you don't care about or request more storage... which brings us to...
• DO use Charge-Backs. Consumption of resources should imply a cost. This is where we get the customer to actually question whether that storage/quota increase is really worth it. Do I want to pay an extra $1K a month, or do I want to delete some old files? This is a valid question that every user should be able to ask themselves... and if they really need those files then that extra $1K might actually be worth it. As I realized recently: If there is no cost to excessive consumption, there is no reason to not consume excessively.
• Consider elevating search. SharePoint has offered "Enterprise Search" for quite some time now, but it has constantly been relegated to "SharePoint Search" because people don't realize just how powerful it is... plus, the SharePoint Admins seem to think that the search functionality should be exposed under some SharePoint-centric hostname. The fact that the Enterprise Search Center lives inside of SharePoint perpetuates this perspective that all search knows is how to search SharePoint. Instead, consider creating a search-specific web application... http://search for example... and let that be nothing but search. This elevates Search as a discrete service that can provide access to all types of content, not just SharePoint. Search is now a first-class citizen... elevate it.

These recommendations are intended to provide the most value from your SharePoint deployment possible, and try to help you avoid a lot of the difficulties that Premier Field Engineering finds a lot with customers who, in the process of trying to protect themselves from FUD or not understanding how to assemble the massive-box-of-legos-that-is-SharePoint into a high value service, manage to create for themselves. We want you to be successful, and avoiding pitfalls before they become sinkholes is the best way we know how.

If you have questions, ask them below. I'm happy to respond as soon as I can.

Hopefully after reading my previous post on how SharePoint and snapshots really, really don't get along well, you're wondering how you might take advantage of this incredibly awesome capability to literally turn back time on your environment in case of an emergency.

Before we discuss how snapshots can help SharePoint, let’s discuss briefly how snapshots should NOT be used:

• Snapshots have almost no place in a regularly scheduled backup plan. That is, unless you've followed the carefully coordinated steps below, there's pretty much nothing you can or should do with snapshots to recover a failed SharePoint server or drive (including SQL LUNs or data storage).
• Snapshots should not be used casually. All of the reasons listed in the previous article imply that snapshots can cause some interesting and extremely difficult to resolve issues in your SharePoint environment. They shouldn't be used as part of any real frequent administrative activities.
• Snapshots should not be used to replicate an environment. Along with all of the potential issues mentioned in the previous article, using snapshots to replicate an environment for testing or pre-production use introduces an entire list of additional risks, such as duplicate names on the network, AD membership failure, unexpected notifications to end users from the new, non-production system, etc. Don't do it. A well designed deployment script (manual or code) is the better answer here.

So… what role can snapshots play?

Warning: I am about to discuss something that violates Microsoft supportability statements. Although the process below SHOULD work as described without any significant problems or errors, you do so at your own risk.

Any use of snapshots must avoid the key elements that present inconsistency in the environment. The use of snapshots must specifically avoid any loss of synchronization with the SharePoint Configuration database. Other issues listed in my previous entry, specifically  storage, network, and domain membership synchronicity should also be avoided.

The unfortunate reality is that there is only one time when we can be absolutely certain that the environment is in a known, stable, unchanging state: When the machines are physically turned off. You can click a thousand buttons, pre-script backup commands, or press the "snapshot" button as fast as you can, and there is still going to be a risk of misalignment. It is only when the machines are turned off and SharePoint is physically incapable of doing work or making request of the databases that you can be certain that any snapshots or backups you make will be consistent between each of the infrastructure components.

Of course, once you power down the machines and take the various snapshots, you must manage that backup as if it was one single atomic unit. That is, if any piece of it is to be restored, all of it is to be restored. You cannot choose to restore one SharePoint server and not the others. You cannot choose to restore one database and not the others. You cannot choose to restore the SharePoint servers and not the database server. They are all linked, cannot be separated, and cannot be restored independently. Restoring one means restoring all. This includes the content databases, which means that if you suddenly decide you need to revert your SharePoint environment to that snapshot, you will also be reverting the state of the SharePoint content to that point in time. This could potentially be a significant amount of data lost and would likely be unacceptable to your end users.

As I mentioned in the first bullet above: Snapshots have no place in a regular backup or disaster recovery strategy, mostly because of the issues listed above. In particular, the requirement that the content be included in this backup/recovery plan makes using snapshots as part of a standard backup strategy nearly impossible (notice the "nearly" ;)). There is only one process that can change this, which I'll discuss at the very end of this post.

So… where are snapshots useful? Where can you use them and have them be something actually useful? There's really only one place that they offer true value for SharePoint: Patching! 

SharePoint (and I might contend Windows and SQL) patches are a one-way proposition. Once you've installed them, there is absolutely no supported way to remove them. Your options are to make them work, or completely reinstall your environment. However, a properly timed and executed snapshot can give you a (reasonably) safe back-out strategy that would otherwise be impossible using the built in features. If you have a production-identical environment and you've done all of your testing, deploying a SharePoint service pack should be fairly simple, flawless, and give you no cause for concern… but no one will hold it against you for having a safety net. Snapshots are the only thing that offers the possibility of sufficiently capturing every aspect of the environment to be that safety net.

This is predicated on the process being followed properly. Assuming you've done adequate testing and are in your scheduled maintenance/outage window, the process should look like this:

1. 1.       Shutdown all of the VMs.
2. 2.       Snapshot the VMs (either in the virtualization or storage tiers).
3. 3.       Backup/Snapshot the SQL Storage.
4. 4.       Power on all services
5. 5.       Apply any updates as normal.
6. 6.       Verify update(s) were completed successfully.

In the event that you find yourself needing to roll back to the snapshot that was taken using the above process, the rollback should look something like this:

1. 1.       Power down all SharePoint VMs.
2. 2.       Restore all snapshots/backups (including SQL and SharePoint VMs)
3. 3.       Power on VMs.

Using this process there is a potential risk of a loss of connectivity to Active Directory. If the machines have changed their account passwords (Yes, every AD joined machine has its own account and account password), then when you revert the snapshot to the state prior to that change, the domain membership will be broken. For this reason, be sure you have the password for a local administrator account. This should allow you to log into the machine, unjoin and re-join the machine from the domain. This resets the machine account password and will allow the system to operate normally.

As with all things, details matter in the backup/snapshot and the revert processes. It is important that the backup and snapshot be appropriately thorough, and that no major elements of the process be missed. This means that, depending on your infrastructure, the number of things and/or places that have to be backed up or snapshotted must be well known and accounted for.

• (A) The easiest option is to simply take a snapshot at the storage tier, being certain to capture the virtual machine hard drives and any pass-through mounted LUNs for SQL Databases (or SharePoint servers, if applicable). In this instance, the SQL Servers should ALSO be powered down so as to prevent them from attempting to modify data during the snap.
• (B) + (D) You may also choose to perform snapshots of the virtual machines in the virtualization tier, and snapshots of SQL at the SAN tier.
• (C) + (D) You may choose to perform a backup copy of the VHDX or VMDK files which contain the virtual machine hard drives. This is the equivalent of a "manual" snapshot, but reduces the potential negative performance impact that true snapshots generally imply. SAN locations must also be backed up/snapshotted.
• (B)/(C) + (E) You may choose to perform snapshots of the VMs in the virtualization layer and perform SQL-based backups of all databases.

The options listed above aren't exhaustive… it's up to you to know what you need to back up/snapshot and where it will best meet your needs to do so. For example, it is entirely possible for your VMs to use a VMDK for their system drive, and pass-through storage directly to your SAN for a non-system "data" drive. You must be aware of the fact that your VM snapshot will NOT snapshot the pass-through storage… it is up to you to either snapshot this at the storage tier or perform a high fidelity, full backup of that volume while the machines are powered down. The number of things you need to consider to be certain your snapshot process captures everything is going to increase as the complexity of your environment increases. As another example, RBS creates yet another point that must be captured as part of this process.

The ultimate goal of all of this to help you understand that when you're using snapshots with SharePoint, you need to not just snapshot a machine, a database, or a set of files… you're attempting to capture the state of the SharePoint farm at a point in time… and you need to capture that state in such a way that it is in perfect sync across all of the components. Shutting the machines down is a way to reduce the number of things that could be altering that state at any given moment, thereby increasing the likelihood that the capture of that state that you do will be a complete capture of that state, and that if you need to revert to that prior state, there are fewer things you need to accommodate, fix, or otherwise deal with that could make your farm completely unstable (and unsupportable).

I've tried to emphasize that the farm backup is really only good if it is intact, and it's only good at one moment in time. There are some reading this that will try to do things I'm NOT suggesting. For example, I'm almost certain I will someday receive a support request that says something like this:

"Customer followed your blog entry on snapshots and did one just before deployment of a service pack. They then learned two weeks later that their customizations are not compatible, so they reverted the farm to the snapshots and then restored their content databases to the most recent backup so as to not lose any data. Now their farm is broken."

Just trying to head this one off at the pass, let me be clear: THIS WILL NOT WORK. You cannot have the farm in a state that is "pre" service pack and databases that are "post" service pack. The pre-SP farm may not know how to talk to the post-SP databases… there is no telling what structural changes may have been made as part of the service pack upgrade, and we cannot support any scenario in which you try to do this or any variation of this process. JUST SAY NO.

And now, I did promise to mention the one caveat to the "snapshots are not backups" statement. It MAY be possible to use snapshots as part of a regular backup process as long as all of the above stated rules of snapshotting are maintained (ex., the machines are powered off, etc.) AND there have been no changes to the SharePoint platform. This means that IF the SharePoint version is exactly the same across all snapshots, no hotfixes or other patches have been deployed to SharePoint, no solutions or features have been deployed, changed, or upgraded, and no logical architectural changes have been made (no new web applications, modifications to service application design, etc.). This caveat also only really applies to the Farm vs. the Content Databases. In this scenario, as long as all other things are equal, it may be possible to use an older snapshot, and then restore a newer content database. The trick to this is forcing SharePoint to treat it as a "new" database, forcing it to somehow sync up to the DB. The following process is an example:

1. 1.       Restore the SharePoint Farm (including all content) as you normally would.
2. 2.       Detach all SharePoint content databases from the farm (in SharePoint, not in SQL).
3. 3.       "Catch up" the databases to the most recent version you're looking for.
4. 4.       Re-attach the newer databases to SharePoint, forcing SharePoint to treat them as if they're brand new and doing all of the normal synchronization and maintenance processes against them.

This possible option though comes with a ton of caveats, limitations, requirements, and details, and is generally NOT RECOMMENDED… even by me. I only mention it in the spirit of full disclosure (and because someone will eventually ask me the 'what if' question) and is not something I'm saying you should do. Pretty much ever. Again, just say no.

Thanks for reading… and if you have any questions please feel free to ask. I'll get back to you as soon as I'm able.

Chris Mullendore

Well, I had exactly this need recently while working with a customer on a performance concern they had in their environment, so I quickly whipped up a utility to do exactly that. It is absolutely simple by design. It does one thing, does it very basically, and accomplishes the goal. It is called quite simply PingUrl.

PingUrl is compiled as a windows Console app (yes, PowerShell is there, but you don't always have PS at your disposal, so this is a bit simpler). Running it is simple: PingUrl followed by the URL to ping. For example:

pingurl http://www.bing.com bing.log

This will cause pingurl to submit a request to the default page at http://www.bing.com and report to the screen exactly how long that request took. This information will also be written to a log file called bing.log which will be created/appended automatically at the current directory. The ability to name the log file allows you to run multiple instances of PingUrl in different command windows but in the same directory. My customer had multiple URLs which forced requests to go to specific servers behind the load balancer, allowing us to identify performance differences between the servers. I found it to be useful and interesting... and just randomly common and esoteric to be worth sharing. So, code listed below.

If you find it useful, let me know. :)

Update 1/25/2013: At Thomas's (good) suggestion, compiled version attached. Thanks for reading!

Due to a quirk in the layout choices made by the MSDN Blog site, the following code may appear cut off on the right side. If you select it, copy, and paste it into another tool such as notepad, it should copy completely and correctly. Good luck!

using System;
using System.Web;

namespace PingUrl
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length == 0)
            {
                Console.WriteLine("Please provide a URL to request. You must have appropriate permissions to request this URL and logfile name.");
                return; // Exit Application
            }

            // Prepare the log file to write
            System.IO.StreamWriter logStream = null; // null assigment resolves an "unassigned variable" error in compiler
            if (args[1] != null)
            {
                Console.WriteLine("Logging to " + args[1]);
                logStream = System.IO.File.CreateText(args[1]);
            }
            

            while (true) // Do Forever -- Control-C to exit application.
            {
                // Response captured in case writing out the response is valuable
                string response;

                // Start datetime for this request.
                DateTime started = DateTime.Now;

                try
                {
                    System.Net.WebClient client = new System.Net.WebClient();
                    client.UseDefaultCredentials = true;
                    
                    started = DateTime.Now;

                    System.IO.StreamReader reader = new System.IO.StreamReader(client.OpenRead(args[0]));

                    response = reader.ReadToEnd();

                    Console.WriteLine(started.ToString("T") + " Duration: " + DateTime.Now.Subtract(started).TotalSeconds.ToString());
                    if (args[1] != null)
                    {
                        logStream.WriteLine(started.ToString("T") + " Duration: " + DateTime.Now.Subtract(started).TotalSeconds.ToString());
                        logStream.Flush();
                    }
                }
                catch (System.Net.WebException ex)
                {
                    // Catches all HTTP errors, such as 404, 500, or access denied.
                    Console.WriteLine("Error accessing the specified URL: " + args[0]);
                    Console.WriteLine("Error Message:");
                    Console.WriteLine(ex.Message);
                }
                catch (Exception ex)
                {
                    // Catches everything else, which is likely a local error.
                    Console.WriteLine("General Execution error: " + args[0]);
                    Console.WriteLine("Error Message:");
                    Console.WriteLine(ex.Message);
                }

                // continue while loop - remember... this continues until control-C is pressed to break it.
            }

        }
    }
}

PingUrl.zip

One of the many joys of our currently heavily virtualized world is the use of Snapshots. Whether they're in the storage system or in the virtualization tier, snapshots can provide an invaluable form of insurance against some risky or irreversible activity that's about to be taken in your environment.

Just not for SharePoint. SharePoint HATES snapshots. They go against everything that SharePoint does to manage itself. They create inconsistency and conflict. They change the rules behind SharePoint's back. They are, simply, evil.

And yet, they're so darned useful!

So… we want to use snapshots. We want to leverage the powers of the tools we've paid for. We want to have a safety net that can help us back out of bad decisions. We want to be able to turn back time and recover something we might have lost. We want the power that we've paid for!

It is possible to use snapshots with SharePoint. First, we have to understand a little about SharePoint, a little about snapshots, a little about networking, and a minor surprise about Active Directory domain membership.

Issue #1: The SharePoint Timer Service
Our first issue is the Timer service that should always be running on every SharePoint server. Always. It is the responsibility of the Timer service to check for work that needs to be done, do it, and then report back that the work has been completed. This can include basic maintenance tasks such as cleaning up a few rows in a database to large tasks, such as creating a new web application in the farm. These can be in request to and end user request, such as deploying a SharePoint farm-based solution, or on a scheduled basis, such as starting a search crawl or synchronizing information between databases and service applications. There are many of them, and they can start at any time (per their configuration) and can run for a significant duration of time. You can think of each timer job as a ball that SharePoint has to keep in the air. Some balls need to be thrown up more frequently, some take longer to return to earth, and some get thrown at SharePoint from nowhere. In all instances, it is the responsibility of the Timer service to keep all of the balls up.

In order to keep all of these balls in the air, the Timer service on a SharePoint server checks the Configuration database once per minute and asks if there is any work that it is responsible for. That is, every server checks in once per minute to find out if it should be working on something, and each one is checking in at a different point within that minute due to various factors. So, Server 1 might check at :12 seconds into the minute… Server 2 at :22 seconds, Server 3 at :55, and Server 4 at :56 seconds. There's no real way of knowing… all you know is that at least once per minute, every one of your servers is attempting to do work.

Example: You ask SharePoint to create a new web application. In order to do this, SharePoint must configure a new IIS site on all SharePoint servers that host the Web Application Server role. So, the Central Admin server creates a new timer job assigned to each WFE, informing that WFE that it is to go through any necessary steps to support that new web application. This is what is happening when you see the "Please wait" dialog in SharePoint; Central Administration is waiting for all of the timer jobs it has created to report as being completed successfully. All is well.

However, you decide that something should have been done differently when you deployed the web application, and you, being knowledgeable in virtualization, took a snapshot of the SharePoint servers before you started. "Simple solution!" you say, and revert the snapshots to the previous state.

Unfortunately, all is not well. Although you did remove the IIS site and SharePoint configuration, the configuration database itself firmly believes that the new web application was created, and believes that it should be maintained. The servers will not retry creating it because the timer jobs that were assigned to them say that the work was done successfully, yet all of the timer jobs targeted at that new web application continue to run. And fail. Constantly. Once per minute per server.

"Oh! Well, I'll just revert the SQL server too!" you say, and you do. Unfortunately, while your SQL server's OS state was reverted, you're using a SAN for the actual data files… and those files are unaffected by your virtualization snapshots. Still no good. Problems abound.

Issue #2: The SAN
"Fine, I'll just do all of them… SharePoint, SQL, and the SAN!" you'll say. However, despite your wizard-like speed with a mouse and keyboard, you just can't click all of the buttons at once… and different versions of snapshots take different amounts of time to begin the snapshotting process and to complete the creation of the snapshot.

Issue #3: The Network
Even if you do manage to get the SAN and VM snapshots perfectly lined up, none of those technologies currently capture the state of the network. While servers are talking to each other, to clients, and to databases, packets of information are whizzing over the wires, and network infrastructure is judiciously trying to deliver those packets. If we use our "ball in the air" analogy, this would be the equivalent of one server throwing a ball at the other server which has previously agreed to catch it. Your timing though is impeccable, and you manage to perform this snapshot at just the moment where one server has sent it but the other has not yet received it. Plus, while your environment is performing it's snapshot, the network that is ferrying the ball tosses it to the previously receiving server. Imagine for a moment that image of a baseball player frozen mindlessly as the ball falls directly onto that player's forehead and to the ground. Fortunately, most network protocols in place today are relatively stateful… and when you revert the snapshot (again, assuming your synchronization was perfect to the nanosecond, which it couldn't be), they'll retry. More likely though is that something got lost in translation… and it was all your fault.

Issue #4: Domain Membership
Did you know your computer has an account in Active Directory? Did you know that it changes its password automatically, without telling you, every 30 days? Did you know that if the machine changes its password and then you revert the machine to just before it initiated its own password change request, you may never be able to log in to that machine using a domain logon again? Now you do.

So… How can you safely use either SAN or VM snapshots in SharePoint?

Tune in next time to find out! ;)

The follow-up article, SharePoint vs. Snapshots (Part 2), is now available!

If both of these are true, why is it that so many organizations take something that has such a variable infrastructure cost, like SharePoint, and offer it to employees for free? They consider it a "sunk cost" or simply the price of doing business, and utilize an investment or funded model for the service, in which IT estimates what it's costs are going to be for the next year, generally based on past usage or growth trends, and then let employees consume, consume, consume. While I'm all for SharePoint getting used, I'm not "for" the problems this model creates. Specifically, it means encouraging end users to put as much information into SharePoint as possible without limits.

To be fair, I'm not calling SharePoint "expensive"; SharePoint itself is no more or less expensive that any other enterprise solution, such as email. In fact, SharePoint itself can be thought of as a "sunk" cost, usually factored as "cost per head". An end user typically only needs one license to access any amount of SharePoint services, and in many, many organizations that license is provided as part of a cost-effective bundle that includes the Windows, Exchange, Office, etc., licenses that any given employee may need. It might be factored into the employee cost just as a desk, telephone, PC, or internet might be. (I'm ignoring the complications of assets vs. non-assets, capitalization and depreciation, etc., but run with it. :))

So what's different? Consider email, for example: Each employee will typically have a single mailbox, and that mailbox will likely have a specific cap or quota applied to it. It's relatively easy to manage, and relatively static... increases in quotas or requests for secondary or shared mailboxes are exceptions, not the norm. On the other hand, SharePoint, if used as expected, allows not only for the services to be used but for them to be essentially self-provisioned. For example, within Microsoft, I can request a new SharePoint Site Collection whenever I feel necessary. I can create several at once if I feel so inclined. I can move files and content into my newly provisioned sites with impunity, and I never see a penny or problem.

The point is, that "sunk" cost-per-head model doesn't work. The infrastructure costs don't scale with the employee... it scales with usage, it scales non-linearly, and predictions of future growth or utilization are simply bad guesses. This means the investment model doesn't work; It puts a proactive funding model into a platform that must be managed reactively to end-user demand and use. Sure, you can look at the company initiatives and do your best... but that does nothing for the random executive that tells everyone in their division mid-year to move everything out of their file shares (this is not a good idea, by the way). Suddenly your use triples while your funding remains static and you're stuck.

The issue isn't only about users putting information in to SharePoint; It's as much about how rarely they remove or delete anything from it. So, information goes in, information does come out, but information is very rarely deleted. This comes from the fear that someday they might need it back and from simple laziness... the site is no longer used or is no longer necessary, but no one actually attempts to delete or remove it.

This where a Chargeback based model has value. Using chargebacks changes the mentality such that the business user can make an educated decision: Remove things and pay less, or request an increase in quota (at a higher cost) and pay more for the additional capacity. There may still be some elements of the system that are managed as operational costs and do not incur a specific charge to the business, or the model can be designed to recover all operating costs... but there must be an incentive to users to operating efficiently. SharePoint isn't the same as a basic file share, and the infrastructure and operations are appropriately more expensive.

In my perfect world, here is what I might see:

• Quotas would be established on a storage-used basis, for example:
• MySite: 100MB
• Free: <500 MB
• Small: 501 - 1,000MB
• Medium: 1,001 - 5,000MB
• Large: 5,001 - 25,000MB
• Isolated: 25,001 - 100,000MB
• Dedicated: >100,000 (multiple site collections or a single dedicated corporate service such as Enterprise Records Management)
• "Site Confirmation Use and Deletion" would be ENABLED, set at 90 or 180 days confirmation, delete after 30 attempts.
• Secondary Site Collection Administrators would be REQUIRED
• Farm-level services (Search, Term stores, etc.) would be directly funded as operational costs, or costs would be recovered in the above quota costs.
• Costs would be based on Quota, NOT on actual storage consumed.

This basic structure resolves nearly all of the problems many customers experience when managing a SharePoint environment:

• Charging immediately for the Quota requested (rather than storage consumed) prevents the requestor from requesting the highest level quota available immediately. Also allows for immediate purchasing of additional infrastructure as necessary.
• Continuing to offer a "free" site allows those that wish to test or evaluate the implementation of their solution to do so without being limited or impacted. Lets IT "Get out of the way" for basic users and collaboration spaces.
• Confirmation and Deletion allows sites to be deleted or archived, recovering space in the environment.

Governance plays a significant role here as well, and any charging model should be balanced against the governance plan that the environment is ultimately intended to support. Other factors, such as high availability requirements and disaster recovery requirements should also be considered and factored into the pricing. Having a fully redundant infrastructure is going to cost more. Having an identical disaster recovery plan is going to cost much more. Longer backup retention schedules are going to cost more.

The end result is an environment that does get heavily used, but only contains information that is actually important to the business, because they're paying for it. If it weren't important, they could delete it. This also limits the "moving the problem" scenario, where customers blindly attempt to move entire file shares into SharePoint. If they want to pay the higher operational costs, that's their business decision. They're funding you to cover it, and you'll be happy to do so because you can now afford the additional capacity required.

What is the single most important job of any System Administrator? (Hint: It’s not installing the software.)

Let me ask a different way: What is the one thing that an end users expects of a system more than anything else?

(seriously… think about it…)

The System Administrator for ANY system, including SharePoint, is very simple: Keep the system up, available, and usable, and secure. It doesn’t matter what the system is… end users expect it to be there for them when they need it (and they always need it now). It’s not installing more software, providing TPS reports (yellow only, please), answering phone calls, sending/responding to emails, updating facebook/twitter, or reading Slashdot or XKCD. If you’re the Sys Admin, you have one job… protect the system. In general, this means maintaining existing functionality.

There are two evil, rogue groups of people (joking) that are your arch enemies in your one purpose in life: Business Users (including management), and Developers. Lets compare…

You could think of it this way: The business users and developers are constantly trying to do more with less, while the SharePoint Administrator is being held accountable for any negative experiences the end user may experience… regardless of cause. The goals of the business users and developers are fundamentally opposed to those of the SharePoint Administrator.

This also means that a good SharePoint Administrator (or really any system administrator) has one job: SAY NO. At least initially. :)

“I can’t say No” you might be saying. I understand… and you might be right. Saying “No” outright can be a “career-limiting decision”. So, instead of outright "No”, it may be possible to help others come to your own conclusions… though you’ll have to do your homework first. In some instances you may find that “Yes” is really the right answer anyway. As you review what you’re being asked to do or implement, document the answers to the following questions:

• What does this do to the OTHER users of the system? Will it or should it be available to them?
• Is the implementation heavily localized? Will the deployment of it be global to all users in the environment?
• What does this do to the performance of the system?
• Always assume that the functionality is to be heavily used.
• What does this do to the security of the system?
• Is the functionality appropriately secure? Does it involve any exceptions to common/standard security policies?
• What does this do to the stability of the system?
• Does the software fail gracefully? What is the impact if the solution does not work? Are there any external systems this solution is dependent on?
• How does this affect the maintainability of the system?
• What is the patching process for this solution? Can it be updated when necessary? Does this introduce any regular maintenance demands?
• How does this affect the disaster recovery and backup strategy for the system?
• If the environment needs to be rebuilt, what is the process? Do existing DR plans safely recover this solution?
• How does this align with vendor best practices for the system?
• Are the correct objects referenced and used appropriately? Are the frameworks used proper?
• How does this affect the upgradeability of the system?
• What are the foreseeable impacts to your ability to upgrade to the next version of the product?
• What is the administrative overhead of this change in the system?
• What is the difference in people-hours required to maintain the system before vs. after this is deployed?
• How does this align with your own operational practices and policies?
• Is there anything your company might care about that isn’t listed above?

What the business understands boils down to one simple statement: Increased Cost ($$$). If you can describe what the financial impact will be to the system and the company for the change being requested, the business can make a better decision about whether to move forward with that change. The business usually views this strictly from the view point of licensing, developer time, or the initial purchase… but in truth, there’s almost always an operational impact to deploying a change. Our job is to quantify that operational impact, in dollars, so that the business can decide whether the new functionality is worth it. If the business decides it is worth it, then you can say yes because you’re going to receive enough funding to compensate for the change. If the business isn’t willing to fund at the level necessary to support the change, then the business is effectively telling you that they don’t want the change, or they are willing to accept a reduction in the level or quality of service. If someone says they are willing to accept the reduction of service, you should point out that you’re going to quantify and advertise the reduction in service to all of the end users. The goal here isn’t to be a jerk… but rather to either set expectations appropriately with the end user community, or to make the acknowledge that they’re not really ok with a reduction in service.

The main point is, if the business is willing to back up their request with funding such that you, the administrator, can meet the current and ongoing expectations of your end users, then by all means, say yes. More often though, when faced with the financial reality of the request being made, many business will simply say “never mind” and move on or find a different way. Occasionally someone else will say yes without factoring all of this in. Look for the person that’s afraid to take a vacation and is constantly complaining about the business making unreasonable requests of them… that would be where the request landed. :P

While onsite with a customer I was recently shown an interesting situation where membership data (the list of links that you are considered a "member" of) does not appear to remove sites that no longer exist, and does not update the URL for sites that do exist. For example, pretend you have a temporary site at http://tempsite/ that you are in the Members group for. SharePoint will automatically create a link to that site for you in your profile within appx. 1 hour. Afterward, you either delete this site (it was temporary, after all), or you do a database move to attach it to http://realsite/ instead. You would expect that in the same way that SharePoint created the temp link, it would now remove it since it is no longer there. Unfortunately, this update never happens. Also, since the "real" site now has the same ID as the "temp" site (because you moved the site via a database move), the real link to the real site may never be updated/created.

There have been several cases opened on this issue, and the SharePoint Product Group is aware that people are experiencing the problem. My customer, however, was more interested in an immediate resolution than a formal fix, as they had recently done a significant migration and wanted to quickly remove the large number of incorrect links on profiles... and my job as a Premier Field Engineer is to make my customer happy, not recite dogma. ;)

In the attachments below is a VisualStudio 2010 project that contains a quick, brute-force solution to the problem. The code written creates a command-line utility that requests a web application and a search prefix. The web application URL should be a link to your My Site Host web application. The search prefix narrows which profiles should have the membership deleted. For example, if the My Site host is at http://mysite and you wish to purge all membership data for users in the Contoso domain, the following command would work:

SP2010RemoveMembershipData.exe http://mysite CONTOSO

Note that this is a brute force purge of all membership data and will delete all membership links, including those added manually by an end user.

In order to rebuild the membership data, all existing content database sync connections must be "reset" so that they do a full sync instead of attempt to update incrementally. The following STSADM command will cause all existing synchronization connections to be removed, and will cause SharePoint to rediscover and rebuild the data (perform a full sync) the next time the timer job is run:

STSADM -o sync -DeleteOldDatabases 0

If you are feeling impatient, you may force the sync job to run immediately by locating the "User Profile to SharePoint Full Synchronization" timer job in Central Administration and selecting to "Run Now". Note that if you have a large number of site collections, this job could take time to run. You will not see new membership data immediately. Running the timer job manually simply expedites the process.

After the timer job completes, you should see the correct membership links in your user profiles. Keep in mind that the Membership link to a given site will only be created if the user is a DIRECT member of the group listed as the "Members" group for a given site. Being in the "Owners" group is not sufficient, nor is it enough to be in an Active Directory group that is in the Members SharePoint group (AD groups are not expanded by the sync process).

DISCLAIMER: This should be considered "sample code" and should be run at your own risk and only after your own testing. Source code is provided to allow you to review and modify the code to meet the needs of your situation and your environment. Feel free to modify, but please link back to this blog post if reposted elsewhere.

SP2010RemoveMembershipData.zip

So... in coming up with a standard, we need to consider some things...

1. Many SQL Servers are shared (for better or worse) with either other applications or other SharePoint farms. This means that any naming convention must help identify which environment the various DBs belong to, and group them together in an orderly fashion.
2. SharePoint has some fairly specific types of databases, and those databases should also be grouped together.
3. SharePoint has some built in categorization systems (i.e., Proxy Groups) that should also be taken into account.
4. SharePoint has some fairly specific limits that the naming convention should help me enforce.
5. Redundancy in a database name is usually not necessary.
6. GUIDs are evil (at least in object names and database indexes)

Here's how I've standardized on addressing as many problems as possible. (Note that this isn't exactly carved in stone... so if something doesn't work for you, then don't do it. :)). The components listed are IN ORDER of appearance in the naming convention, separated by underscores for readability. So, just go down the list and you should be able to stick to an easy to use and understand standard.

Environment Name

Almost all of my customers continually use a common set of names for their environments. Some are more common than others, but on the whole they're very similar and serve purposes. I suggest we condense each down to a 3 digit code. Here's my go-to list for now:

• PRD - Production
• TST - Test
• STG - Staging
• DEV - Development
• UAT - User Acceptance
• PRF - Performance Testing (or Load Testing)
• SBX - Sandbox
• INT - Integration
• MRG - Development Merge
• EVL - Evaluation
• ....or anything else you can come up with.

Examples: "PRD_...", "SBX_...", "UAT_..."

Database Type

Most databases in SharePoint can fall into one of 3 broad categories: It is either related to the Farm as a whole, is a content database, or is related to a service application. So, the following would be next in line:

• Farm - Databases related to the farm itself
• Content - Databases that are content databases containing web content
• SA - Databases that support any service application

Third party customizations or functionality are up to you. If you can fit them into this model then that's fantastic... but that's beyond the scope of this article.

Examples: "PRD_Farm_...", "SBX_Content_...", "UAT_SA_..."

Context

Now things get more fuzzy. As we get more specific, the kind of database should get taken into account and the naming strategy changes slightly. For "Farm" databases, simply listing the database context, content, or purpose works fine. For example "Config" or "Configuration" or "Usage" make perfect sense when reading the database names.

Example: "PRD_Farm_Config" (are you seeing how you can now look at this database and know EXACTLY what it relates to? It's a good thing! )

Content Databases are the next easiest - They should have the hostname (and port number, but only if necessary) followed by a 2- or 3-digit number, starting at "00". This means that we can always associate a content database with the web application it is associated with (it can only be associated to one), we have a running count of how many content databases are on that web application, and we can generally see the order they were created in. There's the natural question of "but how do I know which database a site was in?" here... but I'm happy to respond to that in the comments if someone asks the question. Simple answer: with rare exception, I don't actually care which DB contains which site.

Example: "SBX_Content_MySite_00", "SBX_Content_MySite_01"

Service Applications are a bit more tricky, because each one is different and they have differing numbers of databases. Generally though, I focus on the Proxy Group (and in my world, EVERY application belongs to a proxy group... "custom" is not an option), the service application name (without redundancy), and any additional qualifiers. Because there can be a lot of permuations of this, I'll give several examples...

Examples: "UAT_SA_Default_Metadata", "UAT_SA_Default_Profiles", "UAT_SA_Default_Profiles_Sync", "UAT_SA_Default_Profiles_Social", "UAT_SA_Internet_Search", "UAT_SA_Internet_Search_CrawlStore", "UAT_SA_Internet_Search_PropertyStore"

In these examples (though I'll bet it's obvious... again, it's a good thing!), "Default" and "Internet" are Service Application Proxy Groups (Default being the "default" proxy group in SharePoint), "Metadata", "Profiles", and "Search" are the kind of service application, and any extra parameters the differentiate the different types of DB that may be required.

This is only intended to be a good baseline idea, and is widely applicable to many, many users exactly as it is listed. However, it is not sufficient to solve every customer's needs. For example, hosting providers may need to include a section that focuses on the target customer (though I would bet that could be mostly satisfied with the proxy group). If you see that this doesn't meet your needs, then feel free to modify this method into something that does. I would incourage that you follow some basic rules though:

• Don't be redundant - There is no reason to include the word "Service" in the database name for a service application. You already said it was related to a service application by using the "SA" designation... there is no reason to say "Service" again. Also, don't include "DB" in the name... if I can see it in the SQL Server, there's a high likelihood I already know it is a "DB".
• Meet your needs - Use categories and category placements that best help you manage your infrastructure. For example, if a customer name/code is included, it should probably be BEFORE the database type designation. This would allow the "Acme_SA" and "Acme_Content" databases to be next to each other so that I know exactly where to look when someone from Acme Corporation calls.
• Violate properly - Every rule was made to be broken... but break it well. For example, if you need to restore a database, don't simply call it "_Restore". Name it something proper and useful. If it is going to be around for a long time, give it a name that properly fits the strategy. If it's not, include a "_DeleteAfter_20111231" indicator in the name that says when you're fairly certain you'll no longer need the database. If a DB is to contain only a single site collection, feel free to replace the number with the site name (though I would still contend the numbering is still fine).

SharePoint may occasionally be unhelpful when trying to conform to a naming strategy... but you were looking for an excuse to dig deeper into PowerShell anyway, weren't you? :)

Yes, it's worth it. If nothing else, your DB will call you good names instead of bad ones, and may even include you on their holiday card list. Any day that a DBA says I have the best naming strategy in the company is a VERY good day. :)

Update 7/23/2012:

Good references for those Service Applications that require PowerShell for database naming:

• SharePoint 2010 Configuring Search Service Application using PowerShell: http://blogs.msdn.com/b/russmax/archive/2009/10/20/sharepoint-2010-configuring-search-service-application-using-powershell.aspx
  Russ's blog is my go-to guide because it describes the commands simply and clearly. Important: Russ doesn't point it out, but when you will ABSOLUTELY want to use the -DatabaseName parameter. This will name ALL of the Search DBs with a prefix containing this name. This is CRITICAL to sticking with our convention.
• Starting UPS after creating the User Profile Service Application using Windows PowerShell - http://www.harbar.net/articles/sp2010ups2.aspx#ups5
  Spence gives the only truly thorough reference. I do have a few minor complaints, such as his script not stopping in the event of an error or reporting out what the error might have been. I've updated his script here:

 $sb = {
    Add-PSSnapin Microsoft.SharePoint.PowerShell
    $saAppPool = Get-SPServiceApplicationPool "DefaultServicesApplicationPool"
    $upa = New-SPProfileServiceApplication `
        -Name "Default User Profiles" `
        -ApplicationPool $saAppPool `
        -ProfileDBName "PRD_SA_Default_Profiles" `
        -SocialDBName "PRD_SA_Default_Profiles_Social" `
        -ProfileSyncDBName "PRD_SA_Default_Profiles_Sync" `
        -ErrorAction Stop
    New-SPProfileServiceApplicationProxy -name 'Default User Profiles' -ServiceApplication $upa
   }
   $cred = Get-Credential "litware\sp_farm"
   $job = Start-Job -Credential $cred -ScriptBlock $sb | Wait-Job
   Receive-Job -Job $job #Returns output to screen

First, consider that the ULS logs are designed with a singular purpose in mind: To allow an administrator identify and resolve issues in SharePoint. They are not used for any other purpose within SharePoint, and most administrators won't bother looking at them until there is an issue they are investigating. From this perspective, if your environment is "perfect" (yes it is possible), then the best answer for you is to simply turn them off. They consume system a small amount of system resources, and can consume a considerable amount of disk space. However, turning them off will prevent you from being able to do any kind of retrospective on significant one-time events. Your call.

On the other hand, some customers have them all set to verbose, all the time. This is a little rediculous if you're not actively investigating an issue. You're consuming extremely large amounts of drive space and potentially significant system resources and somewhat degrading performance for end users to capture data that you have no interest in actually looking at. I suggest resetting them to the default settings or even turning them off.

Now, if you ARE investigating a reproducible issue, then by all means, turn the appropriate logging level on to verbose, reproduce the problem (likely several times), then turn them back down. If the problem is difficult to reproduce and/or happens randomly, then leave them at an elevated level until you are able to capture an instance or two of the error, then turn them back down. The main point is, turn them up to capture the data you need, then turn them back down or off if you're not actively researching an issue.

Finally, the one thing you should ABSOLUTELY do is move them off of the system drive. ULS is designed to stop logging if it percieves a disk space issue... but that doesn't excuse you from properly managing both your disk space and your logging methods. Moving the logs off of the system drive ensures that logging isn't going to fill up the system drive, and that ULS logging isn't going to contend with your page file for disk IO. Note that in order to change the location fo the log file, the path must exist on ALL SharePoint servers, and the folder's permissions must be set to allow the SharePoint service to write to it. Errors in which you change the location and don't see log files get created in this location is typically one of these two problems.

So... here's a cheat-sheet for the general recommendation:

• FOR MOST ENVIRONMENTS: Keep the logs at the "Default" logging settings using one of the two following commands.
• SharePoint 2007 : STSADM -o SetLoggingLevel -Default
• SharePoint 2010: (PowerShell) Clear-SPLogLevel
• http://technet.microsoft.com/en-us/library/ee748656.aspx
• For "Perfect" environments - everything works well, few/no user complaints, exceptionally well managed.
• SharePoint 2007 : Central Admin, select "All" categories, and "Error", "Error".
• SharePoint 2010 : (PowerShell) Set-SPLogLevel -TraceSeverity Unexpected -EventSeverity Error
• http://technet.microsoft.com/en-us/library/ff607887.aspx
• For temprary troubleshooting purposes only, use verbose
• SharePoint 2007 : Central Admin, select "All" categories, and "Verbose", "Verbose".
• SharePoint 2010 : (PowerShell) Set-SPLogLevel -TraceSeverity Verbose -EventSeverity Verbose
• http://technet.microsoft.com/en-us/library/ff607887.aspx

Of course, there are various permutations of these combinations. For example, I might want the log files at "Verbose" while maintaining the event logs at "Error" so that I don't flood the event logs. The keys are 1) Know what you're looking for and where you might best find it, 2) Know that increased logging demands increased space and system resources for the logging activity itself, 3) Use the lowest level logging available to meet your needs, adjusting when necessary, 4) Use "Verbose" ONLY when troubleshooting an issue.