In this blog post, I will be sharing some guidance and practical tips in plain English on how I deliver a solution based on my years of experience from the field. The experience that I am sharing here mainly comes from the infrastructure and cloud services based projects which will differ from software development based projects. Nonetheless there are many similarities.

In my experience, the following five major phases fit nicely with almost all of the projects that I have delivered. These phases are common in many methodologies but are represented using different terminologies.

Throughout this blog post, I have kept the guidance to a generic, high level and avoided referring to any specific product or solution in order to provide information as simply as possible. I hope this will give you some basic understanding and allow you to adapt it to your own specific project.

Towards the bottom of this blog post, I have provided a quick snapshot of the overall methodology for a quick reference.

In the following sections, I will expand on each of the phases and provide some practical tips and guidance.

Envision

You know what the customer needs but how do you make sure that both your customer and your team stay aligned and focused on how to meet those needs? You define a Vision. A vision sets the tone of the project. This is the most important phase of the project that provides the foundation to build your project upon.

You define your vision in terms of the goals you are trying to achieve. It brings your entire team together on a common set of criteria that translates the vision into reality. Start by creating a "Vision and Scope document" where you will need to clearly define and confirm the following important aspects of the solution with your customer:

1. Vision Statement
2. Scope and Objectives
3. Use Cases
4. Approach, Timeline and Deliverables
5. Assembling teams and defining their roles, and responsibilities

In the following sections, I will describe those aspects in little more detail.

Vision Statement

Creating a vision statement may seem trivial but I believe it is the most important aspect of the envision phase. It is like a tag line that reminds everyone what the project is about. It works like a guiding light to keep everyone on track. A vision statement should be short and precise. For example:

“Improve productivity and content management by providing users with a central storage and collaboration space for sharing content, information, and ideas.”

Scope and Objectives

Scope and objects are the specific requirements of the solution that you will be developing and delivering. You will need to clearly define the requirements that are in scope and the ones that are out of scope. Make sure that you review and confirm them with the customer to avoid any misunderstanding.

Tip: After reviewing the scope, send an email to your customer with what you have reviewed and discussed to confirm your understanding. It never hurts to over communicate.

Use Cases

Use cases define how the solution or the product is going to be used. Make sure to capture as many use cases as possible by interviewing the stake holders and end users. Depending on the solution, you will need to capture use cases from various different business functions and departments such as Human Resources, Operations, Sales and Marketing, Finance and organize them accordingly. You will also need to capture the level of permission required by each user group to use the product, such as read, write, and full control access. These use cases will also become the bases for the “user acceptance testing” (UAT) during the testing phase of the project.

Demo

Provide a demo of the product/solution to help your customer better understand and how they can fully benefit from using full breadth of features provided by that product or solution. This will also help with establishing use cases.

Approach and Timeline

Provide an estimated schedule for each of the phases and tasks within those phases. Add some contingency to the overall schedule. This will be the bases of your work breakdown structure (WBS) that you will be creating during the planning phase. Due to circumstances beyond your control, schedule can be impacted so make sure that your customer is aware of such risks that could impact the schedule. Clearly define what your team and your customer activities and responsibilities are going to be. Create a RACI chart to outline roles and responsibilities.

Deliverables

Provide a list of deliverables that you will be creating such as Design and Build Guide documentation. Also make sure to confirm what the acceptance criteria and process is going to be.

Assembling team and defining roles and responsibilities

Assemble a team and define their roles and responsibilities. Also identify and confirm customer’s SMEs, and stakeholders that your team will be working with. Create and confirm the reporting structure as to who is going to be directly reporting to whom. Create an org chart diagram to visualize the team structure. Depending on your project, you may also need to include an executive steering committee.

Plan

Pick a time when all or most of the team members are available to kickoff the planning phase. Start by jotting down the main aspects of the planning phase on the whiteboard or if you prefer on your laptop that is projecting on the screen. If your meeting is virtual than use the virtual whiteboard, Microsoft Word, OneNote or whichever application that you prefer to present notes.

The main outcome from the planning exercise should be the first draft of the Plan and Design deliverables and it should cover the following aspects:

• Communication Plan
• Work Breakdown Structure
• Milestones
• Functional Specifications

This will be a gradual process and can span several days.

Communication Plan

The communication plan should consist of communication activities for both the project team and the end users.

The Communication plan for the project team should consist of communicating on activities such as collaboration, risk and issue management, document management, and status reporting.

The Communication plan for end users consists of communicating information on the delivery of the solution, impact, training, orientation, downtime, cutover or delivery date, and support.

Team Collaboration Site

If you have access to SharePoint or similar product, create a project team site for managing and collaborating on the project. This site will help with bringing all the status, communication, and artifacts relevant to the project into one place. On this project site, create list of contacts for all the team members.

Project Issues and Risk Management

Create lists for managing issues and risks, such as project creeps and change requests. You can easily create these lists by using the out of the box list templates available in SharePoint. Each issue and risk should be assigned an owner, status, priority, impact, description, category, and a due date.

Technical Issue Management

Create a separate list for managing technical issues such as bugs, installation and deployment errors.

Change Log

As you are deploying the solution, it goes through many changes until you have the final product. Create a Change Log list to keep track of those changes.

Shared Document Library

Store and share the documentation that you will be creating from a shared document library.

Meetings

Setup weekly status meetings with your project manager and customer. During the initial stages of the project, setup daily 15 minutes standup meeting. These standup meetings will be your opportunity to bond with the team and root out the factors that are common during the “forming” and “storming” phases of the project.

Tip: Setup weekly status meeting with your own project manager before your weekly status meeting with the customer. This will allow you to discuss any issues and be in-sync with your project manager before your meeting with the customer.

Status reports

Provide weekly status report to your project manager and customer. Make sure to inform your project manager and customer of any risks and issues immediately, and document them in the status report.

Change Management

Notify your project manager immediately if the scope of the project could potentially change. Any change to the project will impact both the schedule and cost. Make sure that your customer understands these impacts.

Governance

Identify if the customer has an existing governance model that they would like to use to support and operate the solution. Identify and confirm their standard policy and procedures and assist with establishing Service Level Agreement.

Functional Specs

After you are done with planning the logistics of the project, next you will be working with the customer on the following specification of the solution. This a generic list which I am only providing as an example.

• Usage Scenarios
• User Requirements
• Business Requirements
• Operations Requirements
• System Requirements
• Conceptual Design
• Physical Design

Work Breakdown Structure

Work Breakdown Structure is your project plan for outlining each phase and tasks of your project. I like to use Microsoft Project application to create project plans but you can also simply use a spreadsheet. Break your project down into five distinct phases and provide detailed tasks for each phase and establish timeline and milestones. Again, the following is only a generic list which I am providing as an example:

Build

The Build phase starts with the first iteration of the installation and configuration of a product or development of your solution. You go through multiple iterations until your product or solution is ready for deployment in the production environment.

Test

In the test phase, you will test your product/solution for defects and configurations issues. You will need to do this for each feature that you are developing or a product feature that you deploying and configuring.

After initial rounds of testing and bug fixes, you will then put your product for pilot test, where the rubber meets the road, to get feedback from the actual end users. Identify the users who are willing to spend time and provide good concrete feedback that you can use to refine and optimize your product/solution. These users will test the product for normal use case scenarios. You will then take user feedback to further test the product and fix any addition issues and bugs.

After pilot, you will put your product for user acceptance testing (UAT) which is the last stage of your testing process before you are ready to deploy your product/solution into the production environment. You will again collect feedback from the end users to further optimize and refine your product/solution.

Deploy

Deploy is the final phase of the project. In this phase, you deploy your product in the production environment, complete any remaining knowledge transfer and then hand over the product to the operations team. Your operation team will run and support the product based on the governance and service level agreement (SLA).

High level view of the methodology

Here is a quick snapshot of the entire process, including each phase, tasks, and activities.

When it comes to storage, there are many layers to it, such as:

• Primary storage for your active applications
• Archival storage and less frequently accessed data
• Backup that involve tapes
• Disaster recovery site where the data is replicated
• Data that is shipped to an offsite facility for long term backup purposes

As you can see, how all of these many layers of storage can increase data management complexity and cost.

There is a better solution – Hybrid Cloud Storage. It is a cloud hybrid concept which helps you optimize and prioritize your storage needs based on your active and in-active, less frequently accessed data. To simplify the concept, lets compare the traditional on premises storage with the Hybrid Cloud Storage model.

Traditional Storage

Your servers are using SAN for storage –> snapshots are created –>data gets backed up to VTL (virtual tape library) or NAS –> data then gets backed up to tape –> data is then shipped to offsite facility for long term storage.

Hybrid Cloud Storage

Your servers are using SAN for storage –> snapshots are created –> data is copied to the nearest cloud storage region –> another copy is replicated to a remote cloud storage for disaster recovery.

As you can see, in the hybrid cloud storage scenario, you don’t really need to take backups of the data to VTL and tapes and also don’t need to send it to the offsite facility for long term storage. Your data is getting backed up to the cloud at the same time your active, frequently access data is locally available in the StorSimple storage device for quick access. This results in the following benefits:

• Less equipment and hardware thus reduced TCO
• A single solution for consolidating your primary, backup, and disaster recovery storage

StorSimple Storage Device

The Azure Hybrid Cloud Storage utilizes the StorSimple SAN device to store the active data locally and to store the in-active, less frequently data in the Cloud. It is an Enterprise class iSCSI SAN device and provides the following features:

• Full redundancy that allows for hot swaps and non-disruptive upgrades
• Data auto-tiering utilizing SSD for the most active and SAS for the less active data, and Cloud for archiving
• Automatic data snapshots to the Cloud and copying the same data to another the remote Cloud storage for disaster recovery
• Quick data restore using the snapshots
• Thin provisioning
• Data de-duplication
• Data encryption

To learn more about Hybrid Cloud Storage, click here.

To provide both high availability and disaster recovery, You will at least need three nodes, two at a local site to provide high availability, and the third node at a remote site to provide disaster recovery. For local replication, you will need to use the sync mode and for remote DR site you will need to use the async mode. Two simplify the concept:

Sync = local high availability

Async = remote, disaster recovery

AlwaysOn consists of an “Availability Group” which is basically a container to which you add your SQL instance nodes and the desired databases to replicate to the other nodes in the cluster. Each node in the cluster is known as the “Replica”. You have a “primary replica” from which you replicate database to your “secondary replicas”. Primary replica is “Read-Write” whereas the secondary replica is “Read-Only”. You can setup your replication for Automatic or Manual failover. For sync mode, Automatic and Manual failover is supported and for the async mode only manual failover is supported.

Availability group uses the concept of “Listener” to direct client traffic to the available replica. Listener is basically a virtual name that consists of the IP addresses from each node in the cluster. You create an A record in DNS and map each IP address of your nodes to it.

It was recently announced that most of the service applications database are now supported for async mode, that is you can now replicate your service application databases such as managed metadata, and user profile service databases, over the WAN. Before, you had to use log shipping to replicate these databases which resulted in a more complex replication process. Configuration and Central Administration databases however are still not supported for remote site replication as they are farm specific. Following diagram provides a high level view of the AlwaysOn Availability Group:

You can use the following sample table to plan for your AlwaysOn availability group:

At a high level, following steps are required to setup AlwaysOn. For more detailed steps, click here.

1. Setup Windows Failover Clustering and make each SQL instance from local and remote disaster recovery site member of that cluster.
2. Enable AlwaysOn on each node.
3. Create the Availability Group.
4. Add Replicas to each Availability Group.
5. Verify Endpoints, open inbound port 5022 (default) on each SQL node
6. Verify backup preferences.
7. Create Availability Group Listener, provide port 1433(default), and an IP address from each of the local and remote DR subnets. These IPs should map to an A record for the Listener.

Note: Make sure that the same farm service accounts to each of the Prod and DR SQL Instances login and server roles have been added, otherwise the AlwaysOn synchronization process will fail.

8. Create a shared folder for data synchronization for the Availability Group.
9. After availability group(s) has been created, enable the MultiSubnetFailover databases property to true on each database. To learn more, click here.
10. Install SharePoint .
11. After SharePoint has been installed and configured, change Recovery Model for each of its databases to Full. This is a requirement before a database can be added to the Availability Group.
12. Backup all of the databases. This is a requirement before a database can be added to the Availability Group.
13. Add databases to their Availability Group(s)
14. Test failover and failback on each Availability Group.

To learn more on how to setup AlwaysOn Availability Groups, click here.

It goes without saying that you can’t deploy a solution without documenting it first. When I am creating a design and deployment document, I like to start with the solution summary, followed by the scope, prerequisites, and, and then by providing a quick overview of the step by step process on how I am going to deploy the solution. I would also supplement the overview with a picture; you know they say “picture worth a thousand words”. Here is a quick overview of the step by step planning and deployment process which I will be covering in much more detail later in this article.

Sizing the farm

Start your design by gathering some basic workload characteristics such as total number of users, average daily concurrent users, and total number of requests per day. Also try to estimate the workload distribution across different SharePoint capabilities such as Search crawl, Team site activities, and Office Web Apps. Estimate the database size, number of web apps, site collections, number of lists, libraries, and number of items.

In my experience, there is no magic formula for sizing the farm which will get you the exact number of servers and resources for your farm. It is basically more of an art than a science – it is about coming up with an estimate that you can compare with the reference architecture, deploying it and then testing it to make sure that it meets your performance targets. If for some reason you are unable to estimate your workload characteristics, at least try to estimate the total number of users and storage, and use the guidance and reference architectures provided here to design your startup farm. To learn more click here.

The way I like to size the farm is to start with the recommended number of servers for the estimated workload. For example, for 10,000 users or so, I would start with at least four SharePoint servers, and two SQL Servers. This configuration provides a good, well balanced startup topology consisting of the load balancing and redundancy for front end, batch processing, and application servers. The key steps then are to pilot the farm as the following:

1. Populate the storage with a sample dataset
2. Stress the system with synthetic load
3. Run tests, analyze results, and then optimizing the farm by tuning or adding more servers if performance targets are not met.

Use the guidance provided at this link to learn more about performance testing for SharePoint.

Once you have deployed your SharePoint farm in production, continue analyzing performance against the targeted workload on an on-going bases, and make adjustments to the hardware, settings, and the topology based on those analysis. I would scale out (add more servers), rather than scale up (add CPU, RAM) as that provides better performance. 

I would load balance web traffic using the hardware load balancer to ensure even distribution across all the web servers, and an optimal experience for the end users. To provide high availability for SQL server, I would use the AlwaysOn technology which automatically failover databases to the active SQL nodes in case one of the nodes fail. AlwaysOn is also much easier to deploy and maintain and now supports most, if not all, of the SharePoint databases. To learn more about AlwaysOn, click here.

If you are using SAN (storage area network), make sure to estimate the amount of traffic (IOPS) and allocate enough disks to satisfy that traffic. For DAS (Direct Attached Storage) on the other hand, separate out the Data, Logs, and TempDB on separate drives and make sure that each disk is providing the required IOPS. Here is a link to learn more on about how to estimate IOPS.

As a recommended practice, you should maintain two times as much free space as your RAM in your SharePoint servers. It is better to oversize and overinvest in hardware as the cost is almost always far less than the cumulative expenses related to issues caused by under sizing.

You should maintain less than 1 millisecond of network latency and 1 Gbps of bandwidth between the farm servers.

Software and Hardware requirements and boundaries

System requirements are pretty simple to understand but make sure to understand the software boundaries as well. Click on the following links to learn more:

Software Boundaries and SharePoint Limits

System requirements

Traditional Topologies for SharePoint 2013

With office web apps server, you can allow users to view and edit office documents by just using their browser. These users don’t have to have full version of the office installed on their computer. This is specially useful for scenarios such as kiosk, factory, or those users who want to quickly view and make edits to perform their tasks. Users can easily share and collaborate among themselves on the same files by just using their browser. 

Office Web Apps are designed for mobile devices with touch screen so mobile users can easily use them while they are sitting at a coffee shop or on the road. “View only” capability is available for free so users without Office license can easily view office files by just using their browser.

You can also view file attachments within Outlook Web Access (OWA) and view PowerPoint broadcasted via the Lync communication platform.

Office Web Apps is not just designed for SharePoint but also for viewing files hosted in your network shared folders or other websites using a feature called “Online Viewers”.

With the release of SharePoint 2013, Office Web Apps server is now a Standalone product, and must be installed on a separate server(s). This eliminates that complexity in terms of infrastructure and sizing for the SharePoint Farm, and also with applying updates. This allows for users to continue to use their office files which are hosted outside of SharePoint while the SharePoint servers are being updated.  To learn more, click here.

Planning

In terms of planning, you will need to plan for browser compatibility, authentication, licensing and infrastructure.

Browser Support

Following browsers are supported:

• Internet Explorer 11, Internet Explorer 10, Internet Explorer 9, Internet Explorer 8
• Google Chrome (Latest)
• Mozilla Firefox (Latest)
• Apple Safari (Latest)

Infrastructure

You will need to install Office Web Apps on a separate dedicated server(s). To provide high availability and load balancing, deploy on multiple servers. In terms of sizing the Office Web Apps farm, same requirements as the SharePoint servers apply. Click here to learn more.

It is highly recommended to use HTTPS to secure Office Web Apps traffic as it communicates with the SharePoint, Lync, and Exchange servers. This will require you to use SSL certificates on all your servers. For test environment, HTTP should be sufficient.

Software Requirements

Following operating systems are supported:

• Windows Server 2008 R2 Service Pack 1 (SP1)
• Windows Server 2012

Authentication requirements

As Web Apps are created with Claims as the default mode in SharePoint 2013, this goes without saying that Office Web Apps will only work with Claims-based authentication. You will not be able to view or edit office web apps with Classic mode authentication. If you are migrating your Classic mode authentication Web Apps from your older SharePoint environment, those web apps will need to be converted into Claims-based authentication. 

Licensing requirements

By default, you can view office web apps for free. To be able to edit, each user will require a license. You enable editing during your office web apps farm deployment. Enterprise customers who are licensed for Office 2013 through a Volume Licensing program can enable Office Web Apps editing for SharePoint 2013 on-premises however, do make sure to work with your license SME to make sure that you meet the requirements. Click here to learn more.

By default Office files are opened in the browser after SharePoint 2013 has been configured to use Office Web Apps Server.

Deployment

At first, deploying Office Web Apps server might seem like a daunting task but believe me it is not. Now that it is a standalone product, it is now much easier to deploy it. This section covers the steps for deploying the Office Web Apps server in a test environment. For production, make sure to follow the recommended planning and deployment guidance. Click here to learn more.  Here is a high level view of the deployment process:

Install Prerequisites

If you are planning to install Office Web Apps server on Windows Server 2008 R2 server, download and install the following software first:

· Windows Server 2008 R2 Service Pack 1

· .NET Framework 4.5

· Windows PowerShell 3.0

· Platform update for Windows 7 SP1 and Windows Server 2008 R2 SP1 (KB2670838)

After the above software has been installed, run the following PowerShell commands to install the required server features:

Import-Module ServerManager

For Server 2008 R2

Add-WindowsFeature Web-Server,Web-WebServer,Web-Common-Http,Web-Static-Content,Web-App-Dev,Web-Asp-Net,Web-Net-Ext,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Includes,Web-Security,Web-Windows-Auth,Web-Filtering,Web-Stat-Compression,Web-Dyn-Compression,Web-Mgmt-Console,Ink-Handwriting,IH-Ink-Support

For Server 2012

Add-WindowsFeature Web-Server,Web-Mgmt-Tools,Web-Mgmt-Console,Web-WebServer,Web-Common-Http,Web-Default-Doc,Web-Static-Content,Web-Performance,Web-Stat-Compression,Web-Dyn-Compression,Web-Security,Web-Filtering,Web-Windows-Auth,Web-App-Dev,Web-Net-Ext45,Web-Asp-Net45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Includes,InkandHandwritingServices,NET-Framework-Features,NET-Framework-Core

Install Office Web Apps server

1. Download the setup files from the Microsoft Download Center.
2. Run the Setup file, Accept License, choose a file location, and Click Close to complete the install

3. Download and install the latest service pack and updates from Here
4. Install any additional Language pack if required from Microsoft Download Center.

Creating Office Web App server farm

Since this setup is for a test environment, we will only be deploying a single server using HTTP.

Open PowerShell run the following command:

New-OfficeWebAppsFarm -InternalURL "https://servername" -AllowHttp -EditingEnabled

Press Enter for the EditingEnabled if prompted.

Verify that the farm created successfully by browsing the https://servername/hosting/discovery site. If all is well, you should see the Web Application Open Platform Interface Protocol (WOPI)-discovery XML file in your web browser.

Create the binding between SharePoint 2013 and Office Web Apps Server

Note: Perform the following steps on the SharePoint Server.

Now you will need to move to your SharePoint server and run the following commands to create binding between the SharePoint and Office Web Apps server so users are able to open and edit Office files in their browser. Run the following command to create the binding: 

New-SPWOPIBinding -ServerName <WacServerName> -AllowHTTP

After the command is completed successfully, you will see a list of bindings displayed.

Change the WOPIZone to HTTP

By default SharePoint uses internet-https zone, which will not work in our test environment so we will need to change it to http. Confirm the current WOPIZone by running the following command:

Get-SPWOPIZone

If it show https, run the following command to change it to http:

Set-SPWOPIZone –zone “internal-http”

Run the following command to check again to make sure it was changed to http:

Get-WOPIZone

Set AllowOAuthOverHTTP to True

To use Office Web Apps with SharePoint 2013 over HTTP in a test environment, you will need to set AllowOAuthOverHTTP to True. Otherwise Office Web Apps won’t work in our test environment. You can check the current status by running the following command:

(Get-SPSecurityTokenServiceConfig).AllowOAuthOverHttp

If it shows False, run the following commands to change it to True:

$config = (Get-SPSecurityTokenServiceConfig)

$config.AllowOAuthOverHttp = $true

$config.Update()

Run the following command to verify again:

(Get-SPSecurityTokenServiceConfig).AllowOAuthOverHttp 

Verify Office Web Apps functionality

Verify that Office Web Apps is working by browsing to a library in your SharePoint site that is hosting the office documents and open them in your browser.

Note: If the users are only able to view the files and not able to edit, you will need to map your users group to the OfficeWebAppsEdit license. Click here to learn more.

Following table provides detailed information on each role. To learn more, click here.

In order to create the cert, you will need to use the Makecert tool, available in the Windows 7 and Windows 8 SDK. You can download the SDK from the Microsoft Download center. After you have downloaded and installed the SDK, use the following steps to create the cert:

1. Open Command prompt, go to C:\Program Files (x86)\Windows Kits\8.0\bin\x86 and run the following command. Replace the date with the current date and domain name with your own domain name. 

   makecert -r -pe -e <09/14/2014> -eku 1.3.6.1.5.5.7.3.1 -ss My -n CN=<*.domain.local> -sky exchange -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12 -len 2048 

Above command creates a certificate that is good for a year, and will store it in the CurrentUser certificate store. The key piece in that above command that makes the cert wildcard is the CN=*.domain.local parameter. The next steps are to export and import the cert into your SharePoint servers: 

1. Export the cert to a file path so you can then import it into your SharePoint Server(s). Open MMC and add the "Certificate" snap-in, browse to Personal/Certificates store, right click the cert, click All Tasks > Export. Follow the prompt to complete the wizard.
2. After the file has been exported, copy and import it into your SP Server(s) using IIS manager, and then bind it to the web app. Since these are going to be your dev/test servers, you can optionally save the cert to the local trust authority to avoid the "not trusted authority…" warning messages.