Twitter

LinkedIn 

For SharePoint farm topology, installation and provisioning of all required service applications please see blog one, two and three of this series.

1. Farm Topology and Prerequisites
2. SharePoint Installation, Configuration and Basic Service Applications Deployment
3. User Profile, Search and Distributed Cache Service Applications Deployment
4. Office Web Apps Server farm Implementation and Configuration

OWA Overview

Office Web Apps is the online companion to Office Word, Excel, PowerPoint, and OneNote applications. It enables users to view and edit documents through a browser by providing a representation of an Office document. A SharePoint 2013 farm can be configured to use the Office Web Apps Server farm and provide Office Web Apps functionality to users who create or open Office files from SharePoint libraries.

Office Web Apps (OWA) Server supports two primary installation scenarios: a single-server Office Web Apps Server farm, and a multi-server load-balanced Office Web Apps Server farm. For this blog series, continuing with the desire for high availability, OWA will be implemented in a two-server load-balanced mode, utilising the same hardware load balancing solution as the SharePoint web front end servers. The OWA traffic will be encrypted and the solution will be implemented over HTTPS. The OWA Server will not run any other server applications including Exchange Server, SharePoint Server, Lync Server, and SQL Server. Also no other services or roles that depend on the Web Server (IIS) role on port 80, 443, or 809 should be installed on the OWA servers, as OWA Server periodically removes web applications For this blog series, 

Prerequisites and Assumptions

1. Ensure that ports 443 (HTTPS traffic), 80 (HTTP traffic) and 809 (private traffic between the servers that run Office Web Apps Server in the farm) are not blocked by firewalls on the two Office Web Apps Servers.
2. All Office Web App servers have an additional drive to host Data and Log flies.
3. Copy the OWAScripts.zip to the E:\Install\scripts directory.
4. Download and mount the OWA installation media ISO to drive D:\ on both servers
5. The two OWA servers have Windows Server 2012 installed and are called OWA1 and OWA2 respectively

Import-Module ServerManager
Add-WindowsFeature Web-Server,Web-Mgmt-Tools,Web-Mgmt-Console,Web-WebServer,Web-Common-Http,Web-Default-Doc,Web-Static-Content,Web-Performance,Web-Stat-Compression,Web-Dyn-Compression,Web-Security,Web-Filtering,Web-Windows-Auth,Web-App-Dev,Web-Net-Ext45,Web-Asp-Net45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Includes,InkandHandwritingServices

Set-ExecutionPolicy Unrestricted –force 
E:\Install\scripts\WAC_ConfigureOWAFarm.ps1 -configLocation E:\Install\scripts\WAC_ConfigureOWAFarm.xml

Set-ExecutionPolicy Unrestricted –force 
Import-Module OfficeWebApps 
New-OfficeWebAppsMachine –MachineToJoin OWA1.<FQDN> -Confirm:$false

WACServer = "officeweb.<FQDN>"
$WOPIZone = "internal-https" 
 
#Configure SP Farm to OWA Farm
New-SPWOPIBinding -ServerName $WACServer 
 
#Set the WOPI Zone
Set-SPWOPIZone –zone $WOPIZone
Get-SPWopiZone 
 
$config = (Get-SPSecurityTokenServiceConfig)
$config.AllowOAuthOverHttp = $true
$config.Update()

OWAScripts.zip

• User Profile
• Search
• Distributed Cache

For SharePoint farm topology, installation and provisioning of the basic service applications please see blog one and two of this series.

1. Farm Topology and Prerequisites
2. SharePoint Installation, Configuration and Basic Service Applications Deployment
3. User Profile, Search and Distributed Cache Service Applications Deployment
4. Office Web Apps Server farm Implementation and Configuration

Prerequisites and Assumptions

• Please download and extract the attached files onto a local directory on each SharePoint Server. For the purposes of this blog the directory is suggested to be: E:\Scripts\Install.
• From the downloaded files, update the SPUserProfile.xml file with environment specific information such as, SQL server alias name, service account details, database name, service application name, etc. This will be self-explanatory once you open the file.
• Also update the SPSearchService.xml file with environment specific information such as, SQL server alias name, service account name and password, database name, service application name, Index Location, etc.

Configure Distributed Cache

The Distributed Cache service can be deployed in two modes: dedicated mode or collocated mode. In dedicated mode, all services other than the Distributed Cache service are stopped on the application server that runs the Distributed Cache service. In collocated mode, the Distributed Cache service runs together with other services on the application server. Dedicated mode is the recommended mode in which to deploy the Distributed Cache service if the total number of users exceed 10,000.

When more than one server is used for Distributed Cache, all the servers must have the same cache size configured.

Please see Planning for Distributed Cache service in SharePoint Server 2013 for more information.

Provision Distributed Cache Service in dedicated mode:

Launch SharePoint Management Shell as administrator and execute the following PowerShell script on all Application, Search and Web Application Servers (all servers apart from Distributed Cache servers SPDCache01 and SPDCache02) to remove them from the Distributed Cache cluster and stop the services:

 $instanceName ="SPDistributedCacheService Name=AppFabricCachingService" 
 $serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername} 
 $serviceInstance.Unprovision() 
 Stop-SPDistributedCacheServiceInstance -Graceful 
 Remove-SPDistributedCacheServiceInstance 

Change the Memory Allocation of the Distributed Cache Service

When SharePoint Server 2013 is installed, it assigns the Distributed Cache service 10 percent of the total memory on the server. It is recommended to allocate sufficient amount of memory on each Distributed Cache server. Please see Change the memory allocation of the Distributed Cache service to calculate how much memory can be assigned to the Distributed Cache service. It is important to note that the allocated memory on each server should not exceed 16 GB.

Use the following procedure to update the memory allocation accordingly:

Launch SharePoint Management Shell as Administrator on SPDCache01 and SPDCache02 and run the following scripts:

Stop the Distributed Cache service:

 $instanceName ="SPDistributedCacheService Name=AppFabricCachingService" 
 $serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername} 
 $serviceInstance.Unprovision() 

Reconfigure the cache size of the Distributed Cache service:

 Update-SPDistributedCacheSize -CacheSizeInMB 7168 

Restart the Distributed Cache service:

 $instanceName ="SPDistributedCacheService Name=AppFabricCachingService" 
 $serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername} 
 $serviceInstance.Provision() 

Change Service Account

When the server farm is first configured, the server farm account is set as the service account of the AppFabric Caching service. The Distributed Cache service depends on the AppFabric Caching service. For security purposes it is advised to change the service account from the farm account. To change the service account of the AppFabric Caching service to a managed account, set the Managed account as the service account on the AppFabric Caching service.

At the Windows PowerShell command prompt, run the following command:

 $farm = Get-SPFarm 
 $cacheService = $farm.Services | where {$_.Name -eq "AppFabricCachingService"} 
 $accnt = Get-SPManagedAccount -Identity DomainName\SVC_SPFabric 
 $cacheService.ProcessIdentity.CurrentIdentityType = "SpecificUser" 
 $cacheService.ProcessIdentity.ManagedAccount = $accnt 
 $cacheService.ProcessIdentity.Update() 

Where the ‘DomainName’ is the NetBIOS domain name.

Configure User Profile Services

Login to SPAPP01 (Application Server hosting the Central Administration site) and launch SharePoint Management Shell as administrator. Execute the following command to start and configure User Profile Service:

 E:\Scripts\Install\SPUserProfile.ps1 -configLocation E:\Scripts\Install\SPUserProfile.xml 

When prompted provide the My Site Application Pool account variable “$MySiteAppPool”

Start User Profile Synchronisation

1. Ensure that the Farm Admin account (SVC_SPFarm) is a member of local administrators group on SPAPP01. 
   Please note this is only required while configuring User Profile synchronisation Settings. However, when a backup of the User Profile application is initiated, the synchronization service provisions the User Profile application again. During the course of provisioning the User Profile application, the farm account must stop and start the synchronization service. To do this, the farm account must be a member of the Administrators group on the computer that is running the synchronization service. Due to this you may decide that the farm account will remain a member of the Administrators group on SPAPP01.
2. Ensure that the Farm account is able to logon locally on both SPAPP01 and SPAPP02 (Please see https://technet.microsoft.com/en-us/library/ff182925(v=office.15).aspx#permission)
3. Reboot SPAPP01 after granting above permissions
4. Starting the user profile synchronisation service introduce the same challenge as we had in SharePoint 2010. This needs to be done under the farm account. There are a number of options:
1. Login to SPAPP01 using the Farm Admin account (SVC_SPFarm). Navigate to Central Administration Site and click on Services on Server and start User Profile Synchronization Service.
2. Launch the SharePoint Management Shell as the SVC_SPFarm account and use New-SPProfileServiceApplication Cmdlet.
3. Use the third option described in: Avoiding the Default Schema issue when creating the User Profile Service Application using Windows PowerShell by Spencer Harbar.
4. Perform an IIS reset on SPAPP01 server

The following activities are recommended however are out of scope of this blog post:

1. Add a new synchronisation connection
2. Define exclusion filters for a synchronisation connection
3. Map user profile properties
4. Start profile synchronization and configure Import Schedules

Configure Search Service

Login to SPAPP01 (Application Server hosting the Central Administration site) and launch SharePoint Management Shell as administrator. Execute the following command to start and Search Services and configure the search topology:

 E:\Scripts\Install\SPSearchService.ps1 -configLocation E:\Scripts\Install\SPSearchService.xml 

Please note the following Search post configuration steps are required to be carried out; however, are out of scope of this blog post:

1. Provide the location of the global Search Centre
2. Setup People Search
3. Configure result sources
4. Configure Search Result Exclusions
5. Configure Search Alert Settings
6. Configure Search Crawling Schedule

Lookout for the next blog post, where we implement Office Web Application Components and its integration with SharePoint.

SPServices.zip

Please Note: This technique is only supported in browsers that support the FileReader object of the File API. Currently this includes, but is not limited to, Internet Explorer version 10, Firefox version 3.6 and Chrome version 7 (information about the File API, including the FileReader object can be found here).

There are two specific areas that I will cover in this article; how to extract the file from the user’s machine and how to post this file off to the correct document library (including how to upload into a folder).Please note that this article does not cover the authentication aspect of uploading files over REST. If you are performing the upload from a SharePoint hosted app then this is taken care of by the app model. If you are performing an upload from another source then you will need to include an OAuth token (either as an App or as a user) with the request by setting an additional header (“authorization”).

Getting the file

Traditionally, uploading files is achieved by using a form that POSTs off to a web server which then handles the upload. This means that developers do not have to directly reach into the client’s file system to obtain the file. However this also means that you need to have access to a web server where you can handle the upload. In some situations, SharePoint hosted apps for example, the developer does not have access to this infrastructure and the upload must be handled inside the client’s web browser.

Advancements in the JavaScript File API allow developers to access the local file system using a standard input tag. The magical part is that you can access the file contents without having to perform a post back. The JavaScript code below shows how you can read the contents of a file (assuming that you have an input tag of type file inside the HTML document with an ID of fileSelectorInput).

The example first uses jQuery to obtain a reference to the input tag on the page. A new FileReader object is then created and the readAsArrayBuffer function is called on the FileReader passing in a reference to the first file in the list.

The onload function contains the contents of the file (inside result.target.result) however this is in the format of an ArrayBuffer. To upload the file to SharePoint this must be converted into a string which can then be set as the body of a POST operation. This can be done by iterating through the array and building up a string using the string.fromCharCode() function. Details of this encoding method can be found in this MSDN article.

Performing the upload

Performing the actual upload is relatively simple. In the example code at the end of this article I will be using the cross domain library provided with SharePoint 2013 to perform uploads from within a SharePoint Hosted App. This library allows for cross domain calls from pages in the App web to the parent site. I will also show the endpoints required to upload files without the cross domain library.

Uploading to the root folder of a document library (without the cross domain library)

This is probably the simplest endpoint that I will cover. TargetSite needs to be replaced with the URL of the SharePoint site (e.g. https://sharepoint.com/sites/site). Additionally, the final two parameters (TargetLibrary and TargetFileName) need to be replaced with the appropriate values. The URL is:

TargetSite/_api/web/lists/getByTitle(@TargetLibrary)/RootFolder/Files/add(url=@TargetFileName,overwrite='true')?@TargetLibrary='LibraryName'&@TargetFileName='Target File Name'

Uploading to the root folder of a document library (with the cross domain library)

If you would like to perform the upload from a SharePoint hosted App you will need to use the cross domain library to overcome cross domain issues. This can be done by modifying the start of the URL and adding an additional parameter (TargetSite) which needs to be replaced with the URL of you target site (e.g. https://sharepoint.com/sites/site). You will also need to replace the AppWebUrl section with the URL of your AppWeb. All other parameters should stay the same. The URL is:

AppWebUrl/_api/SP.AppContextSite(@TargetSite)/web/lists/getByTitle(@TargetLibrary)/RootFolder/Files/add(url=@TargetFileName,overwrite='true')?@TargetSite=’Target site URL’&@TargetLibrary=’LibraryName’&@TargetFileName=’Target File Name’ 

Uploading to a folder within a document (without the cross domain library)

Uploading to a folder in a document library is fairly similar to uploading to the root folder. Again I will show two examples (one with and one without the cross domain library). The simplified version (without the cross domain library) can be performed using the URL below. You will need to replace the various components in the same way as you would for uploading to the root folder (mentioned previously). Also note the addition of a TargetFolder parameter and the change to URL to include the folder. The URL is:

TargetSite/_api/web/lists/getByTitle(@TargetLibrary)/RootFolder/folders(@TargetFolderName)/files/add(url=@TargetFileName,overwrite='true')?@TargetLibrary=’LibraryName’&@TargetFolder=’Target Folder Name’&@TargetFileName=’Target File Name’

Uploading to a folder within a document (with the cross domain library)

You can also use the cross domain library to upload files to a folder within a document library. The URL needs to be modified slightly (in the same way as before) to redirect the message to the AppWeb and to include the TargetSite parameter. The URL is:

AppWebUrl/_api/SP.AppContextSite(@TargetSite)/web/lists/getByTitle(@TargetLibrary)/RootFolder/folders(@TargetFolderName)/files/add(url=@TargetFileName,overwrite='true')?@TargetSite='Target site url’&@TargetLibrary=’LibraryName’&@TargetFolder=’Target Folder Name’&@TargetFileName=’Target File Name’ 

Required Headers/Options

In addition to ensuring you use the correct endpoint and encoding, it is important to ensure that the correct headers/options are set on the request. The actual request should be sent as a POST operation and the content type, headers, content length and binary string request body options must be set.

Accept: The Accept option must be set equal to “application/json; odata=verbose” inside the headers of the request.

Digest Value: The digest value should be set equal to the digest value obtained from SharePoint. This should also be set in the headers section of the request. The digest value can be read from SharePoint pages or be requested from the app web. More information about digest values can be found here under the heading “Writing data by using the REST interface”.

Content Type: The content type of “application/json; odata=verbose” should be set on the request

Body: The body should be set equal to the file contents encoded as per the description in the MSDN article here.

BinaryStringRequestBody: The option “binaryStringRequestBody” should be set equal to true. This informs SharePoint that the body of the message contains binary data.

Content Length: The content length of the request should be set equal to the length of the body. Note: if you are using the request executor then the content length is optional, if you are uploading directly (without the request executor) then you must set the content length otherwise SharePoint will reject the request.

Conclusion

In summary, performing uploads via the REST API using only client side technologies is not only possible, but is actually quite simple. We have looked at how to extract the file from the user’s machine, generate the correct endpoint and set the correct headers/options on the request. When you put all of this together you can upload binary files to SharePoint from the end users web browser without having to process the files server side. This means that in situations such as SharePoint hosted Apps where developers do not have access to server side code uploads can still be performed. Furthermore in situations where you do have access to server side code you can still use the endpoints mentioned to upload files using the REST API.

The source code below contains a working example (in the form of a SharePoint hosted App).

Source Code

The following downloadable source code contains a working example of uploading files over REST. The sample is contained within a SharePoint hosted app and you will need to have Visual Studio 2012 installed to be able to run it. To run the example you will need to configure the SiteUrl of the project to point to a SharePoint 2013 developer site. To do this click on the project in solution explorer and open up the properties task pane. Enter the URL of your site into the SiteUrl box and then press F5.

Upload Over REST Example.zip

There’s a plenty of documentation on MSDN and TechNet to get you started with building your first application and how to consume the different capabilities of the platform.

But one thing I struggled to find is when and why you would use a SharePoint App rather than using the object model?

This blog will walk you through that specific question and how to answer it.

Before we start there’s no rights or wrongs here, it all depends on what the customer needs and the best way to tackle the requirements

The Challenges:

1.      Hosting

Solutions

The first factor is where is my code being hosted?

If you are using the object model most of your code will be within the SharePoint host environment:

• Managed solutions also known as farm solutions will be hosted within the main SharePoint worker process (w3wp.exe).
• Sandbox solutions will run within the SharePoint sandbox worker process (SPUCWorkerProcess.exe).

If you are using Apps:

• App code never runs within the SharePoint host environment.

So what difference does it make?

1. If the custom code is not running within the SharePoint farm this will eliminate any kind of custom operation being done within the farm and hence increase the farm stability.
2. It’s easier to upgrade if your code is platform independent, we have seen many cases where the customer will hold all the SharePoint migration until he makes sure that all the custom solution will be working fine on the new version.
3. The deployment, most farm solutions require a Farm Administrator to do the deployment and mostly restarts the IIS on all the WFEs, which causes a disruption in the service. Although the deployment of Sandbox solutions is a lot easier and straight forward but there’s no indication on whether or not it’s safe to activate this sandbox solution without actually activating it and giving the code inside aces to all the site collection’s content.
4. Office 365 Ready, you can always use Sandbox solutions on Off365, but mostly you would develop two versions from your code one that works with On-Premise farms and another one for Office 365. Now the whole point of an App is you don’t worry where it’s hosted or deployed as it should behave the same on both On-Premise and Cloud environments.

Apps

Apps are very different from solutions in that they run 100% outside of the SharePoint server, and their custom code executes either within the context of the client browser or on other servers that are not running SharePoint such as Web servers in the cloud.

There are three high-level types of SharePoint apps:

• SharePoint-Hosted: These types of apps are manifested within SharePoint generally without external dependencies. Any custom business logic implemented with custom code must run within the context of the browser client. Some examples of these types of Apps could be an expense calculator or holiday request. Each of these Apps may implement its own business logic using JavaScript, but they can also access and leverage SharePoint lists and libraries.
• Provider-Hosted: These types of apps may have a SharePoint component but the bulk of the business logic is manifested in some other infrastructure, such as an external Web server or in the cloud. These types of apps are a good option when integrating an existing event registration or help desk ticketing solutions within a SharePoint site.
• Auto-Hosted: These types of apps are similar to the Provider-Hosted Apps in that they have the bulk of their business logic and/or data storage manifested externally. What’s different with an Azure Auto-Hosted app is that the app package contains the website & database in the package. When the app is installed in a site, SharePoint handles the automatic provisioning of the database (as a SQL Azure database) and website (as an Azure hosted website) using an account that has been setup by the SharePoint farm/tenant administrator.

One explicit limitation of the SharePoint App Model is that server side code is explicitly prohibited from residing on the SharePoint farm as part of an App. Any server side code that is utilized in the context of an App must be hosted outside of SharePoint either in the cloud or on-premises.

The Cloud App Model provides a loosely coupled architecture for building Apps in SharePoint 2013. This loosely coupled architecture gives the freedom of choice for developers in the technologies they use to not only host their applications, but also the tools they use to write them. Apps can leverage industry web standards such as HTML, JavaScript, jQuery, JSON, REST, OData and OAuth to provide an integrated user experience.

2.      Scopes

Solutions

• Full trusted solutions can be installed on any scope (Farm, Web App, Site Collection, Sites).
• Sandbox solution  or partially trust solutions is scope is limited to Site Collections

Apps

• Apps has two different scopes :
  • Site Scope: in this case the app is installed and launched within the same scope of the SharePoint site.
  • Tenancy Scope: in this case a special SharePoint site is required called app catalog, once it’s installed users from different sites can choose to connect and consume this App.
• Using the object model whenever we need to scale our solutions either up or out ; We needed to introduce new hardware to the farm that comes with the package of :

3.      Scaling

Solutions

1. Down time considerations
2. Cost
3. High Availability and Disaster recover changes
4. Monitoring

Apps

• Using the App model you can always scale your application without affecting the current SharePoint environment.

Have a look at windows Azure Scalability options

https://www.microsoft.com/en-us/download/details.aspx?id=36837

4.      Integration

Solutions

• Previously our only options to integrate with other applications was rather through BCS (Business Connectivity Service), Web Services or build your own service application, this provided the end users with limited experience integrating with external applications.

Apps

• Using the new App model there are too many options for the integration from Services to actually implementing a fully customized applications against different products SDK.
• All the web skills are reusable in building the Apps, to a very good extent you can write a SharePoint app without needing to know a lot about SharePoint 2013.
• Fully supported templates from visual studio 2012.
• Device free in terms of the device on which the code runs. The possibilities include a server in the SharePoint farm, an external server such as a server in the cloud, a client computer, and a mobile device.

Conclusion

Always develop Apps whenever it’s possible rather than developing a classic Full Trust or Partially trust solution.

Bear in mind that sandbox solutions are still supported but they will be deprecated.

But first understand that the client object models and the REST endpoints that are available in the App model do not duplicate all of the APIs of the server object model that are accessible in a classic SharePoint solution.

For more Information :

Reference documentation

Apps for SharePoint compared with SharePoint solutions

Choose the right API set in SharePoint 2013

https://msdn.microsoft.com/en-us/library/jj164060.aspx#Factors

SharePoint 2013 Development

https://msdn.microsoft.com/en-us/library/jj162979.aspx

1. Farm Topology and Prerequisites
2. SharePoint Installation, Configuration and Basic Service Applications Deployment
3. User Profile, Search and Distributed Cache Service Applications Deployment
4. Office Web Apps Server farm Implementation and Configuration

This blog post will cover:

• Installation of SharePoint 2013
• Farm creation and configuration
• Implementation of:
• Excel Services
• Secure Store
• Usage and Health Data Collection
• Managed Metadata Services
• State Services
• Configuration of services on servers
• Configuration of usage and health data collection
• Configuration of state service
• Configuration of diagnostic logging

Servers in Farm

The following table illustrates the server names and associated server roles for each server:

Installation Prerequisites and Assumptions

• Please download and extract attached files onto local directory. It has been assumed that this directory is local folder on all SharePoint serves created as E:\Scripts\Install
• It is assumed that the SharePoint installation directory is D:\ (the location of the media ISO)
• Update the SilentConfig.xml file with SharePoint product key. Please see Config.xml reference on TechNet for more information.
• Update the Config.xml file with environment specific information such as, SQL server alias name (To improve ease of maintenance, and make it easier to relocate the database if it is required in the future, create DNS aliases that point to the IP address for all instances of SQL Server), farm account details, admin database and configuration database.  
• Update the SPCredentials.xml file with service accounts and passwords information.
• Update the SPServices.xml file with Usage and Health data collection log file location, Usage and Health data collection maximum file size, SQL server name, service application names and related database names.

Install SharePoint (with Internet Connection)

The following steps will only work if each SharePoint server has direct access to the internet. If access is through a proxy, further configuration may be required. Therefore, if internet access is NOT available OR through a proxy, it is recommended to follow the guidelines listed next Install SharePoint (without Internet Connection)

1. Ensure that the SharePoint installation media is mounted and D:\Setup .exe is accessible
2. Microsoft SQL Server 2008 R2 Native Client is installed as part of SharePoint 2013 prerequisites installer, however if you are using SQL Server 2012 to host SharePoint databases install Microsoft SQL Server 2012 Native Client 64-bit edition on all SharePoint servers.
3. Install SharePoint prerequisites and SharePoint binaries by executing the following script on all servers listed above (launch PowerShell as administrator).  This script can be run simultaneously on all servers to save time.

Set-ExecutionPolicy Unrestricted –force
E:\Scripts\Install\SPInstaller.ps1 -installPath D:\ -offline "n"

This script will automatically restart servers as required but you will need to log-in to the machine using the install account when prompted. DO NOT RUN the configuration wizard and un-mount the installation media from each server once the script has run successfully

Please note: Windows PowerShell execution policies let you determine the conditions under which Windows PowerShell loads configuration files and runs scripts. The execution policy is not a security system that restricts user actions. Instead, the execution policy helps users to set basic rules and prevents them from violating them unintentionally. Please see About Execution Policy TechNet article.

Install SharePoint (without Internet Connection)

If a direct internet connection is not available during installation, even if a proxy is available, the SharePoint prerequisites installation is likely to fail. This is because, 46 windows components need to be verified and installed (if missing) as well as an additional 8 features specific to SharePoint. By default the prerequisites installer will look to download the missing binaries online and without an internet connection this will fail. The following screenshot shows the first indication of this failure:

To successfully install the necessary prerequisites perform the following steps

Install Windows Server Features and Roles

1. Mount the Windows Server 2012 installation media (ISO) to drive D:\ and check that directory D:\sources\sxs exists on the image (it will contain lots of folders) on each server in the SharePoint farm
2. Open a PowerShell prompt as an administer and run the following script from each server in the farm to install all 46 Windows components: 

Set-ExecutionPolicy Unrestricted –force
E:\Scripts\Install\WinPreReqs.ps1

      3. Verify that all the roles and features have been successfully installed by executing the following from a PowerShell prompt run as administrator and checking the status the required features listed above are set to installed:

Get-WindowsFeature 

      4. Restart all servers after the script has successfully completed

      5. Un-mount the Windows Server 2012 installation media (ISO) drive from all servers

Install Remaining SharePoint 2013 Prerequisites Offline

Download the following 8 additional prerequisites to a local directory and make available on each server in the farm (network shares are supported):

• MicrosoftSync Framework Runtime v1.0 SP1 (x64)
  https://download.microsoft.com/download/E/0/0/E0060D8F-2354-4871-9596-DC78538799CC/Synchronization.msi
  
• Windows Server AppFabric
  https://download.microsoft.com/download/A/6/7/A678AB47-496B-4907-B3D4-0A2D280A13C0/WindowsServerAppFabricSetup_x64.exe 
  
• Cumulative Update Package 1 for Microsoft AppFabric 1.1 for Windows Server (KB2671763)
  https://download.microsoft.com/download/7/B/5/7B51D8D1-20FD-4BF0-87C7-4714F5A1C313/AppFabric1.1-RTM-KB2671763-x64-ENU.exe
  
• Windows Identity Foundation (KB974405)
  https://download.microsoft.com/download/D/7/2/D72FD747-69B6-40B7-875B-C2B40A6B2BDD/Windows6.1-KB974405-x64.msu
  
• Microsoft Identity Extensions
  https://download.microsoft.com/download/0/1/D/01D06854-CA0C-46F1-ADBA-EBF86010DCC6/rtm/MicrosoftIdentityExtensions-64.msi
  
• Microsoft Information Protection and Control Client
  https://download.microsoft.com/download/9/1/D/91DA8796-BE1D-46AF-8489-663AB7811517/setup_msipc_x64.msi
  
• Microsoft WCF Data Services 5.0
  https://download.microsoft.com/download/8/F/9/8F93DBBD-896B-4760-AC81-646F61363A6D/WcfDataServices.exe
• Microsoft SQL Server 2008 R2 SP1 Native Client
  https://download.microsoft.com/download/9/1/3/9138773A-505D-43E2-AC08-9A77E1E0490B/1033/x64/sqlncli.msi

Open a PowerShell prompt as an administer and run the following script from each server in the farm, when prompted enter the path in which the download prerequisites can be found

1. Ensure that the SharePoint installation media is mounted and D:\Setup .exe is accessible
2. Microsoft SQL Server 2008 R2 Native Client is installed as part of SharePoint 2013 prerequisites installer, however if you are using SQL Server 2012 to host SharePoint databases install Microsoft SQL Server 2012 Native Client 64-bit edition on all SharePoint servers.
3. Install SharePoint prerequisites and SharePoint binaries by executing the following script on all servers listed above (launch PowerShell as administrator).  This script can be run simultaneously on all servers to save time.

Set-ExecutionPolicy Unrestricted –force
E:\Scripts\Install\SPInstaller.ps1 -installPath D:\ -offline "y"

This script will automatically restart servers as required but you will need to log-in to the machine using the install account when prompted. DO NOT RUN the configuration wizard and un-mount the installation media from each server once the script has run successfully

Apply March PU and April CU

Before configuring the farm, two product updates are required to SharePoint 2013 which can be downloaded from the following links (note a server restart is required after each update):

(1)   KB2767999 https://support.microsoft.com/kb/2767999

(2)   KB2726992 https://support.microsoft.com/kb/2726992

The March PU (KB2767999) must be applied before the April CU (KB2726992) and is also a dependency for any future patches or cumulative updates.

Create the Farm

Login to SPAPP01 (Application Server hosting the Central Administration site) using the installer account and launch SharePoint Management Shell as administrator. Execute the following command to create the farm: 

E:\Scripts\Install\SPConfig.ps1 -configLocation E:\Scripts\Install\Config.xml -createJoin create -isCA "y"

Join Servers to the Farm

Login to SPAPP02 (The second Application Server hosting the Central Administration site) and launch SharePoint Management Shell as administrator. Execute the following PowerShell script:

E:\Scripts\Install\SPConfig.ps1 -configLocation E:\Scripts\Install\Config.xml -createJoin join -isCA "y"

Run the following script on the below SharePoint servers:

E:\Scripts\Install\SPConfig.ps1 -configLocation E:\Scripts\Install\Config.xml -createJoin join -isCA "n"

 Register Managed Accounts

Login to SPAPP01 (Application Server hosting the Central Administration site) using the installer account and launch SharePoint Management Shell as administrator. Execute the following command to register managed accounts in the farm:

E:\Scripts\Install\SPCredentials.ps1 -configLocation E:\Scripts\Install\SPCredentials.xml

Configure Basic Services

Login to SPAPP01 (Application Server hosting the Central Administration site) using the installer account and launch SharePoint Management Shell as administrator. Execute the following command to start required services and stop unwanted ones:

E:\Scripts\Install\SPServices.ps1 -configLocation E:\Scripts\Install\SPServices.xml

 The above script will stop unnecessary services from servers and starts the following service:

• Excel Services
• Secure Store
• Usage and Health Data Collection
• Managed Metadata Services
• State Services

Configure Usage and Health Data Collection

Login to SPAPP01 (Application Server hosting the Central Administration site) using the installer account and launch SharePoint Management Shell as administrator. Execute the following command to start and configure Usage Service:

E:\Scripts\Install\SPUsage.ps1 -configLocation E:\Scripts\Install\SPServices.xml

Configure State Service

Login to SPAPP01 (Application Server hosting the Central Administration site) using the installer account and launch SharePoint Management Shell as administrator. Execute the following command to start and configure Usage Service:

E:\Scripts\Install\SPStateService.ps1 -configLocation E:\Scripts\Install\SPServices.xml

You should now have basic services started on all servers in the farm. It is recommended to check Servers in Farm page from Central administrations site to ensure that the correct services are started.

By the end of this blog series the expected farm topology will be as illustrated below:

Lookout for the next blog post, where we complete the farm configuration by provisioning search, user profile services and distributed cache service.

BasicServices_Blog02.zip

Introduction

The REST API in SharePoint 2013 provides developers with a simple standardised method of accessing information contained within SharePoint. It can be used from any technology that is capable of sending standard http requests and is particularly useful for developers who are not familiar with the Client Side Object Model. For a full list of advantages and disadvantages of the REST API, and for a comparison with other API’s click here.

This article does not cover App Permissions. For information about permissions there is a good MSDN article here. Use of the REST API in other hosting options, whilst similar to SharePoint Hosted Apps, is not covered in this article.

Obtaining the site URI

One key prerequisite to interacting with SharePoint lists items is to obtain the correct URI for the SharePoint site the list is hosted in. There are multiple ways to do this:

1. On SharePoint pages _spPageContextInfo.siteAbsoluteUrl contains the URI to the site that the app is installed within (the parent site).
2. You can include a set of standard tokens in the URL attribute of both pages and web parts. Adding {StandardTokens} will include the following 7 parameters to the url; SPHostUrl, SPHostTitle, SPAppWebUrl, SPLanguage, SPClientTag, SPProductNumber, and SenderId. These will need to be split out of the URL to be used. There are multiple ways to do this. The simplest is to loop through all the parameters until you find the parameter you are looking for. The following snippet of code searches for the SPAppWebUrl.

There are several different tokens that you can include in the url's used to open pages within your App. For more information about the different tokens and some basic information about each of them click here. To add the tokens to your start page you can use the visual designer for the App's manifest file. The setting can be found on the ‘General’. It is worth noting that the manifest editor has changed significantly since the original release of the developer tools. If you are using an old version of the tools then this setting may be in a different place.

You can also add the tokens using the source code editor. The following example shows the start page url where the standard tokens have been added (note that ?{StandardTokens} has been appended to the StartPage url):

REST URI Operators

Once you have the correct base URI the next step is to determine the correct URI for the list with which you would like to interact. The REST endpoint exposes the same GetByTitle() method that the Client Side Object Model offers and can be used by appending “/_api/web/lists/getbytitle(title)” to the end of the base URI. More information about the REST endpoint can be found in this MSDN article.

List Item Types

To update or create list items you will need to include the list item type. This is a string automatically created by SharePoint when the list is first created. Two possible ways to obtain this are:

1. Perform a read operation on the list and locate the type within the returned set of elements. In JSON format the type can be found within the __metadata attribute of all of the returned list items. In Atom/xml format (the default format returned if you query from a web browser) you can find it in the category scheme attribute (labelled ‘term’) within the item.
2. You can attempt to generate the value from the list name. In general list item types follow the convention SP.Data.ListNameListItem (e.g. list name is “Test”, list item type is SP.Data.TestListItem). However this is not always the case. For example SharePoint automatically capitalises the first letter of the list name (e.g. list name “test” list item type is SP.Data.TestListItem). The following code snippet shows how to generate the List Item Type based on the list name.

Request Digests

A replay attack occurs when a valid request is sent to the server and a copy of that request is stored. The request is then sent multiple times (replayed). This can result in a variety of different issues the most common of which is duplication of data. To prevent this, SharePoint requires the user to include a request digest value with each create, update and delete operation. This value is then used by SharePoint to identify non-genuine requests. Inside SharePoint Hosted apps, the request digest can be found by reading the value of the “__REQUESTDIGEST” object within the html page.

Note: this will only work on pages that contain the SharePoint Chrome element. The example above requires JQuery to work.

eTags

When updating or deleting items within SharePoint lists via REST you must specify the Entity Tag (eTag) value that was returned with the item during the initial query. This enables SharePoint to determine if the item has changed since it was requested. Alternatively you can tell SharePoint to perform the operation regardless by specifying * as the eTag value. For example:

1. “If-Match”: item.__metadata.etag can be used to specify the actual eTag value (‘item’ is the object returned from SharePoint containing the list item in JSON format).
2. “If-Match”: “*” can be used to match any eTag value resulting in the operation being performed regardless of the actual value.

The examples above can be found in the code sample at the end of this article. They form part of the Ajax call for updating an item. eTags are part of the HTTP Protocol V1.1, more information on the topic can be found here.

Cross Domain Calls

Cross domain calls are blocked by modern web browsers due to security concerns. This is a common issue in web based development and is particularly relevant due to the nature of SharePoint hosted Apps. For example, when accessing data in the parent web (https://Sharepoint.com) from an App (hosted at https://Apps.SharePoint.com/MyApp) the calls will be blocked. To get round this you can use the SP.RequestExecutor.js script to relay messages to SharePoint from within the same domain.

To do this you will need to load the JavaScript file from the host web and then use it to execute the REST queries across the network. The following example shows how to do this.

Note: the middle line of the URL variable is where you would change the address that the REST call is sent to. For example to get information about a list rather than its items you can remove “/items” leaving "/web/lists/getbytitle('" + listName + "')?"

Example Operations

This section contains sample code for all of the CRUD operations. All of this sample code can be found in the source code which is available for download. The request executor is not needed in this example since the site URL is in the same domain as the app.

Create

The following snippet of code shows how to perform a Create operation against a SharePoint list. The first line of the function is a call to a method which returns a list item type based on the name of the list using the rules mentioned previously. The next 4 lines create a JavaScript object which contains the information about the item that should be created including values for each of the fields and the item type. The reminder of the function executes the ajax call and then calls either the success or failure functions (passed into the function as parameters).

Read

Reading items is probably the simplest of all of the CRUD operations. The code below shows the simplest form of a read operation. A simple ajax call is executed against the SharePoint server. If the call succeeds then the success function (passed in as a parameter) is called, if it fails then the failure function (also passed in as a parameter) is called. In both cases the data returned from the server is passed through to the function. The code sample also includes an example of how to retrieve an item based on its ID.

Update

In contrast to reading list items updating them is probably the hardest operation. To update an item you will need to obtain the URL of the item and, depending on your handling of eTags, the eTag value. The simplest way to do this is to perform a get operation and then use the properties of the item. The update method uses the same method mentioned for update operations to generate the list item type.

The first line of this function calls a method to get the list item type for the specified list. The next four lines create a JavaScript object containing details of the list item. The next line is a call to a get operation (similar to the one described previously) which returns the list item to be updated. This allows us to extract the eTag and item url and makes it easier to update. If this is successful then the call to update the item is executed and either the success or failure function is executed depending on the result.

Delete

The code below performs a delete operation. In a similar way to updating items delete operations are performed using the URI of the item and depending on how eTags are handled the eTag value of the item. In this example the item is first retrieved and then its properties (uri and eTag) are used to execute the delete operation.

Source Code

The following downloadable source code contains a working example of all four operations. To run the example you will need to configure the SiteUrl of the project to point to a SharePoint 2013 developer site. To do this click on the project in solution explorer and open up the properties task pane. Enter the URL of your site into the SiteUrl box and then press F5.

The sample contains a page hosted within the App Web which interacts with a list in the Parent Web. When you install the App you will be asked to select a list and trust it. The list you select must contain a Title column (a new custom list will work just fine). To use the App enter the name of the list you want to access data from (this must match the list you selected when you trusted the App) and then use the controls under each of the four headings. The code is all contained within the App.js file of the project.

Please Note: to run this sample you will also need access to a SharePoint server which has been configured for Apps. The simplest way to do this is to create an Office 365 developer account. Alternatively you can install SharePoint on your own server. There is a guide to configuring SharePoint 2013 servers for Apps here.

Example CRUD Operations.zip

Whilst the fundamentals remain the same, there were changes in the product over time that have affected the inter and extra-server communications paths. SharePoint 2010 introduced the Service Application model with Windows Communication Foundation proxies and endpoints. SharePoint 2013 brings additional core-building blocks to the product such as Distributed Cache or optional components such as Office Web Apps Server and Workflow Manager with an underlying Service Bus. These components publish some sort of endpoint to communicate with SharePoint 2013 on different ports. You should not forget to secure communication with these external endpoints with a SSL certificate. SharePoint 2013 uses oAuth for Server to Server (S2S) communication and you absolutely don't want oAuth bearer tokens flying over your network in clear text. They contain sensitive information and any party in possession of a bearer token can use it to get access to the associated resource. You can find more information in The OAuth 2.0 Authorization Framework: Bearer Token Usage RFC 6750 document. Yes, this means you should have SSL certificates deployed on your SharePoint sites as well.

Below is an updated communication diagram for SharePoint 2013. I have combined both inter and extra server communications into a single diagram. I hope you will find it easy to navigate. A large version of this image can be found here.

References

Plan security hardening for SharePoint 2013

Using Windows Internet Explorer Developer Tools Network Capture

Marek Samaj
Senior Consultant
Microsoft Consulting Services UK
marek.samaj@microsoft.com
Click here to see my bio page

You can find part 1 here: Managing Different Styles of Development Team (Part 1)

The two attributes of the team that we’ve looked at are:

1. how skilled the team is
2. their approach to the project

That is, some teams are very experienced with SharePoint, and some not so. Also, some are happy that SharePoint is being used for the project, and some not so.

Using these two attributes as axes on a graph, we’ve identified four types of development teams, as show in the following diagram.

The broad characteristics of these types of teams are:

• Type 1: A capable and proficient team who have good experience of developing successful SharePoint solutions. They are happy to be working on the project and are convinced that the use of SharePoint is appropriate.
• Type 2: A development team who are probably new to SharePoint. They may, or may not, be proficient developers. Even so, they see the use of SharePoint as a good thing.
• Type 3: This team has had little exposure to SharePoint development. They are not convinced that SharePoint is a good fit and would probably prefer to be using some other technology.
• Type 4: This team are skilled and versed in SharePoint development. However, they are not convinced the SharePoint is relevant for the project.

The previous posting discussed the two upper quadrants: when folks are happy that SharePoint is being used for the project. In this posting, we address the two lower quadrants: when people are not happy (or not convinced) that SharePoint is a suitable platform for the project.

Type 3: Novice and Sceptical

This type of team are new to SharePoint development and, for various reasons, they are not convinced that SharePoint is appropriate or relevant for the project.

I’ve come across project teams like this in two main scenarios:

1. The development team prefer to write everything themselves
2. The customer is starting to use SharePoint instead of some other technology

In the first case, I’ve seen customers make the following (wrong) argument:

• SharePoint is built on.NET
• I have.NET Developers
• I have SharePoint Developers

With a development team that prefers to build everything themselves, nothing beats the purity of their own codebase written from scratch. Now, don’t get me wrong: this approach is sometimes needed. But SharePoint provides a whole set of building blocks and in-built capabilities. As such, this is a team that is used to doing things in a particular way and they have to come to terms with an alternate approach. For example, there are recommended ways of doing things and this can feel like a whole set of restrictions on the team – they can feel as if they have all their freedoms taken away from them. Naturally, this results in a degree of frustration and it is no wonder that they are not feeling too great about having to use SharePoint.

In the second case (when they’re moving away from some other technology), the team is in a similar position. They know how to get the job done with the previous mechanisms of the alternate software package. Why do we have to go and spoil everything with the introduction of SharePoint? This can be the situation when upper management has not taken the time to explain why there is the move to SharePoint. After all, the IT Strategy has to be tied to, and serve, the Business Strategy. Consequently, there has probably been some decisions made elsewhere that has driven the move to SharePoint. Sometimes, these drivers don’t get fully, or properly, relayed to the developers and this leaves the wondering, “Why are we doing it this way?” Sometimes, a quick chat is all that is needed.

The trick is to try and move them from being a Type 3 team to a Type 2 team. That is, move them from being “novice and negative” to “novice but positive”. From there, we already know how to get them to Type 1 “export and positive”.

Within any development team, I always have a “holy trinity” of three key roles:

• User Experience Lead
• Lead Developer
• Test Lead

These three people are the ones that I look to, in the first instance. The UX Lead is critical to building the user stories, scenarios and wireframes that define the solution that we are creating. The Lead Developer will be responsible decomposing the solution into its constituent elements and organising the other developers to implement the code and create the necessary configurations. The Test Lead is the one who keeps everyone honest by checking to make sure that the code executes as it should and that the various quality attributes are met.

If I sense that I am in the realms of a Type 3 team, I have always found it most important to take my “holy trinity” aside and deal with them independently. If I convince these folks that SharePoint is both useful and appropriate, I can be pretty sure that they will take the message to their respective teams. Instead of trying to persuade the complete team, I have reduced my problem to a much smaller one: convince the leaders and then let them lead their teams.

And I have been most successful in this task when I’ve concentrated on the things that they don’t have to do rather than trying to explain how good SharePoint is. Sometimes a solution is being used to support a brand new business concept that has never existed before and every single line of code needs to be written from scratch. Most of the time, the solution is well understood and the pattern has been established for a long time.

Applications of SharePoint tend to lie mostly in the second camp: the solution patterns are well known and understood. We typically have forms being used to update custom lists and this may make use of event handlers or workflows to implement some business logic. Sometimes, we have a job to perform some form of automated task on a scheduled basis.

Well, so far, so good. The development team could well argue that ASP.NET can also be used to implement such a solution. And they’d be right. However, it is all the “value-adds” that SharePoint brings that I first discuss with my Leads.

For the UX Lead, their efforts can focus on the screens for the actual solution instead of needing to address administrative elements. For example, user administration and security permissions are provided by SharePoint and so these are screens they do not need to think about and design. In addition, the overall approach to screen layout is also provided by SharePoint: master pages, page layouts, themes, headers, left-hand navigation, etc.

For the Lead Developer, we can address scalability and data integrity because SharePoint is based on ASP.NET, runs on IIS, and uses a database to hold its data. It can have multiple front-end web servers that handle load balancing. The use of SQL Server provides the integrity, scalability, robustness, and security of the underlying lists, data and documents used by the solution. We have the security model that can be applied to lock down elements such as Management Libraries, etc. The site engine can be used to serve different types of web site. We have a mechanism to deploy and retract custom code via Features and Solutions. We have the performance counters in order to provide for monitoring the application once it has been deployed. And the list goes on.

For the Test Lead, again, the argument is that they do not need to spend time testing the administrative paraphernalia of the solution. All they have to focus on is testing the new components in the context of the business solution. Also, as SharePoint provides a rich scripting interface, they can automate their tests to populate test data and users, etc. Furthermore, the use of the solution deployment and site administration means that they can have a controlled test environment that they can tear down and rebuild with relative ease.

So, instead of trying to take on a whole development team, I decompose my problem into smaller elements: convince the Leads and trust them to convince (and lead) their teams.

Type 4: Experienced and Sceptical

Now, you might be forgiven for thinking that there’s no option but to resign your position when faced with a Type 4 team. They’re knowledgeable and skilled SharePoint developers. They probably are leading lights in their respective communities. And they’re convinced that SharePoint is not a good idea for this project.

Oh dear.

The trick, here, is to really understand what their problem is. Because it may well be that, once you start digging into things, that they have concerns in only one or two areas. If we can address these concerns, we will have moved the team from a Type 4 (expert but negative) into a Type 1 (expert and positive). And that’s our goal, isn’t it?

Let’s start with the easiest scenario: the team has well founded concerns about some areas. This can arise for a number of reasons. For example, the team might have past experience in a particular area and they are aware of the problems that can arise. These don’t have to be just technical problems. Problems can arise for organisational reasons, as well.

So, it may well be that you have designed a system with an incorrect assumption. Perhaps, for a given customer, certain features are not enabled due to a variety of concerns. Or it may be that you are relying on an external system that the team knows is not as reliable as you might have been led to believe. There’s a whole host of reasons why you might need to modify your design. Just because you’re the architect, it doesn’t mean that you are always right.

Part of the skill of an architect is to be able to negotiate and compromise. And when there is skilled expertise advising you that a particular course of action is ill-advised (and when you have convinced yourself that they are right), it is time to flex the architecture and record your modifications in a refresh of the Solution Architecture document and an update of the Decisions Log.

However, there are times when the skilled team are not correct about something. I’ve most often noticed this when dealing a desire to use new or updated features and I’ve been championing something less sophisticated. It most often comes about when the team doing the development are not connected to the team that will run the system during its day-to-day usage. Teams can have this “disconnected” make-up when, for example, they have been subcontracted for the purposes of the development and they are expected to hand-over to the customer’s operations team.

In this situation, I am keen for something I call, “Operations Oriented Architecture.” What I mean by this is that one of the foremost considerations is the team that will run, or operate, the final system. Now, as we discussed previously, I always try to run the design past the Operations Team to make sure that they are expecting to on-board the system and that they can get any necessary training in place and organised. Sometimes, these conversations drive changes to the architecture. “No, we can’t use this capability as the Operations Team have not been trained to support it. They would rather we use this alternate method.”

So, it can be that the Operations Team will impose restrictions on the solution. As a simple example, they might define a format for error/log messages and assign your application a unique reference number or code. This is so that their systems monitoring application can identify your application. The Development Team can sometimes balk at these “restrictions” or “impositions” but it’s your job as an architect to persuade them otherwise.

So, I’ve experienced a Type 4 team when there is a disparity between what the Project Team want to do and what the Operations Team will support. And, it’s most often that the Project Team want to make use of some fancy, new feature. After all, they’re skilled, expert and capable. For them, it is the smallest stretch to understand and make use of the new feature. But for the Operations Team, it can be a huge hill to climb: they have to send their people on training, they have to provision and deploy new hardware, and they have to update the configuration of the data centres, and so on.

Once again, a key architect skill of, “negotiate and compromise” comes to the fore. Except, this time, you use your negotiation skills to ensure that it is the Development Team that make the compromise.

Conclusion

I hope you’ve found these two articles useful.

It’s worth noting that, whilst I have addressed this at the level of the Development Team as whole, you would be quite right if you thought that these approaches can also be used for individuals. After all, whilst a team can have an identity and a character of its own, this doesn’t mean to say that all the members of the team are the same.

As a Solutions Architect, it took me some time to fully appreciate that projects do not always have the same shape and form. Each has their own unique challenges. What was easy on one will be hard on another. The enjoyment is that it is all about getting people to perform at their best. As an architect, you need to really engage with the team and genuinely understand them and their concerns in order to have them working with you instead of against you.