Now, sizing is a tricky business. Primarily involves 2 parts – find an optimum server/VM and then find out how many. Let’s take these one by one.

In search world, there is no silver bullet for sizing and most of the time it’s done by rigorous test and observation. For example, Microsoft’s own SharePoint Enterprise Search – product team suggested some sizing benchmark based on no. of items to be crawled and indexed (10 million, 40 million and so on). This had been done based on good amount of tests conducted by product and other groups.

In case of Elasticsearch, we have few information and observation provided on their website which may help us derive to some sizing idea.

Let’s look at the following article: https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html. Based on this we can derive the following:

1. Memory: It advises to use 64 GB. But in Azure world, we have 56 GB and 112 GB. 112 GB would be an overkill and as per the above article more than 64 GB is not advisable. So we stick to 56 GB.
2. CPU: No definitive guide. We will choose 8 cores. One thing to be noticed here. It says between faster CPU and more core CPU, we should choose more cores. So 16 cores machine looks like better bet. But as we do not have specific info about “sweet spot” as with memory, let’s start with 8 core.

Now, for Disk, let’s see this: https://www.elastic.co/blog/performance-considerations-elasticsearch-indexing. Although no definitive guide on IOPS, but it is clearly mentioned and advised high speed SSDs to store index. That too, local SSDs are preferred in comparison to attached one like Premium storage. To store the data/log we can have regular Azure storage. Premium storage is not essential. At least not indicated anywhere or at least I didn’t notice.

This brings us to Azure VMs like DS 13 or D13 V2. Which has 8 core, 56 GB RAM, and 400 GB SSD local storage. In my opinion, this can be considered as a benchmark for my VMs. In other words, if we need to scale higher, we will scale out rather than scale up.

Now, the question is how many VMs? Here I couldn’t find any definitive guide. It looks like things are mostly done on trial and error basis. But for a production setup with Terabytes of data, I’d recommend 3 node cluster to start with and then scale-out as required. 

Regarding the OS, I’d vote for Ubuntu, this is only because I saw at least 2 customers successfully running Elasticsearch on Ubuntu on Azure IaaS. Here is a guide, though a bit old but still may be useful: http://code972.com/blog/2014/07/74-the-definitive-guide-for-elasticsearch-on-windows-azure

As correctly said in the last link mentioned, you don't need load balancer (Azure or 3rd party).

At the end, a disclaimer, this article based on my experience with Elasticsearch and Azure. Your opinion and suggestions are most welcome.

using System;

using System.Collections.Generic;

using System.Text;

using Microsoft.VisualStudio.TestTools.WebTesting;

namespace WebAndLoadTestProject2

{

    [Priority(0)]

    public class testCoded : WebTest

    {

        public testCoded()

        {

            this.PreAuthenticate = true;

            this.Proxy = "default";

        }

        public override IEnumerator<WebTestRequest> GetRequestEnumerator()

        {

            WebTestRequest wtRequest = new WebTestRequest(https://www.contoso.com/APICollection/getFlightInfo/);

            wtRequest.Timeout = 300;

            wtRequest.Method = "POST";

            wtRequest.Headers.Add(new WebTestRequestHeader("Content-Type", "application/json"));

            wtRequest.Headers.Add(new WebTestRequestHeader("Authorization", "bearer pranab.paul@contoso.com"));

            StringHttpBody requestBody = new StringHttpBody();

            requestBody.ContentType = "application/json";

            requestBody.InsertByteOrderMark = false;

            requestBody.BodyString = @"{

    ""itenaryType"":""international"",

    ""routes"": [{

        ""endDate"":""30-05-2015"",

        ""from"":""BANGALORE,INDIA"",

        ""startDate"":""30-05-2015"",

        ""timeBetween"":""10:00-12:00"",

        ""to"":""BOSTON,USA""

    }],

    ""travelType"":""singlecity""

}

";

            wtRequest.Body = requestBody;

            yield return wtRequest;

            wtRequest = null;

        }

    }

}

I started looking into “how to” part and found the first lesson:

http://msdn.microsoft.com/library/gg690195.aspx

I found that everything should start working provide you have a SharePoint site and a site workflow in that site in place. Then you just have to mention the Workflow Extender name which is SPWF – because it is SharePoint and then need to enter the site URL and the Workflow name.

Great, I created 2 site workflow one using SharePoint Designer and one with Visual Studio. We planned to change some value in Customers and tried to check whether it starts the workflow or not. Essentially we can pass values to the workflow I created using Visual Studio.

But, it did not work! Then I found an article which claims that we need to create a workflow extender for MDS Workflow Service exclusively
to do so:

http://social.msdn.microsoft.com/Forums/sqlserver/en-US/a31769eb-948c-4bb2-844d-72d70e34292b/master-data-services-2012-and-sharepoint-workflow

It seems it’s only for SQL Server 2012. Previous versions of SQL has that built in. Unfortunately our demo environment was with SQL Server 2012 SP1 and SharePoint 2013 RTM.

Well, I created the extender, compiled it and tried to implement the same with MDS Workflow Service. But, it started throwing error saying I am using assemblies which are of newer versions than the current environment. What I found was the culprits were “Microsoft.SharePoint.dll” and “Microsoft.SharePoint.WorkfloActions.dll”. This is because I am in SharePoint 2013 version and the Workflow Extender in example was in .Net version 3.5. Also it seems MDS wants it that way. Which was ideally using dlls from SP 2010.

Next thing I did I borrowed the dlls from a SP 2010 server compiled the Extender using them and now the MDS Workflow Service accepted the extender.

But, again in runtime we started receiving error because the SharePoint dlls are not located in the SharePoint Server (remember we are using SP 2010 dlls but it is a SP 2013 server).

So, I was in a catch 22 situation. We have to show a demo in next half a day time where we have a workflow with escalation scenario. Which includes any specific change in customer value will assign a task to a person and then if he does not attend it within 5 mins it will be assigned to the 2^nd person. Also the task title should include the name of the customer where the data is changed.

Then I took a drastic step. I created a MDS Workflow Extender and within it I called the Lists.asmx web service of SharePoint and there I added a task to the Workflow Tasks list for the first person. Then I managed the rest of the show by a Site Workflow created using SharePoint Designer. This waits for few mins and then assign the task to the 2^nd person. Below is the code sample I wrote for the Workflow Extender:

using Microsoft.MasterDataServices.WorkflowTypeExtender;

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml;

namespace Services.MDSWorkflowExtender

{

     public class WorkflowTester : IWorkflowTypeExtender

    {

        #region IWorkflowTypeExtender Members

        public void StartWorkflow(string workflowType, System.Xml.XmlElement dataElement)

        {

            // Extract the attributes we want out of the element data.

            XmlNode NameNode = dataElement.SelectSingleNode("//ExternalAction/MemberData/Name");

            XmlNode CodeNode = dataElement.SelectSingleNode("//ExternalAction/MemberData/Code");

            XmlNode EnteringUserNode = dataElement.SelectSingleNode("//ExternalAction/MemberData/LastChgUserName");

            try

            {

                com.contoso.intranet.Lists listService = new com.contoso.intranet.Lists();

                listService.Credentials = System.Net.CredentialCache.DefaultCredentials;

                string strBatch = @"<Method ID='1' Cmd='New'><Field Name='Title'>Customertype is changed for following customer: " + NameNode.InnerText + "</Field><Field Name='AssignedTo'>18;#contoso\\pranab</Field></Method>";

                //Guid of the Workflow Tasks list

                string listGuid = "{cc49bc7b-ccde-4bfa-8458-f9bcf7a709d4}";

                //Guid for All tasks view

                string activeItemViewGUID = "{6780F7D4-A0B2-426A-AB86-2C4D8EEFC36A}";

                XmlDocument xmlDoc = new System.Xml.XmlDocument();

                System.Xml.XmlElement elBatch = xmlDoc.CreateElement("Batch");

                elBatch.SetAttribute("OnError", "Continue");

                elBatch.SetAttribute("ViewName", activeItemViewGUID);

                elBatch.InnerXml = strBatch;

                XmlNode ndReturn = listService.UpdateListItems(listGuid, elBatch);

            }

            catch (Exception)

            {

                throw;

            }

        }

        #endregion

    }

}

So the learning was if you are using SQL Server 2012, you should use custom Workflow Extender to integrate SharePoint Workflow and use SharePoint 2010 instead of SharePoint 2013 if you don’t want sleepless nights.

We have various options to run the unit test on the .Net code written within controller. But sometimes it’s required to test those JavaScript/JQuery codes as well, to check the sanity of the output especially in the situation mentioned above.

QUnit (http://nuget.org/packages/QUnit-MVC, a NuGet package) can help us to run such Unit Test. As per its description:

“QUnit is a powerful, easy-to-use, JavaScript test suite. It's used by the jQuery project to test its code and plugins but is capable of testing any generic JavaScript code (and even capable of testing JavaScript code on the server-side).”

Let’s see a step by step example of how to use QUnit successfully with ASP.Net MVC project:

Step 1: Let’s create an ASP.Net MVC 4 Web Application.

Step 2: Choose Internet Application Template. No need to create Unit Test Project.

Step 3: Once the application created, add following 2 models (Country.cs and City.cs) under “Models” folder in solution explorer.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace QUnitMvcApp.Models
{
    public class Country
    {
        public string Code { get; set; }
        public string Name { get; set; }

        public static IQueryable<Country> GetCountries()
        {
            return new List<Country>
            {
                new Country
                {
                    Code = "AU",
                    Name = "Australia"
                },
                new Country
                {
                    Code = "UK",
                    Name = "United Kingdom"
                }
            }.AsQueryable();
        }
    }
}
==========================================================================
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace QUnitMvcApp.Models
{
    public class City
    {
        public string Code { get; set; }
        public int CityID { get; set; }
        public string CityName { get; set; }

        public static IQueryable<City> GetCitys()
        {
            return new List<City>
            {
                 new City
                    {
                        Code = "AU",
                        CityID=1,  
                        CityName = "Perth"
                    },
                new City
                    {
                        Code = "AU",
                        CityID=2,
                        CityName = "Sydney"
                    },
                new City
                    {
                        Code = "AU",
                        CityID=3,
                        CityName = "Melbourne"
                    },
                new City
                    {
                        Code = "UK",
                        CityID=4,
                        CityName = "London"
                    },
                new City
                    {
                        Code = "UK",
                        CityID=5,
                        CityName = "Bermingham"
                    },
                new City
                    {
                        Code = "UK",
                        CityID=6,
                        CityName = "Manchester"
                    }
            }.AsQueryable();
        }
    }
}
=======================================================================

Step 4: Add following few methods to the HomeController.cs file:

public ActionResult CascadingList()
        {
            return View();
        }

        public SelectList GetCountrySelectList()
        {

            var countries = Country.GetCountries();
            return new SelectList(countries.ToArray(),
                                "Code",
                                "Name");
        }

        public ActionResult CountryList()
        {
            var countries = Country.GetCountries();

            if (HttpContext.Request.IsAjaxRequest())
                return Json(GetCountrySelectList(), JsonRequestBehavior.AllowGet);

            return RedirectToAction("CascadingList");
        }

        public ActionResult CityList(string ID)
        {
            string Code = ID;
            var cities = from s in City.GetCitys()
                         where s.Code == Code
                         select s;

            if (HttpContext.Request.IsAjaxRequest())
                return Json(new SelectList(
                                cities.ToArray(),
                                "CityID",
                                "CityName")
                           , JsonRequestBehavior.AllowGet);

            return RedirectToAction("CascadingList");
        }
Also add following using statement HomeController.cs:
using QUnitMvcApp.Models;

Step 5: Add a new view “CascadingList.cshtml” under View>Homes folder with following code:

@{
    ViewBag.Title = "CascadingList";
}
 @{Html.EnableUnobtrusiveJavaScript(true);}
<script type="text/javascript">
    $(document).ready(function () {

        getCountry();
    });
    function getCountry() {
        var url = '@Url.Content("~/Home/CountryList")/';

        $.getJSON(url, function (data) {
            var items = "<option>Select a Country</option>";
            $.each(data, function (i, country) {
                if (country.Value.indexOf("\'") > -1) {
                    s = country.Value + " " + country.Text;
                    alert(s + ": Country.Value cannot contain \'")
                }
                items += "<option value='" + country.Value + "'>" + country.Text + "</option>";
            });
            $('#CountriesID').html(items);
        });
    }

    function getCity() {
        var url = '@Url.Content("~/Home/CityList")/' + $('#CountriesID').val();

        $.getJSON(url, function (data) {
            var items = '<option>Select a City</option>';
            $.each(data, function (i, city) {
                items += "<option value='" + city.Value + "'>" + city.Text + "</option>";
            });
            $('#CitiesID').html(items);
        });
    }
   
</script>
<h2>Country</h2>

<div id="CountriesDivID">
    <label for="Countries">Countries</label>
    <select id="CountriesID" name="Countries" onchange="getCity()"></select>
</div>   
<div id="CitiesDivID" >
    <label for="Cities">Cities</label>
    <select id="CitiesID" name="Cities"></select>
</div>

Step 6: Open up “_Layout.cshtml” page under Views>Shared folder and add following line under the tab menu section:
<li>@Html.ActionLink("CascadingList", "CascadingList", "Home")</li>

Also add following line under the head tag of the same file:
        <script src="@Url.Content("~/Scripts/jquery-1.7.1.min.js")" type="text/javascript"></script>

Now press F5 to debug this application and go to CascadingList tab. You can see our JQuery/JavaScript code is working fine and filling up the drop-down list boxes as expected after receiving JSON formatted output from the controller.

Ok, it’s time now to set up some Unit Test on the JavaScript code we used in the CascadingList view.

Step 7: Add a reference of the NuGet package QUnit.

Search and install QUnit:

Step 8: Add following 2 lines to _Layout.cshtml
<script src="@Url.Content("~/Scripts/qunit.js")" type="text/javascript"></script>
<link href="@Url.Content("~/Content/qunit.css")" rel="stylesheet" type="text/css" />

Step 9: Now it’s time to convert our code into more “Unit Testable” Format. Instead of calling the getCity() function from the onchange event of the select “Country”, we will add the code into the OnLoad event handler. Let’s look into the changed CascadingList.cshtml file:
@{
    ViewBag.Title = "Cascading List";
}
 @{Html.EnableUnobtrusiveJavaScript(true);}
<script type="text/javascript">
    var cityCount = 0;
    $(document).ready(function () {

        getCountry();

        /////////////Code Added/////////////////////
        $('#CountriesID').change(function () {

            getCity($('#CountriesID').val());
        });
        /////////////Code Added/////////////////////

    });
    function getCountry() {
        var url = '@Url.Content("~/Home/CountryList")/';

        $.getJSON(url, function (data) {
            var items = "<option>Select a Country</option>";
            $.each(data, function (i, country) {
                if (country.Value.indexOf("\'") > -1) {
                    s = country.Value + " " + country.Text;
                    alert(s + ": Country.Value cannot contain \'")
                }
                items += "<option value='" + country.Value + "'>" + country.Text + "</option>";
            });
            $('#CountriesID').html(items);
        })
    }

    /////////////Code Removed/////////////////////
    @*function getCity() {
        var url = '@Url.Content("~/Home/CityList")/' + $('#CountriesID').val();

        $.getJSON(url, function (data) {
            var items = '<option>Select a City</option>';
            $.each(data, function (i, city) {
                items += "<option value='" + city.Value + "'>" + city.Text + "</option>";
            });
            $('#CitiesID').html(items);
        });
    }*@
    /////////////Code Removed/////////////////////

    /////////////Code Added/////////////////////
    function getCity(countryId) {
        var url = '@Url.Content("~/Home/CityList")/' + countryId;

        $.getJSON(url, function (data) {
            var items = '<option>Select a City</option>';
            $.each(data, function (i, city) {
                items += "<option value='" + city.Value + "'>" + city.Text + "</option>";
                cityCount++;
            });
            $('#CitiesID').html(items);
        });
    }
    /////////////Code Added/////////////////////

</script>
<h2>Country</h2>

<div id="CountriesDivID">
    <label for="Countries">Countries</label>
    @*<select id="CountriesID" name="Countries" onchange="getCity()"></select>*@
    <select id="CountriesID" name="Countries"></select>
</div>   
<div id="CitiesDivID" >
    <label for="Cities">Cities</label>
    <select id="CitiesID" name="Cities"></select>
</div>

Now press F5 and you will see everything is working as it was before.

Step 10: Add code for Unit Testing. This is how the Script block will look like after the change:
<script type="text/javascript">
    var cityCount = 0;
    $(document).ready(function () {

        getCountry();

        /////////////Code Added/////////////////////
        $('#CountriesID').change(function () {

            //getCity($('#CountriesID').val());
            getCity('AU');
        });
        /////////////Code Added/////////////////////

    });
    function getCountry() {
        var url = '@Url.Content("~/Home/CountryList")/';

        $.getJSON(url, function (data) {
            var items = "<option>Select a Country</option>";
            $.each(data, function (i, country) {
                if (country.Value.indexOf("\'") > -1) {
                    s = country.Value + " " + country.Text;
                    alert(s + ": Country.Value cannot contain \'")
                }
                items += "<option value='" + country.Value + "'>" + country.Text + "</option>";
            });
            $('#CountriesID').html(items);
        })
    }

    /////////////Code Removed/////////////////////
    @*function getCity() {
        var url = '@Url.Content("~/Home/CityList")/' + $('#CountriesID').val();

        $.getJSON(url, function (data) {
            var items = '<option>Select a City</option>';
            $.each(data, function (i, city) {
                items += "<option value='" + city.Value + "'>" + city.Text + "</option>";
            });
            $('#CitiesID').html(items);
        });
    }*@
    /////////////Code Removed/////////////////////

    /////////////Code Added/////////////////////
    function getCity(countryId) {
        var url = '@Url.Content("~/Home/CityList")/' + countryId;

        $.getJSON(url, function (data) {
            var items = '<option>Select a City</option>';
            $.each(data, function (i, city) {
                items += "<option value='" + city.Value + "'>" + city.Text + "</option>";
                cityCount++;
            });
            $('#CitiesID').html(items);
        });
    }
    /////////////Code Added/////////////////////

    ////////////////////Test Code/////////////////////////
    test("Perform Country onchange", function () {
        $('#CountriesID').trigger("change");
        alert("Check No. of cities from Australia");
        equals(cityCount, 3, "The no. of cities should be 3");
    });
    //////////////////////////////////////////////////////

</script>
And after the last </div> tag we add:
<div>
    <h1 id="qunit-header">QUnit example</h1>              
    <ol id="qunit-tests"></ol>
</div>
Now press F5 once again, browse to CascadingList and you will see we are actually triggering the onchange event of the Country select box with predefined value to run Unit Test successfully. We have added an alert message just to make sure the effect of onchange gets sufficient time to load.

So, we have successfully implemented Unit Test on JavaScript written within our ASP.Net MVC application. Happy QUnit-Testing.

But for a regular ASP.Net developer, ASP.Net MVC and especially the Razor View Engine (in MVC3/MVC4) sometimes confusing and they get more confused when they are asked to create Unit Tests on top of it. This article describes some tools and mechanisms that can help the developers creating successful Unit Test on their MVC applications.

1 .Let’s first talk about the built in Unit Test Project Template available within Visual Studio. A good example of how to use it for MVC can be found here:

http://www.asp.net/mvc/tutorials/older-versions/unit-testing/creating-unit-tests-for-asp-net-mvc-applications-cs

It’s possible to use this type of Unit Test to test the output of different controller methods. As given in the example, it can be the name of the view got returned, Redirect Action or you can even check the data that is available within the Model associated with the View.

2. Similar but arguably better option can be using NuGet’s NUnit Template:

http://visualstudiogallery.msdn.microsoft.com/7f3bdc82-f418-41aa-ad15-f1d6f89df47a

NUnit has a mocking framework built-in. Also it can run beyond IDE. Which is helpful if you are using a Non-MS Build Server.

3. In both the above cases, usage of ReSharper (http://www.jetbrains.com/resharper/download/) will give additional advantages.

4. You can also take advantage of an independent mocking framework like Moq (http://code.google.com/p/moq/). Here is one such example: http://www.codethinked.com/Simplified-Aspnet-MVC-Controller-Testing-with-Moq.

5. All the tools we discussed above will give you options to test methods in Controller classes. Sometimes, you may need to Test a View or Partial View that is generated as output. NuGet’s RazorGenerator can help you do so. RazorGenerator allows processing Razor files at design time instead of runtime. Here are the links for RazorGenerator:

http://razorgenerator.codeplex.com/

http://nuget.org/packages/RazorGenerator.Mvc

http://nuget.org/packages/RazorGenerator.Testing

http://blog.davidebbo.com/search/label/RazorGenerator

6. Last but not the least, QUnit can help you to test the JavaScript and JQuery codes written within your view files:

http://nuget.org/packages/QUnit-MVC

http://www.dotnetcurry.com/ShowArticle.aspx?ID=400

We have a good number of guidance articles and documents on SharePoint 2007 to SharePoint 2010 migration and also on classic to claim migration for SharePoint 2010 in both Technet and MSDN. But there is no step by step guide to follow if someone wants migrating their existing SharePoint 2007 content with existing users to the cloud and implement ADFS for the same.

Let’s consider the following situation – Fabrikam Inc. already has a Team Portal running on SharePoint 2007 with AD (NTLM) based users. These users also have their individual Mysites and profiles. Contoso Inc. a sister concern of Fabrikam Inc. recently purchased a hosting solution from 3^rd party where they have options to host SharePoint 2010, SQL 2008 R2 etc. As there are some servers are available to use, Fabrikam Inc. wants to migrate their current SharePoint 2007 content into those servers. They have options to replicate their DC on this cloud environment but due to security risk they would prefer to use ADFS 2.0. They have already set up ADFS 2.0 server locally for this purpose. They also did a pilot migration with a test web application. They successfully transferred the web app from SharePoint 2007 to SharePoint 2010 and also published it into the cloud environment. But later they realized that the users cannot authenticate into the system as SharePoint considers same user coming from AD and from ADFS as two different users. It’s also impossible to find out the deep down complex permissions each users has in current environment. So they need a guidance to achieve this goal.

Let’s discuss the entire process in step by step approach.

2 Stage 1- from SharePoint 2007 to SharePoint 2010 migration

We have extensive guidance document on Technet/MSDN on this. Here is the list of articles including guidance on user profile migration we followed for the demo environment:

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=3837

http://technet.microsoft.com/library/cc263447(office.14).aspx

http://technet.microsoft.com/en-us/library/cc263299.aspx

2.1  Implement SSL

After migration to SharePoint 2010 we have demo web application running on classic NTLM based authentication. Next, we implemented SSL (required for ADFS) for both the demo web application and the web application for Mysite. We have DNS entries for both these web applications. Here is how the AAM entries look like:

2.2  Test User Access

We checked with random users and found their permission/access, Mysite is running well. Here is one of the permission screens:

And here is a profile view of one of the users:

3  Stage 2- AD to ADFS migration On-Premise

It’s advised to have a content database backup for both the web application at this stage as following steps are irreversible.

As the first step to migrate users from AD to ADFS, we need to transfer both the web application from Classic to Claims. Here are the PowerShell commands used for migration of both the web applications:

$WebAppName = "https://sp.fabrikam.com"

$account = "fabrikam\spadmin"

$wa = get-SPWebApplication $WebAppName

Set-SPwebApplication $wa -AuthenticationProvider (New-SPAuthenticationProvider) -Zone Default

$wa = get-SPWebApplication $WebAppName

$account = (New-SPClaimsPrincipal -identity $account -identitytype 1).ToEncodedString()

$zp = $wa.ZonePolicies("Default")

$p = $zp.Add($account,"PSPolicy")

$fc=$wa.PolicyRoles.GetSpecialRole("FullControl")

$p.PolicyRoleBindings.Add($fc)

$wa.Update()

$wa = get-SPWebApplication $WebAppName

$wa.MigrateUsers($true)

==============================================

$WebAppName = "https://mysite.fabrikam.com"

$account = "fabrikam\spadmin"

$wa = get-SPWebApplication $WebAppName

Set-SPwebApplication $wa -AuthenticationProvider (New-SPAuthenticationProvider) -Zone Default

$wa = get-SPWebApplication $WebAppName

$account = (New-SPClaimsPrincipal -identity $account -identitytype 1).ToEncodedString()

$zp = $wa.ZonePolicies("Default")

$p = $zp.Add($account,"PSPolicy")

$fc=$wa.PolicyRoles.GetSpecialRole("FullControl")

$p.PolicyRoleBindings.Add($fc)

$wa.Update()

$wa = get-SPWebApplication $WebAppName

$wa.MigrateUsers($true)

======================================================

After migration you may find users cannot access their individual Mysites. A browser refresh will help after receiving the 401 error screen to log them in.

3.1  Introduce ADFS 2.0 as Authentication Provider (Trusted Identity Provider)

We implemented ADFS 2.0 to both the web applications. Here is the PowerShell script we used in SharePoint server:

$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("C:\share\fabrikamcert.cer")

New-SPTrustedRootAuthority -Name "Token Signing Cert" -Certificate $cert

 $map = New-SPClaimTypeMapping -IncomingClaimType http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress -IncomingClaimTypeDisplayName "EmailAddress" -SameAsIncoming

 $map2 = New-SPClaimTypeMapping -IncomingClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/role -IncomingClaimTypeDisplayName "Role" -SameAsIncoming

$realm = "urn:sp:fabrikam"

$ap = New-SPTrustedIdentityTokenIssuer -Name "Fabrikam ADFS Provider" -Description "SharePoint secured by SAML from Fabrikam" -realm $realm -ImportTrustCertificate $cert -ClaimsMappings $map,$map2 -SignInUrl https://fabrikam-adfs.fabrikam.com/adfs/ls -IdentifierClaim "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"

$uri = new-object System.Uri("https://mysite.fabrikam.com/")

$ap.ProviderRealms.Add($uri, "urn:mysite:fabrikam")

$ap.Update()

iisreset

As you can see above, we have used email address and role as incoming claim type. This will help us to identify each user and security groups. Also, we need to create two Relying Party Trusts in the ADFS server -one with identifier (“urn:sp:fabrikam” and "https://sp.fabrikam.com/_trust/") and other with (“urn:mysite:fabrikam” and https://mysite.fabrikam.com/_trust/).

Now we went ahead and add this new identity provider to both the Web Applications.

3.2  Migrate users from AD to ADFS

Let’s add one user and a security group coming from ADFS as members of one of the site collections to check the prefix added by claims. This is required to run the next command in PowerShell.

Now we deleted both the user and the group created in last step.

We created a ps1 file with following content and run it to change AD users to ADFS users for the demo web application. Make sure you keep at least one of the site collection admin ID from each site collection out of transformation in the code. This is just to keep an option open to login using Normal AD prompt in case something goes wrong:

$groupprefix = "c:0-.t|fabrikam adfs provider|"

$userprefix = "i:05.t|fabrikam adfs provider|"

$usersuffix = "@fabrikam.com" 

# Get all of the users in a web application

$url = "https://sp.fabrikam.com"

$users = Get-SPUser -web $url

# Loop through each of the users in the web app

foreach($user in $users)

{

    # Create an array that will be used to split the user name

    $a=@()

    $displayname = $user.DisplayName

    $userlogin = $user.UserLogin

    $username = “”

    if($userlogin.Contains("i:"))
#for users

    {

        $a = $userlogin.split('\')

        $username = $userprefix + $a[1] + $usersuffix

    }

   
elseif($userlogin.Contains("c:")) #for groups

    {

        $a = $displayname.split('\')

        $username = $groupprefix + $a[1]

    }    

   
if(!$userlogin.Contains("spsitecoladmin1")) #leave one of the site collection admin

    {

                #Write-Host $username         

                Move-SPUser –Identity $user –NewAlias $username -ignoresid -Confirm:$false

    }

}

3.3  Change User Profile Sync Connections

Now we removed the AD connections for User Profile Sync and added a new connection to use ADFS connection:

Also map the user properties correctly:

Next we started a full synchronization to get the user profile through. After the sync was over, we tested log-in using one of the user accounts using ADFS login prompt. User movement from demo web application to Mysite is seamless now as ADFS provide us with the WebSSO. Logout is still an issue as logging out from one Web App in the same browser window does not mean logging out from the other. We will discuss about an option to avoid this problem little later.

4   Stage 3- Migration to Cloud

Now our web applications are ready to move to cloud. We stopped SQL Server service and copied .mdf and .ldf files for both the web applications’ content database. We brought those to the SQL Server at Contoso and attached there:

4.1 Setting up the Web Applications

We then created 2 web applications with SSL and following AAM (although I used same port 7171 and 8181 for easiness of tracking but that is not mandatory):

Now we mounted the databases to the web applications after testing them:

Test-SPContentDatabase -Name "WSS_Content_7171" -WebApplication "http://appcentral:7171"

Test-SPContentDatabase -Name "WSS_Content_8181" -WebApplication "http://appcentral:8181"

Mount-SPContentDatabase -Name "WSS_Content_7171" -DatabaseServer "SQL" -WebApplication "http://appcentral:7171"

Mount-SPContentDatabase -Name "WSS_Content_8181" -DatabaseServer "SQL" -WebApplication “http://appcentral:8181”

Also enabled Self-Service Site Creation option in the Web application for Mysite (8181) and checked whether managed path “/personal” is available or not.

4.2  Enabling ADFS

We used following command in the SharePoint Management Shell to enable ADFS:

$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("C:\share\fabrikamcert.cer")

New-SPTrustedRootAuthority -Name "Token Signing Cert" -Certificate $cert

$map = New-SPClaimTypeMapping -IncomingClaimType http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress -IncomingClaimTypeDisplayName "EmailAddress" -SameAsIncoming

$map2 = New-SPClaimTypeMapping -IncomingClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/role -IncomingClaimTypeDisplayName "Role" -SameAsIncoming

$realm = "urn:sp:contoso"

$ap = New-SPTrustedIdentityTokenIssuer -Name "Fabrikam ADFS Provider" -Description "SharePoint secured by SAML from Fabrikam" -realm $realm -ImportTrustCertificate $cert -ClaimsMappings $map,$map2 -SignInUrl https://fabrikam-adfs.fabrikam.com/adfs/ls -IdentifierClaim "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"

$uri = new-object System.Uri("https://mysite.fabrikam.contoso.com/")

$ap.ProviderRealms.Add($uri, "urn:mysite:contoso")

$ap.Update()

Iisreset

As shown above we used same name for the “SPTrustedIdentityTokenIssuer” as in step 3.1. Also we have 2 additional Relying Party Trusts at the ADFS server at Fabrikam:

 Also we added the new Trusted Identity provider for both Web Applications:

4.3  Finally

We then configured the User Profile Sync connection with Fabrikam ADFS server as shown in step 3.3. Now we went ahead and checked user access:

So, we were successful in migration of both the demo web application and the Mysite web application to Cloud environment and we also modified the users to use ADFS based authentication. WebSSO between these two web apps are available due to ADFS. We can additionally implement a Single Sign out option to make sure the users got sign-out from both the Web application simultaneously. There is a codeplex solution for the same. The Visual Studio solution, SingleSignOutModule in the 10SharePoint folder from http://claimsid.codeplex.com, includes a custom HTTP module to deploy to your SharePoint web application that includes this functionality.

5 Annexure

For the current setup we used SharePoint 2010 with June 2011 CU for both the environment (in Fabrikam servers and in Contoso servers). I used SQL Server 2008 R2 for both the cases.

5.1 Credits

Took help from following resources:

http://technet.microsoft.com/en-us/library/gg251985.aspx

http://technet.microsoft.com/en-us/library/ff607729.aspx

http://blog.sharepoint-voodoo.net/?p=68

http://technet.microsoft.com/en-us/library/cc263299.aspx

http://msdn.microsoft.com/en-us/library/hh446525.aspx

http://blogs.technet.com/b/speschka/archive/2010/04/27/how-to-create-multiple-claims-auth-web-apps-in-a-single-sharepoint-2010-farm.aspx

http://marcvaneijk.wordpress.com/2010/06/12/sharepoint-2010-and-adfs-2-0-the-complete-step-by-step-guide/

http://blogs.technet.com/b/speschka/archive/2010/07/30/configuring-sharepoint-2010-and-adfs-v2-end-to-end.aspx

5.2  Caveats

In step 3.3 when we changed the User Profile Sync connections from AD to ADFS, the images uploaded in the profile for the users disappear. This is because the User Profile data refreshes the content from new connections and old data got deleted. There can be alternatives to get the photos back in the User Profile although I didn’t go for it.

Moreover, we may also take backup of the User Profile DBs. This will ensure not losing the data user entered in user profile other than the data received from the AD. Because of the extra time and effort I didn’t venture the path.

What I found after investigation is:

• When a SharePoint Workspace Meeting is initiated by a presenter, it creates a meeting workspace site under given URL and adds presenter with “Full Control” access and attendees with “Contribute” access.
• When an attendee (or a delegate) accepts or decline a SharePoint Workspace meeting from outlook, a web service call to “/_vti_bin/meetings.aspx” of the Workspace site is initiated from outlook. This call does not routed through Exchange.
• When this web service call initiated, outlook sends the credential of the current user to SharePoint. If the user has “Contribute” access to the SharePoint site, it will successfully update the meeting acceptance.
• Essentially a delegate, as he has no access to the SharePoint site cannot change the acceptance status although he has full access to attendee’s calendar.

Let’s look into it step by step:

We have used 3 servers (VMs) with latest software for the PoC with following config:

• Server1: Windows Server 2008 R2, Exchange 2010, Office 2010
• Server 2: Windows Server 2008 R2, SharePoint 2010, Office 2010
• Server 3: Windows Server 2008 R2, SQL 2008 R2, Office 2010 

Step1

User (Administrator) starts a SharePoint workspace meeting and invites User (pranabp).

Step2

Once invitation is sent we can go and see the permissions available in the target meeting workspace site.

Step3

The user (pranabp) has delegated the calendar to user (arthuro) and “arthuro” can accept/decline meeting request on behalf of “pranabp”.

Let’s look at the fiddler trace during the meeting acceptance of “arthuro” on behalf of “pranabp”. As we can see outlook process is directly calling SharePoint web service. Also the call is failing because of permission issue.

Step4

Also if we see the outcome in the SharePoint “Attendees” list, we can see the Response field is not updated.

Step5

Now let’s create another workspace and this time we will add “arthuro” as contributor in the workspace site manually once it is created.

Step6

We will continue and again accept the meeting from the calendar of “arthuro” on behalf of “pranabp”.

Step7

Let’s look at the fiddler trace now. It’s a success. Also the SharePoint “Attendee” list shows accepted by user “pranabp”.

So, in the conclusion we can say, we need to have the delegated users as contributors to the SharePoint Workspace site. As you can see, it’s highly improbable that the user who is inviting other users for a meeting will have much understating about their calendar delegation.

Well, we do not have any straight-forward solution for this situation. There can be multiple workarounds - let’s hope someone the reader community of this blog post can point out such an workaround.

Your CTO may not approve it. Hence, the second choice, Import User Profile Photos from Active Directory. Here is a good article about it:

http://www.tcscblog.com/2010/11/18/import-user-profile-photos-from-active-directory-into-sharepoint-2010/

Wait a little. You got shocking news from the Server Admin team. They do not keep thumbnails (user’s photo) in AD. Instead of that, all the users’ photos (snaps taken during assigning them with an access/identity card) are dumped into a file server. And the references are available in some obscure database.

What to do now? Don’t worry we have a solution. Let’s discuss it:

Let’s say the DB maintains at least 3 fields:

EMP ID       User Name               Photo Name

1001           donf                      1001.jpg

1002          tomg                      1002.jpg

First of all create an IIS site and copy all the photos within the root file location of the site so that you can browse the photos like:

http://somecomputername/1001.jpg

or may be,

http://somecomputername/photos/1001.jpg

Just to avoid conflict give read permission to all users to the physical folder where the photo images are located.

Let’s create a csv file out of the DB. The structure of the csv file goes like:

emp_id,domain_user_name

http://somecomputername/1001.jpg,donf

http://somecomputername/1002.jpg,toms

Now create a Powershell script file (PS1) like this:

===============================================================

[void][system.reflection.assembly]::loadwithpartialname("Microsoft.Office.Server.UserProfiles")

$csvFile = "D:\photo.csv"

$MySiteUrl = http://mysite.contoso.com/

$site = Get-SPSite $MySiteUrl

$context = Get-SPServiceContext $site

$profileManager = New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager($context)

$csv = import-csv -path $csvFile

foreach ($line in $csv)

{

$user_name = "contoso\" + $line.domain_user_name

$up = $profileManager.GetUserProfile($user_name)

if($up)

{

$up["PictureURL"].Value = $line.emp_id

$up.Commit()

write-host $user_name,"--->",$up.DisplayName,"--->",$line.emp_id

$up = $null

}

}

=======================================================================

Run the PS1 file from Powershell.

Now if you go to each user’s profile you can see the images are available but if you hover you can see it’s coming from the location “http://somecomputername/1001.jpg”.

Next step is, give SharePoint a free hand to resize those photos into 3 different sizes. Run this command in Powershell window:

Update-SPProfilePhotoStore -MySiteHostLocation https://mysite.contoso.com/

Now most of the photos must got copied into User profile DB. If you still see the photo url’s pointing to “http://somecomputername/1001.jpg”, just go ahead and change your csv and make the emp_id from “http://somecomputername/1001.jpg” to the URL where Mysite is storing the photos (just check with any photo which pointing to mysite correctly).

Run the PS1 a few times and “Update-SPProfilePhotoStore -MySiteHostLocation https://mysite.contoso.com/” as well.

This should do the job.

But also update your CTO, this is a temporary solution and saving thumbnail in AD is a better solution any day.