Here is the link: https://go.microsoft.com/fwlink/p/?LinkId=260923

As most of you have seen by now that Visual Studio has added capability to execute unit test cases for SharePoint 2010. The details of how it can be achieved are available in this nice video: https://channel9.msdn.com/Blogs/adebruyn/Visual-Studio-SP1-and-SharePoint-2010-Unit-Test

Summary of steps:

1. Install Visual Studio 2010 Service Pack 1. You can only write Unit Tests for SharePoint in VS after installing SP1
2. Create a new C# unit test project
3. In Project Properties change Application ->Target Framework to 3.5 and Build -> Platform Target to x64
4. In Test - Edit Test Settings - local testsettings - Hosts tab : change run tests setting to "Run in 64 bit process on 64 bit machine"

Making these settings changes helps us to run the 64-bit SharePoint code in unit tests created using VS 2010 SP1. Visual Studio 2010 SP1 enables the IntelliTrace debugging technology on 64-bit Microsoft SharePoint solutions.

Once you have written the unit test cases, next thing is to see how much code coverage we are getting by running these unit test cases.  Enabling Data Adapters is not supported with tests that target .Net Framework 3.5 in Visual Studio. Therefore code coverage data adapter will not work from VS as SharePoint projects target .NET 3.5. Details available at VS Test team blog: https://blogs.msdn.com/b/vstsqualitytools/archive/2010/12/19/how-to-re-target-unit-tests-to-net-framework-3-5-in-vs-2010-sp1.aspx

The only solution is to run the tests and coverage from command line tools. This involves instrumenting the libraries for which code coverage has to be checked. The details of this process has been nicely documented at https://blogs.microsoft.co.il/blogs/royrose/archive/2011/08/30/manually-configure-and-run-code-coverage.aspx

I'm summarizing the steps here. Ensure you run 64 bit versions of these utilities. This is something which took me some time to figure out. These are available at  <VSDir>\ Team Tools\Performance Tools \x64

• Instrument the dll's containing code to be tested and for code coverage has to be calculated. You will have to re-sign your assembly if original assembly is signed

VSInstr.exe "AssemlbyFile.dll" /coverage

• Start the VSPerfCmd tool to start logging the coverage

vsperfcmd /start:coverage /output=Custoomfileforcoverage.codecoverage

•  Run the automated tests using MSTest utility

mstest /testcontainer:UnitTests.dll

• Stop logging coverage

vsperfcmd /shutdown

Now you can open this coverage file in VS and analyze the code coverage results.

Happy Coding !

https://blogs.msdn.com/b/taj/archive/2011/01/11/site-collection-backup-restore-and-managed-metadata.aspx

https://blogs.msdn.com/b/taj/archive/2010/10/20/import-spmetadatawebservicepartitiondata-and-bulk-load-problem.aspx

When I posted that blog post, I did all my tests on a single server deployment. Recently my colleague was trying to run the same commands on a multi-server deployment. He was using following command to import TermStore:

Import-SPMetadataWebServicePartitionData -Identity $mms2.Id -ServiceProxy $mms2proxy -path "C:\ mmsdev.bak" -OverwriteExisting;

He got following error message on running the command:

Import-SPMetadataWebServicePartitionData : Cannot bulk load because the file "C:\99f6833d2bac4c53af26b816afca1d55\ECMGroup.dat" could not be opened.

Operating system error code 3(failed to retrieve text for this error. Reason: 15105).

At line:1 char:41

+ Import-SPMetadataWebServicePartitionData <<<<  -Identity $mms2.Id -ServiceProxy $mms2proxy -path "C:\Veda\mmsdev.bak" -OverwriteExisting;
+ CategoryInfo          : InvalidData: (Microsoft.Share...cePartitionData:   SPCmdletImportM...cePartitionData) [Import-SPMetadataWebServicePartitionData], FaultException'1 + FullyQualifiedErrorId : Microsoft.SharePoint.Taxonomy.Cmdlet.SPCmdletImp   ortMetadataWebServicePartitionData


Operating system error code 3 means system is not able to locate the file at path mentioned in command. This command uses BULK UPLOAD functionality and it's not able to find the file at specified path when using BULK Upload from SQL Server database. I reached this conclusion as it was working fine on single box deployment. Thereafter I specified network path to the TermStore backup file instead of local path. I started getting a new error (Operating system error code 5). This is an error code for access denied which meant that code is not allowed to access the file location. Since BULK Upload command is run by SQL Server Service account, I tried to give access to that account. I kept getting the same error message even if I gave Everyone group access to the share. I'm still not able to find the reason why it's giving access denied.

To work around, instead of keeping file on Application Server, I kept it on SQL Server box and used the network path to the file in command. You have to give read write access to this share as PowerShell creates temporary files while importing TermStore. After this the command worked as expected and we were able to import TermStore succesfully. I hope this post will help you save some time.

Wherever you restore the site collection, Managed Metadata Service should be available. Managed metadata columns consult the MMS service whenever we interact with data stored in these columns. If MMS is not there things can break and show unexpected behavior. The easiest option to make things work is to restore the termstore from source farm to target farm.

You can use Export-SPMetadataWebServicePartitionData to export term store from source application.https://technet.microsoft.com/en-us/library/ff607847.aspx. You can import  the same in target MMS using Import-SPMetadataWebServicePartitionData https://technet.microsoft.com/en-us/library/ff607667.aspx. By using this technique the term store is available on the target. The sspId used internally by the Termstore and referred by Managed metadata columns in list is retained.

Go ahead and test restored site collection in target Farm. You should find things working properly. In my random testing everything seems to be working fine. Please report issues, if you see any.

UPDATE

Hari requested me to put a sample as the API name is misleading. The service doesn't need to be partitioned for things to work. It works for un-partitioned  service application also. Service Application Proxy ID has to be passed for -Identity parameter. Example given below and hope it helps.

#Export

$mmsApp = “4a867ce5-d9ee-4051-8e73-c5eca4158bcd”; #this sets the exporting MMS ID

$mmsproxy = Get-SPServiceApplicationProxy | ?{$_.TypeName -eq "ExportTaxonomyProxyName"};

Export-SPMetadataWebServicePartitionData -Identity $mmsApp -ServiceProxy $mmsproxy -Path \\location\exportfile.bak;



#Import 

$mms2 = "d045d3ce-e947-4465-b039-0dfbbe24fb22"   #this sets the importing MMS ID 

$mms2proxy = Get-SPServiceApplicationProxy | ?{$_.TypeName -eq "ImportTaxonomyProxyName"};

Import-SPMetadataWebServicePartitionData -Identity $mms2 -ServiceProxy $mms2proxy -path \\location\exportfile.bak -OverwriteExisting;

https://technet.microsoft.com/en-us/edge/technet-radio-throttling-limits-and-business-connectivity-services-in-sharepoint-2010.aspx

The commands failed with the following error message “You do not have permission to use the bulk load statement”. This was kind of surprising as I was logged in as box admin who is SQL admin also. I was having the appropriate permissions and it should have just worked. There are few guys who used it and it just worked fine for them. The only difference in my environment is that I run my services with separate accounts and I try to mimic things as close to production with respect to security. I checked the error message and cause seem to be somewhere in SQL perms. I checked the permissions of my MMS service account and it was not having the SQL server BULK ADMIN role assigned to it. When I added my service account to this server role, things started working fine. Looks like the command uses bulk insert SQL command to import the terms in SQL DB. Since the service account was not having these permissions, it failed. I checked with a dev and this seems to be a known issue. 

So the workaround is to add service account to BULK ADMIN role when you want to import the term store and remove it after operation is complete.

1. Problem while running User Profile Import Job - This was the first issue I ran into and had to go back and manually edit the url connection for my central administration site. Todd Klindt has blogged about the problem and solution in his blog at  https://www.toddklindt.com/blog/Lists/Posts/Post.aspx?ID=223. This solution works but I'm not sure if it's the supported way to make things work. I haven't seen this as a bug in any KB. Since User Profile SA is a fragile thing, I always feel nervous while making such changes :-). But it has been working fine for me till now. Check with Microsoft support and get it verified.
2. Setting up Content Deployment Job Paths - This was little tough to figure out. I was getting error while setting up job path using the traget CA running on SSL. This was the configuration of my farm
   • It's a single box SharePoint Server 2010 install
   • My Central Admin is running on SSL, port 443. I have used IIS to generate self SSL and used it for the CA. CA is opening fine with no certificate security error message
   • My authoring site is at https://author and prod site is https://prod

    When creating the Content Deployment path, after specifying the source CA url and credentials to connect, while testing the connection I get the following error message - 'The remote Web service request failed with this message : 'The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.'.

The error indicates that there is some problem with cert. I checked that Issuer cert is there in the local trusted CA store. Since CA is opening fine, it means system is trusting the cert properly and there should not be an issue. When I added the Issuer cert to SharePoint trust relationship by using Manage Trust options under Security in CA, everything started working fine and the connection to the CA was succesfully setup. This was surprising as I always thought Manage trust option was to setup cross farm trust but in this case even in single Farm setup I had to do that.

I searched on the web and found a KB which was similar to my problem and seem to be the culprit for this issue also https://support.microsoft.com/kb/982982. I din't try the first option in KB which I believe should also work.

Problem Statement

Many organizations have requirement to implement publishing environment such that production site is not edited by the end user. They want to implement staging-production pattern in which first the content is created on staging environment, reviewed and approved. Once approved, this content is moved to production environment where it’s visible to end users in read only mode. Neither content creators, nor viewers should be able to change content. Content Deployment feature in SharePoint 2010 helps achieve this objective of moving content from authoring site to production site.

One of the new features in SharePoint in social space is tagging and note boards. This let people tag the pages, collaborate and discuss about content of sites. One of the requirements of Content Deployment is that you can’t change the target of deployment. It’s only 1-way. Content can only be created on authoring site and no edits should be done on production. If you try to change content on production, it can lead to unexpected results including deployment job failure. The problem now becomes is how to let people tag on production site without any issues. I tried to mpelmet this setup and analyze the behavior of SharePoint2010.

*This is not a formal guidance from SharePoint team or Microsoft Corporation*. I personally feel its a valid implementation and you should check with Microsoft Support before using it in production.

Pre-requisites

Following is the setup of my SharePoint farm

1.  Central Administration running on https://moss02.taj.local. It’s on SSL
2. Single Farm with both the authoring and production farm
   1. Authoring Farm Url: https://author. Publishing Site template
   2. Production Farm Url: https://prod. Empty Site, no template applied to the site
3. User profile Application configured and profile synchronization working
   1. Profile Database: UserProfile
   2. Profile Sync Database: UserProfileSync
   3. Social Database: SocialTagging 

Setting up Content Deployment

You can follow the steps provided in following link to set up the content deployment

https://technet.microsoft.com/en-us/library/cc627268.aspx 

Setting up User Profile Application and Synchronization

There is information available on TechNet to do this but it’s little spread across and confusing to understand. My fellow MCM Spence Harbar has done good job of documenting the whole process. I will recommend you use his article to setup profile synchronization https://www.harbar.net/articles/sp2010ups.aspx

Read-Only Behavior

In this step we tested how tagging doesn’t affect the content database. Tags go into social database and should not affect the DB. https://author to https://prod , go to the prod database in management studio. Go to the content database for your prod site. In my case it is WSS_CONTENT_PROD. Go to database properties, options, State and set the Database read-only property equal to true.

After running the content deployment successfully once from

What this will do is mark your content database as read only. Now one can’t add/edit content to the database. One of the new improvements in SharePoint is that it can make out when the database is in read-only mode and accordingly trim the UI so that actions only show options for read-only operations. This is cool stuff and when we refresh the https://prod  site, we will see the UI trimmed. Now our content deployment should be working fine and the site is in read only mode.

The next thing to check is the tagging control on top of page. Interesting thing is that it still shows up enabled. That means you can tag stuff on the page. This control connects to the Social DB and figures out that since it’s not in edit mode, it should be shown as enabled. Try to tag this page with a tag. I was successfully able to tag content. This demonstrates that there is no write operation performed on the content database when you use this feature.

The next thing to test was how content deployment will behave after this tagging on page. To test this, I made changes on the page in my https://author site. Approved the page and published it.

I went back to my WSS_CONTENT_PROD database and marked read-only attribute to false. This will again light up the edit actions on the https://prod site

After this I ran content deployment job to move the updated page to production. The job ran successfully. I ran multiple cycles with edits and this thing worked well.

Now if we go back to the production site, all works well and I can see the edits done in author on my prod site. And yes I was able to see my tags also on the page on top. This thing really works like a charm. These are same tags which were created after first run of content deployment.

Test in your environment and give your comments. I hope SharePoint team will officially support it.

Summary: Learn how to use SharePoint Server 2010 Business Connectivity Services (BCS) feature to search content in external system when the number of records in system are more than throttling limits defined for the system.

Introduction

Searching external systems with large number of records is a complex problem because on one end the BCS is trying to avoid bringing down the external system/SharePoint by limiting number of records which can be fetched but on other end it will not be able to crawl the system due to throttling. In this walkthrough, you will see you how properties can be configured on metadatamodel to chunk the calls for crawling the external system. The article will also show the changes in requirements for implementing search for External Systems.

Applies to: Microsoft SharePoint Server 2010

Contents

Overview

Search In BCS

Setup Server Environment

Setup BCS in Farm

Create Metadata Model

Setup Crawl

Test Search

Conclusion

Additional Resources

Overview

In this scenario, AdventureWorks database hosted in SQL Server 2008 is used as an External System containing the records to be crawled. A metadata model file is created with IdEnumerator and a SpecificFinder stereotypes implemented to retrieve the data for an External Content Type(ECT). Properties are configured in model to help BCS get the ids for external items to be crawled in batches while staying under the throttling limits.

Search In BCS

Changes have been made in the search functionality in BCS and IdEnumerator is no a requirement for implementing the crawl. BCS uses one of the following stereotypes for crawling the external system:

• Finder Method: If this stereotype is implemented and it has RootFinder property applied to it, then search will use it to crawl the external source.

<Method Name="ReadList">
              <Properties>
                <Property Name="RootFinder" Type="System.String"></Property></Properties>
              <Parameters>

...

• IdEnumerator:If no Finder method is found with RootFinder property applied to it, IdEnumerator is used to crawl the system.

In addition to above, SpecificFinder stereotype is used if the columns returned by it are more than what's returned by Finder/IdEnumerator.

There are other properties shown in Table 1 which affect the behavior of search on UI and how data gets crawled

Table 1-Metadata model properties for search

Set up Server Environment

Software Requirements

The list of software required to run this scenario is as follows:

• 1. Microsoft SQL Server 2008 x64
• 2. Microsoft SharePoint Server 2010

Create AdventureWorks Database

AdventureWorks sample database available on Codeplex is used in this scenario to represent the external system.

Setup BCS in Farm

First step in this scenario is to setup the BCS infrastructure in the farm. It inlcudes setting up BCS Service Application and Profile pages for the ECT.

Follow the following steps to setup the infrastructure:

• 1. Start the Business Data Connectivity Service in the farm
• 2. Create a Business Data Connectivity Service Application
• 3. Create a Web Application https://bcs.contoso.com and create a site collection on root site
• 4. Create a subsite with url https://bcs.contoso.com/profile. This will be used for hosting the profile pages for all ECTs.
• 5. Give rights on the AdventureWorks database and table Sales.SalesOrderHeader to the account used for accessing ECT

Create Metadata Model

SharePoint Designer 2010 (SPD) provides functionality to create ECTs and save or export them in a metadata model file but the IdEnumerator stereotypes cannot be modelled using SPD. Therefore this stereotype has to be modelled manually. To make things easier and get the basic metadata ready, it's  recommended to create the initial metadata model using SPD. This can be further edited using any xml editor/notepad to add the IdEnumerator or other methods types (which cannot be created using SPD).

In this scenario we will be using IdEnumerator to crawl the system. After getting the list of Id's identifying the external items, search will call SpecificFinder on each Id returned by the enumerator to index the content as the fields in SpecificFinder are more than what gets returned by IdEnuemrator.

LastIdSeen filter type is used to batch the calls to external system. LastIdSeen filter is used by SharePoint Search to pass the identifier of the last external item seen as part of call to IdEnumerator method. It passes this as the filter value while calling the IdEnumerator in a loop. The external system should implement the logic to return the records under the throttling limits by using the filter value as the point from where next set of items needs to be fetched. In this scenario, we are using the value 43658 as default value for filter. This gets used when IdEnumerator is called for first time and Search has yet not seen any Id(LastSeenId).

Snippet below shows method definition for a LastIdSeen Filter.

              <FilterDescriptors>

                <FilterDescriptor Type="LastId" Name="LastIdSeen">

                  <Properties>

                    <Property Name="UsedForDisambiguation" Type="System.Boolean">false</Property>

                    <Property Name="IsDefault" Type="System.Boolean">false</Property>

                    <Property Name="CaseSensitive" Type="System.Boolean">false</Property>

                  </Properties>

                </FilterDescriptor>

              </FilterDescriptors>

              <Parameters>

                <Parameter Direction="In" Name="@SalesOrderId">

                  <TypeDescriptor TypeName="System.Int32" IdentifierName="SalesOrderID" AssociatedFilter="LastIdSeen" Name="SalesOrderID">

                    <DefaultValues>

                      <DefaultValue MethodInstanceName="SalesOrderHeaderReadIds" Type="System.Int32">43658</DefaultValue>

                    </DefaultValues>

                  </TypeDescriptor>

                </Parameter>

Metadata snippet below shows IdEnumerator method. If no default value is passed, this gets the top 1000 rows else gets the records starting from the LastIdSeen parameter(SalesOrderId) passed to this method.

                <Method IsStatic="false" Name="SalesOrderHeaderReadListIds">

              <Properties>

                <Property Name="RdbCommandType" Type="System.Data.CommandType, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">Text</Property>

                <Property Name="RdbCommandText" Type="System.String">IF (@SalesOrderId &lt; 43658) BEGIN

SELECT TOP 1000 SalesOrderId, ModifiedDate FROM [Sales].[SalesOrderHeader] ORDER BY SalesOrderId ASC

END

ELSE

SELECT SalesOrderId, ModifiedDate FROM [Sales].[SalesOrderHeader] WHERE SalesOrderId BETWEEN @SalesOrderId AND (@SalesOrderId+1000) ORDER BY [SalesOrderId] ASC

 </Property>

File containing the metadata is available at the end of article as an attachment.

Setup Crawl

Once ECT has been defined, external data can be crawled and the items can be indexed by SharePoint search. Pre-requisites to enable crawling are:

• ECT created with Finder or IdEnumerator defined
• A SpecificFinder defined if the fields to be crawled are not available as part of Finder/Enumerator
• SharePoint Search Service started and Service Application created for search
• Content Source of type Line of Business Data defined with AdventureWorksForSearch model as external data source to be crawled
• Profile page created for SalesOrder ECT at https://bcs.contoso.com/profile
• Ensure that the crawl account has rights to connect to AdventureWorks database and read permissions for content of site https://bcs.contoso.com/profile

Test Search

Once all of above is done, start a full crawl of the content source and ensure that content is crawled succesfully by verifying into the crawl log.

Search for any SalesOrder which is in present in the system document and verify if the results are returned for that. In our example, search for term "SO43659" and you will get one result from external system as shown. The result item url is the profile page used to show the ECT profile. On selecting the link in search result, system navigates you to the profile page and show details of the external item.

Conclusion

BCS Search has been simplified and now items can be easily crawled by using SpecificFinder stereotypes. In addition using throttling features, governance can be imposed to ensure that external system and SharePoint doesn't get overhauled by users requests.

Model File