Symptom

Suppose you swear you know that there are some log related to the filter you have put - say a Correlation ID - but after you execute the following command:

Merge-SPLogFile -Path C:\temp\ErrorJob.log -Correlation "84FC629D-03AF-A09C-2AC2-C59A1650BB9E"

You receive the message

WARNING: Cmdlet did not return any records in the log file. Check your time range or filters.

And you just stare at the message knowing that it is a LIAR! How to teach it who's the Boss?

Reason

This is because normally Merge-SPLogFile haven't found any log in the most recent files (my assumption is that it only check the latest ULS files) so you need to provide the date filter.

Solution

Instead of just asking for the Correlation ID, provide a date filter, the narrower the time span is, the faster it will execute:

Merge-SPLogFile -Path C:\temp\ErrorJob.log -Correlation "84FC629D-03AF-A09C-2AC2-C59A1650BB9E" -StartTime "2/26/2016 12:00:00" -EndTime "2/23/2016 15:00:00"

Violà!

You now won't see the message and will have your beloved trimmed log.

Enjoy!

This time I wanted to post an article about the new concepts of self-service BI and how Excel plays the lead role in it.

The classic way

First of all, let´s go to the basis about the "classic" BI solutions I was involved back some years ago. To summarize all the elements that are involved I have this simplified diagram:

You have the following elements:

Data Sources
This is where all your original data is stored. Can be any kind of system or intermediate system where your soon-to-analyzed data is stored. From a relational database to a plain text report rendered by any system can be useful as a data source, as long as you have the information you need.

Extract, Transform & Load
Once you have detected your data, location, access, etc., you now need to consolidate this data. For that you need to change the structure of your data, like date formats, identifiers, cross information from different sources, etc. All these are summarized in ETL processes. From the Microsoft technology stack you have SQL Server Integration Services and Visual Studio Shell to create, test and debug this ETL processes. Since Visual Studio is such a powerful tool, you will need a high level technical profile in order to create ETL processes in a reasonable time. These processes usually are executed in background and their frequency determines the freshness of the data to be analyzed.

Data Warehouse
In the previous explanation I mentioned "consolidate", a data warehouse is where you do this. Another way to understand this is that you change the structure or the original data into something that you will easily create reports from. Someone once told me that 80% percent of the issues in BI solutions comes from a poor data warehouse design. So here is not only needed a good database designer but a good understanding of the business need and the information that needs to be analyzed. With Microsoft technologies you have SQL Server to build your data warehouse. Three words you will see very often here are: Facts, Measures and Dimensions. Just search over the internet to find some detailed explanation that is out of the scope of this article.

Multidimensional databases.
After you have all your data consolidated into one single place, you now have to prepare the data to be hammered by reports. In Microsoft technologies you have SQL Server Analysis Services for this. You create, using the Visual Studio Shell (also known as SQL Server Data Tools) multidimensional cubes from relational databases (i.e. data warehouse). You may want to think about multidimensional databases as full indexed tables, but the concepts are way more extended that just this.

Dashboards and reports
There wouldn´t be any analysis without reports, would it be? The final outcome for any BI solution are the reports and dashboards you create. Usually this is the first thing you have to define so you can design all the previous element in a consistent manner. Dashboards are the main reason of a BI solution. The idea of a dashboard is to have a 360 degree of what you want to know about so you can drill down through your reports (i.e. data), understand why and take the corresponding actions to steer your business. When speaking about Microsoft technologies you have SQL Server Reporting Services and in case you haven´t thought about it, Excel is also seen as a reporting tool!

So, this is just a really, really brief explanation of BI, the classic way. But, what about this new experience called self-service BI?

Self-service BI

Reading the pasts paragraphs you may infer that you will need a specialized team to build your BI solution, since there are at least 4 or 5 technologies involved. But the new concept tries to ease your way down to your long awaited dashboards through just one single (or should I say centralized?) tool: Microsoft Excel.

I hesitate to define excel as a single tool because Excel now consolidates a set of tools that are intended to give the end user a centralized experience when building reports or dashboards for data analysis. The tools are:

Power Query
This is a WYSISYG tool that helps you to connect to virtually any data source and transform the data into something you will need for further analysis. Example of data sources are: Oracle, OData feeds, SharePoint Lists, HTML web pages, Azure Marketplace, Facebook, SAP or Salesforce. With Power Query you can have a GUI tool to change the format of the data, split columns, remove rows and any action you may need in order to tune up your raw data.

PowerPivot
When speaking about modelling data, PowerPivot is a handy tool that actually runs sort of a SQL Server Analysis engine in your desktop without any installation needed. Once you have retrieved your data with Power Query, you now can define the relationship between your data, columns name set and tables with friendly names (this concept of "friendly" will be helpful in the future). This new modelled data will be saved along with your Excel Workbook in a compressed format so you can "travel" with this data. Some benefit to store data with PowerPivot is that once you run any pivot table / chart using PowerPivot data, the speed will be notoriously increased when managing medium size data. You can also save as much data as your RAM memory can hold. For this, it is recommended to use the 64bits version of Excel if you intend to you PowerPivot.

Power View
No data is relevant if you can´t see it (at least from a BI perspective). Besides your well known pivot tables and pivot charts, in combination with slicers, time scales, etc. You have now Power View, that enriches the experience of your reports. With Power View you can have richer reports that also features maps, where you can show your data related to a geographical location that can be seen in several ways.

Finally, the previous diagram with the "classic" data stages and tools for BI solutions you now have this equivalent diagram when speaking about Self Service BI:

As you can see, now the only thing that lives outside Excel is your data sources!

I hope this explanation gave you some light about the new concepts on our BI experience, in further articles I will try to explain how SharePoint also has something to say about this!

You have an Excel Workbook that consumes data from a SQL Server Database in your network, like this diagram:

In this case, you live happily ever after because your desktop is in the same network of your SQL Server, so you just create your data connection, input the server name, database, credentials and you are ready to analyze your data! But now let's pull you off your comfort zone and move your excel into a SharePoint Online site under an O365 tenant, then you will see things this way:

This change quite a bit, doesn't it? Now you only see a web page that looks like an Excel Workbook. You can even change some things in your published workbook. But if we are now working in the cloud, how you think your workbook will now connect to your on premise data? Here is where comes the Power BI published OData feeds. If you are curious enough, this would look like this (in a very, very, simplified way):

What will happen, is that your published workbook will connect to a published data source, that will connect to your on premise SQL Server data using a gateway.

Things to take in mind:

• You won't be able to refresh manually from the web page your workbook data. The refresh process is configured in the Power BI site, however, you can force the refresh on demand.
• You will need to setup things in your on premise environment, meaning to install the gateway in your SQL Server or in a place where it is accessible.
• At least one or your boxes in your environment needs access to internet.
• By the time this article has been published there are limited data sources that are supported for the data refresh.

So, from a high-level perspective, in order to consume on premise data form an excel workbook that is published in a Power BI site, you have to:

1. Create your workbook the old fashioned way, by creating your data connection from the excel workbook itself. Do not use Power Query nor the PowerPivot data connection tools.
   
2. Remember, when finish the connection wizard, to check the Add this data to the data model so it will create the PowerPivot data model.
   
3. Once you have your workbook showing your data, publish it to your SharePoint site.
4. In your Power BI Admin Center, follow the instructions to create and install a new Gateway from this article
5. Create a new data source, by copying the same connection string from your workbook, follow this article for more information.
6. Go to your Power BI site, the one that is accessible from the same site where you published your Excel Workbook. Usually you will see it in the Quick Launch, if you can't see it, try "Site Contents":
   
7. From the Power BI, enable Power BI feature from the ellipsis (...) menu in the bottom right of your just uploaded document.
   
8. Once enabled, the same menu will now show you an option "Schedule Data Refresh". You will be redirected to the settings pages for that workbook:
   
9. Setup the refresh schedule, and press the "Save and refresh now" button to force the refresh so you can test it!
   
   
   

Hope this helps, I haven't tested this with the others sources: Oracle, SharePoint Lists and Power Query... will try to find out more time for that!

This time, let's summarize a check list if you have to configure AD FS for a SharePoint 2013 environment. The basic steps are well described in the following TechNet article: https://technet.microsoft.com/en-us/library/hh305235.aspx I will just mention things that you should take care of before and during the configuration process. My first recommendation is to read both this and the mentioned article before starting your configuration.

We will have the following topology:

And the following web application setup:

Checklist for the preparations:

1. Certificates for the AD FS and for every web application (and extension) you create in SharePoint. For this example, we will use just mention one certificate in the form of *.contoso.com. If you don't know how to create a self signed certificate, download this and follow this instructions.
2. The certificates, has to be trusted by every server in SharePoint, AD FS farm and clients that will access the web application.
   1. For testing, just add the certificate in the Trusted Root Authority and Personal folders and get the thumbprint of the certificate (you will need it later in this list).
3. Plan your urls, as the above picture, you will need to know the URL for the AD FS service, lets say: fs.contoso.com.
4. Your DNS needs to have all the URL's pointing to the appropriate servers. Ping everyone and ensure that your AD FS URL points to your AD FS server, and so on.
5. I assume you already have your AD FS running. if you have to create yours, I would recommend Window Server 2012 R2 and have prepared a running SQL Server instance somewhere in your network, because the installation is as easy as this script (note the certificate thumbprint):

#

# Windows PowerShell script for AD FS Deployment
#
Import-Module ADFS
# Get the credential used for the federation service account
$serviceAccountCredential = Get-Credential -Message "Enter the credential for the Federation Service Account."

Install-AdfsFarm `
-CertificateThumbprint:"836CD9B85B16EDE2658D7560103B56CAFC9EAD6A" `
-FederationServiceDisplayName:"SPFarmCreator ADFS" `
-FederationServiceName:"fs.spfarmcreator.local" `
-ServiceAccountCredential:$serviceAccountCredential `
-SQLConnectionString:"Data Source=sql2\sharepoint;Initial Catalog=ADFSConfiguration;Integrated Security=True;Min Pool Size=20"

Checklist for the AD FS configuration:

1. When adding the relying party trust, ensure that you have the realm uri included in the identifiers for your RP, like this image: Keep that uri for later when configuring SharePoint.
   
2. When creating the rules, ensure you map the role claim, in my example I have just mapped the email, upn and role.
   
   
   
3. Be careful when exporting the signing certificate, in our case is simple but in production environments you may need to export any intermediate certificates.

Checklist when creating the trust identity provider:

1. Ensure you at least create the same three claim mappings, in the mentioned article you will include SID but I haven't in this simplified script to create it.

$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("C:\temp\ADFS.cer")

New-SPTrustedRootAuthority -Name "Token Signing Cert" -Certificate $cert

$emailClaimMap = New-SPClaimTypeMapping -IncomingClaimType "https://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress" -IncomingClaimTypeDisplayName "EmailAddress" –SameAsIncoming
$upnClaimMap = New-SPClaimTypeMapping -IncomingClaimType "https://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn" -IncomingClaimTypeDisplayName "UPN" –SameAsIncoming
$roleClaimMap = New-SPClaimTypeMapping -IncomingClaimType "https://schemas.microsoft.com/ws/2008/06/identity/claims/role" -IncomingClaimTypeDisplayName "Role" -SameAsIncoming

$realm = "urn:sharepoint:spfarmcreator"

$signInURL = "https://fs.spfarmcreator.local/adfs/ls"

$ap = New-SPTrustedIdentityTokenIssuer -Name "ADFS for Extranet" -Description "SharePoint secured by SAML" -realm $realm -ImportTrustCertificate $cert -ClaimsMappings $emailClaimMap,$upnClaimMap, $roleClaimMap -SignInUrl $signInURL -IdentifierClaim $emailClaimMap.InputClaimType

1. Remember to double and triple check the realm you are passing to the new trusted IP, it has to be the same identifier in the step 1 of the previous list.

Checklist when assigning the trusted IP (Identity Provider) to the web applications:

1. Remember that if you are modifying an existing web application, from the web application management list, click on the big ribbon button "Authentication Providers" and then click on the "Default" zone. You will now have the options you are looking for.
2. If this is the first time you are testing all of this, leave your web application with both Windows Authentication and AD FS, so you can still have access to your web application if your AD FS is not well configured. Once you have ensure AD FS works like a charm, uncheck the Enable Windows Authentication checkbox.
3. If you finally decide to just enable AD FS for the default zone of your web application, extend it and enable in the extended zone windows authentication. This is required for search, and in case AD FS doesn't work you still have a "back door".

Checklist when configuring User Profiles

1. Unless necessary, use the Lightweight option for synchronizing, not the FIM. If you don't know what is this about, read this.
2. Ensure you map the claim user identifier for the user profiles to be updated the right way. In my case I used the email for that, this article will explain how to it: Mappping User Profiles for SAML Users

Checklist when adding my sites:

1. You have to create a new Relying Party Trust for each web application in your farm that requires AD FS. This means you will need a new uri for your realm.
2. The realm also needs to be added to your Trusted Identity Provider, this simple script will do the trick:

$ti = Get-SPTrustedIdentityTokenIssuer "You identity name"
$uri = New-Object System.Uri("https://mysites.contoso.com")
$ti.ProviderRealms.Add($uri, "urn:contoso:mysites")
$ti.Update()

Checklist for Search:

1. As mentioned above, ensure you have at least one extension in you web application that allows Windows Authentication. So your crawler can happily index your content.
2. Change your content sources to map your extended url's.
3. If needed, add a mapping to your search application so it will render the public url (i.e. the default zone url) in the user's search results.

Checklist for using groups:

1. Ensure the group is a security group.
2. Ensure is global or universal.
3. When you grant access a group, ensure that the tooltip that will appear over the group name in the people picker says "Role", as they say, a picture is worth a thousand words:

Things that don't work like in AD old times:

1. You can't have Kerberos negotiation, this is because the Claim to Window Token Service only understands windows claims, not SAML-tokens. Consider using Secure Store Service with SSO.
2. People picker will not find the names and/or groups, you have to create a custom claim provider for that.
3. RSS webparts mapped to SharePoint feeds require authentication, since this can be done with Kerberos, with AD FS it will not work.
4. There might be some limitations with the size of the SAML token or at least from the traffic perspective. Since you map a claim that include the roles (user's membership), ensure that the user doesn't belong to a big amount of groups, nested or not.

Finally! I think this is it!

The high level steps are:

1. Install SSRS in SharePoint mode (and the add-in) in the farm server you want to be the Reporting Server.
2. Install the SSRS add-in in the remaining farm servers.
3. Activate the service instances.
4. Create the SSRS Service Application in your farm
5. Activate the PowerView feature in your site collections

Things you should take in mind:

Currently there is a bug in some ISO's where SQL Server 2012 includes the Service Pack 2. I recommend to download the service pack 2 itself as well and install it after step 2. I have done it after all the process, though.

• SSRS needs a SQL Server Instance, ensure both the SSRS and SQL Server have the same edition, same patch level.
• Execute the Install-SPRSService and Install-SPRSServiceProxy only on the server you want the role. Then you are good to start them.
• Ensure you have granted access the SSRS service identity to the web application by using this script:
  Get-SPWebApplication | % {$_.GrantAccessToProcessIdentity("your reporting service account")}
• Ensure the SSRS service identity has access to the PowerPivot Service Application. You do this by selecting the Service Application and managing the Permissions (the big button in the ribbon).
  
• Ensure Excel Service service identity has access to the SSRS Service, by the same way (from the service application permissions).
• Ensure the SSRS service identity has access to the SSAS PowerPivot instance

Currently I am in the middle of a deployment with PowerPivot in SharePoint 2013, for the sake of have as much new stuff as possible, at least from the SharePoint end. My scenario looks like this:

For the installation the high-level steps are:

1. Install a SQL Server Analysis Services for SharePoint in any server (doesn't need to be a farm server). This will install a SSAS instance named PowerPivot.
2. Install the PowerPivot add-in for SharePoint in all the farms.
3. Execute the PowerPivot configuration tool once in any of the servers.

So, the checklist itself to ensure your PowerPivot for SharePoint installation works smoothly:

Least-privilege user layout

If you want a very granular user account setup, you will need the following users:

• Setup: This will be local admin on every box in your farm.
• Excel Services: This can be shared for Visio Graph services as well for example.
• Unattended Account: For miscellaneous purposes, I would recommend one. This will be the user that will be impersonated when accessing external data through the "unattended account" in Excel, Visio, PerformancePoint, for instance. The idea of this user is to be different from any SharePoint managed account.
• Web Application Pool: For the intranet web application pool itself.
• SQL Server Service Account. SQL Server Analysis Services for SharePoint needs a domain account.
• Claims to Windows Token Service. This user needs to have the following privileges under the local policies/User rights assignments (secpol.msc): Act as part of the operative system, log on as a service, impersonate a client after authentication.
• Secure Store Service: This is just a recommendation, but a good one!
• Common Service Application Account: For example, for Managed Metadata, etc.

The mentioned users should be the minimal accounts. One of the main reasons for so many are the Kerberos delegation, you may want to just allow the Excel Service Account for Kerberos delegation, or the C2WTS (Claim to Windows Token Service) who needs specific privileges.

For SQL Server Analysis Services for SharePoint Installation

Ensure you install SQL Server 2012 with Service Pack 2, I've found an issue when trying to connect to the PowerPivot instance with the management studio that is solved in SP1/SP2. Follow the instruction on this article: Install Analysis Services Server in SharePoint Mode for SharePoint 2013 (SQL Server 2012 SP1). Two important points here:

• Add the excel service identity account as administrator in the PowerPivot instance, as well as the unattended (if you want one).
• The user that will execute the Analysis Service (PowerPivot) instance has to be a domain account.

For the SharePoint Servers Installation

• Ensure you have installed both the Microsoft® SQL Server® 2008 R2 ADOMD.NET and the Microsoft® SQL Server® 2008 R2 Analysis Management Objects from the Microsoft® SQL Server® 2008 R2 Feature Pack. Do this on every SharePoint server. (This may be not required in the future).
• Ensure the Secondary Logon service is up and running. See this for more information: https://msdn.microsoft.com/en-us/library/hh230899(v=sql.110).aspx
• Disable the loopback check on the web front end servers. TIP: I followed the article https://support.microsoft.com/kb/896861/en-us but the method 2 didn't work for me, only the method 1 (the one that creates the BackConnectionHostNames value). The catch on the first method is that you have to explicitly put the urls.

This, as mentioned earlier, have been tested over a Windows 2012 R2 installation.

Before executing the PowerPivot configuration tool

• Ensure you have the following Services Application up and running (with the users mentioned earlier)
  • Secure Store Service, with a master key already added
  • Excel Services
• For each web application, grant access to the Excel Service identity by using the following script:
  Get-SPWebApplication -IncludeCentralAdministration | % {$_.GrantAccessToProcessIdentity("your excel service account")}
• Ensure you have assigned the C2WTS user to the service under https://centraladminserver/_admin/FarmCredentialManagement.aspx, and started it on every SharePoint server in the farm.
• Add the PowerPivot instance to the excel service as mentioned in the article. Install Analysis Services Server in SharePoint Mode for SharePoint 2013 (SQL Server 2012 SP1). and wait for the job to add it. In my experience, sometimes it won't show any log unless you open a powerpivot file.

Testing the whole thing

There is a lot of ways to test this, but my way would be:

1. Test if a simple Excel Workbook can be viewed on your browser.
2. Create an Excel with PowerPivot data in it, with no external connections.
3. If you feel courageous enough, create a PowerPivot table from an external connection and test it. In this case ensure you configure the external data connection with the right  authentication settings.

For the Service Application:

• You have started at least one User Profile Service Instance
• You have included the service application in the connection list for the web applications, or put it in the default associations

For the Sync Service Instance

• You have the farm account included in the local Administrators group. You need this only in the server that will run the service instance.
• If you haven´t and you have just done it, restart the server

For the  Syncrhonization to be completed:

• Ensure you have created a connection
• When creating the connection, ensure the user you use to populate the tree in the connection form, has the right privileges. (https://technet.microsoft.com/en-us/library/hh296982(v=office.14).aspx#RDCdomain)
• Perform first a full synchronization.

For the my sites:

• Create the web application and a site collection in its root managed path (/) using the my site host template. (There are more options, but I leave this as the most common).
• The web application should have the "personal" wildcard managed path and the self service site creation enabled.
• At the user profile service application configuration page, ensure you have entered the web application url in the my site host.

For the pictures from AD

• Check that you have created the map for the picture profile property to the thumbnailPhoto AD attribute.
• If you haven't, do so and perform a full synchronization again. Check that you have pictures at the my site host site collection (by browsing into it's content).
• After that, if the pictures still does not appear, execute the Update-SPProfilePhotoStore command under the farm account credentials, or a user that has enough access to it or you will have a "Object reference not set to an instance of an object".

My scenario:

I have a publishing site collection in SharePoint 2010 with three labels: English and another two languages. Let's say that the English is the source label. After you update an existing page in the root label, after the execution of the job, it will update all the target labels. This is the same if you create a new page.

Migration process

The migration was done by using the Test-SPContentDatabase and Mount-SPContentDatabase explained in the following TechNet article:

https://technet.microsoft.com/en-us/library/cc263299.aspx

Unexpected behavior

If you update any page in the root label, it won't be reflected in any target, but the variation log reflects that the job actually started the update process! What happened?

Explanation

After some testing, I fully upgraded the site collection to SharePoint 2013 "Experience" using the following procedure.

https://technet.microsoft.com/en-us/library/jj219474.aspx

And noticed that the variations label settings page has totally changed from this:

to this:

Problem is that you can't change the values shown in the SharePoint 2013 page in "2010 mode".

Solution

I have prepared a little script so you can change it without the need to fully upgrade to SharePoint 2013 mode and give you some oxygen until you fully upgrade it!

$url = https://mysite.midomain.com

#
#Get the site collection
$rootsite = Get-SPSite $url

#
#Get the GUID of the variation label list
$guidlista = New-Object Guid($rootsite.RootWeb.AllProperties["_VarLabelsListId"])

#
#Get the list with the variation labels itself
$variationlabelsList = $rootsite.RootWeb.Lists[$guidlista]

#
#Retrieves all the label except the source one
$etiquetasdestino = $variationlabelsList.Items | ? {!$_["Is_x0020_Source"]}

#
#For each target label
$etiquetasdestino | % {

Write-Host "Updating label $($_.Title)"
$_["NotificationMode"] = $false
$_.Update()

}

 Enjoy!!

You want to install SharePoint 2013 in a server that has no connection to the internet. You have downloaded all your prerequisites according to the following articles:

https://technet.microsoft.com/en-us/library/ff686793.aspx
https://technet.microsoft.com/en-us/library/cc262485.aspx

If you want an easier version (in spanish) refer to this article by Matias Rama:

https://blogs.technet.com/b/pfespain/archive/2013/08/07/sharepoint-2013-instalaci-243-n-de-requisitos-de-software-y-v-237-nculos.aspx

Symptom:

When you run the prerequisite. The process fails while trying to activate the Web App role (IIS). This also happen if you manually try to add this role to the server.

Solution

The problem, usually, is because the binaries are in the CD, which might be in a different drive when installed. You need to specify manually where are your binaries.

When you clic on the image, check on the message:

"Source files for .net Framework 3.5 are not installed as part of a typical installation [...]"

Put the path

E:\Source\SxS

In the text box, press ok and add the role to the server. This shouldn´t last too much.

Now you are ok to continue to install all the prerequisites.