Insights

Developing Stream Analytics Jobs in Visual Studio

MutazAG — Tue, 30 May 2017 21:50:50 GMT

Stream Analytics Tools for Visual Studio came out a couple months ago (here) and today I got a chance to try it out to prep some work for a project I am on. After getting the tools and making sure I am ready to go I wanted to build and run the job locally, this required that I provide a local input file. My input in this case is stream from an event hub. If you already setup an input points to an event in the stream analytics job using Azure Portal, you can use the Input Details blade in the portal to download sample data:

Click on Sample Data button
Select date range for data sample and click OK
Wait ...
Once you get the notification that data is ready, click on the notification and download the data file.

Or, you can use Service Bus Explorer to prepare an events data file based on real events coming into the event hub:

Connect to Event Hub
Create a listener on the consumer group, I created a consumer group just for this purpose
On the events tab, select all events in the list
Right click on the selected events and click "Save Selected Events"

Now I needed to prep the saved file in a format that ASA project will accept as a valid local input, for that I processed the content of the file into a new one using the following code:


string filePath = ConfigurationManager.AppSettings.Get("eventfile");
string jsonData = File.ReadAllText(filePath);
var eventsArray = JArray.Parse(jsonData);
var streamwriter = File.AppendText(ConfigurationManager.AppSettings.Get("outputfile"));
foreach (var item in eventsArray)
{
streamwriter.WriteLine(item["body"]);
}
streamwriter.Flush();
streamwriter.Close();

Back to the Stream Analytics project in Visual Studio, Add a new item of type "Local Input" and point it to the output file with the processed content. With this, I can develop and test stream analytics jobs locally using real data captured from event hub.

Keep your own MachineKey values in SharePoint 2010 web.config

MutazAG — Tue, 29 May 2012 21:26:21 GMT

In some cases you may need to have specific MachineKey values in the SharePoint web applications web.config. SharePoint will accept this change briefly but will will soon change it to a different value midnight.

This timer job: Health Analysis Job (Daily, Microsoft SharePoint Foundation Web Application, All Servers) will override any modifications to the values in the web.config of web applications.

The timer job runs daily at mid night and will execute a number of health rule checks (for full list of health rule checks in the farm run Get-SPHealthAnalysisRule).

One particular health rule is responsible for ensuring that values are kept in synch across all web applications, the health rule name is ViewStateKeysAreOutOfSync.

The issue with this health rule is that it will revert the values of the to some values stored in the config db which stops us from using our custom machine key values.

The solution is to disable the ViewStateKeysAreOutOfSync health analysis rule by running the following PowerShell cmdlet:

Get-SPHealthAnalysisRule ViewStateKeysAreOutOfSync | Disable-SPHealthAnalysisRule

Also check this article for background information on the issue: https://blogs.msdn.com/b/allenwang/archive/2012/03/23/sharepoint-2010-health-analyser-timer-job-changed-custom-web-config-machine-key.aspx

cheers,
- mutaz

Add User Policy using Powershell

MutazAG — Mon, 28 May 2012 17:03:40 GMT

# This script will add to user policy for a web application in SharePoint 2010

# and will add few more things to get rid of this event in the event logs:

# Object Cache: The super reader account utilized by the cache does not have sufficient permissions to SharePoint databases.

# See, https://technet.microsoft.com/en-us/library/ff758656.aspx

Add-PSSnapin Microsoft.SharePoint.PowerShell -erroraction SilentlyContinue

function SetPolicy ($userOrGroup, $wa)

{

$user = New-SPClaimsPrincipal -IdentityType WindowsSamAccountName -Identity $userOrGroup

$policy = $wa.Policies.Add($user.ToEncodedString(), $userOrGroup)

$policy.PolicyRoleBindings.Add($wa.PolicyRoles.GetSpecialRole([Microsoft.SharePoint.Administration.SPPolicyRoleType]::FullControl))

}

$superUserAccount = "Domain\SuperUserAccount"

$superReaderAccount = "Domain\ServiceMOSSInternet"

$wa = Get-SPWebApplication -Identity "https://www.contoso.com"

SetPolicy -userOrGroup $(whoami) -wa $wa

SetPolicy -userOrGroup $superUserAccount -wa $wa

SetPolicy -userOrGroup $superReaderAccount -wa $wa

$wa.Properties["portalsuperuseraccount"] = $superUserAccount

$wa.Properties["portalsuperreaderaccount"] = $superReaderAccount

$wa.Update()

New-SPSite and New-SPClaimPrincipal

MutazAG — Mon, 28 May 2012 16:46:29 GMT

Self note: new a new site and set my account as an owner:

$web = "https://localdev/sites/site1"

$user = New-SPClaimsPrincipal -IdentityType WindowsSamAccountName -Identity $(whoami)

New-SPSite -Name "site1" -Url $web -OwnerAlias $user -Template "CMSPUBLISHING#0"

Import IIS Logs into SQL Server

MutazAG — Wed, 28 Sep 2011 23:06:00 GMT

I often need to analyse SharePoint IIS logs for different purpose, mainly for capacity planning. The first step to do so (after collection the logs from the servers) is to bulk import the logs into SQL.

To import the IIS logs into SQL I use the logparser tool (Download Details - Microsoft Download Center - Log Parser 2.2).

The following script will iterate through folders of log files and import them into a SQL Server table named IISLogs

$IISLogFolders = Get-ChildItem | ? { $_.PSIsContainer -eq $true}

foreach ($Workloadfldr in $IISLogFolders) {
    "Processing: {0}, {1}" -f $Workloadfldr.Name, $Workloadfldr.FullName
     $logfiles = Test-Path $($Workloadfldr.FullName + "\*") -Include *.log

     if ($logfiles -eq $true)
     {

        $QueryStr = "Select 'TypeYourName' As Customer, '$($WorkloadFldr.Name)' as Workload,
                            LogFilename, LogRow,
                            to_date(to_localtime(to_timestamp(date,time))) as date,
                            to_localtime(to_timestamp(date,time)) as time,
                            c-ip, cs-username, s-sitename, s-computername, s-ip,
                            s-port, cs-method, cs-uri-stem, cs-uri-query, sc-status, sc-substatus, sc-win32-status,
                            sc-bytes, cs-bytes, time-taken,
                            cs-version, cs-host, cs(User-Agent), cs(Cookie),
                            cs(Referer), s-event, s-process-type, s-user-time,
                            s-kernel-time, s-page-faults, s-total-procs, s-active-procs,
                            s-stopped-procs
                            INTO IISLogs from '$(($Workloadfldr.FullName)+'\*.log')'"
        $cmdLine = "logparser -i:IISW3C ""$QueryStr"" -q -o:SQL -server:SQLServerName -database:IISLogs -createTable:ON"
        Invoke-Expression $cmdLine
      }
      else
      {
        "No log files found in folder"
      }

}

I will try and post more about analysis of IIS logs after importing them into SQL.

- mutaz

New SharePoint 2010 training content

MutazAG — Fri, 02 Sep 2011 18:52:14 GMT

Microsoft just published two new training content for BI and Composites in SharePoint 2010. also don't forget to review the available content for ECM and Internet Implementers trainings.

SharePoint Server 2010 Business Intelligence Training

View the course: https://technet.microsoft.com/en-us/sharepoint/hh126809

Learn how you can lay a deep foundation for your SharePoint Server 2010 Business Intelligence (BI) skills in this training course. This course teaches how to use SharePoint Server 2010 as your BI platform and covers the following topics: Business Intelligence Overview, Excel Services, PerformancePoint Services, Visio Services, Reporting Services Integration with SharePoint 2010, and PowerPivot for SharePoint Server 2010. The content is cross referenced with available Advanced IT Pro training (Ignite) per module.

SharePoint Server 2010 Business Composites Training

View the course: https://technet.microsoft.com/en-us/sharepoint/hh126810

Learn how you can lay a deep foundation for your SharePoint Server 2010 Business Composites skills in this training course. This course covers a SharePoint Server 2010 Composites overview and all of these aspects for SharePoint Server 2010: Access Services, InfoPath Forms Services, Workflows in SharePoint 2010, Business Connectivity Services, and SharePoint BI and Composites Service Application Configuration.

- mutaz

Installing SP1 and June CU for SharePoint 2010 and Office Web Apps

MutazAG — Sun, 07 Aug 2011 01:58:10 GMT

(cross posting from: https://blogs.technet.com/b/office_resource_kit/archive/2011/07/12/patching-office-web-apps-to-sp1-and-june-cus-including-sharepoint-server-2010.aspx)

Here is the install order and required patches for SharePoint 2010 deployments with Office Web apps:

if your environment includes:

SharePoint Server 2010 + Office Web App Farm

Step 1: SharePoint Server 2010 SP1: https://support.microsoft.com/kb/2460045

Step 2: Office Web Apps SP1: https://support.microsoft.com/kb/2460073

Step 3: SharePoint Server 2010 June CU: https://support.microsoft.com/kb/2536599

Step 4: Office Web Apps June CU: https://support.microsoft.com/kb/2553919/

(NOTE: The Office Web Apps updates for Excel Web App are included in the SharePoint Server 2010 June Cumulative update.)

Step 5: Once you have finished applying all the updates, run the SharePoint 2010 Products Configuration Wizard or "psconfig –cmd upgrade –inplace b2b -wait” one time on every server in the farm.

Optional patch:

2553935 Description of the Office Web Apps hotfix package (wacproof-sv-se.msp): June 28, 2011 *NOTE: Only needed if you have Swedish language pack installed.

SharePoint Server 2010 + Project Server 2010 + Office Web Apps Farm

Step 1: Project Server 2010 SP1: https://support.microsoft.com/kb/2460047

(NOTE: This update, (2460047) Project Server 2010 SP1, includes SharePoint Server 2010 SP1 and Project 2010 SP1.)

Step 2:Office Web Apps SP1: https://support.microsoft.com/kb/2460073

Step 3: Project Server 2010 June CU: https://support.microsoft.com/kb/2536600

Step 4: Office Web Apps June CU: https://support.microsoft.com/kb/2553919/

Step 5: Once you have finished applying all the updates, run the SharePoint 2010 Products Configuration Wizard or "psconfig –cmd upgrade –inplace b2b -wait” one time on every server in the farm.

Optional patch:

2553935 Description of the Office Web Apps hotfix package (wacproof-sv-se.msp): June 28, 2011 *NOTE: Only needed if you have Swedish language pack installed.

SharePoint Foundation 2010 + Office Web Apps

Step 1: SharePoint Foundation 2010 SP1: https://support.microsoft.com/kb/2460058

Step 2: Office Web Apps SP1: https://support.microsoft.com/kb/2460073

Step 3: SharePoint Foundation 2010 cumulative update https://support.microsoft.com/kb/2536601

Step 4: Apply the following 3 patches for Office Web Apps:

2553924 Description of the Excel Web App hotfix package (xlwacwfe-x-none.msp): June 28, 2011 - https://support.microsoft.com/kb/2553924

2544029 Description of the Excel Web App hotfix package (wacmui-xx-xx.msp, xlwacmui-xx-xx.msp): June 28, 2011 - https://support.microsoft.com/kb/2544029

2553919 Description of the Office Web Apps hotfix package (wacwfe-x-none.msp): June 28, 2011 - https://support.microsoft.com/kb/2553919

Step 5: Run the SharePoint 2010 Products configuration wizard.

Optional patch:

2553935 Description of the Office Web Apps hotfix package (wacproof-sv-se.msp): June 28, 2011 *NOTE: Only needed if you have Swedish language pack installed

- mutaz

Office 365 Jump Start Videos

MutazAG — Mon, 25 Jul 2011 15:33:22 GMT

Microsoft published great Office 365 Readiness and Training videos at: https://technet.microsoft.com/en-us/edge/office-365-jump-start-01-microsoft-office-365-overview-for-it-pros

here is the list of available jump start videos:

- mutaz

Check the build level status of components within your SharePoint 2010 farm

MutazAG — Thu, 21 Jul 2011 18:08:00 GMT

To check the upgrade status of your farm after applying the latest SP and CU:

Get-SPProduct - this CmdLet will check the build version of the binaries on all servers in the farm

PS C:\> $spprod = Get-SPProduct

PS C:\> $spprod.Servers | select ServerName, Products, InstallStatus

ServerName Products                           InstallStatus
---------- --------                           -------------
WFE2       {Microsoft SharePoint Server 2010} NoActionRequired
WFE1       {Microsoft SharePoint Server 2010} NoActionRequired

Here is what you get when a patch is missing on one of the servers

PS C:\> $spprod.Servers | select ServerName, Products, InstallStatus

ServerName Products                           InstallStatus
---------- --------                           -------------
WFE2       {Microsoft SharePoint Server 2010} InstallRequired
WFE1       {Microsoft SharePoint Server 2010} NoActionRequired

you can drill down on the server missing the update can find out which specific patches are missing:

PS C:\> $spprod.Servers | ? { $_.InstallStatus -eq "InstallRequired" } | % { $_.RequiredButMissingPatches }

PatchName          : Hotfix for Office (KB2352342)
PatchGuid          : f58f97af-113a-4ccb-bc8d-f733ddc7cfd2
Version            : 14.0.5123.5000
Link               : https://support.microsoft.com/kb/2352342
ServersMissingThis : {WFE2}

....

Get-SPContentDatabase - check the status of dbs that are still upgrading using this cmdlet

PS C:\> Get-SPContentDatabase | select Name, NeedsUpgrade

Name                                          NeedsUpgrade
----                                          ------------
WSS_Content_81                                True
WSS_Content_1894898f3e174d98b67bb59257bbbbf7 True
WSS_Content_e550737e9fa5447194a02e5ea5228add  True
WSS_Content_Portal                            True
wss_content_1                                 True

You can of course still use the central admin to Check Product and Patch Installation Status.

- mutaz

Talking about SharePoint 2010 large scale deployments, identity, ALM and deployment best practices at MS tech.ed 2010

MutazAG — Wed, 15 Sep 2010 18:36:00 GMT

My colleague Kuldip Obhan and my self presented at tech.ed 2010 in Australia on lessons learned SharePoint 2010 deployments in which we focused our presentation on the following topics:

Designing a large scale SharePoint 2010 deployment
Identity and federation in SharePoint 2010
ALM recommended practices
deployment and configuration practices

a copy of the presentation is now available on the tech.ed web site here: https://www.msteched.com/2010/Australia/OFS205.