Rodney Viana's (MSFT) Blog

Step by step video on how to fix the SharePoint Workflow issue caused by .NET patch

Rodney Viana - MSFT - SharePoint — Fri, 12 Oct 2018 15:05:52 GMT

This post is a continuation of this previous post. It is also an extension of KB4465015. The video assumes you read the previous post and/or KB. The page to the scripts mentioned in the video can be found here: https://gist.github.com/joerodgers/2302b394796c865818839d843bae2dad [embed]https://www.youtube.com/watch?v=l25Gp08egC0[/embed]

After installing .NET security patches to address CVE-2018-8421, SharePoint workflows stop working (KB 4457916/4457035 and others)

Rodney Viana - MSFT - SharePoint — Thu, 13 Sep 2018 16:58:18 GMT

* FINAL UPDATE VI * SharePoint CUs are out

The Nov 2018 SharePoint CUs

The SharePoint with fixes are available. Some environments using third-party solution may still need to follow the steps on this post: Description of the security update for SharePoint Enterprise Server 2016: November 13, 2018 (KB4461501) Description of the security update for SharePoint Foundation 2013: November 13, 2018 (KB4461511) Description of the security update for SharePoint Server 2010 Office Web Apps: November 13, 2018 (KB4461527)

Important

As some companies cannot make changes based on Blog posts, we worked on a public KB that can be found here: https://support.microsoft.com/en-us/help/4465015/sharepoint-workflows-stop-after-cve-2018-8421-security-update The KB has a watered down version of Joe's script. If you have Nintex workflows, favor the scripts on this post. We put together a video with step-by-step. It does not need to be a confusing thing. Found the video here (you can go straight to the video instead of following instructions here): https://blogs.msdn.microsoft.com/rodneyviana/2018/10/12/step-by-step-video-on-how-to-fix-the-sharepoint-workflow/

Symptom

After applying .NET Security Only patch to resolve CVE-2018-8421 (Remote Code Execution Vulnerability) , all SharePoint out of the box Workflows fail to execute and the log will show an error like this: 09/13/2018 01:59:07.57 w3wp.exe (0x1868) 0x22FC SharePoint Foundation Workflow Infrastructure 72fs Unexpected RunWorkflow: Microsoft.SharePoint.SPException: /> The error suggest that System.CodeDom.CodeBinaryOperatorExpression is not in the authorized types.

Cause

Workflow Foundation (WF) will only run workflows when all the dependent types and assemblies are authorized in the .NET config file (or added explicitly via code) under this tree:

However, after the update, the following lines are necessary for SharePoint 2013 and beyond: And for SharePoint 2007 and 2010, use these lines:

Solution

The solution is to add explicitly the types to all web applications' web.config: Or (for SharePoint 2007 and 2010): Please notice that sometimes SharePoint Timer Service (SPTimerV4) runs workflows. If you notice that the application showing the error is ULS logs in OWSTIMER.EXE, you should also include the authorized types in [SharePoint Hive Folder]\bin\OWSTIMER.EXE.config. The Hive Folder will change by version of SharePoint. For SharePoint 2016, it is normally at c:\program files\common files\microsoft shared\web server extensions\16. For 2013, at c:\program files\common files\microsoft shared\web server extensions\15.

Additional Information

My colleague Joe Rodgers, who is Sr. PFE, put together this PowerShell script: https://gist.github.com/joerodgers/2302b394796c865818839d843bae2dad There are two scripts. Normally, the only necessary script is: Add-CodeDomAuthorizedType.ps1 Uncomment this line to make the changes: Add-CodeDomAuthorizedType If you have Nintex workflows you should run like this: Add-CodeDomAuthorizedType -IncludeNintexWorkflow To undo the changes, run: Remove-CodeDomAuthorizedType The script needs to run only once on any WFE. All web.config files related to SharePoint on all servers will be modified. New web applications created after that will also include the changes. Even if a new WFE is added to the farm, the entries will also be included in web.config. The change is a permanent requirement from now on since the WF patch. You do not need to undo the change before applying the SharePoint patch addressing it. There is a second script to update OWSTIMER.exe.config. This one should only run if you see the symptoms in ULS logs with process OWSTIMER.EXE. Otherwise, you do not need to update. if you have the problem though, you need to rerun the script if a new machine is added to the farm. No line needs to be uncommented for this one. The script name is: Add-CodeDomAuthorizedTypeToOWSTimerConfig.ps1

Note

Microsoft is aware of this issue and patches for SharePoint 2010, 2013 and 2016 are being worked as of 9/17/2018. I will update when we have an ETA. I had confirmation from the product team on 9/18/2018 that this information and solution on this post is in the line with the future patch and it is the recommended action plan until the patch is out. If anything change, I will update the post.

Note 2

Some people using third-party workflows (like Nintex) need to also include this: If you are using SharePoint 2010, use this instead: Using the script, you need to add to the line defining types: CodeTypeReferenceExpression Example: $typeNames = @( "CodeBinaryOperatorExpression", "CodePrimitiveExpression", "CodeMethodInvokeExpression", "CodeMethodReferenceExpression", "CodeFieldReferenceExpression","CodeThisReferenceExpression", "CodePropertyReferenceExpression", "CodeTypeReferenceExpression")

Note 3

Joe updated his script to add a switch for Nintex workflows. Call this way to include the extra type required by Nintex: Add-CodeDomAuthorizedType -IncludeNintexWorkflow

Note 4

Some very rare cases require that you add All entries that exist in your web.config (include the new ones added by the script) to OWSTIMER.EXE.config I explain this (narration by Keith Richie, thanks Keith Richie for adding narration) in the video referenced in the beginning of this post.

SharePoint 2013/2016 Usage Reports fail to show the correct number of clicks from anonymous access

Rodney Viana - MSFT - SharePoint — Wed, 28 Mar 2018 11:04:17 GMT

Symptom

When using a Search Center with anonymous access, click statistics do not compute correctly for anonymous access but registers other user's accesses properly. ULS logs will show entries like this: w3wp.exe SharePoint Foundation CSOM ajwqj Medium Request does not have SPBasePermissions.UseRemoteAPIs permission. Need to check it when each API is accessed

Repro Steps

Set up web application for anonymous users:
From CA / Application Management, select the web Application and click on Authentication Providers, then the default Zone
From the Edit Authentication form, select Enable Anonymous Access and click Save
Navigate to the site collection / Site Settings / Site Permissions and click the Anonymous Access button on the ribbon
Configure anonymous access for the Entire Web Site
Close the browser and start a browser anonymous

NOTE: When you set up anonymous access for the site, you will need the Require Use Remote Interfaces Permission checked

Results

When using the Search Center as anonymous, the clicks will not be computed.

Expected Results

Anonymous and non-anonymous clicks should be computed.

Cause

The permission UseRemoteAPIs is somehow disabled on Search Center web site.

Solution

Run the following script: [ps] Add-PSSnapin Microsoft.SharePoint.PowerShell -erroraction SilentlyContinue $webUrl = 'https://portal.contoso.com/searchcenter' # please replace this url by your search center web URL try { $web = Get-SPWeb $webUrl if($web.AnonymousPermMask64.ToString().Contains('UseRemoteAPIs')) { Write-Host "Your web '$($web.Url)' is all set. No change needed"; Write-Host "Effective rights for anonymous: $($web.AnonymousPermMask64)"; } else { Write-Host "Your web '$($web.Url)' does not include the appropriate rights to anonymous" Write-Host "Current Effective rights for anonymous: $($web.AnonymousPermMask64)"; Write-Host "Adding the appropriate permission"; $web.AnonymousPermMask64 = $web.AnonymousPermMask64.ToString() + ", UseRemoteAPIs"; $web.Update(); Write-Host "New effective rights for anonymous: $($web.AnonymousPermMask64)"; } $web.Dispose(); } catch { Write-Error "Error: Fix could not be applied" Write-Error $Error[0] } [/ps]

Leveraging the new Time Travel Trace API in Debugging tools to find when one or more SharePoint event happened

Rodney Viana - MSFT - SharePoint — Tue, 27 Mar 2018 18:33:05 GMT

In my previous post, I showed a proof-of-concept script to list all occasions a process opened a file. JavaScript is easy to program and works for most cases, however in some occasions you need to access resources not available from JavaScript and only a full-fledged debugging extension will do. In this post I will show some highlights of a debugging extension using the new Time Travel Debugging and the new API which at the time of writing was still in preview and may change before final release. The extension contains some other commands as I wrote this for training purposes and decided to leave "as is", so if you are also only interested in writing your extension, you can get some ideas from it. The command for Time Travel Trace (TTD) is "!idnauls". The idea is to list all occasions when SharePoint logs something and show the time, event id (tag), TTD position, category, severity and message of the log entries. It can also filter by tag, category and message. The logic behind this is explained on a previous post. This command only works with TTD dumps (.run) and it requires the new debugging tool which is also in preview version. There is no need to use this extension for other end than learning. NetExt includes a similar command (!widnauls) for this purpose. Special thanks to Ken Sykes and Bill Messmer for the help with new API. Download project here.

The command to enumerate all calls to SharePoint log

Knowing that most ULS log is done by calling these two export functions "onetnative!ULSSendFormattedTrace", "Microsoft_Office_Server_Native!ULSSendFormattedTrace", and that tag ai108 translates to 0x21b6a2 per my previous post, this command would do the trick:

dx /g @$cursession.TTD.Calls("onetnative!ULSSendFormattedTrace", "Microsoft_Office_Server_Native!ULSSendFormattedTrace").Where(c => c.Parameters[0] == 0x21B6A2).Select(c=> new { param1 = c.Parameters[0], param2 = c.Parameters[1], param3 = c.Parameters[2], param4 = c.Parameters[3], start = c.TimeStart, sequence = c.TimeStart.Sequence, steps = c.TimeStart.Steps } )

Parameters

Parameters[0] – Contains the tag in numeric form and is used to filter
Parameters[1] – Contains the id of the product/category. This information is in SPLogLevel object and I used this snippet to get the code id and description:

Add-PSSnapin Microsoft.SharePoint.Powershell -ErrorAction SilentlyContinue

$logLevels = (Get-SPLogLevel | Sort Id)

$lines = New-Object 'System.Collections.Generic.List[string]';

foreach($level in $logLevels)

{

$lines.Add("`tcatMap[0x$($level.Id.ToString(""x"))]=""$($level.Area)|$($level.Name.Replace(""|"", "":""))"";");

Write-Host "`tcatMap[0x$($level.Id.ToString(""x"))]=""$($level.Area)|$($level.Name.Replace(""|"", "":""))"";";

}

$lines | Out-File spcat.txt

Parameters[2] – Contains the severity which is defined here.
Parameters[3] – Contains the pointer to the message string. There is a caveat here. A TTD string is only resolved when the context moves to the time of occurrence.

Logic in a nutshell

Run the command to list all times when one of the two functions are called.
Go to each position and retrieve the message since it can only be read when the context is moved to the position.
Calculate the current time (at target machine).
Transform the id of product/category into string using a simple table (this does not require moving the context)
Transform the severity id into string (also no context change)
Print the result with the TTD position

Extension command code (partial)

[cpp] #define IfFailedReturn(x) if(FAILED(x)) return E_FAIL; void DisplayFound(std::string &FullMessage, std::string& Part) { if (Part.size() == 0) { g_ExtInstancePtr->Out("%s", FullMessage.c_str()); return; } size_t i = 0; size_t p = 0; while (i < FullMessage.size() - Part.size()) { p = FullMessage.find(Part, i); if (p == std::string::npos) { g_ExtInstancePtr->Out("%s", FullMessage.substr(i).c_str()); break; } g_ExtInstancePtr->Out("%s", FullMessage.substr(i, p - i).c_str()); g_ExtInstancePtr->Dml("%s", Part.c_str()); i = p+Part.size(); } } HRESULT MoveTo(IModelObject *spStart) { // // SeekTo is a key on the object just like anything else. The value of the key is a method. // CComPtr spSeekToMethod; IfFailedReturn(spStart->GetKey(L"SeekTo", &spSeekToMethod, nullptr)); // // Before we arbitrarily go about using it as a method, do some basic validation. // ModelObjectKind mk; IfFailedReturn(spSeekToMethod->GetKind(&mk)); if(mk != ObjectMethod) { return E_FAIL; } // // ObjectMethod indicates that it is an IModelMethod packed into punkVal. You can QI to be extra // safe if desired. // VARIANT vtMethod; IfFailedReturn(spSeekToMethod->GetIntrinsicValue(&vtMethod)); //ASSERT(vtMethod.vt = VT_UNKNOWN); // guaranteed by ObjectMethod CComPtr spMethod; // or whatever mechanism you want to guarantee the variant gets cleared. variant_ptr, … spMethod.Attach(static_cast(vtMethod.punkVal)); // // Call the method (passing no arguments). The result here is likely to be ObjectNoValue (there is no return value). // CComPtr spCallResult; IfFailedReturn(spMethod->Call(spStart, 0, nullptr, &spCallResult, nullptr)); return S_OK; } EXT_COMMAND(idnauls, "Command to list ULS position and tag and can be filtered by message or category", "{nomessage;b,o;;Do not show the ULS log message (faster processing).}" "{tag;s,r;;Tag to search for (e.g.: -tag b4ly). Use * for all tags. Required}" "{category;b,o;;Search text in Category or Product and not in message (e.g. -category Claims). Faster processing. Severity not searched}" "{message;b,o;;Search text in message and not in category or product (e.g. -message disk is full). Slower processing.}" "{;x,o;;Optional filter for message (-message) or category (-category) (e.g.: -message disk is full). Must be the last parameter}") { wasInterrupted = false; UINT64 startTime, endTime; std::string tag = GetArgStr("tag"); std::string mess; if (HasUnnamedArg(0)) mess = GetUnnamedArgStr(0); bool nomess = HasArg("nomessage"); bool message = HasArg("message"); bool catonly = HasArg("category"); if (catonly && message) { Out("Error: You have to use either -category or -message. Never both\n\n"); Out("No search was performed\n"); return; } if (message && tag == "*") { Out("Warning: When you combine -tag * and -message, it creates a very inefficient query.\n"); Out(" -tag * will retrieve all ULS log entry and -message will require to move to an iDNA possition every time.\n"); Out(" -tag will only retrieve the ULS logs with this tag and then move to the position to retrieve the message.\n"); Out("Information: Notice that -tag * and -category is ok and still very fast as category is also filtered without moving to the position.\n\n"); } if ((catonly || message) && mess.size() == 0) { Out("Error: -category or -message require a filter pattern\n"); Out("Example: !idnauls -tag ag9cq -message User was authentticated\n"); Dml("3dbba3:1254 SharePoint Foundation Claims Authentication High ag9cq\tUser was authenticated. Checking permissions.\n\n"); Out("Example: !idnauls -tag * -category Web Content Management\n"); Dml("3de9ae:14c0 Web Content Management Publishing Cache High ag0ld\n"); Out("No search was performed\n"); return; } map catVector; if (catonly) { catVector = SPCategories::GetListAreaName(mess); if (catVector.size() == 0) { Out("No category/product contains '%s'\n", mess.c_str()); Out("No search was performed\n"); return; } Out("Warning: The string '%s' will only be searched on Category/Product, not in message\n", mess.c_str()); mess.clear(); } if (tag != "*" && (tag.size() < 4 || tag.size() > 5)) { Out("Tag: '%s' is invalid\n", tag.c_str()); Out("It can be either '*' for all or be between 4 and 5 bytes\n"); return; } unsigned int tagBin = StrToTag(tag); if (tagBin == 0 && tag != "*") { Out("Tag: '%s' is invalid\n", tag.c_str()); Out("It does not contain a valid tag sequence\n"); return; } if (tag == "*") { swprintf_s(Buffer, MAX_MTNAME, L"@$cursession.TTD.Calls(\"onetnative!ULSSendFormattedTrace\", \"Microsoft_Office_Server_Native!ULSSendFormattedTrace\").Select(c=> new { param1 = c.Parameters[0], param2 = c.Parameters[1], param3 = c.Parameters[2], param4 = c.Parameters[3], start = c.TimeStart, sequence = c.TimeStart.Sequence, steps = c.TimeStart.Steps } )"); } else { swprintf_s(Buffer, MAX_MTNAME, L"@$cursession.TTD.Calls(\"onetnative!ULSSendFormattedTrace\", \"Microsoft_Office_Server_Native!ULSSendFormattedTrace\").Where(c => c.Parameters[0] == 0x%p).Select(c=> new { param1 = c.Parameters[0], param2 = c.Parameters[1], param3 = c.Parameters[2], param4 = c.Parameters[3], start = c.TimeStart, sequence = c.TimeStart.Sequence, steps = c.TimeStart.Steps } )", tagBin); } std::wstring query(Buffer); #if _DEBUG Out("%s = ", tag.c_str()); Out("%x \n", tagBin); Out("dx %S\n", query.c_str()); #endif CComPtr client; HRESULT Status; REQ_IF(IHostDataModelAccess, client); CComPtr pHost; CComPtr pManager; if (FAILED(client->GetDataModel(&pManager, &pHost))) { Out("Data Model could not be acquired\n"); return; } CComPtr hostEval; CComPtr spObject; pHost->QueryInterface(IID_PPV_ARGS(&hostEval)); startTime = GetTickCount64(); if (!SUCCEEDED(hostEval->EvaluateExtendedExpression(USE_CURRENT_HOST_CONTEXT, query.c_str(), nullptr, &spObject, nullptr))) { Out("Expression could not be evaluated\n"); return; }; CComPtr pListOfBreaks; CComPtr spIterable; if (SUCCEEDED(spObject->GetConcept(__uuidof(IIterableConcept), (IUnknown**)&spIterable, nullptr))) { CComPtr spIterator; if (SUCCEEDED(spIterable->GetIterator(spObject, &spIterator))) { // // We have an iterator. Error codes have semantic meaning here. E_BOUNDS indicates the end of iteration. E_ABORT indicates that // the debugger host or application is trying to abort whatever operation is occurring. Anything else indicates // some other error (e.g.: memory read failure) where the iterator MIGHT still produce values. std::vector queryResult; // It will store the list of parameters UINT32 Index = 0; // for (;;) { CComPtr pBreakItem; CComPtr spContainedMetadata; HRESULT hr = spIterator->GetNext(&pBreakItem, 0, nullptr, &spContainedMetadata); if (hr == E_BOUNDS || hr == E_ABORT) { break; } if (FAILED(hr)) { Out("There was a failure at an Item\n"); continue; // // Decide how to deal with failure to fetch an element. Note that pBreakItem *MAY* contain an error object // which has detailed information about why the failure occurred (e.g.: failure to read memory at address X). // } // // Read the values // CComPtr tag; CComPtr SevLevel; CComPtr Category; CComPtr Message; CComPtr Start; CComPtr Sequence; CComPtr Steps; VARIANT vt_tag, vt_sevlevel, vt_category, vt_message, vt_sequence, vt_steps; if (FAILED(hr = pBreakItem->GetKeyValue(L"param1", &tag, NULL /* &spContainedMetadata */))) { Out("Error reading param1"); continue; } hr = tag->GetIntrinsicValue(&vt_tag); //hr = tag->GetIntrinsicValueAs(VT_INT_PTR, &vt_tag); if (FAILED(hr = pBreakItem->GetKeyValue(L"param2", &Category, NULL /* &spContainedMetadata */))) { Out("Error reading param2"); continue; }; hr = Category->GetIntrinsicValue(&vt_category); if (FAILED(hr = pBreakItem->GetKeyValue(L"param3", &SevLevel, NULL /* &spContainedMetadata */))) { Out("Error reading param3"); continue; }; hr = SevLevel->GetIntrinsicValue(&vt_sevlevel); if (FAILED(hr = pBreakItem->GetKeyValue(L"param4", &Message, NULL /* &spContainedMetadata */))) { Out("Error reading param4"); continue; }; hr = Message->GetIntrinsicValue(&vt_message); if (hr = FAILED(pBreakItem->GetKeyValue(L"start", &Start, NULL /* &spContainedMetadata */))) { Out("Error reading start"); continue; }; //hr = Start->GetIntrinsicValue(&vt_start); // It fails here because the type in ObjectSynthetic if (FAILED(hr = pBreakItem->GetKeyValue(L"sequence", &Sequence, NULL /* &spContainedMetadata */))) { Out("Error reading sequence"); continue; }; hr = Sequence->GetIntrinsicValue(&vt_sequence); if (FAILED(hr = pBreakItem->GetKeyValue(L"steps", &Steps, NULL /* &spContainedMetadata */))) { Out("Error reading steps"); continue; }; hr = Steps->GetIntrinsicValue(&vt_steps); if (IsInterrupted()) { break; } UlsInstance obj; obj.Category = vt_category.uintVal; obj.Message = vt_message.llVal; //vt_category.llVal); obj.SevLevel = vt_sevlevel.uintVal; obj.Sequence = vt_sequence.uintVal; obj.Steps = vt_steps.llVal; obj.tag = vt_tag.uintVal; obj.Index = Index; if (catVector.size() > 0) { if (catVector.find((int)obj.Category) == catVector.end()) { continue; } } // Only move if necessary bool show = true; string fullMess; if ((mess.size() > 0 || !nomess) && SUCCEEDED(MoveTo(Start))) { CComBSTR stringConv; CComPtr context; CComPtr memory; if (SUCCEEDED(hr = pHost->QueryInterface(__uuidof(IDebugHostMemory), (void**)&memory))) { ULONG64 BytesRead = 0; Location loc; loc.HostDefined = 0; loc.Offset = obj.Message; if (SUCCEEDED(hr = memory->ReadBytes(USE_CURRENT_HOST_CONTEXT, loc, Buffer, MAX_MTNAME * 2, &BytesRead))) { Buffer[MAX_MTNAME - 1] = L'\0'; fullMess = CW2A(Buffer); if (mess.size() > 0) { show = fullMess.find(mess) != std::string::npos; } } } if (hr != S_OK) { fullMess = "*** Unable to read memory ***"; show = true; } } if (show) { Index++; Dml("%S\t", obj.IDnaPosition().c_str(), obj.IDnaPosition().c_str()); string area; string prod; string sev = SPCategories::GetSevLevel(static_cast(obj.SevLevel)); SPCategories::GetAreaName(obj.Category, area, prod); if (!nomess) { SYSTEMTIME time; if (!GetTime(time, true)) { Out("??/??/???? ??:??:??.??\t"); } else { Out("%02i/%02i/%04i %02i:%02i:%02i.%02i\t", time.wMonth, time.wDay, time.wYear, time.wHour, time.wMinute, time.wSecond, time.wMilliseconds / 10); } } Out("%s\t", area.c_str()); Out("%s\t", prod.c_str()); Out("%s\t", sev.c_str()); Out("%s\t", TagToStr(obj.tag).c_str()); if (!nomess) { if (mess.size() > 0 && !catonly) DisplayFound(fullMess, mess); else Out("%s", fullMess.c_str()); } queryResult.push_back(obj); Out("\n"); } } Out("%u Instances\n", Index); endTime = GetTickCount64(); Out("Search took %f seconds\n", (float)(((float)endTime - (float)startTime) / (float)1000)); } } } [/cpp]

Example 1 – Looking for a particular tag and message

Example 2 – Listing all instances

Leveraging the new WinDbgX and Time-Travel-Trace –Script to list all access to files

Rodney Viana - MSFT - SharePoint — Thu, 08 Mar 2018 18:38:33 GMT

WinDbg Preview a.k.a. WinDbgX aesthetically looks like a marriage between Visual Studio (VS) and WinDbg, however VS and WinDbg have not many things in common. For me, this is the good news. The bad news is that the support for managed code is as limited as WinDbg’s. You can download the preview here: https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools

The most impressive feature of the new debugger is by far the ability to create dump files to later be analyzed as live debugging. This is called Time-Travel-Debugging (TTD). The idea is that you can record an actual live process (at a performance penalty) to later debug going back and forth in time, thus the name. A good use for this is to capture a ‘live dump’ on a production server and analyze the dump later on another environment with access to source code and symbols. If you are a software provider, this is golden. Sometimes, only a particular environment causes the error and your logs do not show much that can help you identify the problem. Recording on a place and debugging on your development lab does not require that you ship your private symbols and source code to a customer’s environment. You may learn more here: https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/time-travel-debugging-overview

The analysis is done via a LinQ like syntax that can be used directly from the debugger command window or automated via a classic debugger extension or a JavaScript extension. In this post, I am presenting a proof-of-concept script that shows all instances when a file is accessed. More on WinDbg scripting here: https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/javascript-debugger-scripting

Proof of concept OpenFileList.js (this script only WORKS with TTD dumps – .run files)

"use strict";     var FileAccess =    {        FILE_READ_DATA: 0x0001,    /* file & pipe */        FILE_LIST_DIRECTORY: 0x0001,    /* directory */        FILE_WRITE_DATA: 0x0002,    // file & pipe        FILE_ADD_FILE: 0x0002,    // directory         FILE_APPEND_DATA: 0x0004,    // file        FILE_ADD_SUBDIRECTORY: 0x0004,    // directory        FILE_CREATE_PIPE_INSTANCE: 0x0004,    // named pipe          FILE_READ_EA: 0x0008,    // file & directory         FILE_WRITE_EA: 0x0010,    // file & directory         FILE_EXECUTE: 0x0020,    // file        FILE_TRAVERSE: 0x0020,    // directory         FILE_DELETE_CHILD: 0x0040,    // directory         FILE_READ_ATTRIBUTES: 0x0080,    // all         FILE_WRITE_ATTRIBUTES: 0x0100,    // all         DELETE: 0x10000,        WRITE_DAC: 0x40000,           WRITE_OWNER: 0x80000,         SYNCHRONIZE: 0x100000,         ACCESS_SYSTEM_SECURITY: 0x11F01FF,          GENERIC_READ: 0x80000000,        GENERIC_WRITE: 0x40000000,        GENERIC_EXECUTE: 0x20000000,        GENERIC_ALL: 0x10000000    };     var FileShare =     {        FILE_SHARE_NONE: 0,        FILE_SHARE_READ: 0x00000001,        FILE_SHARE_WRITE: 0x00000002,        FILE_SHARE_DELETE: 0x00000004     }; function PrintFileAccess(Access){    var flag = "";    if(Access == 0)      return "None";    if(Access & FileAccess.FILE_ADD_FILE)       flag = "FILE_ADD_FILE";    if(Access & FileAccess.FILE_ADD_SUBDIRECTORY)       flag = flag.concat(" FILE_ADD_SUBDIRECTORY");    if(Access & FileAccess.FILE_APPEND_DATA)       flag = flag.concat(" FILE_APPEND_DATA");    if(Access & FileAccess.FILE_CREATE_PIPE_INSTANCE)       flag = flag.concat(" FILE_CREATE_PIPE_INSTANCE");    if(Access & FileAccess.FILE_DELETE_CHILD)       flag = flag.concat(" FILE_DELETE_CHILD");    if(Access & FileAccess.FILE_EXECUTE)       flag = flag.concat(" FILE_EXECUTE");    if(Access & FileAccess.FILE_LIST_DIRECTORY)       flag = flag.concat(" FILE_LIST_DIRECTORY");    if(Access & FileAccess.FILE_READ_ATTRIBUTES)       flag = flag.concat(" FILE_READ_ATTRIBUTES");    if(Access & FileAccess.FILE_READ_DATA)       flag = flag.concat(" FILE_READ_DATA");    if(Access & FileAccess.FILE_READ_EA)       flag = flag.concat(" FILE_READ_EA");    if(Access & FileAccess.GENERIC_ALL)       flag = flag.concat(" GENERIC_ALL");    if(Access & FileAccess.GENERIC_EXECUTE)       flag = flag.concat(" GENERIC_EXECUTE");    if(Access & FileAccess.GENERIC_READ)       flag = flag.concat(" GENERIC_READ");    if(Access & FileAccess.GENERIC_WRITE)       flag = flag.concat(" GENERIC_WRITE");    if(Access & FileAccess.FILE_TRAVERSE)       flag = flag.concat(" FILE_TRAVERSE");    if(Access & FileAccess.FILE_WRITE_DATA)       flag = flag.concat(" FILE_WRITE_DATA");    if(Access & FileAccess.FILE_WRITE_EA)       flag = flag.concat(" FILE_WRITE_EA");    if(Access & FileAccess.DELETE)       flag = flag.concat(" DELETE");    if(Access & FileAccess.WRITE_DAC)       flag = flag.concat(" WRITE_DAC");    if(Access & FileAccess.WRITE_OWNER)       flag = flag.concat(" WRITE_OWNER");    if(Access & FileAccess.SYNCHRONIZE)       flag = flag.concat(" SYNCHRONIZE");    if(Access & FileAccess.ACCESS_SYSTEM_SECURITY)       flag = flag.concat(" ACCESS_SYSTEM_SECURITY");       return flag;} function PrintFileShare(Share){    var flag = "";    if(Share == 0)      return "None";    if(Share & FileShare.FILE_SHARE_DELETE)       flag = "FILE_SHARE_DELETE";    if(Share & FileShare.FILE_SHARE_READ)       flag = flag.concat(" FILE_SHARE_READ");    if(Share & FileShare.FILE_SHARE_WRITE)       flag = flag.concat(" FILE_SHARE_WRITE");    return flag;       } function findFileHandle(){    var fileOper = host.currentSession        .TTD.Calls("KERNELBASE!CreateFileW", "KERNELBASE!CreateFileA");    var locations = [];    for(var loc of fileOper)    {        var posObj = {Start: loc.TimeStart, FileNameAddr: loc.Parameters[0],              Access: loc.Parameters[1], ShareMode: loc.Parameters[2],              Result: loc.ReturnAddress, Ansi: loc.Function.includes("CreateFileA") };                locations.push(posObj);            }    return locations;}    function printFileName(location){    host.diagnostics.debugLog("------------------------------------------------\n");     location.Start.SeekTo();     host.diagnostics.debugLog((location.Result > 0 ? "(SUCCESS) " : "(Failure) ") +                       (location.Ansi ? host.memory.readString(location.FileNameAddr) : host.memory.readWideString(location.FileNameAddr)) + "\n");    var access = host.evaluateExpression(location.Access.toString(16)+" & 0xffffffff")    var share = host.evaluateExpression(location.ShareMode.toString(16)+" & 0xffffffff")     // host.diagnostics.debugLog(access.toString(16) + " " + share.toString(16)+" ");                            host.diagnostics.debugLog("Access: ["+access.toString(16)+"] "+PrintFileAccess(access)+" Share: ["+                    share.toString(16) + "] "+PrintFileShare(share) + "\n");      return;                   } function invokeScript(){    host.diagnostics.debugLog("Collecting File Access information...\n");       var currentPosition = host.currentThread.TTD.Position;     var locations = findFileHandle();    for (var location of locations)    {        printFileName(location);    }     currentPosition.SeekTo();}   

This is an example of the output:

Collecting File Access information...------------------------------------------------Setting position: 1DB4F:4D(2370.2184): Break instruction exception - code 80000003 (first/second chance not available)Time Travel Position: 1DB4F:4DKERNELBASE!CreateFileW:00007ffe`d0258460 4883ec58        sub     rsp,58hCollecting File Access information...------------------------------------------------Setting position: 1DB4F:4D(2370.2184): Break instruction exception - code 80000003 (first/second chance not available)Time Travel Position: 1DB4F:4DKERNELBASE!CreateFileW:00007ffe`d0258460 4883ec58        sub     rsp,58hCollecting File Access information...------------------------------------------------Setting position: 1DB4F:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.SharePoint\v4.0_15.0.0.0__71e9bce111e9429c\Microsoft.SharePoint.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------Setting position: 42094:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------Setting position: 4211F:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------Setting position: 1DBCD:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.SharePoint\v4.0_15.0.0.0__71e9bce111e9429c\Microsoft.SharePoint.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------Setting position: 414C5:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.SharePoint.Publishing\v4.0_15.0.0.0__71e9bce111e9429c\Microsoft.SharePoint.Publishing.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------Setting position: 4219E:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------Setting position: 1DC4B:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.SharePoint\v4.0_15.0.0.0__71e9bce111e9429c\Microsoft.SharePoint.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------Setting position: 4221C:4D(SUCCESS) C:\windows\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dllAccess: [0x4c4b400]  WRITE_DAC ACCESS_SYSTEM_SECURITY Share: [0x1]  FILE_SHARE_READ------------------------------------------------(...)

After updating SharePoint 2013 to November 2017 CU or later you may not be able to open documents with Office

Rodney Viana - MSFT - SharePoint — Tue, 05 Dec 2017 19:45:40 GMT

*** UPDATED TO INCLUDE Stefan Gossner and Adam Soreson's INPUT ***

Symptom

After updating SharePoint 2013 to November 2017 CU from a previous version you will not be able to open Office documents (Word, Excel, PowerPoint) but other types of documents like .txt and .pdf files can be opened. There is no indication of the error in the ULS log. Observing the network activity using a tool like Telerik's Fiddler, you will notice that the response from cellstorage.svc is 400 (bad format) or 401 (unauthorized). This is why the Office Integration piece fails and other file types succeed. The failed request will look like:

POST /_vti_bin/cellstorage.svc/CellStorageService

Cause

November CU brings a breaking update to stssoap.dll. This dll is normally cached in the _app_bin folder of the web application and it does get updated at the 15 hive bin folder but not at the local web applications bin folder. For example, imagine a web application for portal.contoso.com, on location C:\inetpub\wwwroot\wss\VirtualDirectories\portal.contoso.com80. You will find stssoap.dll in C:\inetpub\wwwroot\wss\VirtualDirectories\portal.contoso.com80\_app_bin\stssoap.dll version not to match the version in C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\CONFIG\BIN\stssoap.dll. The main cause for this is that the update was not done properly (i.e. via Wizard or using all the appropriate commands via PSConfig).

Resolution

The DLL version in the 15 hive should be at 15.0.4981.1000 (10/17/2017) or higher. The local version in local bin should be dated before 10/17/2017. If this is not the case, this solution does not apply to you. The solution also applies to you if the version in 15 hive is newer than the version of the local bin for future stssoap.dll changes. Make sure you run the update properly. Always prefer to use Configuration Wizard over PSConfig to update. This command should update only the web applications _app_bin, but this is not the best solution:

PSConfig.exe -cmd applicationcontent -install

See more details here. The best solution is to run the configuration Wizard or to emulate all the steps of the configuration wizard using this:

PSConfig.exe -cmd upgrade -inplace b2b -wait -cmd applicationcontent -install -cmd installfeatures -cmd secureresources -cmd services -install

SharePoint 2013 and 2016 does not respect the user zone when sending alerts but the Cross Firewall Access Zone (an unusual scenario)

Rodney Viana - MSFT - SharePoint — Mon, 30 Oct 2017 13:04:02 GMT

Symptoms

Consider you have a one Web Application with at least two zones. The extranet zone (https://sp2016 in this example) and an intranet zone (http:sp2016:100 in this example). The cross firewall zone is the intranet zone (http:sp2016:100 in this example). Notice that this is an unusual scenario as extranet is the natural candidate to the cross firewall access zone.

A user with access to the extranet site navigates to a document library on the zone where he/she has access, and then add an alert for all changes for a document. The first e-mail will point to the zone where customer created the alert, subsequent alerts will point to the intranet zone which user cannot access from the extranet.

Steps to reproduce

Go to Library ribbon –> Alert me -> Set alert on this library

The first e-mail will point to https://sp2016
When a change occurs in the document the new alerts will point to the Cross Firewall Access zone (https://sp2016:100 in this example) .

Cause

This is by design in 2013 and 2016 but it breaks with the behavior when compared to SharePoint 2010 which would send an e-mail based on where it was created.

In SharePoint 2010 the alert link goes unmodified pointing to the location it was created. In 2013 and 2016, the direct link was modified and the link goes mediated by Redirect.aspx which will reroute the call to the Cross Firewall Access Zone.

Workaround:

It is possible to override the behavior by enabling a special flag in the Farm. Below is the PowerShell script to do so.

$farm = Get-SPFarm$farm.ServerDebugFlags.Add(34) #Flag to disable ClickRedirect$farm.Update()

Tool to generate DebugDiag rule for SharePoint based on event id (Tag) and optionally a partial message

Rodney Viana - MSFT - SharePoint — Wed, 30 Aug 2017 17:44:58 GMT

When I updated my post on how to create DebugDiag rule based on event id (Tag), I received some feedback that modifying the scripts is quite error prone and more often than not the resulting script would not work as expected. To that end, I decided to create a visual tool that will create the rule template based on the parameters that matter. This rule template can be imported directly into DebugDiag. Please use DebugDiag 2 Update 2 (or later) which you can download from Microsoft. So let’s see two different scenarios. Download the tool here: Tool Binaries Unzip the files to your tool folders. NOTE: Normally when you download an EXE (zipped or not) from the Internet, Windows will block the execution. To resolve this, right click on CreateSPDebugRule.exe and check ‘Unblock’ and click OK:

When the ULS log entry points to w3wp.exe

We plan to generate a dump file based on this fresh ULS entry: 08/30/2017 14:48:18.80 w3wp.exe (0x3820) 0x2714 SharePoint Foundation DistributedCache air4a Monitorable Token Cache: Failed to get token from distributed cache for '0).w|s-1-5-21-3258600628-1467426315-3527673178-500'.(This is expected during the process warm up or if data cache Initialization is getting done by some other thread). aa74149e-4d3f-b0b8-39ea-4e89c1505cc6 IMPORTANT: The entry needs to be fresh because the process id changes when IIS resets. Along with the process name, ULS logs include the process id (PID) in hex format. In this case, the thread is 0x3820 which is 14368 in decimal. There is an easy way to identify the application pool via command prompt. Run from an elevated command prompt (IT WILL NOT WORK ON POWERSHELL): %windir%\system32\inetsrv\appcmd list wp So we learned the application pool name is ‘SharePoint – 80’. You can fill out the form as below. Notice that I add a ‘partial message’ – this is necessary when a tag is too generic and you just want to capture when the message contains some pattern. The tag bz7l which I discussed in several occasions for example is very specific, it only occurs when a claim credential cannot be converted to Windows credential, so there is no need to add a partial message. In doubt, add a partial message. I can choose the number of dump files to be created by the rule (the default is 5). You do not need to run the tool on the server where the dump will be captured, but if you do you have the advantage of Select the Application Pool by clicking the Select… button (you need to run the tool in administration mode for this). The selection window looks like that. If you have everything in place you click on Generate Template… and choose a name (the name will not affect the name of the rule name in DebugDiag).

To import the rule into DebugDiag

Run ‘DebugDiag 2 Collect’.
Click ‘Cancel’ to close the Rule Wizard.
Click the ‘Import’ button to load the rule you created with the tool

Click Ok to accept the rule settings

Notice that the rule will not be active when you import. You need to right-click the rule and choose Activate Rule. If this is the first time you activate a rule there will be several pop-ups from DebugDiag, just accept all defaults.
Once the rule is active, the dump will be generated when the condition occurs.

When the ULS log entry points to a process

For this scenario,, we plan to generate a dump file based on this ULS entry: 08/30/2017 14:48:29.28 OWSTIMER.EXE (0x1A94) 0x04E4 SharePoint Server Search Administration ajzc7 Medium Cleanup of Orphan Systems in server CONTOSOWFE ac74149e-4d87-b0b8-39ea-409643feb4b7 Notice that this time, you do not need a fresh ULS log because we will capture based on a service (actually based on a process name). OWSTIMER.exe is the name for the SPTimerV4 service, The tool needs the process name which is how it shows in ULS log already. You can fill out this rule like below. Notice I chose ‘Application/Service’ this time. You can also use the Select… button to choose the process if you are running the tool on the server. This is the form filled out: Feel free to play around and adapt the tool by changing the source code if you want to. The project zip is somewhere at: https://github.com/rodneyviana/blogdemos

PowerShell CmdLet to check OneDrive for Business or OneDrive Personal Status

Rodney Viana - MSFT - SharePoint — Tue, 06 Jun 2017 19:29:13 GMT

My good friend Neil Hodgkinson gave me a challenge: some partners and companies were asking him how to verify the status of OneDrive. He gave me the idea to check the icon of the OneDrive folder and depending on the icon, tell the status. I created a proof-of-concept and then helped some folks at Microsoft Consulting Services to have a PowerShell cmdlet to do so. Not being enough the status of OneDrive using the regular OneDrive synchronization, I also included the status of OneDrive when synchronized via Groove (yes, people still use it).

The project is open source and available in GitHub. To get only the PowerShell cmdlet, download OneDriveLib.dll here. License is MIT and can be found here. It is an open source project so it is not entitled for Microsoft support but I will do my best to respond questions in this post. Just ask them in comments.

How to use:

Open PowerShell is normal mode (it will not work in Elevated mode for OneDrive own design)
Import the module OneDriveLib.dll
Run Get-ODStatus

Syntax:

Get-ODStatus [-Type ] [-ByPath ]

Where:

-Type let you filter by OneDrive type (e.g. Personal or Business) [optional]

-ByPath if the folder information is not in the registry you can test a particular folder (this form will return only a status string)

Examples:

Get-ODStatus –Type Business

Get-ODStatus –ByPath c:\user\rodneyviana\onedrive

Example of output

PS C:\ODTool>; Import-Module OneDriveLib.dll PS C:\ODTool>; Get-ODStatus  StatusString : UpToDateLocalPath    : E:\MicrosoftOnedrive\OneDrive - ContosoUserSID      : S-1-5-21-124012345-708012345-1543012345-802000UserName     : CONTOSO\rodneyvianaServiceType  : Business1 StatusString : UpToDateLocalPath    : D:\OnedriveUserSID      : S-1-5-21-124012345-708012345-1543012345-802000UserName     : CONTOSO\rodneyvianaServiceType  : Personal

SharePoint 2013/2016 RSS Viewer not working with external sites when through a proxy

Rodney Viana - MSFT - SharePoint — Tue, 25 Apr 2017 18:45:17 GMT

Recently I worked on a case for a Fortune 500 company that was having problems to render the RSS Viewer Web Part in their SharePoint 2013 farm. They still have a SharePoint 2010 farm in the same network environment and this farm was not having problems rendering the RSS Viewer. The constraint was this: access to outside sites requires going through a proxy. Default proxy configuration is a tricky thing. It can be done in several ways: DHCP settings, DNS settings, registry settings, Internet Explorer (for the user logged in only) or for .NET applications via configuration. In production environment is always a good idea to inform the default proxy explicitly since auto-discovery may take a few seconds the first time the process try it for an endpoint. So, initially we configured their proxy explicitly by adding this in the web.config file of the SharePoint application. So, for the local domain ‘contoso.com’, proxy address ‘https://proxy.contoso.com:8080’ this is the configuration:

  <system.net>

    <defaultProxy>

        <proxy

            usesystemdefault="false"

            autoDetect="false"

            proxyaddress="https://proxy.contoso.com:8080"

            bypassonlocal="true"

        />

      <bypasslist>

        <add address="[a-zA-Z0-9\.]+\.contoso\.com$" />

      bypasslist>

     defaultProxy>

  system.net>

This configuration is better placed one line below or one line above . Another important thing concerning bypass list: one should never use * at the beginning of an exclusion item. System.Net uses RegEx to check whether the address is in exclusion or not and will fail if wildcard is used (specially if it starts with '*' which is a no-no in .NET RegEx. The regex equivalent to * is ‘[a-zA-Z0-9\.\-]+’. They added the suggested configuration and analyzing the dump file we could confirm the proxy configuration was applied. Still we were seeing the error below in the ULS logs: DATE TIME w3wp.exe (0x15B4) 0x4878 SharePoint Portal Server Web Parts 8imh High RssWebPart: Exception handed to HandleRuntimeException.HandleException System.Net.Sockets.SocketException (0x80004005): No such host is known at System.Net.Dns.GetAddrInfo(String name) at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6) at System.Net.Dns.GetHostAddresses(String hostNameOrAddress) at Microsoft.SharePoint.Utilities.CommonUtility.IsIntranetAddress(String hostName) at Microsoft.SharePoint.Utilities.SPWebRequest.PreventIntranetCalls(Uri requestUri, SPContext context, String sharePointMarkValue, String appContextLoggingValue) at Microsoft.SharePoint.Utilities.SPWebRequest.SafeCreate(Uri requestUri, SPContext context, Type callingType, String callingInstanceTag) at Microsoft.SharePoint.WebControls.XmlUrlDataSource.FetchData(String requestUrl) at Microsoft.SharePoint.WebControls.BaseXmlDataSource.Execute(String request) at Microsoft.SharePoint.WebControls.BaseXmlDataSource.GetXmlDocument() at Microsoft.SharePoint.WebControls.SingleDataSource.GetXPathNavigatorInternal() at Microsoft.SharePoint.WebControls.SingleDataSource.GetXPathNavigator() at Microsoft.SharePoint.WebPartPages.DataFormWebPart.GetXPathNavigator(String viewPath) at Microsoft.SharePoint.WebPartPages.DataFormWebPart.PrepareAndPerformTransform(Boolean bDeferExecuteTransform) We captured a dump file and could confirm that .NET was indeed applying the proxy configuration. We did some more troubleshooting and we tested in a lab farm. Things worked like a charm for us. Also the SharePoint 2010 farm worked as expected. We noticed that the last public method called before failure was this one: System.Net.Dns.GetHostAddresses(String hostNameOrAddress) And this was odd, since proxy resolution will first try to request the page via the proxy server and then if, and only if, it fails it will try DNS resolution. Checking the source code (you may achieve the same result with reflection) we were able to find out the culprit: if SPFarm.Properties.DisableIntranetCalls or SPFarm.Properties.DisableIntranetCallsFromApps (if it is an App application) is set to true, SharePoint will use DNS to verify whether the address is intranet or internet. As the address cannot be resolved because the rss feed site is not in the DNS and can only be resolved via proxy which would be the next step after this, SharePoint throws the exception you see in the ULS log entry above. So, how to resolve this issue? The solution is very counter-intuitive: set DisableIntranetCalls and DisableIntranetCallsFromApps to false and it will work. Then you may ask yourself, if the call is extranet why enabling internet calls will resolve the issue? The answer is the fact that if these properties are set to false the call to resolve the address via DNS will not be called so there will be no exception and the retrieval of the page will occur without a problem. Is this change introducing security risk to SharePoint? Not at all. This property is rather to force people to use better web parts to list a feed of a local site. This PowerShell script will disable the undesired DNS search if you are facing this issue:

# These commands will prevent SharePoint from trying to resolve the RSS name host in DNS which may cause problem with proxies 

# 

 Add-PSSnapin Microsoft.SharePoint.PowerShell

 $farm = Get-SPFarm

 $farm.Properties.DisableIntranetCalls = $false

 $farm.Properties.DisableIntranetCallsFromApps = $false

 $farm.Update()

How to verify if you are facing this problem in your environment. Run this in PowerShell (change the URL for the RSS feed to test):

$url = New-Object System.Uri "https://blogs.msdn.microsoft.com/rodneyviana/feed/"

[System.Net.Dns]::GetHostAddresses($url.DnsSafeHost)

If it throws an exception, then the RSS web part will not render unless a proxy is set and the DisableIntranetCalls* properties are set to false. Special thanks to Tony DeVere from Distributed Services team for his invaluable help in this case.

Additional information on Proxy discovery

The .Net Framework proxy detecting/loading, is using the following setting combination https://msdn.microsoft.com/en-us/library/sa91de1e.aspx By default, autoDetect is set as unspecified, which equals to enabled. It means the proxy is automatically detected. The proxy auto detection follows this white paper for design. https://msdn.microsoft.com/en-us/magazine/cc300743.aspx “Automatic configuration is a two-step process. First, the user agent (Internet Explorer or the .NET Framework) tries to discover the URL of a configuration script. Typically, the configuration script is a JavaScript file that contains logic to determine whether to use a proxy and what proxy to use for a given target resource. The protocol for automatic detection is documented at Web Proxy Auto-Discovery Protocol. The user agent first tries a DHCPINFORM message. The reply to this message contains the URL of the configuration script. If that doesn't work, a DNS lookup for a well-known alias, WPAD, is issued. Then the configuration script's location is determined as https://>/wpad.dat where the machine name is the result of the DNS lookup. The second step is to download the configuration script from the URL and run the script. The specification of what the configuration script looks like is at Navigator Proxy Auto-Config File Format. And then parse and run the proxy auto configuration (PAC) script.” This detection implementation is expensive, take time and also require you have either DHCP server or DNS set up correctly. If you are using a middle tier server type of client, for performance consideration, we will need to disable it to save the time spent on proxy detection routine. The 2nd proxy detection is using IE proxy. It is controlled through setting, usesystemdefault="true|false|unspecified " usesystemdefault -- > Specifies whether to use Internet Explorer proxy settings. If set to true, subsequent attributes will override Internet Explorer proxy settings. The default value is unspecified, which equals to true. But this setting will not work well for a service. A service typically doesn’t have any IE profile with proxy setting. Creating a IE profile for a service account is like a hacking. Usesystemdefault setting is more designed for a log on user type of application instead of service. IE proxy connection is implemented through wininet.dll, but wininet.dll is not supported in service. https://support.microsoft.com/kb/238425 INFO: WinInet Not Supported for Use in Services So, we will discourage using the IE proxy setting and creating an IE profile for a service account to load proxy in .Net as it will make the proxy loading behavior hard to be predicted if a service has some user impersonation implemented. For the middle tier or service type of client app, we can set the proxy use the following choice: bypassonlocal="true|false|unspecified" proxyaddress="uriString" scriptLocation="uriString" We can using proxy address to well specify the proxy information combined with proxy bypass rule. If you have to use the proxy auto configuration (PAC) script rule, you can use the choice in scriptLocation. This way, we can well control the proxy loading behavior to avoid delay on Proxy detection or unpredictable result for using impersonated account to load IE proxy setting.

Rodney Viana's (MSFT) Blog

Step by step video on how to fix the SharePoint Workflow issue caused by .NET patch

After installing .NET security patches to address CVE-2018-8421, SharePoint workflows stop working (KB 4457916/4457035 and others)

*** FINAL UPDATE VI *** SharePoint CUs are out

The Nov 2018 SharePoint CUs

Important

Symptom

Cause

Solution

Additional Information

Note

Note 2

Note 3

Note 4

SharePoint 2013/2016 Usage Reports fail to show the correct number of clicks from anonymous access

Symptom

Repro Steps

Results

Expected Results

Cause

Solution

Leveraging the new Time Travel Trace API in Debugging tools to find when one or more SharePoint event happened

The command to enumerate all calls to SharePoint log

Parameters

Logic in a nutshell

Extension command code (partial)

Example 1 – Looking for a particular tag and message

Example 2 – Listing all instances

Leveraging the new WinDbgX and Time-Travel-Trace –Script to list all access to files

Proof of concept OpenFileList.js (this script only WORKS with TTD dumps – .run files)

After updating SharePoint 2013 to November 2017 CU or later you may not be able to open documents with Office

Symptom

Cause

Resolution

SharePoint 2013 and 2016 does not respect the user zone when sending alerts but the Cross Firewall Access Zone (an unusual scenario)

Symptoms

Steps to reproduce

Cause

Workaround:

Tool to generate DebugDiag rule for SharePoint based on event id (Tag) and optionally a partial message

When the ULS log entry points to w3wp.exe

To import the rule into DebugDiag

When the ULS log entry points to a process

PowerShell CmdLet to check OneDrive for Business or OneDrive Personal Status

How to use:

Syntax:

Where:

Examples:

SharePoint 2013/2016 RSS Viewer not working with external sites when through a proxy

Additional information on Proxy discovery

* FINAL UPDATE VI * SharePoint CUs are out