<?xml version="1.0" encoding="utf-8"?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
  version="2.0">
  <channel>
    <title>Zsolt Illes [MSFT]</title>
    <atom:link
      href="https://docs.microsoft.com/archive/blogs/zsoltilles/feed.xml"
      rel="self"
      type="application/rss+xml" />
    <link>https://docs.microsoft.com/archive/blogs/zsoltilles/feed.xml</link>
    <description>Thoughts on SharePoint</description>
    <lastBuildDate>Tue, 16 Apr 2019 06:18:46 GMT</lastBuildDate>
    <language>en-US</language>
    <sy:updatePeriod>hourly</sy:updatePeriod>
    <sy:updateFrequency>1</sy:updateFrequency>
    <item>
      <title>SPFx DEV Environment offline installation</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/spfx-dev-environment-offline-installation</link>
      <pubDate>Fri, 06 Oct 2017 22:02:35 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=695</guid>
      <description><![CDATA[This post is about how to create a Development Environment for SharePoint Framework (SPFx) if your...]]></description>
      <content:encoded><![CDATA[This post is about how to create a Development Environment for SharePoint Framework (SPFx) if your box does not have access to the Internet.
<h1>Exposition</h1>
Microsoft recently published the <a href="https://blogs.technet.microsoft.com/wbaer/2017/09/12/announcing-availability-of-feature-pack-2-for-sharepoint-server-2016/">Feature Pack 2 for SharePoint 2016</a>. The biggest shot in this update is that it makes the long waited <a href="https://blogs.office.com/en-us/2016/05/04/the-sharepoint-framework-an-open-and-connected-platform/?eu=true">SharePoint Framework (SPFx)</a> available for the on-prem customers, which was available for SharePoint Online users since <a href="https://blogs.office.com/en-us/2017/02/23/sharepoint-framework-reaches-general-availability-build-and-deploy-engaging-web-parts-today/?eu=true">23<sup>rd</sup> February 2017</a>. The installation of the framework is a relatively straightforward process, you just have to install Node.JS, Visual Code, Yeoman Generator, and the SharePoint Framework specific toolkit.

The only problem is, that the instructions available currently (<a href="https://www.youtube.com/watch?v=_fxYexlUhe0">like this one from Vesa Juvonen</a>) shows only how to install that if your box has access to the Internet. Of course, since if you are working with SharePoint Online, you're not likely to work on a box without access to the net, but if you are a developer in a locked-down corporate environment, you might have a bit of issue. Let's see how to get around that.

If you want to skip the drama, just jump to the <a href="#FallingAction">Falling Action</a> section of this post.
<h1>Climax</h1>
<h3>Act 1</h3>
To expand the action plan of the SPFx Toolkit installation on a clean machine, this is what you have to do:
<ol>
 	<li>Install the latest version of <a href="https://nodejs.org/en/">Node.JS</a>.</li>
 	<li>In a PowerShell window install the <a href="https://github.com/yeoman/yo">Yeoman Generator</a>.
<ul>
 	<li>npm install -g yo</li>
</ul>
</li>
 	<li>Still in a PowerShell window install <a href="https://github.com/gulpjs/gulp">Gulp</a> (for example) for previewing and testing your solution.
<ul>
 	<li>npm install -g gulp</li>
</ul>
</li>
 	<li>Still in the PowerShell install the Microsoft SharePoint Framework toolkit
<ul>
 	<li>npm install -g @microsoft/generator-sharepoint</li>
</ul>
</li>
 	<li>Finally install the <a href="https://code.visualstudio.com/">Visual Studio Code</a> (or whatever dev environment you want to use).</li>
</ol>
My first thought was that I'm just going to use <a href="https://www.telerik.com/download/fiddler/fiddler4">Fiddler</a> to see what is being downloaded when I run the npm installer. Oh, boy, was I wrong... While Fiddler is a great tool most of the times, certain modules are not using the INetProxy configuration, so obviously it won't be aware that there is something trying to sniff the wire, it will simply get around that, so I had to find something else.
<h3>Act 2</h3>
After a few hours of search I've found a nice little project by Glen R. Goodwin <em>(co-authored by Dan Bornstein and a few others)</em>, called <a href="https://github.com/arei/npmbox">NPMBox</a>, which is an "<span style="font-family: verdana"><em>npm addon utility for creating and installing from an archive file of an npm install, including dependencies. This lets you create a "box" of an installable package and move it to an offline system that will only install from that box.</em></span>"

The tool really does work. It indeed is capable of fetching all the packages, and dependencies <em>(almost, but let's not run that fast...)</em>, and put it into a nice little file that you can copy onto your locked-down environment and install them offline.

The installation of the Yeoman and Gulp packages went without problem, but the installation of the SharePoint Framework toolkit failed with a naughty error message:

<span style="color: #ff0000">Installing @microsoft/generator-sharepoint...</span>
<span style="color: #ff0000">An error occurred while installing @microsoft/generator-sharepoint.</span>
<span style="color: #ff0000">@microsoft/generator-sharepoint was not installed.</span>
<h3>Act 3</h3>
This is where I had to come back to the original question, how can I sniff what npm is doing on the network? Fortunately Luke Brendt figured this out already. Well... Not exactly this, but how to <a href="http://lukeberndt.com/2013/using-node-js-behind-a-proxy/">use Node.JS behind a proxy</a>.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npm config set strict-ssl false
npm config set registry "http://registry.npmjs.org/"
npm --proxy http://username:password@cacheaddress.com.br:80 install packagename</strong></pre>
Given that Fiddler is nothing else, but a proxy, I could use the instructions in his blog entry to configure my machine to use Fiddler as a man in the middle. Awesome... Next installation attempt revealed that the NPMBox package is missing a single component. (http://registry.npmjs.org/postcss-modules-extract-imports/-/postcss-modules-extract-imports-1.1.0.tgz).

How to put this into the package. Well... That's not an easy job, but why would I bother with that, when Fiddler has a nice <a href="http://docs.telerik.com/fiddler/knowledgebase/autoresponder">AutoResponder function</a>? The only thing needed to be done is saving this file somewhere and creating a nice rule pointing to this file. Something like this:

<a href="https://msdnshared.blob.core.windows.net/media/2017/10/FiddlerAutoresponseRule.jpg"><img width="300" height="30" class="alignnone size-medium wp-image-705" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/10/FiddlerAutoresponseRule-300x30.jpg" /></a>

&nbsp;

<span style="color: #ff0000"><em>Originally this was the point where I sat back and thought I was the king of the jungle. Until I saw an email from my colleague <a href="https://www.linkedin.com/in/koenzomers">Koen Zomers</a> highlighting the fact that the Yeoman template will always (or at least from time-to-time) try to connect to the Internet to validate the version of the cached packages. Which of course won't work in a locked down environment. So I pushed forward to pin this one to the ground as well.</em></span>

<span style="color: #ff0000"><em>I have to admit, the solution I describe here is even more crazy than the previous one. I can but hope that there is an easier solution, or at least something is in the works, because this is really clumsy. Should any of you smart people have an idea, please link it in the comment.</em></span>
<h3>Act 4</h3>
Starting the new package creation, when you have a clean environment will give you some spooky error messages, like this:

<a href="https://msdnshared.blob.core.windows.net/media/2017/10/SPFX_NoInternet_Error.jpg"><img width="300" height="72" class="alignnone size-medium wp-image-716" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/10/SPFX_NoInternet_Error-300x72.jpg" /></a>

<span style="color: #ff0000;font-size: x-small">npm ERR! registry error parsing json</span>
<span style="color: #ff0000;font-size: x-small">.. </span>
<span style="color: #ff0000;font-size: x-small">npm ERR! Unexpected token F in JSON at position 1 </span>
<span style="color: #ff0000;font-size: x-small">... </span>
<span style="color: #ff0000;font-size: x-small">npm ERR! The connection to 'registry.npmjs.org' failed. &lt;br /&gt;Error: TimedOut (0x274c). &lt;br /&gt;System.Net.Sockets.SocketException A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 151.101.36.162:80</span>

&nbsp;

Obviously this means <em>(as Koen highlighted)</em> that there's no access to the Internet, so certain packages cannot be downloaded.

If you look closely, you can see that the packages are attempted to be downloaded by <strong>npm</strong>, the same thing that we use for installing packages. <em>This is the Node Package Manager after all.</em> Tiny little problem that we cannot explain Yeoman that it should use the proxy directive, when kicking off npm.

Here came a little digging. Long story short, it seems npm is trying to connect to the Internet directly, and if it cannot, only then it starts looking around for proxy configuration, and attempts to do a second try. But where do we set the proxy configuration? There are two places:
<ol>
 	<li>The system environment variables, where you specify it like this:
<ul>
 	<li>HTTP_PROXY    http://10.2.0.8:8888</li>
 	<li>HTTPS_PROXY  http://10.2.0.9:8888</li>
 	<li><a href="https://msdnshared.blob.core.windows.net/media/2017/10/SPFX_System_Proxy_Setting.jpg"><img width="300" height="67" class="alignnone size-medium wp-image-725" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/10/SPFX_System_Proxy_Setting-300x67.jpg" /></a></li>
</ul>
</li>
 	<li>The WinHTTP configuration
<ul>
 	<li>NETSH WINHTTP SET PROXY 'http://10.2.0.8:8888'</li>
</ul>
</li>
</ol>
But... And this is where I used woodoo... For the testing, I've used Azure Virtual Machines. Most of you should be familiar with the fact that by default Azure VMs have direct access to the Internet. So what I did, I configured two VMs.
<ol>
 	<li>One that was acting as a proxy.
<ul>
 	<li>This has Fiddler installed, and</li>
 	<li>Configured to "Allow remote computers to connect"
<a href="https://msdnshared.blob.core.windows.net/media/2017/10/SPFX_Fiddler_AllowExternalConnections.jpg"><img width="300" height="201" class="alignnone size-medium wp-image-735" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/10/SPFX_Fiddler_AllowExternalConnections-300x201.jpg" /></a></li>
</ul>
</li>
 	<li>Another one that was creating the package.
<ul>
 	<li>This one is using the above mentioned proxy configurations <em>(both the system variable and the WinHTTP)</em>, and</li>
 	<li>is creating the new SPFx package with the good old <strong>yo @microsoft/sharepoint</strong> command.</li>
</ul>
</li>
</ol>
The result is a nice, currently 97MB trace on the first machine that contains all the calls that Yeoman does when creating a new SPFx package. This trace then can be fed to Fiddler on any machine on the AutoResponder tab as we have discussed it earlier.

I've tested it with two different machines and it works as expected. Even more, subsequent projects created on the same day won't even try to connect to the Internet, but at this point I do not know how often it checks for updates. (Daily maybe?)

If it looks stupid, but it works, it ain't stupid...

&nbsp;
<h1><a>Falling Action</a></h1>
To summarize, you have two options to install the bits required on a SharePoint Framework DEV Environment:

&nbsp;
<h4>Method 1: Capture the whole installation network traffic and play it back with Fiddler.</h4>
<ul>
 	<li>This is probably the easier way to do it, although it does not make it possible to automate the installation, should you need to create installation packages.
<ol>
 	<li>Configure NPM to use HTTP traffic.</li>
</ol>
</li>
</ul>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npm config set strict-ssl false
npm config set registry "http://registry.npmjs.org/"
</strong></pre>
<ol>
 	<li style="list-style-type: none">
<ol start="2">
 	<li>Start Fiddler and start capturing.</li>
 	<li>Install your package using Fiddler as a proxy.</li>
</ol>
</li>
</ol>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npm --proxy http://username:password@cacheaddress.com.br:80 install packagename</strong></pre>
<ol>
 	<li style="list-style-type: none">
<ol start="4">
 	<li>Save all packets from Fiddler.</li>
 	<li>Transfer the .SAZ file to your locked down box.</li>
 	<li>Start Fiddler on the locked down box.</li>
 	<li>Import the .SAZ file into the Autoresponse rules.
<a href="https://msdnshared.blob.core.windows.net/media/2017/10/FiddlerAutoresponseImport.jpg"><img width="300" height="83" class="alignnone size-medium wp-image-706" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/10/FiddlerAutoresponseImport-300x83.jpg" /></a></li>
 	<li>Configure the locked down box as you did with the previous one (step 1 in this section).</li>
 	<li>Run the installation using Fiddler as a proxy, just as you did with the previous one (step 3 in this section.)</li>
</ol>
</li>
</ol>
&nbsp;
<h4>Method 2: Use NPMBox to package most of the stuff and use Fiddler to replay only the single file that is missing from the package.</h4>
<ul>
 	<li>Personally, I favor this option as this is cleaner and could be packaged in the future, when the guys fix that tiny little problem of the missing file.
<ol>
 	<li>Install NPMBox.</li>
</ol>
</li>
</ul>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npm install -g npmbox</strong></pre>
<ol>
 	<li style="list-style-type: none">
<ol start="2">
 	<li>Create an npmbox packate for Yeoman, Gulp and the SharePoint Framework Toolkit.</li>
</ol>
</li>
</ol>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npmbox --proxy='http://127.0.0.1:8888' -t c:\Temp\NPMBox NPMBox
npmbox --proxy='http://127.0.0.1:8888' -t c:\Temp\Yo yo
npmbox --proxy='http://127.0.0.1:8888' -t c:\Temp\Gulp gulp
npmbox --proxy='http://127.0.0.1:8888' -t c:\Temp\SPFx @microsoft/generator-sharepoint</strong></pre>
<ol>
 	<li style="list-style-type: none">
<ol start="3">
 	<li>Configure NPM to use HTTP traffic.</li>
</ol>
</li>
</ol>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npm config set strict-ssl false
npm config set registry "http://registry.npmjs.org/"</strong></pre>
<ol>
 	<li style="list-style-type: none">
<ol start="4">
 	<li>Re-install the SharePoint Framework toolkit using the NPMBox package.</li>
</ol>
</li>
</ol>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npmunbox --proxy='http://127.0.0.1:8888' -g SPFx</strong></pre>
<ol>
 	<li style="list-style-type: none">
<ol start="5">
 	<li>Take note of the URL of the extra package that is loaded by the installer. It's going to be a version of the <em>postcss-modules-extract-imports-1.1.0.tgz</em> module. <em>(Note: this step is required only until the NPMBox guys fix the issue with the missing packet.)</em></li>
 	<li>Save the module as a file.</li>
 	<li>Copy the NPMBox packages and the extra file to the locked down box.</li>
 	<li>Unpack the NPMBox file. For this you will need to rename the NPMBox.npmbox file to NPMBox.tar. On Windows machines, you'll need to use something like <a href="http://7-zip.org/download.html">7ZIP</a>.</li>
 	<li>Change to the folder where you unpacked the NPMBox package, then install the NPMBox offline on the locked down box</li>
</ol>
</li>
</ol>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npm install --global --cache .\.npmbox.cache --optional --cache-min 99999999999 --shrinkwrap false npmbox</strong></pre>
<ol>
 	<li style="list-style-type: none">
<ol start="10">
 	<li>Install Fiddler and configure an AutoResponder rule for the missing packet with the URL in step 5. <em><em>(Note: this step is required only until the NPMBox guys fix the issue with the missing packet.)
</em></em><a href="https://msdnshared.blob.core.windows.net/media/2017/10/FiddlerAutoresponseRule.jpg"><img width="300" height="30" class="alignnone size-medium wp-image-705" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/10/FiddlerAutoresponseRule-300x30.jpg" /></a></li>
 	<li>Use NPMUnbox to install the offline packages.</li>
</ol>
</li>
</ol>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>npmunbox --proxy='http://127.0.0.1:8888' -g Yo
npmunbox --proxy='http://127.0.0.1:8888' -g Gulp
npmunbox --proxy='http://127.0.0.1:8888' -g SPFx</strong></pre>
&nbsp;

Don't forget that you'll need to have a Fiddler trace available with the packages that are required for Yeoman when creating a new SPFx package. See Act 4 for the details.

&nbsp;
<h1>Dénouement</h1>
Now that you have a way to deploy a SharePoint Framework (SPFx) environment in any locked down environment, you can test, try, use and spread the word that this is really some good stuff for SharePoint let it be Online or On-prem installation.

Once again, I admit this is hacky. If you have a better solution, I'm open to suggestion. Hopefully someone will implement offline functionality for Yo. Also, this whole thing is only required if you have a locked down environment. But maybe your company will go to the cloud sooner or later and you won't need this hacking. Future is the Cloud, after all. ;)

&nbsp;]]></content:encoded>
    </item>
    <item>
      <title>Myths and truth about the SharePoint Workflow History list</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/myths-and-truth-about-the-sharepoint-workflow-history-list</link>
      <pubDate>Wed, 24 May 2017 22:15:40 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=665</guid>
      <description><![CDATA[This post is about demystifying the Workflow History list and some of the confusion around...]]></description>
      <content:encoded><![CDATA[This post is about demystifying the Workflow History list and some of the confusion around it.

&nbsp;
<h1>Exposition</h1>
To be honest, this is a second try around this topic. The whole story started with a question from my colleague <a href="https://www.linkedin.com/in/christophersscott/">Christopher Scott</a>. His customer is in SharePoint Online now and using Workflows to automate their business logic. For various reasons they need to keep the Workflow History entries for a long time. While they've been using an on-prem installation, they simply disabled the Workflow History Cleanup timer job, but obviously in SPO this is not something you could do. So the question was, how to keep the history entries with a single click to the users?

Here comes why this is the second try. Even I got confused a bit. Let's see how.
<h1>Rising Action</h1>
<h3>Act 1</h3>
Before I dig deep into this, a little clarification for those who are not familiar with what the Workflow History Cleanup timer job does. Despite of what the name suggests, it <span style="text-decoration: underline">does not</span> clean up the Workflow History list. <em>(This is where the log entries you add in your workflow are stored.)</em> It does, however, two other things:
<ul>
 	<li>removes the SP2010 Workflow History link from the list item
<a href="https://msdnshared.blob.core.windows.net/media/2017/05/WFHistory1.jpg"><img width="300" height="53" class="alignnone wp-image-685 size-medium" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/05/WFHistory1-300x53.jpg" /></a></li>
 	<li>removes the Workflow Instance reference from the workflow service's register.</li>
</ul>
This also means that the items will ever be in the Workflow History list, unless you start manually deleting them. <em>(Which you should, by the way. Stand by for a post on that...) </em>

As for the SP2013 style workflows, the reference is not stored in the SharePoint Workflow infrastructure, but in the Workflow Manager. So there is a different logic for that, but the behavior is kind of the same, the result is that your Workflow instance information is gone after 30 days.
<h3>Act 2</h3>
Given the fact that the entries in the Workflow History list are there, you might think, if you save the link for the Workflow History page, you can access the history information forever. Well... <a href="http://knowyourmeme.com/memes/that-s-where-you-re-wrong-kiddo">That's where you are wrong, kiddo!</a>

If you are familiar with the Workflow History page <em>(of course you are, that's why you are reading this entry)</em>, you know there is a field called Internal Status. It is there for both SP2010 and SP2013 style workflows.
<a href="https://msdnshared.blob.core.windows.net/media/2017/05/201WFInternalStatus.jpg"><img width="300" height="96" class="alignnone size-medium wp-image-675" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/05/201WFInternalStatus-300x96.jpg" /></a>

The value of this field is also responsible for displaying some context information, like stopping the workflow, list of the related Tasks, etc.

This is where the Workflow History Cleanup timer job comes into play again. As I mentioned it earlier, it cleans the reference in the SP2010 style Workflow Infrastructure, leaving you with a nice <em>"<span id="ctl00_PlaceHolderMain_LabelMessage">Invalid WorkflowInstanceID parameter in URL.</span>"</em> error message.
<a href="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFInvalidWFInstanceID.jpg"><img width="300" height="91" class="alignnone size-medium wp-image-676" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFInvalidWFInstanceID-300x91.jpg" /></a>

The error with the SP2013 style workflows is a bit different, but the result is the same, you cannot see the WF History entries.

<a href="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFCleanedWFInstance.jpg"><img width="300" height="79" class="alignnone size-medium wp-image-686" alt="" src="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFCleanedWFInstance-300x79.jpg" /></a>

That's pretty much it. The built-in Workflow History page is not usable. It would be a lot easier if - instead of the above error messages - the page would simply put "Already cleaned up" into the Internal Status field, and still display the rest of the page, but it does not.

&nbsp;
<h1>Climax</h1>
<h3>Act 3</h3>
Does this mean that the information we've stored in the Workflow History list is useless? Well... Not entirely. While there is no built-in way to display the content, it does not mean that we cannot dig out this information. Let's take a quick look at the actual WF History List that you can reach on a similar URL  <a href="https://www.contoso.com/sites/TestSite/Lists/Workflow%20History/AllItems.aspx">https://www.contoso.com/sites/TestSite/Lists/Workflow%20History/AllItems.aspx</a>. You would see something like this:

[caption id="attachment_436" align="alignnone" width="300"]<a href="https://msdnshared.blob.core.windows.net/media/2017/05/WFHistoryList.jpg"><img width="300" height="136" class="size-medium wp-image-436" alt="This is how the actual Workflow History List looks like" src="https://msdnshared.blob.core.windows.net/media/2017/05/WFHistoryList-300x136.jpg" /></a> This is how the actual Workflow History List looks like[/caption]

Let's observe the columns a bit:
<table border="1">
<tbody>
<tr style="padding-left: 5px">
<th>Workflow engine</th>
<th>Workflow History Parent Instance</th>
<th>Workflow Association ID</th>
<th>Workflow Template ID</th>
<th>List ID</th>
<th>Primary Item ID</th>
</tr>
<tr style="padding-left: 5px">
<td>2010 Style WF</td>
<td>GUID of the WF instance</td>
<td>GUID of the WF association</td>
<td>GUID of the WF association version</td>
<td>GUID of the List</td>
<td>Number of the item in the List</td>
</tr>
<tr style="padding-left: 5px">
<td>2013 Style WF</td>
<td>GUID of the WF instance</td>
<td>Name of the WF association</td>
<td bgcolor="#f78181">N/A</td>
<td bgcolor="#f78181">N/A</td>
<td bgcolor="#f78181">N/A</td>
</tr>
</tbody>
</table>
How to get back the information then?
<h3>Act 4</h3>
For the SP2010 style workflows it is relatively easy. You either know the GUID of your Workflow's association, or you will have to look it up. Hopefully you are in the lucky situation that you have not deleted the Workflow from the list yet, so you can get it with a simple REST call to the Workflow service. <em>(Should you need help with playing with the REST calls, <a href="https://blogs.technet.microsoft.com/zsoltilles/2017/05/21/calling-sharepoint-rest-api-with-post-from-fiddler/">read this entry</a>.)</em>
<ul></ul>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><b>https://www.contoso.com/sites/testsite/</b>_api/Web/Lists/GetByTitle('MyList')/WorkflowAssociations?$filter=Name eq 'My2010WFName'&amp;$Select=BaseID,ListID</strong></pre>
<ul>
 	<li>The BaseID will be equal to the WorkflowTemplateID.</li>
 	<li>While the ListID is the GUID of the original list where the item is.</li>
</ul>
Thus, if you know the ID <em>(not the GUID)</em> of the item, you can use the below REST call to get all the related entries from the Workflow History list.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>https://www.contoso.com/sites/testsite/_api/Web/Lists/GetByTitle('Workflow%20History')/Items?$filter=((substringof('<em>&lt;ListGUID&gt;</em>',List)) and (substringof('<em>&lt;BaseID&gt;</em>',WorkflowTemplate)) and (Item eq <em>&lt;ItemID&gt;</em>))&amp;$orderby=ID desc</strong></pre>
To understand the above query, keep in mind that:
<ul>
 	<li>Filter for the WorkflowTemplateID, where it equals the BaseID,</li>
 	<li>Order the results by Modified descending.</li>
 	<li>Should you have run the workflow more than once, it will give you all the entries for that Workflow definition.</li>
 	<li><em>(Note that the { and } characters are forbidden in the Workflow definition, so we have to use the SubstringOf method to filter for the List and WorkflowTemplate columns.)</em></li>
</ul>
&nbsp;
<h3>Act 5</h3>
The SP2013 style workflow is a bit different. Remember, the link is not cleaned up by the Workflow History timer job, so at least we have the WorkflowInstanceName in the URL.
<table border="1">
<tbody>
<tr>
<th>WF Style</th>
<th>Link schema</th>
</tr>
<tr>
<td align="center">2013 Style WF</td>
<td style="padding-left: 5px">/sites/testsite/_layouts/15/wrkstat.aspx?<span style="color: #ff6600">List=8d56ca92-0687-4032-972c-10df5ad0d82e</span>&amp;<span style="color: #ff0000">WorkflowInstanceName=3bf2ef56-d100-4a61-9354-2b08f4a0aacc</span></td>
</tr>
</tbody>
</table>
This information we can use in a similar REST query:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong>https://www.contoso.com/sites/testsite/_api/web/lists/<b>GetByTitle('Workflow%20History')</b>/Items?$filter=WorkflowInstance eq '<em>&lt;WFInstanceGUIDFromURL&gt;</em>'$orderby=ID desc</strong></pre>
Luckily as here we are using an actual Workflow instance GUID, we won't have information about the previous runs. I assume you can guess, this means the information about previous runs is lost forever. ... ... Unless... ... By any chance you put the item's ID into one of the comment fields, in which case you can still dig the info out. But then we really are getting into the archeology of Workflows.

&nbsp;
<h1>Falling Action</h1>
So getting the information from the Workflow History page is possible even after the related workflow instance has been cleaned up. This is not too convenient and if you really want to use it, you should create your own Workflow History page.

&nbsp;
<h1>Dénouement</h1>
It would be a lot more comfortable if the built-in Workflow History page would not fail if the Workflow instance ID has already been cleaned up. But given the fact that both the SP2010 and the Sp2013 style workflows are already in deprecated status and that Microsoft is focusing on MSFlow which does not have a default history list anyway, it would be a good idea to get used to creating your own workflow history in case you need it for legal reasons.

(Stand by for the next entry around that topic.)]]></content:encoded>
    </item>
    <item>
      <title>The workflow instance exceeded the CPU usage limit of throttle...</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/the-workflow-instance-exceeded-the-cpu-usage-limit-of-throttle</link>
      <pubDate>Mon, 22 May 2017 17:12:15 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=625</guid>
      <description><![CDATA[This is a quick one act drama about what happens if you do an infinite loop in a SharePoint 2013...]]></description>
      <content:encoded><![CDATA[This is a quick one act drama about what happens if you do an infinite loop in a SharePoint 2013 Workflow.
<h3>Act 1</h3>
Iterating through a set of items in a SharePoint 2013 style workflow is not a one-step action. The infrastructure does not have a ForEach cycle, it can only do <em>Loop with Condition</em> and <em>Loop n Times</em>.

How to iterate through all items? Easy.
<ol>
 	<li>You get the number of items you want to iterate through.</li>
 	<li>Set a pointer.</li>
 	<li>Do your magic.</li>
 	<li>Increment the pointer.</li>
</ol>
Your Workflow section would look something like this:

<a href="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFLoop.jpg"><img width="300" height="125" class="alignnone size-medium wp-image-626" alt="Standard Loop to iterate through items." src="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFLoop-300x125.jpg" /></a>

In this case it is pretty straight forward as the loop runs only a predefined number of times. Thus worst case scenario; you keep doing operation on the same item.

Let's say, however, that you do not know which item you are looking for. Let's assume the conditions are so complex that you cannot resolve it with a REST call, and you have to iterate through each of the items until you find what you need. In this case you need to use the other loop; <i>Loop with Condition</i>. Your loop would look something like this:

<a href="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFLoop2.jpg"><img width="300" height="159" class="alignnone wp-image-635 size-medium" alt="Standard Loop to iterate through items." src="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFLoop2-300x159.jpg" /></a>

It might happen though that you forget to increment the counter. Which means your cycle is doing operation on the same item over and over again, and unless the first item is the winner, you get into an infinite loop.

The Workflow Manager has a nice little feature to discover this and will give you this error message:

[caption id="attachment_616" align="alignnone" width="300"]<a href="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFSuspended.jpg"><img width="300" height="117" class="wp-image-616 size-medium" alt="The workflow instance exceeded the CPU usage limit of throttle of 00:00:01.2000000 and could not be unloaded because it was not persistable." src="https://msdnshared.blob.core.windows.net/media/2017/05/2013WFSuspended-300x117.jpg" /></a> The workflow instance exceeded the CPU usage limit of throttle of 00:00:01.2000000 and could not be unloaded because it was not persistable.[/caption]

Thus... If you see this, most probably you have some infinite loop somewhere.]]></content:encoded>
    </item>
    <item>
      <title>Calling SharePoint REST API with POST from Fiddler</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/calling-sharepoint-rest-api-with-post-from-fiddler</link>
      <pubDate>Sun, 21 May 2017 21:04:12 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=555</guid>
      <description><![CDATA[This is a quick one act drama about how to use Fiddler to issue POST requests to a SharePoint REST...]]></description>
      <content:encoded><![CDATA[This is a quick one act drama about how to use <a href="http://www.telerik.com/fiddler">Fiddler</a> to issue POST requests to a SharePoint REST API.
<h3>Act 1</h3>
Fiddler has a nice feature called <a href="http://docs.telerik.com/fiddler/Generate-Traffic/Tasks/CreateNewRequest">Composer</a> which can be used to create your own request with your own verb, header and content information.

Certain SharePoint web services will require GET, others POST calls. The POST call is just slightly different from a GET. There are a few things you need to keep in mind:
<ol>
 	<li>Fiddler cannot authenticate against POST, so you should have a cookie available.</li>
 	<li>You need to post an X-RequestDigest header.</li>
 	<li>You might need to have a payload in the Message Body.</li>
</ol>
How you do the authentication, is up to you, as for me, I just issue a query from a browser, record the traffic, steal the cookie information and there you go. In theory Fiddler does the authentication for you <em>(If you have the checkbox Automatically Authenticate set on the Options tab.) </em>but that did not work reliably for me for SPO, so I dropped it.

So to get the Request Digest token, you just call a POST query against the _api/contextinfo web service.

<a href="https://msdnshared.blob.core.windows.net/media/2017/05/FiddlerContextInfo.jpg"><img width="300" height="51" class="alignnone size-medium wp-image-565" alt="Calling ContextInfo from Fiddler" src="https://msdnshared.blob.core.windows.net/media/2017/05/FiddlerContextInfo-300x51.jpg" /></a>

The information will be in the FormDigestValue attribute:

<a href="https://msdnshared.blob.core.windows.net/media/2017/05/FiddlerFormDigestValue.jpg"><img width="300" height="85" class="alignnone size-medium wp-image-566" alt="FormDigestValue in the response" src="https://msdnshared.blob.core.windows.net/media/2017/05/FiddlerFormDigestValue-300x85.jpg" /></a>

Now you can put this information into your next POST request's header and there you go, you can test the API's behavior.

<a href="https://msdnshared.blob.core.windows.net/media/2017/05/FiddlerPOSTWebService.jpg"><img width="300" height="114" class="alignnone size-medium wp-image-575" alt="POST call to an SPO web service" src="https://msdnshared.blob.core.windows.net/media/2017/05/FiddlerPOSTWebService-300x114.jpg" /></a>]]></content:encoded>
    </item>
    <item>
      <title>Generate Random Content for SharePoint – 4</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/generate-random-content-for-sharepoint-4</link>
      <pubDate>Tue, 28 Mar 2017 17:03:14 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=385</guid>
      <description><![CDATA[This is the last chapter of the Generate Random Content For SharePoint series. This time, detailing...]]></description>
      <content:encoded><![CDATA[This is the last chapter of the Generate Random Content For SharePoint series. This time, detailing the challenges you might face when you want to create PowerPoint presentations.

<strong>Chapters:</strong>
<ul>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/02/26/generate-random-content-for-sharepoint/">Generate random content for SharePoint</a> – About how to create big files for upload testing.</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/05/generate-random-content-for-sharepoint-2/">Generate random content for SharePoint 2</a> – About how to generate Word files for Search and integration testing.</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/12/generate-random-content-for-sharepoint-3/">Generate random content for SharePoint 3</a> – About how to generate Excel files for Search and integration testing.</li>
 	<li>Generate random content for SharePoint 4 – About how to generate PowerPoint files for Search and integration testing. (This article.)</li>
</ul>
(If you don’t want to read through all the drama, jump directly to the Falling Action section.)

&nbsp;
<h1>Exposition</h1>
Last, but not least we got to the last chapter, in the generation of random content for our stress test. We have big files, we have Word documents, Excel Sheets and here we will discuss the automatic creation of PowerPoint presentations.

Just as in the earlier posts, let's see what do we have in a PowerPoint presentation. We have slides with text, maybe pictures, tables here and there and charts perhaps. I could have gone really deep into creating all these objects in my presentation, but I think simple texts and pictures will do for most of the scenarios. If not, at least you'll have a baseline to start with, and setting up these extra functions should not be more than a few minutes of coding.
<h1>Rising Action</h1>
<h3>Act 1</h3>
Even though we are talking about the same suit, there are important differences between the Office Applications, as you might have already noticed. For example, while in Word and Excel you can use simple string for defining 'True' and 'False' values, in PowerPoint, you have to use explicit casting, otherwise you'll get scary error messages:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$msoTrue</span> = [Microsoft.Office.Core.MsoTriState]::msoTrue
<span style="color: #ff0000">$msoFalse</span> = [Microsoft.Office.Core.MsoTriState]::msoFalse

<span style="color: #ff0000">$PPTDocument</span> = $PPTApplication.Presentations.Add(<span style="color: #ff0000">$msoFalse</span>)
</span></span></span></span></span></strong></pre>
For the full list of MsoTriStates check out the <a href="https://msdn.microsoft.com/en-us/library/office/microsoft.office.core.msotristate.aspx">MsoTriState Enumeration</a> MSDN article.

&nbsp;
<h3>Act 2</h3>
Due to the special format of the slides, it does not worth generating the text in a text file as we used it with Word or Excel. Luckily however the content of a PowerPoint presentation is not that big to worry that it would take too long to generate even a few hundreds of files.

In fact, in my test generating 100 files with 20 slides in each, with 10 words on each slide and randomly inserting pictures on the slides took only <strong>79</strong> seconds, which is not that bad.

&nbsp;
<h1>Climax</h1>
<h3>Act 3</h3>
Otherwise the generation of the PowerPoint slides themselves is not much different than the one we have seen with the Excel sheets: we have to address the Document, then the Slide (in Excel the Sheet), then the Shape (in Excel the cell).

Because you can create slides from the various slide templates available in the Slide Master gallery, when creating the slide, you have to choose it too. Just as with the 'True' and 'False' values, these have to be explicitly casted. In the below example you can see a simple Text Slide:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$PPTSlideLayout</span> = [microsoft.office.interop.powerpoint.ppSlideLayout]::ppLayoutText
<span style="color: #ff0000">$Slide</span> = $PPTDocumet.Slides.Add(<span style="color: #ff0000">$SlideCounter</span>,<span style="color: #ff0000">$PPTSlideLayout</span>)
</span></span></span></span></strong></pre>
A list of the OOB available slide templates can be found on MSDN of course under the <a href="https://msdn.microsoft.com/en-us/library/office/microsoft.office.interop.powerpoint.ppslidelayout(v=office.14).aspx">ppSlideLayout Enumeration</a> article.

The $SlideCounter variable above is the actual number of the slide that you want to use. The counter starts from one and goes all the way up to the number of slides you want to create, of course. Be aware, if you use a number that is already used, for example you already have two slides and $SlideCounter = 2, then the new slide will be inserted to be the second slide and the rest of the slides will be shifted.

Adding text to the slide itself is not magic, but given the fact that in PowerPoint you can do way more with a Shape than what you can do with an Cell in Excel, adding a text to a Title or simple Text shape is a bit more complex:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$Slide</span>.Shapes.Title.TextFrame.TextRange.Text = <span style="color: #ff0000">$TitleText</span>
<span style="color: #ff0000">$Slide</span>.Shapes.Item(2).TextFrame.TextRange.Text = <span style="color: #ff0000">$SlideText
</span></span></span></span></span></strong></pre>
Of course, the $TitleText and $SlideText variables are filled out with the help of the dictionary file that we have used in the earlier posts.

&nbsp;
<h3>Act 4</h3>
Adding a picture to the slide is a bit more tricky though. First and foremost, the picture size is not measured in pixels as you might think, but points. What the reason is behind this, I can't tell, but this is not the focus of my blog post anyway. The important thing you need to learn, however, is that the base reference is 72 dots per inch. What does this mean? It means that to be able to address the size correctly in your code, you will have to do some math:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$PictureWidth</span> = [<span style="color: #008080">math</span>]::<span style="color: #000000">Round((</span><span style="color: #ff0000">$Picture</span>.<span style="color: #000000">Width</span> / <span style="color: #ff0000">$Picture<span style="color: #999999">.</span></span><span style="color: #000000">HorizontalResolution</span> * <span style="color: #800080">72</span><span style="color: #000000">)</span>,<span style="color: #800080">4</span><span style="color: #000000">)
</span><span style="color: #ff0000">$PictureHeight</span> = [<span style="color: #008080">math</span>]::<span style="color: #000000">Round((</span><span style="color: #ff0000">$Picture</span>.<span style="color: #000000">Height</span> / <span style="color: #ff0000">$Picture</span>.<span style="color: #000000">VerticalResolution</span> * <span style="color: #800080">72</span><span style="color: #000000">)</span>,<span style="color: #800080">4</span><span style="color: #000000">)
</span></span></span></span></span></strong></pre>
This way the numbers you put into your code to size of the picture will not distort it at all. Otherwise you might see that your pictures grow a bit bigger than you expect.

Speaking of resizing... I put in a little rutin that scales the pictures according to the maximal size you want to use on your slides. The process is pretty straight forward, the only thing needs to be done is we need to make sure the picture is scaled both horizontally and vertically:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #000080">If</span> <span style="color: #000000">(</span><span style="color: #ff0000">$PictureWidth</span> -gt <span style="color: #ff0000">$PicturesMaxWidth</span><span style="color: #000000">)</span>
<span style="color: #000000">{</span>
    <span style="color: #ff0000">$Multiplier</span> = <span style="color: #ff0000">$PicturesMaxWidth</span> / <span style="color: #ff0000">$PictureWidth</span>
    <span style="color: #ff0000">$PictureHeight</span> = <span style="color: #ff0000">$PictureHeight</span> * <span style="color: #ff0000">$Multiplier</span>
<span style="color: #000000">}</span>
<span style="color: #000080">If</span> (<span style="color: #ff0000">$PictureHeight</span> -gt <span style="color: #ff0000">$PicturesmaxHeight</span>)
<span style="color: #000000">{</span>
    <span style="color: #ff0000">$Multiplier</span> = <span style="color: #ff0000">$PicturesmaxHeight</span> = <span style="color: #ff0000">$PictureHeight</span>
    <span style="color: #ff0000">$PictureWidth</span> = <span style="color: #ff0000">$PictureWidth</span> * <span style="color: #ff0000">$Multiplier</span>
<span style="color: #000000">}
</span></span></span></span></span></strong></pre>
Of course, this is just half of the solution as adding the picture requires some extra directives, such as whether we want to reference to the original picture or not, or if we want to save the picture with the file or not.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$pic</span> = <span style="color: #ff0000">$Slide</span>.<span style="color: #000000">Shapes</span>.<span style="color: #000000">AddPicture((</span><span style="color: #ff0000">$RandomPicture</span>.<span style="color: #000000">FilePath)</span>,<span style="color: #ff0000">$msoFalse</span>, <span style="color: #ff0000">$msoTrue</span>,<span style="color: #ff0000">$PicturesLeft</span>,<span style="color: #ff0000">$PicturesTop</span>, <span style="color: #ff0000">$RandomPicture</span>.<span style="color: #000000">Width</span>, <span style="color: #ff0000">$RandomPicture</span>.<span style="color: #000000">Height)
</span></span></span></span></span></strong></pre>
Details of the <a href="https://msdn.microsoft.com/en-us/library/office/ff745953.aspx">AddPicture method</a> can be found on MSDN.

Additionally you might wish to insert only pictures of a specific MIME type. To do that you will have to use the <a href="https://msdn.microsoft.com/en-us/library/system.drawing.imaging.imageformat.aspx">ImageFormat Class</a> Enumeration.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$Imgage</span>.<span style="color: #000000">RawFormat</span>.<span style="color: #000000">Equals(</span>[<span style="color: #008080">System.Drawing.Imaging.ImageFormat</span>]::<span style="color: #000000">Png)
</span></span></span></span></span></strong></pre>
&nbsp;
<h3>Act 5</h3>
There's one more thing that could be interesting and that is the transitions of the slides. This might seem like an obsolete feature to put into some test documents, but remember, these are not only for search, but maybe other purposes. For example, you could test the Office WebApps, or Office Online server's functionality with these nice transitions.

As you know, the transition can be bound to shapes and slides as well, so you can assign the animation to both.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$Slide</span>.<span style="color: #000000">SlideShowTransition</span>.<span style="color: #000000">EntryEffect</span> = <span style="color: #800080">769


</span><span style="color: #ff0000">$Slide</span>.<span style="color: #000000">Shapes</span>.<span style="color: #000000">Title</span>.<span style="color: #000000">AnimationSettings</span>.<span style="color: #000000">TextLevelEffect</span> = <span style="color: #800080">1</span>
<span style="color: #ff0000">$Slide</span>.<span style="color: #000000">Shapes</span>.<span style="color: #000000">Title</span>.<span style="color: #000000">AnimationSettings</span>.<span style="color: #000000">EntryEffect</span> = <span style="color: #800080">769
</span><span style="color: #ff0000">$Slide</span>.<span style="color: #000000">Shapes</span>.<span style="color: #000000">Title</span>.<span style="color: #000000">AnimationSettings</span>.<span style="color: #000000">Animate</span> = <span style="color: #ff0000">$msoTrue
</span></span></span></span></span></strong></pre>
If you paid enough attention, you might have noticed that the EntryEffect property is using digit value. These numbers are coming from the <a href="https://msdn.microsoft.com/en-us/library/office/ff746794.aspx">PpEntryEffect Enumeration</a> MSDN article, but unfortunately I have not found a way to get only the animations available in PowerPoint, so I had to do some dirty casting:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$Animations</span> = <span style="color: #800080">3844</span>, <span style="color: #800080">769</span>, <span style="color: #800080">770</span>, <span style="color: #800080">3074</span>, <span style="color: #800080">3073</span>, <span style="color: #800080">1025</span>, <span style="color: #800080">1026</span>, <span style="color: #800080">3845</span>, <span style="color: #00ff00">... ...</span> , <span style="color: #800080">3347</span>, <span style="color: #800080">3348
</span></span></span></span></span></strong></pre>
Then using this array to create an animation is pretty simple:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$RandomAnimation</span> = <span style="color: #000000">(</span><span style="color: #ff0000">$TextAnimationEffects</span> | <span style="color: #0000ff">Get-Random</span><span style="color: #000000">)</span>
<span style="color: #ff0000">$Slide</span>.<span style="color: #000000">Shapes</span>.<span style="color: #000000">Title</span>.<span style="color: #000000">AnimationSettings</span>.<span style="color: #000000">EntryEffect</span> = <span style="color: #ff0000">$RandomAnimation
</span></span></span></span></span></strong></pre>
&nbsp;
<h1>Falling Action</h1>
<h3>Act 6</h3>
There is one single thing that bothers me a lot and that is proper cleanup. I was not able to find a way to properly close the COM objects and as such, the PowerPoint application stays in the memory, even after I closed the PowerShell window. This is what I used:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999">[<span style="color: #008080">gc</span>]::<span style="color: #000000">collect()
</span>[<span style="color: #008080">gc</span>]::<span style="color: #000000">WaitForPendingFinalizers()
</span>[<span style="color: #008080">gc</span>]::<span style="color: #000000">collect()</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">WaitForPendingFinalizers()
</span><span style="color: #ff0000">$PPTDocument</span>.<span style="color: #000000">Saved</span> = <span style="color: #ff0000">$msoTrue</span>
<span style="color: #ff0000">$PPTDocument</span>.<span style="color: #000000">Close()</span>
<span style="color: #ff0000">$null</span> = [<span style="color: #008080">System.Runtime.InteropServices.Marshal</span>]::<span style="color: #000000">FinalReleaseComObject(</span><span style="color: #ff0000">$PPTDocument</span><span style="color: #000000">)</span>
<span style="color: #ff0000">$PPTDocument</span> = <span style="color: #ff0000">$null</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">collect()</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">WaitForPendingFinalizers()</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">collect()</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">WaitForPendingFinalizers()</span>
<span style="color: #ff0000">$PPTApplication</span>.<span style="color: #000000">quit()</span>
<span style="color: #ff0000">$null</span> = [<span style="color: #008080">System.Runtime.InteropServices.Marshal</span>]::<span style="color: #000000">FinalReleaseComObject(</span><span style="color: #ff0000">$PPTApplication</span><span style="color: #000000">)</span>
<span style="color: #ff0000">$PPTApplication</span> = <span style="color: #ff0000">$null</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">collect()</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">WaitForPendingFinalizers()</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">collect()</span>
[<span style="color: #008080">gc</span>]::<span style="color: #000000">WaitForPendingFinalizers()
</span></span></span></span></span></strong></pre>
If any of you guys have a better - and working - way to do a proper cleanup, please feel free to share it in the comments.

The scripts are still available on Codeplex (<a href="https://randomfilegenerator.codeplex.com/">link</a>). Of course this solution still has room for improvement, but that is up for you guys to play with.
<h1>Dénouement</h1>
Now that we have different files available, we can start our precious performance testing. Keep an eye out for those entries soon.]]></content:encoded>
    </item>
    <item>
      <title>Generate Random Content for SharePoint - 3</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/generate-random-content-for-sharepoint-3</link>
      <pubDate>Sun, 12 Mar 2017 18:09:36 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=325</guid>
      <description><![CDATA[This is the third chapter of the Generate Random Content For SharePoint series. This time, detailing...]]></description>
      <content:encoded><![CDATA[This is the third chapter of the Generate Random Content For SharePoint series. This time, detailing the challenges you might face when you want to create Excel workbooks. <strong>Chapters:</strong>
<ul>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/02/26/generate-random-content-for-sharepoint/">Generate random content for SharePoint</a> – About how to create big files for upload testing.</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/05/generate-random-content-for-sharepoint-2/">Generate random content for SharePoint 2</a> – About how to generate Word files for Search and integration testing.</li>
 	<li>Generate random content for SharePoint 3 – About how to generate Excel files for Search and integration testing. (This article.)</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/28/generate-random-content-for-sharepoint-4/">Generate random content for SharePoint 4</a> – About how to generate PowerPoint files for Search and integration testing.</li>
</ul>
(If you don’t want to read through all the drama, jump directly to the Falling Action section.)
<h1>Exposition</h1>
If you've been following my previous posts, by now you should know how to generate unique, big files for upload test, and Word documents with random words to test the indexing functionality. In this post I'll show you the challenges you might face when creating Excel files.

&nbsp;
<h1>Rising Action</h1>
<h3>Act 1</h3>
An Excel table is relatively simple with column- and row headers and some numbers. Unfortunately we do not have the "Lorem ipsum" generator as in Word, but in the previous section we already discussed that this would not be good anyway. We could however use the methods learned in the previous post; get the header colums' content from a dictionary and use a simple random generator for the numbers.
<pre style="padding-left: 30px;font-family: 'Courier New';font-size: 10pt;background-color: #f4f4f4"><strong><span style="color: #ff0000"><span style="color: #99cc00"># Generate the first line with the headers</span>
<span style="color: #000000">For (</span>$Column <span style="color: #999999">=</span> <span style="color: #800080">1</span><span style="color: #000000">;</span> $Column <span style="color: #999999">-le</span> $Columns<span style="color: #000000">;</span> $Column<span style="color: #999999">++</span><span style="color: #000000">)</span>
<span style="color: #000000">{</span>
    $RandomWordLine <span style="color: #999999">=</span> <span style="color: #0000ff">Get-Random</span><span style="color: #000080"> -Minimum</span> <span style="color: #800080">1</span> <span style="color: #000080">-Maximum</span> $DictionaryFileRows
    $RandomWord <span style="color: #999999">=</span> $DictionaryFileContent<span style="color: #999999">[</span>$RandomWordLine<span style="color: #999999">]</span>
    $FileStream<span style="color: #000000">.Write(</span><span style="color: #993300">"`t</span>$RandomWord<span style="color: #993300">"</span><span style="color: #000000">)</span>
<span style="color: #000000">}</span>
<span style="color: #99cc00"># And the rows with the content</span>
<span style="color: #000000">For (</span>$Row <span style="color: #999999">=</span> 1<span style="color: #000000">;</span> $Row <span style="color: #999999">-le</span> $Rows<span style="color: #000000">;</span> $Row<span style="color: #999999">++</span><span style="color: #000000">)</span>
<span style="color: #000000">{</span>
    $FileStream<span style="color: #000000">.Write(</span><span style="color: #993300">"`r`n"</span><span style="color: #000000">)</span> <span style="color: #99cc00"># New line</span>
    $RandomWordLine <span style="color: #999999">=</span> <span style="color: #0000ff">Get-Random</span> <span style="color: #000080">-Minimum</span> <span style="color: #800080">1</span> <span style="color: #000080">-Maximum</span> $DictionaryFileRows
    $RandomWord <span style="color: #999999">=</span> $DictionaryFileContent<span style="color: #999999">[</span>$RandomWordLine<span style="color: #999999">]</span>
    $fileStream<span style="color: #000000">.Write(</span><span style="color: #993300">"</span>$RandomWord<span style="color: #993300">`t"</span><span style="color: #000000">)</span> <span style="color: #99cc00"># Row first column</span>
    <span style="color: #000000">For (</span>$Column <span style="color: #999999">=</span> <span style="color: #800080">1</span><span style="color: #000000">;</span> $Column <span style="color: #999999">-le</span> $Columns<span style="color: #000000">;</span> $Column<span style="color: #999999">++</span><span style="color: #000000">)</span>
    <span style="color: #000000">{</span>
        $NumberTXT <span style="color: #999999">= [</span>string<span style="color: #999999">]</span><span style="color: #000000">(</span><span style="color: #0000ff">Get-Random</span> <span style="color: #000080">-Minimum</span> <span style="color: #800080">1</span> <span style="color: #000080">-Maximum</span> <span style="color: #800080">10000</span><span style="color: #000000">)</span> <span style="color: #999999">+</span> <span style="color: #993300">"`t"</span>
        $FileStream<span style="color: #000000">.Write(</span>$NumberTXT<span style="color: #000000">)</span> <span style="color: #99cc00"># The actual numbers in the table</span>
    <span style="color: #000000">}</span>
<span style="color: #000000">}</span></span></strong></pre>
<em>I hope you found the <span style="color: #993300;font-size: 14px"><b>"`t"</b></span> and <span style="color: #993300;font-size: 14px"><b>"`r`n"</b></span> directives interesting. This is how you tell PowerShell to put in a Tab and a new row entry.</em> Also, because we want to use the search refiners, we update the <em>Creator</em> and <strong></strong><em>Las Modified By<strong></strong></em> fields just as we learned earlier. Cheesy easy. Yeah... Not quite.
<h3>Act 2</h3>
As I've said earlier, while using a Office COM Objects from PowerShell is possible, the performance is not the best. The same applies to Excel as well, of course. Imagine you have to create a table with 100 columns and 1000 rows. This is how long it would take:
<table>
<tbody>
<tr>
<th></th>
<th>  COM Object</th>
<th>  Text Writer</th>
</tr>
<tr>
<td><strong>Time Taken</strong></td>
<td style="text-align: center">324 s</td>
<td style="text-align: center">18 s</td>
</tr>
</tbody>
</table>
Even if we have to do the conversion from TXT to XLSX, we will be well below how long it takes with the COM Object. I guess it's a no-brainer what we are going to choose. With Word documents the method was simple: we generated a long string, then pasted it into a document, saved the document as some name, and that was it. It would be good to use this with Excel, wouldn't it? Well... Yes, it would.
<h1>Climax</h1>
<h3>Act 3</h3>
Tiny little problem, that Excel does not work as Word does. There's no Selection object, so channeling information directly into an Excel file directly is not possible. Why is this a problem? Two reasons:
<ul>
 	<li>If we do not have a template Excel file, then we have to create TXT files, then convert them into XLSX. Not a big burden, but it has a little performance impact.</li>
</ul>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><b><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$ExcelWorkBook</span> = <span style="color: #ff0000">$ExcelApplication</span>.<span style="color: #000000">Workbooks.Open(</span><span style="color: #ff0000">$TempTXT</span><span style="color: #000000">)</span>
<span style="color: #ff0000">$ExcelWorkBook</span>.<span style="color: #000000">SaveAs(</span><span style="color: #ff0000">$ExcelFile</span>, [<span style="color: #339966">Microsoft.Office.Interop.Excel.XlFileFormat</span>]::<span style="color: #000000">xlWorkbookDefault)</span>
<span style="color: #ff0000">$ExcelWorkBook</span>.<span style="color: #000000">Saved</span> = <span style="color: #ff0000">$true</span>
<span style="color: #ff0000">$ExcelWorkBook</span>.<span style="color: #000000">Close()</span></span></span></span></span></b></span></strong></pre>
<ul>
 	<li>If we do have a template file, then we have to copy-paste the content into the file. Yes, copy-paste. It is working, but if you want to use the machine while the files are being generated, then it might cause some problems.</li>
</ul>
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><b><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$TempDocument</span> = <span style="color: #ff0000">$ExcelApplication</span>.<span style="color: #000000">Workbooks</span>.<span style="color: #000000">Open(</span><span style="color: #ff0000">$TempTXT</span>,<span style="color: #ff0000">$null</span>,<span style="color: #ff0000">$true</span><span style="color: #000000">)</span>
<span style="color: #ff0000">$TempSheet</span> = <span style="color: #000000">(</span><span style="color: #ff0000">$TempDocument</span>.<span style="color: #000000">Sheets)</span>[<span style="color: #800080">1</span>]
<span style="color: #ff0000">$CopiedContent</span> = <span style="color: #ff0000">$TempSheet</span>.<span style="color: #000000">UsedRange</span>.<span style="color: #000000">Copy()</span>
<span style="color: #ff0000">$Pasted</span> = <span style="color: #ff0000">$ExcelSheet</span>.<span style="color: #000000">Range(</span><span style="color: #993300">"A1"</span><span style="color: #000000">)</span>.<span style="color: #000000">PasteSpecial()</span>
</span></span></span></span></b></span></strong></pre>
Now there are a few things you might have noticed:
<ul>
 	<li>The <span style="color: #999999;font-size: 14px"><b>[</b></span><span style="color: #339966;font-size: 12x"><b>Microsoft.Office.Interop.Excel.XlFileFormat</b></span><span style="color: #999999;font-size: 14px"><b>]::</b></span><span style="color: #000000;font-size: 14px"><b>xlWorkbookDefault</b></span> directive is defining the output file format, so if you want to generate different files, you could change it to fit your needs. The full list of possible output formats in the <a href="https://msdn.microsoft.com/en-us/library/office/microsoft.office.interop.excel.xlfileformat.aspx">XlFileFormat enumeration article</a>.</li>
 	<li>The <span style="color: #000000;font-size: 14px"><b>(</b></span><span style="color: #ff0000;font-size: 14px"><b>$TempDocument</b></span><span style="color: #999999;font-size: 14px"><b>.</b></span><span style="color: #000000;font-size: 14px"><b>Sheets)</b></span><span style="color: #999999;font-size: 14px"><b>[</b></span><span style="color: #800080;font-size: 14px"><b>1</b></span><b><span style="color: #999999;font-size: 14px">]</span></b> definition is not as any other array references as we do not start from zero, but from one.</li>
 	<li>Last, but not least the <span style="color: #ff0000;font-size: 14px"><b>$TempSheet</b></span><span style="color: #999999;font-size: 14px"><b>.</b></span><span style="color: #000000;font-size: 14px"><b>UsedRange</b></span><span style="color: #999999;font-size: 14px"><b>.</b></span><span style="color: #000000;font-size: 14px"><b>Copy()</b></span> call is the one that copies the content to the clipboard, and this is where you might wish to not use your computer, because you might end-up losing some information.</li>
</ul>
<h3>Act 4</h3>
The rest should be pretty straightforward, as the property bag of an excel file is the same as a Word documents. Well... It is. In a way. But not when you are doing the simple TXT to XLSX conversion. For some reason Excel decides that in this case the Creator property should be empty. Even more... Not just empty, but not in the core.xml file at all.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><span style="color: #3366ff">&lt;?xml version="1.0" encoding="UTF-8" standalone="true"?&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">cp:coreProperties</span> <span style="color: #ff0000">xmlns:xsi</span><span style="color: #3366ff">="</span><strong><span style="color: #ff0000">http://www.w3.org/2001/XMLSchema-instance</span></strong><span style="color: #3366ff">"</span><span style="color: #ff0000"> xmlns:dcmitype</span><span style="color: #3366ff">="</span><strong><span style="color: #ff0000">http://purl.org/dc/dcmitype/</span></strong><span style="color: #3366ff">"</span> <span style="color: #ff0000">xmlns:dcterms</span><span style="color: #3366ff">="</span><span style="color: #ff0000"><strong>http://purl.org/dc/terms/</strong></span><span style="color: #3366ff">"</span> <span style="color: #ff0000">xmlns:dc</span><span style="color: #3366ff">="</span><strong><span style="color: #ff0000">http://purl.org/dc/elements/1.1/</span></strong><span style="color: #3366ff">"</span> <span style="color: #ff0000">xmlns:cp</span><span style="color: #3366ff">="</span><span style="color: #ff0000"><strong>http://schemas.openxmlformats.org/package/2006/metadata/core-properties</strong></span><span style="color: #3366ff">"&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">cp:lastModifiedBy</span><span style="color: #3366ff">&gt;</span><strong>Zsolt Illes</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">cp:lastModifiedBy</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dcterms:created xsi:type</span><span style="color: #3366ff">="</span><strong><span style="color: #000000">dcterms:W3CDTF</span></strong><span style="color: #3366ff">"&gt;</span><strong>2017-03-02T07:17:00Z</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">dcterms:created</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dcterms:modified xsi:type</span><span style="color: #3366ff">="</span><strong><span style="color: #000000">dcterms:W3CDTF</span></strong>"<span style="color: #3366ff">&gt;</span><strong>2017-03-02T07:17:00Z</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">dcterms:modified</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;/</span><span style="color: #993300">cp:coreProperties&gt;
</span></pre>
Of course we could set all these properties in the Excel COM Object directly, but as we discussed it a few times earlier, this would have some serious performance impact. This means that we have to create it from PowerShell. It is a pretty straight-forward operation:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><b><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$CreatorNameSpace </span>= <span style="color: #993300">'http://purl.org/dc/elements/1.1'</span>
<span style="color: #ff0000">$CreatorElement </span>= <span style="color: #ff0000">$CoreXML</span><span style="color: #000000"><span style="color: #999999">.</span>CreateElement(</span><span style="color: #993300">'dc'</span>,<span style="color: #993300">'creator'</span>,<span style="color: #ff0000">$CreatorNameSpace</span><span style="color: #000000">)</span>
<span style="color: #ff0000">$null </span>= <span style="color: #ff0000">$CoreXML</span>.<span style="color: #000000">DocumentElement</span>.<span style="color: #000000">AppendChild(</span><span style="color: #ff0000">$CreatorElement</span><span style="color: #000000">)</span>
</span></span></span></span></b></span></strong></pre>
For reasons unknown to me PowerShell 5 creates the new element like this:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><span style="color: #3366ff">&lt;</span><span style="color: #993300">dc:creator <span style="color: #ff0000">xmlns:dc</span></span><span style="color: #3366ff">="<strong><span style="color: #ff0000">http://purl.org/dc/elements/1.1/</span></strong></span><span style="color: #3366ff">" /&gt;</span><span style="color: #993300">
</span></pre>
Instead of this:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><span style="color: #3366ff">&lt;</span><span style="color: #993300">dc:creator</span><span style="color: #3366ff"> /&gt;</span><span style="color: #993300">
</span></pre>
Purely from XML perspective there is little difference between the two, as the first one means the same, just a bit more noisy. Excel however does not like the first option, and if we leave it in the core.xml like this, the Excel file becomes invalid. To work around that we have to clear that namespace reference from the Element. The only way I found for this is to rip it off on a string level:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #0000ff"><b><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000">$CreatorNameSpaceToNull</span> = <span style="color: #993300">' xmlns:dc="'</span> + <span style="color: #ff0000">$CreatorNameSpace</span> + <span style="color: #993300">'"'</span>
<span style="color: #ff0000">$CoreXML</span> = <span style="color: #ff0000">$CoreXML</span>.<span style="color: #000000">OuterXml</span>.<span style="color: #000000">Replace(</span><span style="color: #ff0000">$CreatorNameSpaceToNull</span>,<span style="color: #993300">''</span><span style="color: #000000">)</span>
</span></span></span></span></b></span></strong></pre>
If you know a more elegant solution, please share it in the comment section.

<span style="color: #ff0000">[Edit: 2017.03.14]</span>

I got the solution for the above from my colleague Roman Lutz.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><span style="color: #0000ff"><span style="color: #0000ff"><span style="color: #993366"><span style="color: #ff0000"><span style="color: #999999"><span style="color: #ff0000"><b><span style="color: #999999">[</span><span style="color: #339966">System.Xml.XmlNamespaceManager</span><span style="color: #999999">]</span>$NameSpaceManager <span style="color: #999999">=</span> <span style="color: #0000ff">New-Object</span> <span style="color: #800080">System.Xml.XmlNamespaceManager</span> $</b><span style="color: #ff0000"><b>CoreXML</b></span><b><span style="color: #999999">.</span>NameTable
</b><b>$NameSpaceManager<span style="color: #999999">.</span><span style="color: #000000">AddNamespace(</span><span style="color: #993300">'dc'</span><span style="color: #999999">,</span> <span style="color: #993300">'http://purl.org/dc/elements/1.1/'</span><span style="color: #000000">)</span>
$DCNameSpace <span style="color: #999999">=</span> <span style="color: #ff0000">$NameSpaceManager</span><span style="color: #999999">.</span><span style="color: #000000">LookupNamespace(</span><span style="color: #993300">'dc'</span><span style="color: #000000">)</span></b><b>
$NewElement <span style="color: #999999">=</span> $<span style="color: #ff0000">CoreXML</span><span style="color: #999999">.</span><span style="color: #000000">CreateElement(</span><span style="color: #993300">'dc'</span><span style="color: #999999">,</span> <span style="color: #993300">'creator'</span><span style="color: #999999">,</span> <span style="color: #ff0000">$DCNameSpace</span><span style="color: #000000">)</span></b></span>
</span></span></span></span></span></pre>
&nbsp;
<h1>Falling Action</h1>
<h3>Act 5</h3>
We have now everything to put the solution together.
<ul>
 	<li>Dictionary for the Row- and Column headers,</li>
 	<li>Random generator for the numbers,</li>
 	<li>A list for users and dates to update the document properties,</li>
 	<li>Way to push all this information into an Excel file,</li>
 	<li>I’ve not detailed it in this article, but my solution also contains a routine to use a pre-defined Excel file template. This might come handy if you want to fine tune your search experience even more. (Just think about search driven solutions for example.)</li>
</ul>
The scripts are still available on Codeplex (<a href="https://randomfilegenerator.codeplex.com/">link</a>). Of course this solution still has room for improvement, but that is up for you guys to play with.
<h1>Dénouement</h1>
With these files on your environment you can test your search infrastructure in a near-life scenario. Stay tuned for the last episode of the series where I detail the challenges with creating PowerPoint presentations.]]></content:encoded>
    </item>
    <item>
      <title>Generate random content for SharePoint - 2</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/generate-random-content-for-sharepoint-2</link>
      <pubDate>Sun, 05 Mar 2017 10:21:11 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=275</guid>
      <description><![CDATA[This is the second chapter of the Generate Random Content For SharePoint series. Yes, it turns out...]]></description>
      <content:encoded><![CDATA[This is the second chapter of the Generate Random Content For SharePoint series. Yes, it turns out to be a series, as I was not 100% satisfied with what I've written in the previous post.

<strong>Chapters:</strong>
<ul>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/02/26/generate-random-content-for-sharepoint/">Generate random content for SharePoint</a> - About how to create big files for upload testing.</li>
 	<li>Generate random content for SharePoint 2 - About how to generate Word files for Search and integration testing. (This article.)</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/12/generate-random-content-for-sharepoint-3/">Generate random content for SharePoint 3</a> - About how to generate Excel files for Search and integration testing.</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/28/generate-random-content-for-sharepoint-4/">Generate random content for SharePoint 4</a> - About how to generate PowerPoint files for Search and integration testing.</li>
</ul>
(If you don’t want to read through all the drama, jump directly to the Falling Action section.)
<h1>Exposition</h1>
So in my previous post I explained how to create random big files for upload testing. Part of the solution was that the content gets uniquely indexed. This made me thinking though. Biggest problem is, that even though I've used GUIDs to generate the content, and these GUIDs would be indexed individually, this does not smell like real life. Why not? Simply because who on earth would search for massive number of GUIDs? Also, this will have an impact on the Index component, as there will be no duplicates in there, so it falsifies the numbers in various ways.

So how could one create random content that makes a bit more sense than this one?
<h1>Rising Action</h1>
<h3>Act 1</h3>
Those who have read my previous post around the topic, or are familiar with Word know well, that there is the famous <i>=rand(x,y)</i> function. In previous versions of Word the function generated a text starting with:
<blockquote>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</blockquote>
The Word 2016 however is using an English text beginning with:
<blockquote>Video provides a powerful way to help you prove your point.</blockquote>
Now this would do, you might say, but I still have a problem with it, as there is a repeating pattern in the text. As a matter of fact, Word is generating 239 words, 1,063 characters (1,301 with spaces), then starts repeating it. as many times as you instructed it with the above function. Still far from smelling real life.

I could use an English corpus from the Internet. There are plenty available, some are for money, some might be free for use. Sounds good, but I might bounce into some licensing issues, so I thought to myself, why not creating a script, that is using words of the English language (or in fact any other language) to generate random content with it. Of course the text in it will be gibberish, since we're putting random words next to each other, but it would be way closer to real life than simply using the GUIDs. Wouldn't it? Yes, it would.

Where can I get a list of all the English words? If you are reading this article, then you most probably are familiar with this thing called the Internet. A simple query for the term "<em>list of English words download</em>" in your favorite search engine will give you loads of results. Also, if you query for other languages, with a few minutes of browsing you'll find similar downloads.
<h3>Act 2</h3>
I was so excited that things like this (that is a list of all English words) are available on the internet. I thought maybe there are corpuses available for analysis. <em>(Yeah, if I would be a linguist, this would probably be obvious, but since I am not...)</em> I assume you already guessed there are. So out of mere curiosity I ran a quick query against the ONAC <a href="http://www.anc.org/"><em>(Open American National Corpus)</em></a> database to see the separator statistics. You know, to know how many spaces and periods and so on you have in a text. This is what I've found:
<table width="274" style="border: 1px solid black">
<tbody>
<tr>
<th>Punctuation mark</th>
<th>Ratio</th>
</tr>
<tr>
<td>Space</td>
<td>87.5819</td>
</tr>
<tr>
<td>Full stop</td>
<td>3.7542</td>
</tr>
<tr>
<td>Comma</td>
<td>3.9734</td>
</tr>
<tr>
<td>Colon</td>
<td>0.2435</td>
</tr>
<tr>
<td>Semicolon</td>
<td>0.1832</td>
</tr>
<tr>
<td>Apostrophe</td>
<td>1.6572</td>
</tr>
<tr>
<td>Exclamation mark</td>
<td>0.0342</td>
</tr>
<tr>
<td>Question mark</td>
<td>0.1124</td>
</tr>
<tr>
<td>Doublequotes</td>
<td>0.7048</td>
</tr>
<tr>
<td>Hyphen</td>
<td>1.7455</td>
</tr>
<tr>
<td>Underscore</td>
<td>0.0098</td>
</tr>
</tbody>
</table>
Interesting, isn't it. How would this come handy? Well... The SharePoint search infrastructure could be used for a lot of things. You could drag statistics, modify relevancy, results sets, etc. Would it not be nice to have this incorporated in our document? I though it would.

What would be the best way to randomly choose a punctuation mark based on their weight in the above table. Maybe there are other ideas as well, as for me, I've chosen to create a file that contains 10,000 rows, each containing one of the punctuation marks. For example it would have 8,758 rows with space, 375 full stop, etc. To support other languages, I've created a script that would create such a separator file, you just need to use the parameters with the ratio of your choice. <em>(See the <a href="#falling">Falling Action section </a>below.)</em>

&nbsp;
<h1 id="climax">Climax</h1>
<h3>Act 3</h3>
So I have a corpus, I have a file that contains the punctuations as separators. Next question is, how to choose items from them randomly? I'm still using PowerShell, so I asked my fried <a href="https://www.linkedin.com/in/daniel-vetro-7793b748">Daniel Vetro</a>, what would be the best way. He came up with a really elegant solution:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$DictionaryFile </span><span style="color: #999999"><b>=</b></span> <span style="color: #0000ff">Get-Content</span> <span style="color: #993366">'.\words_en.txt'</span>
<span style="color: #ff0000">$<b><span style="color: #ff0000">DictionaryFile</span></b></span> <span style="color: #999999"><b>|</b></span> <span style="color: #0000ff">Get-Random</span>
</strong></pre>
Tiny little problem, that this turned out to be slow. Run it against a corpus of 500,000 words and you'll find that generating documents of 82,000 words will take just too long. So I had to go back to the basics and use the "good old" addressing of arrays. Something like this:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$DictionaryFileRows</span> <span style="color: #999999"><b>=</b></span> <span style="color: #ff0000"><b>$<span style="color: #ff0000">DictionaryFile</span></b></span><b><span style="color: #999999">.</span>Count</b>
<span style="color: #0000ff"><span style="color: #ff0000"><b>$RandomWordLine </b></span><span style="color: #999999"><b>=</b></span><b> <span style="color: #0000ff">Get-Random <span style="color: #000000">-ProviderName</span> <span style="color: #993366">1 <span style="color: #000000">-Maximum</span> <span style="color: #ff0000">$DictionaryFileRows
$RandomWord <span style="color: #999999">=</span> $DictionaryFileContent<span style="color: #999999">[</span>$RandomWordLine<span style="color: #999999">]</span></span></span></span></b></span></strong></pre>
<h3>Act 4</h3>
Awesome. So now that we are able to randomly choose words and separators, we should put them into a document. Obviously PowerShell cannot create a Word file directly <em>(well, technically it could as a DOCX is a ZIP file after all, but the structure and the content would be a bit too complicated)</em>, but we have COM Objects that we can address. Something like this:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$WordApplication </span><span style="color: #999999"><b>=</b></span> <span style="color: #0000ff">New-Object <span style="color: #000080">-ComObject</span></span> <span style="color: #993366">"Word.Application"</span>
<span style="color: #ff0000">$WordDocument</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$WordApplication</span><span style="color: #999999">.</span>Documents<span style="color: #999999">.</span>Add()
<span style="color: #ff0000">$WordSelection</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$WordApplication</span><span style="color: #999999">.</span>Selection
<span style="color: #ff0000"><b>$WordSelection<span style="color: #000000"><span style="color: #999999">.</span>InsertAfter(<span style="color: #ff0000">$RandomWord</span>)
<span style="color: #ff0000">$WordSelection</span><span style="color: #999999">.</span>InsertAfter(<span style="color: #ff0000">$RandomSeparator</span>)</span></b></span>
</strong></pre>
Too easy to be true, right? Right. It is too easy to be true. Just as earlier, it technically works. It is just slow like hell. A document with 120 words creates in about a minute. Go up to some thousand words and you sit there for hours. Not really what one would be looking for...

Luckily the Above COM object allows inserting a whole set of words. So we could create a long string, then add that to the Word document:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #99cc00"><span style="color: #999999">[</span><span style="color: #339966">string</span><span style="color: #999999">]</span><span style="color: #ff0000">$LongString</span> <span style="color: #999999">=</span> <span style="color: #800000">''</span>
<span style="color: #000000">For(</span><span style="color: #ff0000">$i</span><span style="color: #999999">=</span><span style="color: #993300">1</span><span style="color: #000000">;</span> <span style="color: #ff0000">$i</span> <span style="color: #999999">-le</span> <span style="color: #ff0000">$WordCount</span><span style="color: #000000">;</span> <span style="color: #ff0000">$i</span><span style="color: #999999">++</span><span style="color: #000000">)</span>
<span style="color: #000000">{</span>
    <span style="color: #ff0000">$RandomWordLine</span> <span style="color: #999999">=</span> <span style="color: #0000ff">Get-Random</span> <span style="color: #000080">-Minimum</span> <span style="color: #993300">1</span> -<span style="color: #000080">Maximum</span> <span style="color: #ff0000">$DictionaryFileRows</span>
    <span style="color: #ff0000">$RandomWord</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$DictionaryFileContent</span><span style="color: #999999">[</span><span style="color: #ff0000">$RandomWordLine</span><span style="color: #999999">]</span>
    <span style="color: #ff0000">$RandomSeparator</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$Separators</span><span style="color: #999999">[</span><span style="color: #000000">(</span><span style="color: #0000ff">Get-Random</span> <span style="color: #000080">-Minimum</span> <span style="color: #993300">0</span> <span style="color: #000080">-Maximum <span style="color: #000000">$(</span></span><span style="color: #ff0000">$Separators</span><span style="color: #000000"><span style="color: #999999">.</span>Count)</span><span style="color: #000000">)</span><span style="color: #999999">]</span>
    <span style="color: #ff0000">$RandomWord</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$RandomWord</span> <span style="color: #999999">+</span> <span style="color: #ff0000">$RandomSeparator</span>
    <span style="color: #ff0000">$LongString</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$LongString</span> <span style="color: #999999">+</span> <span style="color: #ff0000">$RandomWord</span>
<span style="color: #000000">}</span>
<span style="color: #ff0000">$WordSelection</span><span style="color: #000000"><span style="color: #999999">.</span>Select()</span>
<span style="color: #ff0000">$WordSelection</span><span style="color: #000000"><span style="color: #999999">.</span>Delete()</span>
<span style="color: #ff0000"><b>$WordSelection</b></span><b><span style="color: #000000"><span style="color: #999999">.</span>InsertAfter(</span><span style="color: #ff0000">$LongString</span></b><span style="color: #000000"><b>) </b></span>
</span></strong></pre>
<h3>Act 5</h3>
Now we know how to generate random words, put random separators afterwards, and how to put it into a Word document. But is there more we could do. Of course there is. Remember that the SharePoint Search results have a few nice built-in refiners?

<a href="https://msdnshared.blob.core.windows.net/media/2017/03/searchrefiners1.jpg"><img width="101" height="300" class="size-medium wp-image-295 alignleft" alt="Search Refiners" src="https://msdnshared.blob.core.windows.net/media/2017/03/searchrefiners1-101x300.jpg" /></a>

These refiners are coming from the document's property bag. Some part of this property bag is being updated by system automatically, another part manually by the actual users. The information is being saved within the DOCX file.

Would it not be great if we could somehow hack into that property bag and update something? I don't know, maybe fake who created or last modified the document? That would definitely give us something to play with in the search results...

Now, for those of you who wouldn't know, an Open Office XML file is technically a ZIP file with a nice little structure and a bunch of XML files. The structure looks like this:

<a href="https://msdnshared.blob.core.windows.net/media/2017/03/wordfile1.jpg"><img width="300" height="100" class="alignnone size-medium wp-image-305" alt="wordfile1" src="https://msdnshared.blob.core.windows.net/media/2017/03/wordfile1-300x100.jpg" /></a>  and  <a href="https://msdnshared.blob.core.windows.net/media/2017/03/wordfile2.jpg"><img width="300" height="66" class="alignnone size-medium wp-image-315" alt="wordfile2" src="https://msdnshared.blob.core.windows.net/media/2017/03/wordfile2-300x66.jpg" /></a>

&nbsp;

When you open the XML file you should see something like this:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><span style="color: #3366ff">&lt;?xml version="1.0" encoding="UTF-8" standalone="true"?&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">cp:coreProperties</span> <span style="color: #ff0000">xmlns:xsi</span><span style="color: #3366ff">="</span><strong><span style="color: #ff0000">http://www.w3.org/2001/XMLSchema-instance</span></strong><span style="color: #3366ff">"</span><span style="color: #ff0000"> xmlns:dcmitype</span><span style="color: #3366ff">="</span><strong><span style="color: #ff0000">http://purl.org/dc/dcmitype/</span></strong><span style="color: #3366ff">"</span> <span style="color: #ff0000">xmlns:dcterms</span><span style="color: #3366ff">="</span><span style="color: #ff0000"><strong>http://purl.org/dc/terms/</strong></span><span style="color: #3366ff">"</span> <span style="color: #ff0000">xmlns:dc</span><span style="color: #3366ff">="</span><strong><span style="color: #ff0000">http://purl.org/dc/elements/1.1/</span></strong><span style="color: #3366ff">"</span> <span style="color: #ff0000">xmlns:cp</span><span style="color: #3366ff">="</span><span style="color: #ff0000"><strong>http://schemas.openxmlformats.org/package/2006/metadata/core-properties</strong></span><span style="color: #3366ff">"&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dc:title</span><span style="color: #3366ff">/&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dc:subject</span><span style="color: #3366ff">/&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dc:creator</span><span style="color: #3366ff">&gt;</span><strong>Zsolt Illes</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">dc:creator</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">cp:keywords</span><span style="color: #3366ff">/&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dc:description</span><span style="color: #3366ff">/&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">cp:lastModifiedBy</span><span style="color: #3366ff">&gt;</span><strong>Zsolt Illes</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">cp:lastModifiedBy</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">cp:revision</span><span style="color: #3366ff">&gt;</span><strong>3</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">cp:revision</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dcterms:created xsi:type</span><span style="color: #3366ff">="</span><strong><span style="color: #000000">dcterms:W3CDTF</span></strong><span style="color: #3366ff">"&gt;</span><strong>2017-03-02T07:17:00Z</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">dcterms:created</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;</span><span style="color: #993300">dcterms:modified xsi:type</span><span style="color: #3366ff">="</span><strong><span style="color: #000000">dcterms:W3CDTF</span></strong>"<span style="color: #3366ff">&gt;</span><strong>2017-03-02T07:17:00Z</strong><span style="color: #3366ff">&lt;/</span><span style="color: #993300">dcterms:modified</span><span style="color: #3366ff">&gt;</span>
<span style="color: #3366ff">&lt;/</span><span style="color: #993300">cp:coreProperties&gt;
</span></pre>
So what do we need to know about the above structure?
<ul>
 	<li>The <span style="color: #993300">creator </span>and <span style="color: #993300">lastModifiedBy</span> fields should be presented in SIP format, so that the presence information</li>
 	<li>The <span style="color: #993300">created</span> and <span style="color: #993300">modified</span> fields are in <a href="https://msdn.microsoft.com/en-us/library/az4se3k1(v=vs.110).aspx#Sortable">Sortable Time Format</a>.</li>
</ul>
Thus what we need is two files; one that contains a list of people, and another one that contains the dates. Getting random data from these two files works just as we learned above. We - however - might wish to make sure that the time of modification is at least the same or later than the time of creation. Somehow like this:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$RandomDateNr</span> <span style="color: #999999">=</span> <span style="color: #0000ff">Get-Random</span> <span style="color: #000080">-Minimum</span> <span style="color: #993300">0</span> <span style="color: #000080">-Maximum</span> <span style="color: #ff0000">$DatesCount</span>
<span style="color: #ff0000">$RandomDate</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$DatesList</span><span style="color: #999999">[</span><span style="color: #ff0000">$RandomDateNr</span><span style="color: #999999">]</span>
<span style="color: #ff0000">$CoreXML</span><span style="color: #999999">.</span>coreProperties<span style="color: #999999">.</span>created<span style="color: #999999">.</span><span style="color: #993300">'#text'</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$RandomDate</span><span style="color: #999999">.</span>ToString()
<span style="color: #99cc00"># Here we need a trick, as we have to make sure the last modifed</span>
<span style="color: #99cc00"># date is not earlier than the date creation. (This is why we ordered the list.)</span>
$<span style="color: #ff0000">RandomDateNr</span> <span style="color: #999999">=</span> <span style="color: #0000ff">Get-Random</span> -Minimum <span style="color: #ff0000">$RandomDateNr</span> -Maximum <span style="color: #ff0000">$DatesCount</span>
<span style="color: #ff0000">$RandomDate</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$DatesList</span><span style="color: #999999">[</span><span style="color: #ff0000">$RandomDateNr</span><span style="color: #999999">]</span>
<span style="color: #ff0000">$CoreXML</span><span style="color: #999999">.</span>coreProperties<span style="color: #999999">.</span>modified<span style="color: #999999">.</span><span style="color: #993300">'#text'</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$RandomDate</span><span style="color: #999999">.</span>ToString()
</strong></pre>
Now, because - as we already discussed - a DOCX file is just a ZIP, we could simply write into them and update the core.xml file directly.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$WordFile</span> <span style="color: #999999">= [</span><span style="color: #339966">System.IO.Compression.ZipFile</span><span style="color: #999999">]::</span>Open( <span style="color: #ff0000">$WordFilePath</span><span style="color: #999999">,</span> <span style="color: #993300">'Update'</span> )
<span style="color: #ff0000">$ZippedCoreXMLFile</span> <span style="color: #999999">= [</span>System<span style="color: #999999">.</span>IO<span style="color: #999999">.</span>StreamWriter<span style="color: #999999">]</span>(<span style="color: #ff0000">$WordFile.Entries</span> <span style="color: #999999">|</span> Where-Object { $_.FullName <span style="color: #999999">-match</span> <span style="color: #993300">'docProps/core.xml'</span> })<span style="color: #999999">.</span>Open()
<span style="color: #ff0000">$ZippedCoreXMLFile</span><span style="color: #999999">.</span>BaseStream<span style="color: #999999">.</span>SetLength(<span style="color: #993300">0</span>)
<span style="color: #ff0000">$ZippedCoreXMLFile</span><span style="color: #999999">.</span>Write($Script:CoreXML.OuterXml)
<span style="color: #ff0000">$ZippedCoreXMLFile</span><span style="color: #999999">.</span>Flush()
<span style="color: #ff0000">$ZippedCoreXMLFile</span><span style="color: #999999">.</span>Close()
<span style="color: #ff0000">$WordFile</span><span style="color: #999999">.</span>Dispose()
</strong></pre>
&nbsp;
<h1 id="falling">Falling Action</h1>
<h3>Act 6</h3>
We have everything now to put the solution together.
<ul>
 	<li>Dictionary for words,</li>
 	<li>Separator list,</li>
 	<li>A list for users and dates to update the document properties,</li>
 	<li>and  a nice way to push all this information into a Word document directly.</li>
 	<li>I've not detailed it in this article, but my solution also contains a routine to use a pre-defined Word file template. This might come handy if you want to fine tune your search experience even more. (Just think about search driven solutions for example.)</li>
</ul>
The scripts are still available on Codeplex (<a href="https://randomfilegenerator.codeplex.com/">link</a>). Of course this solution still has room for improvement, but that is up for you guys to play with.
<h1 id="denouement">Dénouement</h1>
With these files on your environment you can test your search infrastructure in a near-life scenario.

Stay tuned for the next episode of the series where I detail the challenges with creating Excel files. In the same way. Then another one to create PowerPoint presentations.]]></content:encoded>
    </item>
    <item>
      <title>Generate random content for SharePoint</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/generate-random-content-for-sharepoint</link>
      <pubDate>Sun, 26 Feb 2017 12:18:32 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=235</guid>
      <description><![CDATA[This post is about how to create dummy files that you can use in SharePoint for various...]]></description>
      <content:encoded><![CDATA[This post is about how to create dummy files that you can use in SharePoint for various purposes.

<strong>Chapters:</strong>
<ul>
 	<li>Generate random content for SharePoint - About how to create big files for upload testing.(This article.)</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/05/generate-random-content-for-sharepoint-2/">Generate random content for SharePoint 2</a> - About how to generate Word files for Search and integration testing.</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/12/generate-random-content-for-sharepoint-3/">Generate random content for SharePoint 3</a> - About how to generate Excel files for Search and integration testing.</li>
 	<li><a href="https://blogs.technet.microsoft.com/zsoltilles/2017/03/28/generate-random-content-for-sharepoint-4/">Generate random content for SharePoint 4</a> - About how to generate PowerPoint files for Search and integration testing.</li>
</ul>
(If you don’t want to read through all the drama, jump directly to the Falling Action section.)
<h1>Exposition</h1>
When you are building your on-prem SharePoint environment <em>(yes, there are people who still do that)</em>, you might wish to know how your farm behaves under pressure. There are a number of blog posts out there on how to perform stress tests, I am not planning to create another one. There is one thing that you need to keep in mind though; how file uploads and content crawls are handled in SharePoint 2013 and later versions.

First and foremost, I'm sure you've heard about the Shredded Storage feature that Microsoft introduced with full power in SharePoint 2013. <i>(Should you not, </i><a href="https://www.microsoft.com/en-us/download/details.aspx?id=39719"><i>here's the White Paper</i></a><i> and </i><a href="https://blogs.technet.microsoft.com/wbaer/2012/11/12/introduction-to-shredded-storage-in-sharepoint-2013-rtm-update/"><i>here's Bill Baer's blog</i></a><i> about how it works in detail.)</i>

Long story short, the WFE servers chop up every files uploaded to SharePoint <em>(yes each and every one of them)</em> into 64k chunks, and send these chunks to the SQL server. What it also does, it caches these chunks and if there's a new chunk that should be sent to the SQL, it does not do the transfer, only instructs SQL to reuse one that it saved earlier.

Also, while stress testing file uploads makes sense, it's just a little part of your solution as the content crawl and query processing might be just as important.

Let's see them pieces then.

&nbsp;
<h1>Rising Action</h1>
<h2>Act 1</h2>
So this Shredded storage is chopping files up to 64k pieces? So what? As I stated earlier, this causes the shreds to be cached on the WFE server, so we cannot re-use the same file with a different name, because the number of roundtrips to the SQL server is halved. Cool, so we need to use unique content for the files. At this point you might think, what's the big fuzz about this topic that I decided to create a whole post about it, since there are plenty of tools are available to generate files with random garbage in it.

True... However, remember that you want to use these files for other purposes. For testing the whole Search lifecycle <em>(Crawling, Content Processing, Index Replication and Query Processing)</em> of course. Which means that your content should somewhat make sense. Now, those of you who are familiar with Word would say: "Yeah, but there is the famous <em>=rand(x,y)</em> function, which generates random text for you.".

True... However, there are two things that invalidates this approach:
<ol>
 	<li>Remember that you need unique content. While this function is useful to generate excessive number of paragraphs, the content in them will be repeating, and we already know, this is not useful for us because of the Shredded Storage.</li>
 	<li>Since Microsoft introduced the Office Open XML file format, the actual text is being compressed into a simple ZIP file, so generating big files <em>(above a few 100KBs)</em> will be quite hard. Yes, we could fill the content with picture, but this would be once again not be the brightest idea if we want to process the file content with the Search.</li>
</ol>
It's getting confusing, isn't it. So many things to keep in mind, so many variables to pay attention to. No wonder stress testing is considered an art.

So to summarize... We need files that:
<ul>
 	<li>Have unique content.</li>
 	<li>Have sensible content that we can search for.</li>
 	<li>Vary by size.</li>
</ul>
What is it that would comply to these criteria?
<h2>Act 2</h2>
GUIDs seem to be a good candidate. Let's see how a GUID looks like then:
<blockquote>
<pre>a3284fbb-6fd6-49bc-a0de-f12886a3e648</pre>
</blockquote>
As you can see, this is a set of alphanumeric characters, divided by hyphens. Wait a second... Is a hyphen a word breaker character? Yes it is. What is not? Well, an underscore. Cool, so we can generate GUIDs, then swap the hyphens with underscores...
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$GUID</span> <span style="color: #999999">=</span> (<span style="color: #999999">[</span><span style="color: #339966">guid</span><span style="color: #999999">]</span>::NewGuid()).ToString()
<span style="color: #ff0000">$StringToDump</span> <span style="color: #999999">=</span> <span style="color: #ff0000">$GUID</span>.Replace(<span style="color: #993366">'-'</span>,<span style="color: #993366">'_'</span>)</strong></pre>
... and put them into files. Now here we want to use continuous file writing, otherwise we would need to create a 1GB file in the memory, then save it, which might kill the server.
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$FileStream</span> <span style="color: #999999">=</span> <span style="color: #0000ff">New-Object</span> <span style="color: #cc99ff">System.IO.FileStream</span>(<span style="color: #ff0000">$FullFilePath</span>,<span style="color: #999999">[</span><span style="color: #339966">System.IO.FileMode]::</span>CreateNew)
<span style="color: #ff0000">$StreamWriter</span> <span style="color: #999999">=</span> <span style="color: #0000ff">New-Object</span> <span style="color: #cc99ff">System.IO.StreamWriter</span>(<span style="color: #ff0000">$FileStream</span>,<span style="color: #999999">[</span><span style="color: #339966">System.Text.Encoding</span><span style="color: #999999">]::</span>ASCII,<span style="color: #993366">128</span>)

<span style="color: #99cc00"># Logic to generate n number of GUIDs.</span>
...
<span style="color: #ff0000">$StreamWriter</span>.Write(<span style="color: #ff0000">$StringToDump</span>)
...
<span style="color: #99cc00"># End of logic <b>to generate n number of GUIDs.</b></span>

<span style="color: #ff0000">$StreamWriter</span>.Close()
<span style="color: #ff0000">$FileStream</span>.Close()

</strong></pre>
Awesome, isn't it? It is. It would mean, that we have 36 characters long "words" that are completely unique. <em>(Yes, technically it is possible to create the same GUID twice, but the chances are astronomical as the possible variation is 5.3x10<sup>36</sup> (5.3 undecillion) per <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_.28random.29">Wikipedia</a>.)</em>

Things are shaping form out of the purple mist, I assume, so let's start putting it together. I could write a console application, but since I'm not a coder, I am staying with good old PowerShell. So we need a script, that accepts a few parameters:
<ul>
 	<li>A target folder where we're going to save the files.</li>
 	<li>Maybe a file prefix parameter.</li>
 	<li>A set of parameters for different pre-defined size of files.
<ul>
 	<li>Some pre-defined sizes:  5KB, 10KB, 20KB, 50KB, 100KB, 200KB, 500KB, 1MB, 2MB, 5MB, 10MB, 20MB, 50MB, 100MB, 200MB, 500MB, 1GB.</li>
</ul>
</li>
 	<li>A parameter for custom sizes.</li>
 	<li>And maybe a switch parameter to create a folder for each file size.</li>
</ul>
Cheesy easy. You wish...

&nbsp;
<h2>Climax</h2>
<h2>Act 3</h2>
Let's see a little math first. A GUID is 36 characters long. Plus we need a space for separator, so one GUID will consume 37 characters in total. How does that look like in the view of the proposed file sizes?
<table style="width: 100%">
<tbody>
<tr>
<th>File size</th>
<th>Number of characters</th>
<th>Number of GUIDs</th>
<th>Remaining characters</th>
</tr>
<tr>
<td>5KB</td>
<td>5,120</td>
<td>138</td>
<td>14</td>
</tr>
<tr>
<td>10KB</td>
<td>10,240</td>
<td>276</td>
<td>28</td>
</tr>
<tr>
<td>20KB</td>
<td>20,480</td>
<td>553</td>
<td>19</td>
</tr>
<tr>
<td>50KB</td>
<td>51,200</td>
<td>1,383</td>
<td>29</td>
</tr>
<tr>
<td>100KB</td>
<td>102,400</td>
<td>2,767</td>
<td>21</td>
</tr>
<tr>
<td>200KB</td>
<td>204,800</td>
<td>5,535</td>
<td>5</td>
</tr>
<tr>
<td>500KB</td>
<td>512,000</td>
<td>13,837</td>
<td>31</td>
</tr>
<tr>
<td>1MB</td>
<td>1,048,576</td>
<td>28,339</td>
<td>33</td>
</tr>
<tr>
<td>2MB</td>
<td>2,097,152</td>
<td>56,679</td>
<td>29</td>
</tr>
<tr>
<td>5MB</td>
<td>5,242,880</td>
<td>141,699</td>
<td>17</td>
</tr>
<tr>
<td>10MB</td>
<td>10,485,760</td>
<td>283,398</td>
<td>34</td>
</tr>
<tr>
<td>20MB</td>
<td>20,971,520</td>
<td>566,797</td>
<td>31</td>
</tr>
<tr>
<td>50MB</td>
<td>52,428,800</td>
<td>1,416,994</td>
<td>22</td>
</tr>
<tr>
<td>100MB</td>
<td>104,857,600</td>
<td>2,833,989</td>
<td>7</td>
</tr>
<tr>
<td>200MB</td>
<td>209,715,200</td>
<td>5,667,978</td>
<td>14</td>
</tr>
<tr>
<td>500MB</td>
<td>524,288,000</td>
<td>14,169,945</td>
<td>35</td>
</tr>
<tr>
<td>1GB</td>
<td>1,073,741,824</td>
<td>29,020,049</td>
<td>11</td>
</tr>
</tbody>
</table>
Practically this means that to create a 1GB file we need to create twenty nine million, twenty thousand and forty nine GUIDs. Now that will take some time. Luckily this is not something you would do every day, so you can leave a machine running over a few days to create all the files for you.

The next thing is that you cannot use a parameter called -5kfiles for example, because you cannot use an argument name that starts with a number. <em>(If you want to know why, read <a target="_blank" href="http://blogs.technet.microsoft.com/zsoltilles/2017/02/25/why-you-cannot-use-an-argument-that-starts-with-a-number/">this</a> blog post.) </em>Here you can decide to use some other characters as a starter (ex: -_5kfiles), or just change the size and the scale identifier (ex: -k5files). I used the first option.

Last, but not least we have to make sure that the script is not filling up the destination drive, because it would not be too good. If the target is pointing to the System drive, we should also make sure that there is enough space left for the memory dump in case of <a target="_blank" href="https://en.wikipedia.org/wiki/Blue_Screen_of_Death">BSOD</a>. (Of course it won't be our script that causes this, but we do not want to be part of the discussion anyway. This can be achieved with a simple function.

In this function we need to get the Disk information
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #99cc00"># For network drives:</span>
<span style="color: #ff0000">$Disk</span> <span style="color: #999999"><b>=</b></span> <span style="color: #0000ff">Get-WmiObject</span> <span style="color: #cc99ff">Win32_LogicalDisk</span> <span style="color: #999999">|</span> <span style="color: #0000ff">?</span>{(<span style="color: #ff0000">$_</span>.ProviderName) <span style="color: #999999">-and</span> (<span style="color: #ff0000">$DestinationFolder</span> <span style="color: #999999">-like</span> <span style="color: #993366">"*</span>$(<span style="color: #ff0000">$_<span style="color: #999999"><b>.</b></span></span>ProviderName)<span style="color: #993366">*"</span>)}
<span style="color: #99cc00"># For local drives:</span>
<span style="color: #ff0000">$Disk</span> <span style="color: #999999"><b>=</b></span> <span style="color: #0000ff">Get-WmiObject</span> <span style="color: #cc99ff">Win32_LogicalDisk</span> -Filter $(<span style="color: #993366">"</span>DeviceID=<span style="color: #993366">'"</span> <span style="color: #999999">+</span> <span style="color: #ff0000">$DestinationDrive</span> <span style="color: #999999">+</span> <span style="color: #993366">"'"</span>)
</strong></pre>
And the memory information:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New';font-size: 10pt"><strong><span style="color: #ff0000">$<span style="color: #ff0000"><b>PhysicalMemory</b></span><b> </b><span style="color: #999999"></span></span><span style="color: #999999">=</span> (((<span style="color: #0000ff">Get-CimInstance</span> -ClassName <span style="color: #993366">"cim_physicalmemory"</span>)<span style="color: #999999">.</span>Capacity) <span style="color: #999999">|</span> <span style="color: #0000ff">Measure-Object</span> -Sum)<span style="color: #999999">.</span>Sum</strong></pre>
&nbsp;
<h1>Falling Action</h1>
<h3>Act 4</h3>
Putting together the above is not complicated, it's merely a few hours of typing. I'm going to be nice and caring and link it for you. (<a href="https://randomfilegenerator.codeplex.com/">link</a>).

Just remember... Depending on the number and size of files you want to generate, you might need some time, so don't leave this to the last minute.
<h1>Dénouement</h1>
Now that we have a bunch of files available, you can start your performance testing. I might also create a blog entry on that. But maybe another before that an entry on how to create random Word files for testing the search functionality in a better way.

Special thanks go to my friend <a href="https://www.linkedin.com/in/daniel-vetro-7793b748">Daniel Vetro</a>, who's a real PowerShell ninja, and without whom this entry would not be here.]]></content:encoded>
    </item>
    <item>
      <title>Why you cannot use an argument that starts with a number?</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/why-you-cannot-use-an-argument-that-starts-with-a-number</link>
      <pubDate>Sat, 25 Feb 2017 13:36:21 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=265</guid>
      <description><![CDATA[This is a quick, one Act drama about how the past defines the present.
Exposition
I recently had...]]></description>
      <content:encoded><![CDATA[This is a quick, one Act drama about how the past defines the present.
<h2><span class="mw-headline" id="Exposition">Exposition</span></h2>
I recently had the requirement to create a PowerShell script and was planning to use parameters that begin with a number. Something like this
<pre style="padding-left: 30px;background-color: #f4f4f4"><strong><span style="margin: 0px;color: #000080;font-family: 'Courier New';font-size: 9pt">function</span><span style="margin: 0px;font-family: 'Courier New';font-size: 9pt"><span style="color: #000080"> </span><span style="color: #800080">MyFunction
</span></span></strong><strong><span style="margin: 0px;color: #000080;font-family: 'Courier New';font-size: 9pt"><span>{</span>
</span><span style="margin: 0px;font-family: 'Courier New';font-size: 9pt"><span style="color: #000000">    </span><span style="color: #000080">param</span></span><span style="margin: 0px;color: #000080;font-family: 'Courier New';font-size: 9pt">
<span>    </span><span>(</span>
</span><span style="margin: 0px;font-family: 'Courier New';font-size: 9pt"><span style="color: #000000">        </span><span style="color: #999999">[</span><span style="color: #339966">int</span><span style="color: #999999">]</span><span style="color: #ff0000">$5turns</span></span>
<span style="margin: 0px;font-family: 'Courier New';font-size: 9pt"><span style="color: #000000">    </span><span style="color: #000080">)</span></span>
<span style="margin: 0px;font-family: 'Courier New';font-size: 9pt"><span style="color: #000000">    </span><span style="color: #0000ff">Write-Host</span><span style="color: #ff0000"><span style="color: #0000ff"> </span>$5turns</span></span>
</strong><span style="margin: 0px;color: #000080;font-family: 'Courier New';font-size: 9pt"><span><strong>}</strong> </span></span></pre>
This turned out to be a bad idea. I got a dirty error message when calling my function:
<pre style="padding-left: 30px;background-color: #f4f4f4;font-family: 'Courier New'"><span style="color: #ff0000">MyFunction: Cannot process argument transformation on parameter '5turns'. Cannot convert value "-5turns" to type "System.Int32". Error: "Input string was not in a correct format."</span></pre>
<h2>Act 1</h2>
I was surprised to see that finding an answer is not that easy. No matter which search engine I used, all of them offered results that explained how argument conversion works. Finally I found a loooong thread that explained that coders sometime are lazy, and this is the cause of the problem. Imagine the following declaration:
<pre style="background-color: #f4f4f4;font-family: 'Courier New'"><strong>int a = 1
int 2a = 3
int b = 2a</strong></pre>
Should <span style="font-family: Courier New">b</span> be equal to 2 or should it be 3?

Well... As you can see, this is pure laziness, as we assume <strong><span style="font-family: Courier New">2a</span></strong> actually means two times a, which could be represented as <strong><span style="font-family: Courier New">2*a</span></strong>.

Now, let's assume I understand that it would be confusing in certain languages, but in PowerShell <strong><span style="font-family: Courier New">2$a</span></strong> should never be considered as <strong><span style="font-family: Courier New">$2a</span></strong>.

&nbsp;
<h2>Dénouement</h2>
Moral of the story: pay attention to the programming techniques prof. Or if you did not attend such seminars (like myself), remember, that using a parameter beginning with a number is not an option. Ehh...]]></content:encoded>
    </item>
    <item>
      <title>Application Lifecycle Management with SharePoint Workflows</title>
      <link>https://docs.microsoft.com/archive/blogs/zsoltilles/application-lifecycle-management-with-sharepoint-workflows</link>
      <pubDate>Wed, 08 Feb 2017 09:06:24 GMT</pubDate>
      <dc:creator><![CDATA[Zsolt Illés]]></dc:creator>
      <guid
        isPermaLink="false">http://blogs.technet.microsoft.com/zsoltilles/?p=215</guid>
      <description><![CDATA[This post is mainly for those who want to achieve email sending from SharePoint Designer Workflows....]]></description>
      <content:encoded><![CDATA[This post is mainly for those who want to achieve email sending from SharePoint Designer Workflows. (If you don’t want to read through all the drama, jump directly to the Falling Action section.)
<h2><span class="mw-headline" id="Exposition">Exposition</span></h2>
In an enterprise environment developing SharePoint Workflows on the production environment might not be the best thing you can do. Of course if you are a superuser and you are developing a workflow for your small team, it might still be a solution, but if your solution is business critical... Well... You might wish to have some kind of Application Lifecycle Management (ALM) for your workflow.

(<a href="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg"><img width="13" height="17" class="alignnone size-full wp-image-72" alt="notesign" src="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg" /></a> Don't Be Afraid <a href="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg"><img width="13" height="17" class="alignnone size-full wp-image-72" alt="notesign" src="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg" /></a>)
<h2><span class="mw-headline" id="Rising_action">Rising action</span></h2>
With the introduction of SharePoint 2013 Microsoft introduced the interchangeability feature between SharePoint Designer and Visio 2013 Professional. What does it mean? It means that you are able to create your Workflow in Visio 2013, then import it into SharePoint Designer and the other way around. Sounds fancy, but what does it really mean to us?
<h2><span class="mw-headline" id="Climax">Climax</span></h2>
It means two things:
<ol>
 	<li>One does not need to be a SharePoint superuser to be able to create a business logic that is to be implemented in SharePoint.</li>
 	<li>One can port their precious Workflows between environments with ease.</li>
</ol>
Too good to be true, you say? Well... Indeed there are some things you need to keep in mind.
<h3>Act 1</h3>
First and foremost, the SharePoint 2013 style Workflows are bound to GUIDs. This post's target audience is the superusers so I won't go into the details on why and where. When porting your Workflow between DEV, TEST, UAT and PROD environments, you have to use the same GUID for :
<ul>
 	<li>the actual list where you deploy your workflow to,</li>
 	<li>the Workflow History List,</li>
 	<li>the Workflow Task List.</li>
</ul>
Also, you have to make sure that:
<ul>
 	<li>the association scope between the SharePoint site and the Workflow Manager farm is different, because that's another GUID.</li>
</ul>
<h3>Act 2</h3>
So let's put it together. What steps do I need to make this thing work?
<ol>
 	<li>Create the basic structure of your solution in your DEV environment, including the
<ul>
 	<li>List(s) that you will use to store your data</li>
 	<li>Workflow History List</li>
 	<li>Workflow Task List</li>
</ul>
</li>
 	<li>Port your list to the other environments (TEST, UAT and PROD). For this you can use
<ul>
 	<li>Content Database transfer</li>
 	<li><del datetime="2017-02-09T14:10:45+00:00">SharePoint solution that deploys the above lists with the same GUID throughout the environments. <em><em>(This is the option you have to choose when you are in a Hybrid scenario, as you cannot just port the ContentDB to SharePoint Online.)</em></em> </del>
<em>(Seems this is not an option unfortunately. Thanks to my colleagues Westin Kirebel who highlighted this, and <a href="https://www.linkedin.com/in/istvan-fabian-9976871a">Istvan Fabian</a>, who helped with the clarification. Sorry.)</em></li>
</ul>
</li>
 	<li>Have your Business Champ create the Workflow in Visio 2013. <em>(This is an optional step, if you are the Site Administrator and you know what you want to achieve.)</em></li>
 	<li>Import the saved Workflow into SharePoint Designer - or create your own if you skipped Step 3 - and bound it to the List. Map the fields and do whatever other modifications are needed to make the Workflow working in SharePoint.</li>
 	<li>Export the Workflow from SPD.</li>
 	<li>Copy the VSDX file to your other environments.</li>
 	<li>Import the VSDX file in your other environments and bound them to the same lists.</li>
 	<li>Enjoy your solution.</li>
</ol>
&nbsp;
<h3>Act 3</h3>
Doing it once is not a big deal, since ALM is all about constant changes and updates, right? Yes, exactly. The beauty is that from this point whatever modification you do in your DEV environment can be easily ported to your other farms.

Those of you who paid attention so far might have noticed that I did not mention the columns so far. Indeed I did not, because the GUID of the columns are irrelevant. Only the internal name matters. How cool is that?

When you do subsequent export-imports between the environments, you will get a nice warning window in SharePoint Designer, reminding you that the VSDX file that you are about to import contains an update to an existing Workflow and asks you if you want to update it. Of course you do, since this is exactly why we are here, no? Of course we are.

&nbsp;
<h2><span class="mw-headline" id="Falling_action">Falling Action</span></h2>
The above described action works just fine with SP2010 and SP2013 style Workflows as well. Unfortunately it works with on-prem only as you cannot create lists with a specific GUID and you cannot port the ContentDB to the Cloud.

Life is good. (<a href="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg"><img width="13" height="17" class="alignnone size-full wp-image-72" alt="notesign" src="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg" /></a> I feel good <a href="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg"><img width="13" height="17" class="alignnone size-full wp-image-72" alt="notesign" src="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg" /></a>)
<h2><span class="mw-headline" id="D.C3.A9nouement.3B_resolution.2C_revelation_or_catastrophe">Dénouement</span></h2>
As you can see, the built-in SharePoint Workflow infrastructure is flexible enough to implement an Application Lifecycle Management easily. For how long? That we do not know, as - at the time of the writing of this post - Microsoft has no clear statement of the future of the built-in Workflow infrastructure, the only thing we know is both the 2010 and 2013 style Workflows are deprecated.

One thing we know for sure though; Future is the Cloud after all.

(<a href="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg"><img width="13" height="17" class="alignnone size-full wp-image-72" alt="notesign" src="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg" /></a> Blue skies <a href="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg"><img width="13" height="17" class="alignnone size-full wp-image-72" alt="notesign" src="https://msdnshared.blob.core.windows.net/media/2016/01/notesign.jpg" /></a>)]]></content:encoded>
    </item>
  </channel>
</rss>