Phase 1: Collect crash or hang dump

Phase 2: Install debugging tools

1. Download the Windows 10 SDK from: https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk
2. Please note that this SDK works for all Windows versions starting Windows Server 2008, not only for Windows 10.
3. Run the installer
4. At first step, specify an easily accessible location, since we'll need to work from the Command Prompt
5. At the step that asks for features to install, select only "Debugging Tools for Windows"
6. Optional: Set symbol path. Debugging symbols are files generated at compile time which link variable and method addresses to names. By default, the debugger downloads symbols from the Microsoft public symbols server. To cache symbols locally, you may specify a System Environment Variable:       _NT_SYMBOL_PATH=srv*c:\Symbols*https://msdl.microsoft.com/download/symbols. More information here: https://msdn.microsoft.com/en-us/library/windows/hardware/ff552208(v=vs.85).aspx

Phase 3: Analyze DMP

• What is the crashing signature?
• What is the call stack?

• Faulting process / application name
• Faulting module
• Faulting offset
• Exception code

1. Run WinDBG. Should be located under: C:\debuggers\Debuggers\x64\windbg.exe
2. File - Open Crash Dump…
3. Search for the DMP file stored during Phase 1 - collection
4. Now the DMP file is loaded in WinDBG and commands can be executed to perform the analysis

1. Find the faulting module. It should be shown as the penultimate line in the command window. Running the .ecxr command also displays it. In this example, the faulting module is ntdll
2. Find the faulting offset: ?eip - <faulting module name> 
3. Find the faulting process and its command line: !peb
4. Find the call stack that leads to the crash: k
5. Find the exception code: .lastevent
6. Please refer to this post, which explains how to retrieve the X++ call stack.

!analyze -v

Phase 4. Take action

1. Determine frequency of particular crash

2. Determine if the crash is resolved

3. Call Support

Since then I've been implementing the user interface and, obviously, the RevisionStream also needed some readjustment. Here are some major changes that were done:

- detecting encoding of text file

- support for Unicode (2-bytes characters)

- some bugs related to addition and deletion

Until now I've managed to repair most of the visible bugs but I'm pretty sure there are some also remaining. Which is why, apart from the UserControl that holds the large file editor, I've also created a Form for testing the control. The source code is available so everyone interested and go ahead and fix the bugs they discover. I will also appreciate if you post comments with feedback.

The UserControl LargeTextBox contains a RichTextBox and a vertical scroll bar.

Apart from the RevisionStream, the LargeTextBox control also contains:

 -  long topPosition; // The position in the changed stream of first line in RTB

 - char[][] displayedLines; // The list of currently displayed text lines

 - int[] lineLengths; // The list of lengths of currently displayed text lines     

The most delicate part of the whole process is to detect the changes done by the user in the RichTextBox and update them in the Revision Stream. This is implemented in the method SynchDifferences where the cached lines are compared with the lines from the RichTextBox and Insertions and Deletions are added accordingly.

Let's see how this works on an Excel file. I create a Workbook with as many rows as Excel allows, namely 1,048,576. I saved this to Book2.xlsx and extracted the sheet1.xml. I then indented the XML to make more human-readable and loded it into the editor.

I then scrolled down to row 511,386 and added a new cell on the B column with the value of "0".

I saved the XML as sheet2.xml and then renamed it back to sheet1.xml (minor bug remaining - can't save to same file name). After repackaging the file in Excel, here's the result:

LFETester.zip

The basic idea behind a large file viewer is that you can’t load a very large file into a TextBox or RichTextBox control because the memory usage as well as the time to load the file will be enormous. Moreover, the user can only see a very small portion of the file on the screen at once, so it doesn’t make sense to load all the data in the file at once.

As such, what is needed from a large file viewer is to only display some small parts of the file in order to fill at least a screen. This is pretty straightforward and will be addressed in the second post. What is a bit more complex is to enable the user to edit the file (i.e. make insertions and deletions). The problem with editing the file is that edits need to be taken into account when computing the position in the stream. If the user deletes say the first 10 characters and then wants to read two characters at offset 4, what actually needs to be read is the two characters at offset 14 from the original file:

• Original string: "0123456789abcdefgh"
• String after deleting 10 characters: "abcdefgh"
• Character at offset 4 (zero-based): "e"
• Offset in initial string: 14.

Obviously, things get more complicated when the user starts to randomly delete and insert text at various locations in the file.

The solution I’m proposing is the class RevisionStream (see attached code). This class is wrapper around the FileStream class and supports four public methods:

• AddDeletion – the caller signals that a deletion has been made by the user
• AddInsertion – the caller signals that the user has inserted some text
• Read – reads some bytes from the revised stream
• SaveTo – saves revised stream to file

The basic idea behind this approach is that normally the largest part of the text will remain identical to the original stream and only some minor changes will be done by the user. So it makes sense to leave the original text on disk and only load to memory the parts that we need together with the changes the user makes.

Under the hood, the RevisionStream uses a list of Segments. A segment is either a reference to a contiguous section of the original text or a newly added text. The really difficult part is to adjust the segments from the RevisionStream when an insertion or a deletion is made.

Getting back to the previous example, the RevisionStream initially contains all the text from the original file, which is "0123456789abcdefgh". The list of segments will contain only one segment, [0, 17] (where 17 = the offset of the last character).

Scenario #1: The user deletes the first 10 characters

Resulting list of segments: [10, 17].
Resulting string: "abcdefgh"

Scenario #2: The user deletes the character ‘b’, which is located at offset 11.

Resulting list of segments: [0, 10], [12, 17].
Resulting string: "0123456789acdefgh".

Scenario #3: The user deletes the character ‘b’, which is located at offset 11, and writes "ZZZ" at the same offset.

Resulting list of segments: [0, 10],["ZZZ"],[12, 17].
Resulting string: "0123456789aZZZcdefgh".

The real challenge here is to deal with all the possible scenarios of deleting and inserting text (delete from more consecutive segments, delete at end of segment, etc.). Hopefully I’ve covered all the possible scenarios in the implementations of the private methods AdjustForInsertion and AdjustForDeletion but if you find any case that hasn’t been covered, please post a comment here.

There’s still some work to do here: using only long indices to ensure really large files can be read, making the Insertions encoding-independent, etc. However, up to this point, we can go ahead and build a pretty decent Large File Editor based on this Revision Stream.

RevisionStream.cs

 
If we uncheck “Allow background saves” from Word Options – Advanced – Save, this is what appears in Process Monitor:

 
 The problem with Process Monitor is that we can’t leave it running for a long time because the page file will become too large and it will not be able to continue the capture.
 
So the alternative is Registry Auditing.
 
What we need to do is:
 

1.       Run the following command from Command Prompt:

auditpol /set /subcategory:"Registry" /success:enable
 

Note: if the OS has a different language pack, the name “Registry” might differ. For instance, on a German Windows, the name is “Registrierung”. To see what the name of the subcategory is you can run:
 
auditpol /list /subcategory:*
 

2.       Open Registry Editor and navigate to the key which we want to audit (HKEY_CURRENT_USER\Software\Microsoft\Office\14.0\Word)
3.       Right-click on the key and choose “Permissions…”
4.       Click “Advanced” and switch to the Auditing tab
5.       Add a user or group and select Access: Set Value
6.       Apply settings
 
Now the registry changes are visible in the Event Viewer under Windows Logs\Security:

More detailed information on this topic:

- https://support.microsoft.com/kb/921469 How to use Group Policy to configure detailed security auditing settings for Windows Vista-based and Windows Server 2008-based computers in a Windows Server 2008 domain, in a Windows Server 2003 domain, or in a Windows 2000 domain.

- https://technet.microsoft.com/en-us/library/cc755264(WS.10).aspx Auditpol set

- https://support.microsoft.com/kb/324739 How to use Group Policy to audit registry keys in Windows Server 2003
 

What's different is obviously the rotation. What Office saves for a shape is the Left and Top coordinates together with a Rotation. So the Top coordinate, for instance, is not necessarily the uppermost part of a shape.

So let's solve the following problem: to which shape is the following arrow pointing?

Input data:

Arrow:                  Left =  98.74763      Top = 67.08575      Width = 162           Height = 60            Rotation = 45

Upper shape:       Left = 264                Top = 36                 Width = 168           Height = 72            Rotation = 0

Lower shape:       Left = 264                Top = 114               Width = 168           Height = 72            Rotation = 0

The code for computing the "From" and "To" points:

 public static void GetFromAndTo(RectangleF bounds, double rot, 
 ref PointF from, ref PointF to)
 {
 
 double La, Ta, Wa, Ha, alpha;
 La = bounds.Left;
 Ta = bounds.Top;
 Wa = bounds.Width;
 Ha = bounds.Height;
 
 alpha = (rot * Math.PI) / 180;
 
 from.X = (float)(La + Wa * (1 - Math.Cos(alpha)) / 2);
 from.Y = (float)(Ta + (Ha / 2) - (Wa * Math.Sin(alpha)) / 2);
 to.X = (float)(La + Wa * (1 + Math.Cos(alpha)) / 2);
 to.Y = (float)(Ta + (Ha / 2) + (Wa * Math.Sin(alpha)) / 2);
 }
 

 The "From" point has coordinates: (122.512 ; 39.8101) and the "To" point has coordinates:(237.0633 ; 154.3614)

What needs to be determined now is the distance between the "To" point and each rectangle. For that, we'll need to determine where the segment between "To" and the center of the rectangle intersects the bounds of the rectangle.

The line between "To" and the center intersects the bounds in found points at most. So we'll need to determine which one of these points is inside the bounds and between "To" and the center.

We start off by computing the line equation for the line that contains "To" and the center (Y = m * X + b). Then we find out the intersection by substituting the points on the bounds (e.g. Y = Top or X = Left + Width).

//Line equation between point and center
 double m = (cen.Y - src.Y) / (cen.X - src.X);
 double b = cen.Y - (cen.X * m);
 
 PointF[] pis = new PointF[4];
 pis[0] = new PointF((float)((bounds.Top - b) / m), bounds.Top);
 pis[1] = new PointF((float)(bounds.Left + bounds.Width), (float)((bounds.Left + bounds.Width) * m + b));
 pis[2] = new PointF((float)((bounds.Top + bounds.Height - b) / m), (float)(bounds.Top + bounds.Height));
 pis[3] = new PointF(bounds.Left, (float)(bounds.Left * m + b));
 sqdist = double.NaN;
 
 //Select the intersection with the bounds which is inside the bounds and between the two points
 for (int i = 0; i < 4; ++i)
 {
 
 if (IsInsideAndBetwen(pis[i], src, cen, bounds))
 {
 sqdist = Math.Pow(src.X - pis[i].X, 2) + Math.Pow(src.Y - pis[i].Y, 2);
 
 return;
 }
 }

The method IsInsideAndBetween(pi, src,cen, bounds) returns true if pi is between cen and src and if pi is inside the bounds.

 private bool IsInsideAndBetwen(PointF pi, PointF src, PointF cen, RectangleF bounds)
 {
 
 if (pi.X < bounds.Left ||
 pi.X > bounds.Left + bounds.Width ||
 pi.Y < bounds.Top ||
 pi.Y > bounds.Top + bounds.Height)
 return false; //It's outside bounds
 
 if (cen.X == pi.X)
 {
 if ((src.Y - cen.Y) / (pi.Y - cen.Y) < 1)
 return false;
 }
 
 else
 if (cen.Y == pi.Y)
 {
 if ((src.X - cen.X) / (pi.X - cen.X) < 1)
 return false;
 }
 else
 if ((src.X - cen.X) / (pi.X - cen.X) < 1 ||
 (src.Y - cen.Y) / (pi.Y - cen.Y) < 1)
 return false; //It's not between the two points.
 
 return true;
 }

Disclaimer: Sample Code is provided for the purpose of illustration only and is not intended to be
used in a production environment. THIS SAMPLE CODE AND ANY RELATED INFORMATION
ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE. We grant You a
nonexclusive, royalty-free right to use and modify the Sample Code and to
reproduce and distribute the object code form of the Sample Code, provided
that. You agree: (i) to not use Our name, logo, or trademarks to market Your
software product in which the Sample Code is embedded; (ii) to include a valid
copyright notice on Your software product in which the Sample Code is embedded;
and (iii) to indemnify, hold harmless, and defend Us and Our suppliers from and
against any claims or lawsuits, including attorneys’ fees, that arise or result
from the use or distribution of the Sample Code

Looking for some code samples how to pass DDE messages from a .NET program I found out that there‘s a great library already in place that facilitates working with DDE: the NDde project from Codeplex: https://ndde.codeplex.com/

However, I just needed to make one call and including a new DLL into my code would have been too much.

So I just created a sample C# program derived from the NDde code that imports only the necessary functions from the Win32 API to pass an Open DDE message to Excel for opening a workbook.

Workbook wb = DDEHelper.OpenWorkbook(_excel, args[0]);

Compilation:

D:\Projects\DDE>csc /r:C:\Windows\assembly\GAC_MSIL\Microsoft.Office.Interop.Excel\14.0.0.0__71e9bce111e9429c\Microsoft.Office.Interop.Excel.dll DDEOpen.cs

DDEOpen.cs

1. Write a source file:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Runtime.InteropServices;

using System.Windows.Forms;



namespace InteropTest

{

    [Guid("BEB2C62D-5646-4B08-81DF-670314BFEE4A")]

    public class InteropTestClass

    {

        public void Help()

        {

            MessageBox.Show("Hello From .NET!");

        }



    }

}

The GUID needs to be unique.

2. Generate a strong key to sign the assembly:

sn -k MyKey.snk

3. Compile the C# file using CSC:

csc /target:library /out:tstint.dll MyClass.cs /keyfile:MyKey.snk

We now have a DLL that can be exported to another machine with the .NET Framework installed. On the target machnie, the assembly needs to be registered:

regasm tstint.dll /codebase /tlb:tstint.tlb

Now the library can be used in Office by selecting it from the list of references in VBA:

The PPT file format is very well documented as per Microsoft’s Open Specification Promise. However, accessing any part of a PPT file isn’t really a snap.

I’ve created a small C# program which parses the \Pictures stream from a PPT file. For that, one needs to:

• Extract the Pictures stream from the PPT OLE storage
• Parse the stream according to the OfficeArtFBSE format
• Extract all the binary pictures according to format: WMF, EMF, PNG, JPG

This can be used for damaged PPT files where the shapes that contain pictures can’t be viewed anymore.

ParsePictures.zip

Editing Excel OpenXML documents using the OpenXML 2.0 SDK offers better performace and improved supportability compared to normal Office automation (e.g. VBA). There are also great examples on the Internet to get started: Office Open XML Formats: Inserting Values into Excel 2007 Cells.

What I haven’t found until now is some clarifications as to how one can edit multiple Excel cells without corrupting the workbook. This is what I’m presenting in this blog post.

The fact is that whenever rows and cells need to be added to an Excel worksheet, they need to be sorted upon insertion. Take for instance the XML for the following worksheet where A1 = 1, B1 = 2 and B2 = 3. Here is the only way the XML should be saved like:

If we inserted say cell B1 before cell A1, Excel would show a corruption error when opening the XLSX.

The program that I’m attaching here does exactly this: the input is a worksheet, a row and a column and the output is a modified worksheet that contains the row and the cell required for that cell to be edited.

Other useful features:

- Convert column index to colum name (e.g. 28 to AB)

- Get the column index from cell reference (e.g. AB23 to 28)

- Selecting cells based on argument value using LINQ

Sample edited worksheet:

Note: The sample program does not create a blank workbook, you will need to save one from Excel and pass the full path and the name of the worksheet to the program.

ModifyWorksheet.zip

The operation of indenting XML files is pretty trivial when using the classes from the System.XML namespace. However, I often find myself trying to read canonical XML code from files. As such, a created a very simple tool that converts XML files to indented XML.

All you need to do is to drag and drop an XML file into the WPF window and there you have it!

And the difference:

Canonical.zip