Random.Next()

Interop: marshaling double pointers that can be null

Carlos [MSR] — Wed, 15 Jun 2011 12:04:00 GMT

I recently had to implement IMediaBuffer in managed code. The GetBufferAndLength method takes a BYTE** ppBuffer argument, so I initially went with the following straightforward C# declaration:

public void GetBufferAndLength(out IntPtr ppBuffer, out uint cbLength)
{

ppBuffer = myBuffer;
…
}

This seemed to work fine, until I noticed a lot of first time NullReferenceExceptions were scrolling by in the debugger output window. Puzzled, I turned on breaking on all exceptions in VS and found that the native code calling my IMediaBuffer implementation would sometimes call with a NULL first argument in cases where it was only interested in obtaining the buffer length (cbLength). Apparently the interop layer was silently handling such cases. This was less than ideal, so I spent some time searching for a better way to do the managed declaration, looking at the various MarshalAs and such attributes but with no luck. Finally it dawned on me that a double pointer is also just a pointer and that I could do the dereferencing myself:

public void GetBufferAndLength(IntPtr ppBuffer, out uint cbLength)
{

     if (ppBuffer != IntPtr.Zero)

          Marshal.WriteIntPtr(ppBuffer, myBuffer);

...

Voila! This can handle the double pointer being NULL and no more NullReferenceExceptions :)

Silverlight anti-aliasing woes

Carlos [MSR] — Thu, 03 Dec 2009 09:36:53 GMT

I recently ran into an interesting problem. The designer in my team gave me a piece of XAML for my UI containing horizontal lines of the form:

I put this in the application and to our surprise, these lines showed twice as thick (2px) on my machine but looked fine on his. After researching this for a while I found it can happen when elements are not positioned on whole pixel boundaries. In this cases Silverlight has to do some anti-aliasing, which can result in additional pixels being rendered. For example, notice the difference between this rectangle when positioned at an integer and non-integer pixel positions (zoomed in):

So this was happening in my case and my 1 pixel line was becoming 2 pixels thick. However, the problem was still mystery to me because I only use integers for all the sizes and positions in the UI. Finally it dawned on me: I have some code to programmatically center a parent of this element on a canvas. When doing the math to center this, depending on the browser window size, I would get a non-integer position (which explained the variation across machines).

A simple Math.Round fixed the problem.

Deferred TLS/SSL handshakes on windows mobile

Carlos [MSR] — Tue, 15 Sep 2009 12:57:00 GMT

I recently received an email and blog comment from João Almeida, who was trying to use the SslHelper class to make a secure connection to an SMTP server and was having some trouble. After doing a little research I found this is a little different from doing an HTTPS connection. The SMTP protocol requires the connection to start in plain text and the TLS handshake to happen after the STARTTLS command is issued. Here’s how a typical secure SMTP session would start (bolded text is what the SMTP client needs to send, normal text are server responses):

>telnet smtp.live.com 587

220 BLU0-SMTP8.blu0.hotmail.com Microsoft ESMTP MAIL Service, Version: 6.0.3790.3959 ready at Mon, 14 Sep 2009 16:44:33 -0700
EHLO
250-BLU0-SMTP8.blu0.hotmail.com Hello [131.107.0.75]
250-TURN
250-SIZE 35840000
250-ETRN
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-8bitmime
250-BINARYMIME
250-CHUNKING
250-VRFY
250-TLS
250-STARTTLS
250 OK
STARTTLS
220 2.0.0 SMTP server ready

Up to this point everything needs to be in plain text. But once the 220 status code is received the TLS handshake needs to be started.

So how does the SslHelper code need to change to accommodate this? We need to ask winsock to do a deferred handshake as follows (please refer to the full listing of SslHelper). First, we need to bring in a couple of constants from the mobile SDK:

private const int SSL_FLAG_DEFER_HANDSHAKE = 0x0008;
private const int _SO_SSL_PERFORM_HANDSHAKE = 0x0d;
private const long SO_SSL_PERFORM_HANDSHAKE = _SO_SSL | _SO_SSL_PERFORM_HANDSHAKE;

Next, we need to tell the socket that we want it to use a deferred handshake by calling socket.IOControl with the SSL_FLAG_DEFER_HANDSHAKE. We do this in the constructor after setting the ssl cert validation hook:

socket.IOControl((int)SO_SSL_SET_VALIDATE_CERT_HOOK, inBuffer, null);
if (deferHandshake)
    socket.IOControl((int)SO_SSL_SET_FLAGS, BitConverter.GetBytes(SSL_FLAG_DEFER_HANDSHAKE), null);

Finally, we need to add a method that will let us start the handshake when we need to:

        public void DoHandshake()
        {
            socket.IOControl((int)SO_SSL_PERFORM_HANDSHAKE, BitConverter.GetBytes(ptrHost.ToInt32()), null);
        }

With this in place, here’s a quick and dirty example of how to start a TLS session with an SMTP server:

string SmtpServer = "smtp.live.com";
int port = 587;
IPHostEntry IPhst = Dns.GetHostEntry(SmtpServer);
IPEndPoint endPt = new IPEndPoint(IPhst.AddressList[0], port);
Socket s = new Socket(endPt.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
using (SslHelper helper = new SslHelper(s, SmtpServer, true))
{
    string line = null;
    s.Connect(endPt);
    StreamReader sr = new StreamReader(new NetworkStream(s), Encoding.ASCII);                    
    s.Send(Encoding.ASCII.GetBytes("EHLO\r\n"));
    while ((line = sr.ReadLine()) != "250 OK") ;
    s.Send(Encoding.ASCII.GetBytes("STARTTLS\r\n"));
    while (!(line = sr.ReadLine()).StartsWith("220")) ;
    helper.DoHandshake();
    Debug.WriteLine("success");
}

Again, thanks to João for the question and for helping me validate the approach.

Handling DOM events in a C# ActiveX

Carlos [MSR] — Fri, 28 Aug 2009 19:38:35 GMT

I recently had to implement an ActiveX object (or I guess just a COM object since it had no UI) in C#. There are several articles out there about how to do this (bing it), but I couldn’t find much information about handling DOM events in managed code. After some digging and experimentation I found a way to make this work, which is the subject of this post. I’ll assume you have some familiarity with getting a basic managed object running in IE and calling its methods from Javascript.

The first step in handling a DOM event is getting a pointer to the IHtmlDocument2 interface in your C# code. The easiest way is to just pass it in explicitly from Javascript. To do this first you’ll need to add a COM reference to mshtml:

Then you add a method to your COM object to receive this pointer (I decided to call it Initialize):

using System;
using System.Runtime.InteropServices;
using mshtml;
 
namespace ManagedActiveX
{ 

    public interface ITestObject
    {
        void Initialize(IHTMLDocument2 doc);
    }
 
    [ComVisible(true)]
    [ClassInterface(ClassInterfaceType.None)]    
    public class TestObject : ITestObject
    {
        public void Initialize(IHTMLDocument2 doc)
        {                        
        }
    }
}

And you can call this from your Javascript code to pass in the document:

var testObj = new ActiveXObject("ManagedActiveX.TestObject");

function Init() {

testObj.Initialize(document);

}

script>

At this point your C# object has a pointer to the document and can get to the various events the DOM exposes, but how do you register a callback? Let’s take window.onunload for example:

        public void Initialize(IHTMLDocument2 doc)
        {                     
            doc.parentWindow.onunload = ??
        }

The onunload member is of type object, which doesn’t give much of a clue about what needs to go in there. I tried a delegate but that didn’t work. Then I looked at the documentation for this member which describes what you need to pass as:

VARIANT of type VT_DISPATCH that specifies the IDispatch interface of an object with a default method that is invoked when the event occurs.

So how do you do that from C#? After doing a little reading I found all COMVisible managed objects automatically implement IDispatch. That is, the COM Callable Wrapper generated by .NET to expose your object to COM creates an implementation of IDispatch for you. Now we need to figure out how to tell the wrapper which method is the default IDispatch method. I found you specify this by marking the method with a Dispatch Id of 0:

    [ComVisible(true)]
    [ClassInterface(ClassInterfaceType.None)]    
    public class OnUnloadHandler 
    {
        IHTMLDocument2 doc;
 
        public OnUnloadHandler(IHTMLDocument2 document)
        {
            doc = document;
        }
 
        [DispId(0)]
        public object DefaultMethod()
        {
            doc.parentWindow.alert("onunload event handled in managed code");
            return null;
        }
    }

Then you simply create an instance of this type and set it as the onunload handler:

        public void Initialize(IHTMLDocument2 doc)
        {
            doc.parentWindow.onunload = new OnUnloadHandler(doc);
        }

If you host this object on a page with the script code shown above and then navigate away from it you should get the alert to pop up. Drop me a line if this doesn’t (or does :)) work for you.

New name: Random.Next()

Carlos [MSR] — Mon, 24 Aug 2009 00:50:45 GMT

I decided to rename the blog. Given that it just started I figured this wouldn’t be too bad. Plus “Carlos’” blog was lacking some creativity.

So why “Random.Next()”? Well, I don’t work on any particular Microsoft product or research area so the blog content will largely be dictated by the projects that get thrown my way. As such I feel there will be a little bit of randomness to the technologies I’ll cover here.

Plus I like the fact that Random.Next() always returns something positive :)

Getting a managed socket to talk SSL

Carlos [MSR] — Sun, 23 Aug 2009 00:05:40 GMT

This is a follow up to my previous post, where I showed a helper class I wrote to enable a managed socket to communicate via SSL on windows mobile. It was my first technical blog post and a friend gave me some very good feedback: blogs can be more interesting if they not only show the solution to a problem, but also how the solution was found. So here I’d like to tell you a little bit of what I tried before finally making SSL work on my mobile device.

I started searching MSDN and came across this article. The article talks about a native implementation using winsock and basically talks about calling setsockopt and WSAIoctl on the socket to enable SSL and register a certificate validation callback, respectively. I found the managed Socket class has corresponding methods: SetSocketOption and IOControl and at this point I thought this would be straightforward. To a certain extent it was, but I still had some work to do.

My first surprise was that SetSocketOption takes a SocketOptionName enum value as the second parameter, but this enum doesn’t have the equivalent of SO_SECURE. However, C# was nice enough to let me cast an arbitrary integer value to the enum I needed:

            //The managed SocketOptionName enum doesn't have SO_SECURE so here we cast the integer value
            socket.SetSocketOption(SocketOptionLevel.Socket, (SocketOptionName)SO_SECURE, SO_SEC_SSL);

Next in line was calling socket.IOControl to register a certificate validation callback. Turns out this method is a very general purpose one that takes a control code and then simply a buffer whose interpretation depends on such code. For SO_SSL_SET_VALIDATE_CERT_HOOK the first four bytes in this buffer are a function pointer to the certificate validation callback and the next 4 bytes are a pointer to a string containing the name of the host. Since we want to implement the validation callback in C# (more on why later) we need to first get a pointer to a delegate that we can pass to unmanaged code:

            hookFunc = Marshal.GetFunctionPointerForDelegate(new SSLVALIDATECERTFUNC(ValidateCert));
 

Note that it is important to maintain a reference to this IntPtr, otherwise the GC could collect it while the unmanaged code still holds a reference to it. I missed this detail the first time around and it seemed to work fine. But this was just because the GC was not reclaiming that pointer right away and that could change for any number of reasons so I was risking running into a very hard to find bug later on.

Now we need to write the host name into an unmanaged buffer, which we first need to allocate:

            //Allocate the buffer for the string
            ptrHost = Marshal.AllocHGlobal(host.Length + 1);
            WriteASCIIString(ptrHost, host);
 

This buffer is freed when the object is disposed since it needs to stick around until the certificate validation happens. Finally, we put these 8 bytes into a buffer and call socket.IOControl:

//Now put both pointers into a byte[]
            var inBuffer = new byte[8];
            var hookFuncBytes = BitConverter.GetBytes(hookFunc.ToInt32());
            var hostPtrBytes = BitConverter.GetBytes(ptrHost.ToInt32());
            Array.Copy(hookFuncBytes, inBuffer, hookFuncBytes.Length);
            Array.Copy(hostPtrBytes, 0, inBuffer, hookFuncBytes.Length, hostPtrBytes.Length);
 
            unchecked
            {
                socket.IOControl((int)SO_SSL_SET_VALIDATE_CERT_HOOK, inBuffer, null);
            }

That’s it as far as enabling SSL over the socket. All that remains is writing the ValidateCert function. This turns out to be relatively simple because .NET has a X509Certificate2 class that can parse the certificate blob passed to us. You can see the implementation of this function in the previous post, where I do the required validations. But I want to make a comment about a particular piece of code that had me scratch my head for a little while. This is the code that takes the unmanaged pointer to the certificate passed in and reads the data into managed code:

            //NOTE: The documentation says pCertChain is a pointer to a LPBLOB struct:
            //
            // {ulong size, byte* data} 
            //
            //in reality the size is a 32 bit integer (not 64).
            int certSize = Marshal.ReadInt32(pCertChain);
            IntPtr pData = Marshal.ReadIntPtr(new IntPtr(pCertChain.ToInt32() + sizeof(int)));
 
            byte[] certData = new byte[certSize];

When I read in the documentation that size was a long, I was thinking C#, so I was trying to read the size as a 64 bit integer which gave back a huge number. Needles to say the allocation of the byte[] threw an OutOfMemoryException. What I was missing was the fact that while a C# long is 64 bits, a C++ long is only 32 bits. And now that I think about it, it wasn’t just the C++ side of my brain that was turned off at that point, but also my common sense. Why on earth would you need 64 bits to indicate the size of the certificate? :)

Lucky for me I decided to just try reading 32 bits instead and then it all worked. However, I came to the wrong conclusion that the docs were mistaken when in fact I was failing to think about the differences in data types across managed and unmanaged. It wasn’t until I was re-reading the code for this post and I realized the mistake I had made.

Here are links to some of the material I read when putting this together:

https://msdn.microsoft.com/en-us/library/system.net.sockets.socket_members.aspx

https://msdn.microsoft.com/en-us/library/aa916117.aspx

https://windowsmobilepro.blogspot.com/2006/03/windows-mobile-secure-socket_25.html

Enable SSL for managed socket on windows mobile

Carlos [MSR] — Fri, 21 Aug 2009 22:33:10 GMT

I recently had to implement an application on windows mobile that would talk to a service via an SSL socket. This wasn’t just doing https requests so HttpWebRequest was not an option. And unfortunately the compact framework doesn’t include the SslStream class (which would have made my life easier).

I found windows mobile sockets support SSL through winsock so I decided to explore this route. Since my application was mostly managed it took a little interop dancing but I finally got it to work. What I came up with is a helper class that can take an existing socket and will enable SSL on it. For example:

var socket = new Socket(ipaddr, SocketType.Stream, ProtocolType.IP);
using (var sslHelper = new SslHelper(socket, host))
{
socket.Connect(ep);
     //at this point the SSL handshake has occurred and you can
    //send and receive normally
} 

Below is the code for this class. This is not meant to be production ready but can serve as an illustration on how to hook things up. In my case this was used as part of a proof of concept prototype.

It took me some time to figure out how all the pieces work together since the documentation is a little scattered but hopefully if you have to do something similar it can save you some time.

Another option might be to use SSPI, although I haven’t looked into that yet.

using System;
using System.Diagnostics;
using System.Net.Sockets;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Globalization;
 
namespace SslTest
{
    class SslHelper : IDisposable
    {
        private const ushort SO_SECURE = 0x2001;
        private const ushort SO_SEC_SSL = 0x2004;
        private const int _SO_SSL_FLAGS = 0x02;
        private const int _SO_SSL_VALIDATE_CERT_HOOK = 0x08;
        private const int SO_SSL_FAMILY = 0x00730000;
        private const long _SO_SSL = ((2L << 27) | SO_SSL_FAMILY);
        private const uint IOC_IN = 0x80000000;
        private const long SO_SSL_SET_VALIDATE_CERT_HOOK = (IOC_IN | _SO_SSL | _SO_SSL_VALIDATE_CERT_HOOK);
        private const long SO_SSL_SET_FLAGS = (IOC_IN | _SO_SSL | _SO_SSL_FLAGS);
 
        private const int SSL_CERT_X59            = 1;
        private const int SSL_ERR_OKAY            = 0;
        private const int SSL_ERR_FAILED          = 2;
        private const int SSL_ERR_BAD_LEN         = 3;
        private const int SSL_ERR_BAD_TYPE        = 4;
        private const int SSL_ERR_BAD_DATA        = 5;
        private const int SSL_ERR_NO_CERT         = 6;
        private const int SSL_ERR_BAD_SIG         = 7;
        private const int SSL_ERR_CERT_EXPIRED    = 8;
        private const int SSL_ERR_CERT_REVOKED    = 9;
        private const int SSL_ERR_CERT_UNKNOWN    = 10;
        private const int SSL_ERR_SIGNATURE       = 11;
        private const int SSL_CERT_FLAG_ISSUER_UNKNOWN = 0x0001;
 
 
        public delegate int SSLVALIDATECERTFUNC(uint dwType, IntPtr pvArg, uint dwChainLen, IntPtr pCertChain, uint dwFlags);        
        private IntPtr ptrHost;
        private IntPtr hookFunc;
 
        public SslHelper(Socket socket, string host)
        {
            //The managed SocketOptionName enum doesn't have SO_SECURE so here we cast the integer value
            socket.SetSocketOption(SocketOptionLevel.Socket, (SocketOptionName)SO_SECURE, SO_SEC_SSL);
 
            //We need to pass a function pointer and a pointer to a string containing the host
            //to unmanaged code
            hookFunc = Marshal.GetFunctionPointerForDelegate(new SSLVALIDATECERTFUNC(ValidateCert));
 
            //Allocate the buffer for the string
            ptrHost = Marshal.AllocHGlobal(host.Length + 1);
            WriteASCIIString(ptrHost, host);
 
            //Now put both pointers into a byte[]
            var inBuffer = new byte[8];
            var hookFuncBytes = BitConverter.GetBytes(hookFunc.ToInt32());
            var hostPtrBytes = BitConverter.GetBytes(ptrHost.ToInt32());
            Array.Copy(hookFuncBytes, inBuffer, hookFuncBytes.Length);
            Array.Copy(hostPtrBytes, 0, inBuffer, hookFuncBytes.Length, hostPtrBytes.Length);
 
            unchecked
            {
                socket.IOControl((int)SO_SSL_SET_VALIDATE_CERT_HOOK, inBuffer, null);
            }
        }
 
        private static void WriteASCIIString(IntPtr basePtr, string s)
        {
            byte[] bytes = Encoding.ASCII.GetBytes(s);
            for (int i = 0; i < bytes.Length; i++)
                Marshal.WriteByte(basePtr, i, bytes[i]);
 
            //null terminate the string
            Marshal.WriteByte(basePtr, bytes.Length, 0);
        }
 
        #region IDisposable Members
 
        ~SslHelper()
        {
            ReleaseHostPointer();
        }
 
        public void Dispose()
        {
            GC.SuppressFinalize(this);
            ReleaseHostPointer();
        }
 
        private void ReleaseHostPointer()
        {
            if (ptrHost != IntPtr.Zero)
            {
                Marshal.FreeHGlobal(ptrHost);
                ptrHost = IntPtr.Zero;
            }
        }
 
        #endregion
 
        private int ValidateCert(uint dwType, IntPtr pvArg, uint dwChainLen, IntPtr pCertChain, uint dwFlags)
        {  
            //According to https://msdn.microsoft.com/en-us/library/ms940451.aspx:
            //
            //- dwChainLen is always 1
            //- Windows CE performs the cert chain validation
            //- pvArg is the context data we passed into the SO_SSL_SET_VALIDATE_CERT_HOOK call so in our
            //- case is the host name
            //
            //So here we are responsible for validating the dates on the certificate and the CN
 
 
            if (dwType != SSL_CERT_X59)
                return SSL_ERR_BAD_TYPE;
 
            //When in debug mode let self-signed certificates through ...
#if !DEBUG
            if ((dwFlags & SSL_CERT_FLAG_ISSUER_UNKNOWN) != 0)
                return SSL_ERR_CERT_UNKNOWN;
#endif
 
            Debug.Assert(dwChainLen == 1);
 
            //Note about the note: an unmanaged long is 32 bits, unlike a managed long which is 64. I was missing 
            //this fact when I wrote the comment. So the docs are accurate.
            //NOTE: The documentation says pCertChain is a pointer to a LPBLOB struct:
            //
            // {ulong size, byte* data} 
            //
            //in reality the size is a 32 bit integer (not 64).
            int certSize = Marshal.ReadInt32(pCertChain);
            IntPtr pData = Marshal.ReadIntPtr(new IntPtr(pCertChain.ToInt32() + sizeof(int)));
 
            byte[] certData = new byte[certSize];
 
            for (int i = 0; i < certSize; i++)
                certData[i] = Marshal.ReadByte(pData, (int)i);
 
            X509Certificate2 cert;
            try
            {
                cert = new X509Certificate2(certData);
            }
            catch (ArgumentException) { return SSL_ERR_BAD_DATA; }
            catch (CryptographicException) { return SSL_ERR_BAD_DATA; }
 
            //Validate the expiration date
            if (DateTime.Now > DateTime.Parse(cert.GetExpirationDateString(), CultureInfo.CurrentCulture))
                return SSL_ERR_CERT_EXPIRED;
 
            //Validate the effective date
            if (DateTime.Now < DateTime.Parse(cert.GetEffectiveDateString(), CultureInfo.CurrentCulture))
                return SSL_ERR_FAILED;
 
            string certName = cert.GetName();
            Debug.WriteLine(certName);
 
            //Validate the CN
            string host = ReadAnsiString(pvArg);
            if (!certName.Contains("CN=" + host))
                return SSL_ERR_FAILED;
 
            return SSL_ERR_OKAY;
        }
 
        private static string ReadAnsiString(IntPtr pvArg)
        {
            byte[] buffer = new byte[1024];
            int j = 0;
            do
            {
                buffer[j] = Marshal.ReadByte(pvArg, j);
                j++;
            } while (buffer[j - 1] != 0);
            string host = Encoding.ASCII.GetString(buffer, 0, j - 1);
            return host;
        }
    }
}

New to the blogosphere

Carlos [MSR] — Fri, 21 Aug 2009 14:00:19 GMT

Finally, I decided to start a blog. This is something I’ve been thinking about for a while but kept putting off. This week I attended Devscovery and the keynote by Scott Hanselman was all about social networking for developers and I guess it gave me the nudge I needed to get started. By the way, if you can attend one of this events by Wintellect I’d recommend it.

So who am I? My name is Carlos Garcia Jurado Suarez and I’m a developer in Microsoft Research. I’ve been with Microsoft for 9 years and I spent the first half in Visual Studio working on things like the Class Designer and Application Designer. Nowadays I work on a variety of projects from prototypes to tech transfers as part of Research’s Advanced Development Team. It’s a lot of fun as I get to explore a very wide range of technologies and work with some very interesting people.

I expect most of my posts to be about interesting technical problems I run into. To get started I’ll be posting shortly about some fun I had with SSL on windows mobile which I hope people will find useful.