Avoiding C++ types in your P/invokes

It's easier to explain this in an example. Let’s say you have a export function in your native DLL:

 void WINAPI MyFunction(_com_ptr_t<IMyInterface> ptr)
 {
    // ... do something with ptr
    ptr->Func();
 }

And the DllImport would be something like this:

 [DllImport("mydll.dll")]
void MyFunction(IMyInterface ptr);

Everything seems to be perfectly valid, except that there is a extra release, and you'll end up crashing later when the RCW finalizes.

The problem here is the _com_ptr<IMyInterface> argument. This is a C++ smart COM pointer, and it AddRef the pointer when it is created, and Release the pointer when it goes out of scope. Normally, when you are calling it in C++, C++ will take care of everything, making sure the contruction of _com_ptr is done correctly, making sure the AddRef is done. For example, when you are calling MyFunction with a IMyInterface* in C++, the actual code generated will look like this:

 IMyInterface *p;
 _com_ptr_t<IMyInterface> ptr(p);     // -------> C++ inserts this code for you
MyFunction(ptr);  

However, when you are calling this as a P/invoke, the construction of ptr is not done at all - CLR is not C++, and has absolutely no knowledge about your _com_ptr_t. Whatever CLR passes as the interface pointer, becomes the _com_ptr_t. This happens to work (except for the extra release part), because _com_ptr_t only has one pointer field. As a result, AddRef is not done, and _com_ptr_t does an extra Release on the interface pointer. Then when CLR is trying to release the COM object, CLR will crash because the COM object will be gone/deleted before the last release from CLR. And people typically blame CLR for that because CLR dll shows up on the stack...

The underlying problem here, is that C++ types come with additional semantics that is not part of the P/invoke signature and is not understood by CLR, and there is no way CLR could follow C++ semantics and make sure they are done correctly. Some common mistakes that people make are:

1. Assume constructors are called
2. Don't realize that C++ classes might come with v-tables that change the layout of other fields
3. Assume C++ 'standard' type like std::string is understood by CLR

We've already discussed #1. #2 is fairly obvious if you think about it. #3 is a bit interesting - potentially CLR could support a MarshalAs(CPPString), but std::string not standardized in terms of binary memory layout, CLR can't marshal them reliably if CLR don't know for sure what they look like in memory. So it is always a good idea to avoid C++ types in your native functions that you need P/invoke to.

BTW, this actually isn't really specific to .NET interop. Any time you are 'interop'ing with other languages, you better make sure you only pass data that other languages understand.