CAS and Native Code

  • Article

CAS is complicated enough to understand when all of the moving parts are written in managed code (and therefore have all the associated managed meta-information like grant sets, etc).  However, once native code comes into play things can get even more confusing.  Let's take a look at how CAS works when there's native code on the call stack.

For this discussion, lets assume we have a call stack (growing down) like so:

Managed code: M1

Managed code: M2

Native code: N1

Managed code: M3

Managed code: M4

AppDomain: AD

So in this example, M1 calls M2, which calls N1, which calls back to managed code M3 and M4.  All of the managed objects live in AppDomain AD.

Full Demands

Now lets consider what happens when M4 triggers a full stack walk for a demand.  In this case, the CLR essentially ignores the native code on the stack, and does a stack walk looking at M3, M2, M1, and AD.  Intuitively, this is done because the native code doesn't have a grant set associated with it, and therefore including it on a CAS stack walk wouldn't make much sense.

It's important to note however that the stack walk proceeds through N1, rather than stopping at it.  So in this case if M3 and M2 are fully trusted, but M1 is partial trust and M4 triggered a FullTrust demand, that demand would fail.

Stack walk modifiers like Assert continue to work just as you would expect as well, so it's perfectly legal for M2 to Assert FullTrust, which would make M4's demand succeed.

Link demands are much more interesting when native code gets involved.  In our example, imagine that managed method M3 has a LinkDemand for FullTrust on it.  Normally, LinkDemands are evaluated at JIT time against the caller of the method with the demand.  But in this case, the caller of M3 is native code which is not JITed (and doesn't have a grant set to evaluate against in any event).

The obvious solution then is to evaluate the LinkDemand against the last managed frame on the call stack before we transitioned to native code.  At the time we're JITing this method however, we only know that it's calling a native method -- we don't know what managed code that native code is going to turn around and call later on.  In this example, when M2 is being JITed we know that it may call N1, however we don't know that N1 will turn around and call M3 and therefore don't know that M3 has as a LinkDemand to evaluate against M2.

However, it turns out that we don't need to know what M3 is going to LinkDemand of M2.  Since M2 is calling N1, it must have UnmanagedCode permission, and since UnmanagedCode permission is never handed out without FullTrust, we can reason that M2 must be fully trusted.  If M2 is fully trusted, then it will satisfy any LinkDemand that M3 will require.

With this analysis, we could say that LinkDemands are implicitly satisfied by native code callers.  However, that may not be the effect that we want in all cases.  For instance, one common scenario might be to expose a managed object via COM Interop to a script.  (For instance, make a managed object available via COM to some JavaScript in a web page).

From the CLR's perspective, this script is just native code, so our reasoning would lead to any link demands on methods the script calls being satisfied.  However, we may not want the script to satisfy all link demands.  In order to solve this problem, the CLR treats calls to managed code from native via COM Interop differently.

If native code uses a COM interface to call a managed method protected with a LinkDemand, then that LinkDemand is evaluated against the AppDomain which the managed object lives in.

Let's go back to our example again, and see how these rules apply.  Let's suppose that N1 calls M3 through COM Interop, M3 has a LinkDemand for FullTrust, and the AppDomain AD is also fully trusted.  In this case, the call succeeds because the AppDomain's grant set satisfies M3's LinkDemand.

Now let's consider the case where this AppDomain is hosted by Internet Explorer, and therefore has only the Internet grant set.  When N1 calls into M3, we'll see that the object M3 is being called on lives in AppDomain AD, and that causes us to look up the grant set of that domain.  We then check M3's LinkDemand for FullTrust against the AppDomain's grant set of Internet.  Since the AppDomain's grant set does not satisfy the LinkDemand, we throw a SecurityException.  Note that this is true even if the last managed frame on the stack (M2) is fully trusted.

If we make one more change to the scenario, and have N1 call M3 via reverse P/Invoke we a different result.  Even though the AppDomain is partially trusted, since M3 was not invoked from COM Interop, we allow the call to succeed.

3 Rules for Native Code CAS Evaluation

That's a ton of complexity, but thankfully we can boil it down to three rules depending upon your scenario:

  1. If a full demand is done, native stack frames are ignored and the stack walk proceeds exactly as if there were only managed frames on the stack.
  2. If a link demand is done via COM Interop, the link demand is evaluated against the grant set of the AppDomain that the managed object lives in.
  3. If a link demand is done via reverse P/Invoke, the link demand is satisfied and the call succeeds.