Anyway, I wanted to let you all know that the CLR Profiling API is shifting ownership to some other fantastic folks on the CLR team.  So after many years of working on this facet of the CLR, it's time for me to say goodbye.

Please continue to monitor the .NET Team blog (https://blogs.msdn.com/b/dotnet/) for exciting new happenings in the CLR, including the profiling API.

So long, and thanks for the memories!

Dave

https://go.microsoft.com/fwlink/?LinkID=271485

The whitepaper provides recommendations for all the changes you’ll need to make so your profiler can profile Windows Store apps.  As you read through the whitepaper, keep in mind that the source code to CLRProfiler 4.5 can also be used to illustrate many of these changes you’ll need to make.

Happy coding!

• Parsing a stream of IL bytes into a linked list of editable structures
• Writing that linked list back out into a new stream of IL bytes
• Using the metadata API to add AssemblyRefs, TypeRefs, and MemberRefs to modules you instrument
• Using the metadata API to add brand new methods into mscorlib.dll
• Using the new RequestReJIT and RequestRevert APIs.

Click on the ILRewrite10Source download from the Downloads tab.  You can find some basic documentation on the Documentation tab, and more detailed documentation in the readme.txt distributed as part of the source.

If you’re interested in using CLRProfiler to diagnose memory issues with your managed app, including managed Windows Store apps, all you need is the CLRProfiler45Binaries download from the Downloads tab.  Please be sure to read the installation instructions first.  You will find a link to the installation instructions from the Downloads tab.

If you’re interested in writing your own profiler to diagnose Windows Store apps, you may find CLRProfiler 4.5 useful as an example.  You’ll want the CLRProfiler45Source download from the Downloads tab.  (Note, in early December we expect also to release a white paper documenting what you’ll need to know about writing a profiler that analyzes Windows Store apps.)

In the coming weeks we expect to publish more complete information on CLRProfiler 4.5, but I wanted to get this small post out there now so you’re aware that CLRProfiler 4.5 is available.

Some profilers manually advance through objects on the heap, inspecting their field values, by starting at an ObjectID and moving forward by its size to the next ObjectID, repeating this process, for all the reported generation ranges (via GetGenerationBounds or MovedReferences/SurvivingReferences).  If your profiler doesn’t do this, then this blog entry will be of no interest to you, and you can skip it.  But if your profiler does do this, you need to be aware of the alignment rules that the CLR employs as it allocates and moves objects around on the GC heap.

• On x86: All objects are 4-byte aligned, except for objects on the large-object-heap, which are always 8-byte aligned.
• On x64: All objects are always 8-byte aligned, in all generations.

And the important point to note is that GetObjectSize does NOT include alignment padding in the size that it reports.  Thus, as your profiler manually skips from object to object by using GetObjectSize() to determine how far to skip, your profiler must manually add in any alignment padding necessary to achieve the alignment rules listed above.

The overall guidance is that if you plan to dereference an ObjectID or pass it to an ICorProfilerInfo(2,3,4) method, then you must do so either:

1. From inside a GC, from a thread doing the GC (e.g., in response to one of the GC callbacks, in which case you're guaranteed that the GC is blocked by this call), OR
2. From a callback that gave you the ObjectID (in which case you're guaranteed that the GC is blocked by the callback that gave you the ObjectID)

Of course, taking an ObjectID that you were given and caching it away somewhere is a big no-no, unless you take pains to update all the ObjectIDs in your cache on every GC, by using the SurvivingReferences/MovedReferences callbacks.  And even with a well-updated cache such as this, the CLR will still require that you pass ObjectIDs to Info methods only in the above two circumstances, or else you will receive an error HRESULT.  This extra checking was added in .NET 4.0.

The reason for this is that some code paths in the CLR assume that their thread is already blocking the GC (to ensure referenced objects stay put), but there was no enforcement in place to ensure this.  So we added this enforcement in .NET 4.0.  Without checks like this, it could be possible to cause nondeterministic GC heap corruptions or to reference the wrong memory.  For example, calling GetObjectSize on a thread that you create (i.e., not a manage thread) does not intrinsically block the GC, and thus is considered unsafe.

In this post, I write about the two primary kinds of IDs your profiler deals with, when each kind is appropriate to use, how to convert between those two types of IDs, and some gotchas with those conversions—particularly in how they may invoke the type loader.

The two kinds of IDs

Profilers have to deal with two kinds of IDs.  The first kind are IDs from metadata, a.k.a., metadata tokens.  These are the mdToken values, like mdMethodDef or mdTypeDef, which are read straight out of the metadata of managed modules.  These values do not change for a given module from process to process.  They are placed in the module by the language compiler that generates the IL (e.g., csc.exe).  Profilers typically use metadata tokens in order to look up symbolic information from the metadata (e.g., for pretty-printing names of methods or classes), and for performing IL rewriting.  Metadata tokens are also fantastic for deferring symbolic lookup to a post-processing phase.  For example, a sampling profiler could log metadata tokens for classes and functions encountered on a sample at run-time and defer looking up the names of those classes and functions to a post-processing phase that occurs after the profiled process has exited.  This keeps the profiler’s data collection lightweight, and is only possible because metadata tokens don’t change so long as the managed modules defining those tokens don’t change.

The second kind of IDs are run-time IDs, such as FunctionID or ClassID which are defined in corprof.idl.  These values do change from process to process, and they represent internal data structures that the CLR builds up at run-time as it loads modules, loads types, JIT compiles functions, etc.  Profilers use these values as its main currency between ICorProfilerInfo* and ICorProfilerCallback* methods.  The CLR uses these values when it notifies profilers of various events (ICorProfilerCallback* methods), and the profiler passes these values back into the CLR (ICorProfilerInfo* methods) in order to get further information about them.  These IDs are handy because they are your profiler’s key to unlocking class layout, generated code, object addresses, and everything else that the CLR maintains about the actively executing managed code at run-time.  See this post for more info about what these IDs really are.

Converting between metadata tokens and run-time IDs

Since metadata tokens are good for some things and run-time IDs are good for others, you will inevitably find yourself in situations where you have one kind of ID handy, but you really need the other kind of ID.  Can you convert from one kind of ID to another?  Yes, but there are some caveats!

It’s always safe to go this direction: run-time ID –> metadata token.  Just use methods such as GetFunctionInfo2 and GetClassIDInfo2, which take run-time IDs as input, and provide their module + metadata token as (part of) the output.

However, it is problematic going the opposite direction: metadata token –> run-time ID.  Why?  Because a given type may not be loaded yet, and thus the run-time ID may not exist.  There exist methods on the ICorProfilerInfo* interfaces that go this direction, namely GetFunctionFromToken(AndTypeArgs) and GetClassFromToken(AndTypeArgs).  However, they are dangerous to use (see below), and should be avoided.  Instead, it’s preferable that your profiler build up its own run-time ID –> metadata token map as it encounters run-time IDs, and then performing reverse lookups in that map as necessary.  For example, as your profiler encounters ClassIDs via callbacks like ClassLoadFinished, it goes the “safe” direction (run-time ID –> metadata token), to build up its map.  When it later encounters an mdTypeDef for a class, it checks to see if that mdTypeDef exists yet in its map—if so, your profiler uses that map to find the corresponding ClassID.  Safe and easy.

“Dave, stop telling us to do impossible things.  You know full well that profilers which attach to a process after it has started up don’t have the benefit of seeing all the ClassLoad* notifications.  Also, if regular NGEN’d images are used, ClassLoad* notifications are not reliably sent.”

True.  Though you will come across ClassIDs other ways.  Memory profilers will encounter ObjectIDs on the heap, and can call GetClassFromObject to start filling up its map of ClassIDs and thus mdTypeDefs.  Similarly, sampling profilers encounter FunctionIDs during stack walks, and can then get the ClassIDs containing those FunctionIDs and thus build up its map that way.

“You’re a dreamer, man.  There will still be cases where I have a metadata token, but have not yet encountered the ClassID.  Think about deep inspection of embedded structs!”

Yes, that is a good example.  You are an astute reader.  Memory profilers that wish to deeply inspect values of classes and structures on the heap need to know the ClassIDs in order to call GetClassLayout.  This works great when you’re dealing with reference types whose fields point to other reference types: as you bounce from object to object, you can take the ObjectID (i.e., the location in memory where the object starts), pass it to GetClassFromObject, and there’s your ClassID.  But what happens when a struct is embedded inside an object?  Sure, you can get the layout of the object, and determine the offset into the object where the embedded struct lives.  But then what?  How to inspect and report on the values of fields inside the embedded struct?  At this point, all you can get is the mdTypeDef for the struct (from the metadata of the containing class), but you may never have seen the ClassID for that struct.

“Told you so.”

Going from metadata token to run-time ID

As I mentioned above, the safest way to do this is to build up your own map and do reverse-lookups as necessary.  If that scheme meets your needs, then by all means do that, and stop reading!  But in the cases where this is insufficient, you may need to resort to using GetFunctionFromToken(AndTypeArgs) and GetClassFromToken(AndTypeArgs).  There is no simple, foolproof way to use these APIs safely, but here is your guideline:

Never call GetFunctionFromToken(AndTypeArgs) and GetClassFromToken(AndTypeArgs) unless you’re certain the relevant types have been loaded.  (“Relevant types” include the ClassID containing the FunctionID whose mdMethodDef you pass to GetFunctionFromToken(AndTypeArgs), and the ClassID whose mdTypeDef you pass to GetClassFromToken(AndTypeArgs).)  If these types have not been loaded, you may cause them to be loaded now!  This is bad because:

• This is an easy way to crash the app.  Trying to load a type at the wrong time could cause cycles, causing infinite loops (depending on what your profiler does in response to class load notifications) or outright crashes.  For example, trying to load a type while its containing assembly is still in an early phase of loading is a great and fun way to crash the CLR.
• You will impact the behavior of the app.  If you’re lucky enough not to crash the app, you’ve still impacted its behavior, by causing types to get loaded in a different order than they normally would.  Any impact to app behavior like this makes it difficult for your users to reproduce problems that they are trying to use your tool to diagnose, or may hide problems that they don’t discover until they run their application outside of your tool.

Determining whether a class was loaded

So how do you know a class has been fully loaded?

Unfortunately, receiving the ClassLoadFinished callback does not necessarily mean that ClassID has been fully loaded yet, as the MSDN documentation warns us.

Basically, the CLR type loader is one of the laziest things on this planet.  It doesn’t want to do anything unless it really, really has to.  The best guideline I can give you is this:  If the app is currently executing managed code that uses a type, then the type is loaded.  For example, if you do a stackwalk, and determine that the app is executing inside of

MyRetType MyClass::MyFunction(MyArgumentType myArgumentType)

then you can be reasonably assured that the following are loaded:

• MyClass
• MyArgumentType (if it’s a value-type)
• MyRetType (if it’s a value-type)
• For any class you know is loaded, so should be:
  • its base class
  • its value-type fields (not necessarily reference-type fields!)
  • implemented interfaces
  • value-type generic type arguments (and even reference-type generic type arguments in the case of MyClass)

So much for stacks.  What if you encounter an instance of a class on the heap?  Surely the class is loaded then, right?  Well, probably.  If you encounter an object on heap just after GC (inside GarbageCollectionFinished, before you return), it should be safe to inspect the class’s layout, and then peek through ObjectIDs to see the values of their fields.

But what if you encounter an object earlier than that?  For example, if you receive an ObjectAllocated callback, and call GetClassFromObject on the allocated ObjectID, can you be certain the ClassID has been fully loaded?  Well, usually.  But I have seen cases in the past, with types stored in NGENd images, where the CLR may issue an ObjectAllocated callback just before the type has been fully loaded from the NGENd image.  I’ve recently tried to get this to happen again but couldn’t, which probably means this is rather unlikely, but not necessarily impossible.  Ugh.

In general, a lot of the uncertainty above comes from types stored in NGENd modules.  If we actually JIT-compile a function at run-time and load the types it uses from non-NGENd modules, then you can have much greater certainty about the above types being loaded.  You can even make further assumptions about locals and types from signatures of direct callees being loaded.

Interlude: Remember the Unloads!

Now is a good time remind you that, not only is it dangerous to inspect run-time IDs too early (i.e., before they load); it’s also dangerous to inspect run-time IDs too late (i.e., after they unload).  For example, if you store ClassIDs and FunctionIDs for later use, and use them “too late”, you can easily crash the CLR.  The profiling API does pretty much no validation of anything (in many cases, it’s incapable of doing so without using up significant amounts of memory to maintain lookup tables for everything).  So we generally take any run-time ID that you pass to ICorProfilerInfo* methods, cast it to an internal CLR structure ptr, and go boom if the ID is bad.

There is no way to just ask the CLR if a FunctionID or ClassID is valid.  Indeed, classes could get unloaded, and new classes loaded, and your ClassID may now refer to a totally different (valid) class. 

You need to keep track of the unloads yourself.  You are notified when run-time IDs go out of scope (today, this happens at the level of an AppDomain unloading or a collectible assembly unloading—in both cases all IDs “contained” in the unloading thing are now invalid).  Once a run-time ID is out of scope, you are not allowed to pass that run-time ID back to the CLR.  In fact, you should consider whether thread synchronization will be necessary in your profiler to maintain this invariant.  For example, if a run-time ID gets unloaded on thread A, you’re still not allowed to pass that run-time ID back to the CLR on thread B.  So you may need to block on a critical section in thread A during the *UnloadStarted / AppDomainShutdown* callbacks, to prevent them from returning to the CLR until any uses of the contained IDs in thread B are finished.

Take a look at the docs is for more info.

TypeRefs

So far I’ve been talking about how to go from a typeDef to its run-time ID, and by now that should seem hard enough that we don’t need to throw a monkey wrench into the works.  But the sad fact is we’re rarely lucky enough even to have a typeDef.  A class’s fields or even base type, might have their types defined in other modules, in which case the metadata tells us the fields or base type might actually be typeRefs, and not typeDefs.  Ugh.  Whaddya do with that?!

I’ll tell you what you don’t do.  You don’t call the enticingly-named IMetaDataImport::ResolveTypeRef.  On the surface, it seems like ResolveTypeRef would do exactly what you want: starting from a typeRef, please find the referenced module and return an IMetaDataImport on that module, along with the typeDef in that target module to which the typeRef refers.  But the problem lies with how ResolveTypeRef determines the module to which a typeRef refers.

I think ResolveTypeRef was originally designed for use at build-time (by language compilers), though I don’t know if it’s even used in that scenario anymore.  It is certainly not good for use at run-time, where the loader’s decision on how to locate a referenced assembly can be arbitrarily complex.  Different AppDomains in the same process may have different rules on how to locate the referenced assembly due to varying permission sets, host settings, or assembly versions.  In the limit, the CLR may even call into the user’s managed code to dynamically influence the decision of where the referenced assembly exists (see AppDomain.AssemblyResolve Event).

ResolveTypeRef doesn’t know about any of this—it was never designed to be used in a running application with all these environmental factors.  It has an extremely simple (and inaccurate) algorithm to iterate through a set of “known modules”, in an arbitrary order, looking for the first one that matches the reference.  What does “known modules” mean?  It’s a set of modules that have been opened into the metadata system, which is NOT the same as the list of modules already loaded by the assembly loader (and thus notified to your profiler).  And it’s certainly not the same as the set of modules installed onto the disk.

If you absolutely need to resolve refs to defs, your best bet may be to use your own algorithm which will be as accurate as you can make it, under the circumstances, and which will never try to locate a module that hasn’t been loaded yet.  That means that you shouldn’t try to resolve a ref to a def if that def hasn’t actually been loaded into a type by the CLR.  Consider using an algorithm similar to the following:

1. Get the AssemblyRef from the TypeRef to get to the name, public key token and version of the assembly where the type should reside.
2. Enumerate all loaded modules that the Profiling API has notified you of (or via EnumModules) (you can filter out a specific AppDomain at this point if you want).
3. In each enumerated module, search for a TypeDef with the same name and namespace as the TypeRef (IMetaDataImport::FindTypeDefByName)
4. Pay attention to type forwarding!  Once you find the TypeDef, it may actually be an “exported” type, in which case you will need to follow the trail to the next module.  Read toward the bottom of https://blogs.msdn.com/b/davbr/archive/2009/09/30/type-forwarding.aspx for more info.

The above can be a little bit smarter by paying attention to what order you choose to search through the modules:

• First search for the TypeDef in assemblies which exactly match the name, public key token and version for the AssemblyRef.
• If that fails, then search through assemblies matching name and public key token (where the version is higher than the one supplied – this can happen for Framework assemblies).
• If that fails, then search through all the other assemblies

I must warn you that the above scheme is not tested and not supported.  Use at your own risk!

Future

Although I cannot comment on what will or will not be in any particular future version of the CLR, I can tell you that it is clear to us on the CLR team that we have work to do, to make dealing with metadata tokens and their corresponding run-time type information easier from the profiling API.  After all, it doesn’t take a rocket scientist to read the above and conclude that it does take a rocket scientist to actually follow all this advice.  So for now, enjoy the fact that what you do is really hard, making you difficult to replace, and thus your job all the more secure.  You’re welcome.

Special thanks to David Wrighton and Karel Zikmund, who have helped considerably with all content in this entry around the type system and metadata.

If none of the above makes any sense to you, you’ve probably stumbled onto the wrong blog.  Indeed, both “rejit” and “ejit” have other definitions (some rather unfriendly) that are completely unrelated to the CLR.  I’m talking about that ReJIT thing you do when you want to instrument and then JIT-compile code that has already been JIT-compiled before.

This post is organized in chronological order, telling what your profiler should be doing at the following times in the process:

• Startup Time
• ModuleLoadFinished Time
• RequestReJIT Time
• Actual ReJIT Time
• RequestRevert Time

Startup Time

The first thing your profiler will do is get itself loaded on startup of a managed application—the old environment variable way, not the new attach way.  I’m sure you’ve already read up on the limitations!

Inside your profiler’s Initialize() method, it will of course call SetEventMask().  In that call, your profiler must include (COR_PRF_ENABLE_REJIT | COR_PRF_DISABLE_ALL_NGEN_IMAGES) in the bitmask.  COR_PRF_ENABLE_REJIT is required to use any of the ReJIT APIs later on (they’ll fail immediately otherwise).  COR_PRF_DISABLE_ALL_NGEN_IMAGES causes the CLR’s assembly loader to ignore all NGENd images (even NGEN /Profile images), and thus all code will be JITted from scratch, and all classes loaded from scratch.  If you try to be tricky and specify only COR_PRF_ENABLE_REJIT (without COR_PRF_DISABLE_ALL_NGEN_IMAGES), then SetEventMask will fail.  Conversely, though, you’re perfectly welcome to specify COR_PRF_DISABLE_ALL_NGEN_IMAGES without COR_PRF_ENABLE_REJIT if you want.

At this time you will likely want to set other flags that control optimizations, particularly inlining (COR_PRF_DISABLE_OPTIMIZATIONS, COR_PRF_DISABLE_INLINING), or at least subscribe to the inlining callbacks (COR_PRF_MONITOR_JIT_COMPILATION).

Typically, your profiler will also create a new thread at this point, call it your “ReJIT Thread”.  The expected use-case of ReJIT is to perform instrumentation “on demand”, triggered by some user action (like fiddling with dials in your profiler’s out-of-process GUI).  As such, you’ll need an unmanaged thread of your own creation to receive and act on these requests from out-of-process.  Perhaps you already have such a thread to service other kinds of requests.  It’s perfectly acceptable for such a thread to now also act as your ReJIT Thread.

ModuleLoadFinished Time

Metadata Changes

As each module loads, you will likely need to add metadata so that your future ReJITs will have the tokens they need.  What you do here heavily depends on the kind of instrumentation you want to do.  I’m assuming you’re doing instrumentation that adds some calls from the user code into brand new profiler helper methods you will add somewhere.  If you plan to instrument mscorlib, you will likely want to add those profiler helper methods into mscorlib (remember, mscorlib is not allowed to contain an AssemblyRef that points to any other assembly!).  Otherwise, perhaps you plan to ship a managed helper assembly that will sit on your user’s disk, and all your profiler helper methods will reside in this on-disk managed helper assembly.

So…

IF the module loading is mscorlib AND you plan to add your profiler helper methods into mscorlib, THEN use the metadata APIs now to add those methods.

IF the module loading contains methods that you might possibly ever want to instrument, THEN use the metadata APIs to add any AssemblyRefs, TypeRefs, MemberRefs, etc., which point to your profiler helper methods, that you might possibly need later when you potentially instrument methods from this loading module.  The guiding principle here is that metadata changes may be done at ModuleLoadFinished time, and not later.  So you need to assume you might possibly want to ReJIT methods in the loading module eventually, and proactively add to the loading module whatever metadata you will eventually need (should you actually perform the ReJIT later), and add that metadata now, just in case.

Re-Request Prior ReJITs

This won’t make much sense until you’ve read the next section, but I’m placing it here to keep it in chronological order.  If you’ve made a prior call to RequestReJIT for an unshared (non-domain-neutral) ModuleID, AND if you want that request to apply to the mdMethodDef that appears in all other unshared copies of the module, AND if you’re inside ModuleLoadFinished for the load of a new ModuleID that is just such a new unshared copy of the module, THEN you’ll want to explicitly call RequestReJIT on this newly-loaded ModuleID with that mdMethodDef.  Note that this is optional—if you want to treat AppDomains differently and want, say, only one unshared copy of the function to be ReJITted, then you’re perfectly welcome to cause that behavior and not to call RequestReJIT on any new ModuleIDs relating to the module.  Come back and re-read those last two sentences after you’ve read the next section.

RequestReJIT Time

Now imagine your user has turned some dial on your out-of-process GUI, to request that some functions get instrumented (or re-instrumented (or re-re-instrumented (or …))).  This results in a signal sent to your in-process profiler component.  Your ReJIT Thread now knows it must call RequestReJIT.  You can call this API once in bulk for a list of functions to ReJIT.  Note that functions are expressed in terms of ModuleID + mdMethodDef metadata tokens.  A few things to note about this:

• You request that all instantiations of a generic function (or function on a generic class) get ReJITted with a single ModuleID + mdMethodDef pair.  You cannot request a specific instantiation be ReJITted, or provide instantiation-specific IL.  This is nothing new, as classic first-JIT-instrumentation should never be customized per instantiation either.  But the ReJIT API is designed with this restriction in mind, as you’ll see later on.
• ModuleID is specific to one AppDomain for unshared modules, or the SharedDomain for shared modules.  Thus:
  • If ModuleID is shared, then your request will simultaneously apply to all domains using the shared copy of this module (and thus function)
  • If ModuleID is unshared, then your request will apply only to the single AppDomain using this module (and function)
  • Therefore, if you want this ReJIT request to apply to all unshared copies of this function:
    • You’ll need to include all such ModuleIDs in this request.
    • And… any future unshared loads of this module will result in new ModuleIDs.  So as those loads happen, you’ll need to make further calls to RequestReJIT with the new ModuleIDs to ensure those copies get ReJITted as well.
    • This is optional, and only need be done if you truly want this ReJIT request to apply to all unshared copies of the function.  You’re perfectly welcome to ReJIT only those unshared copies you want (and / or the shared copy).
    • Now you can re-read the “Re-Request Prior ReJITs” section above.  :-)

More on AppDomains

This whole shared / multiple unshared business can get confusing.  So to bring it home, consider your user.  If your user expresses instrumentation intent at the level of a class/method name, then you pretty much want to ReJIT every copy of that function (all unshared copies plus the shared copy).  But if your user expresses instrumentation intent at the level of a class/method name plus AppDomain (think one single AppPool inside ASP.NET), then you’d only want to ReJIT the copy of the function that resides in the single ModuleID associated with that AppDomain.

The SharedDomain can make that last alternative tricky, though.  Because if the ModuleID ends up belonging to the SharedDomain, and you ReJIT a method in that ModuleID, then all AppDomains that share that module will see your instrumentation (whether you want them to or not).  This is due to the very nature of SharedDomain / domain-neutrality.  There’s only one shared copy of this function to instrument, so if two domains share the function, they both see it, either with or without instrumentation.  It doesn’t make sense to instrument the function from the point of view of only one of those two domains.

Pre-ReJIT

Obviously, the main coolness of RequestReJIT is that you can call it with a function that has already been JITted.  But one of the niceties of RequestReJIT is that you don’t actually have to wait until a function is first JITted to use it.  You can request a ReJIT on a function that has never been JITted before (I call this “Pre-ReJIT”).  Indeed, with generics, there’s no way to know if all the instantiations that will ever be used in an AppDomain have been JITted or not.  There may always be some important instantiation that has not been JITted yet.  RequestReJIT takes all this into account as follows:

If a function (or generic instantiation) has already been JITted, it is marked for ReJIT next time it is called.

If a function (or generic instantiation) has not yet been JITted, then it is marked internally for “Pre-ReJIT”.  This means that once it is called, its original (non-instrumented) IL gets JIT-compiled as usual.  Immediately after, it is then ReJITted.  In this way, a Pre-ReJIT request works exactly like a ReJIT request.  Original IL is compiled first, and then instrumented IL is compiled later.  This ensures we can easily “revert” back to the original code at a later time using the same revert mechanism.  (See below.)

Actual ReJIT Time

You may have noticed that you have read a whole lot of words so far, but we haven’t yet provided the instrumented IL to the CLR.  This is because the function hasn’t ReJITted yet.  You’ve only requested that it be ReJITted.  But the actual ReJITting happens the next time the function is called.   Until then, any threads already executing inside functions you requested to be ReJITted stay in those functions, and don’t see the instrumented code until they return and call the functions again.  Once a function is finally called for the first time after its RequestReJIT, you get some callbacks.

IF this is the first generic instantiation to ReJIT, for a given RequestReJIT call (or this is not a generic at all), THEN:

• CLR calls GetReJITParameters
  • This callback passes an ICorProfilerFunctionControl to your profiler.  Inside your implementation of GetReJITParameters (and no later!) you may call into ICorProfilerFunctionControl to provide the instrumented IL and codegen flags that the CLR should use during the ReJIT
  • Therefore it is here where you may:
    • Call GetILFunctionBody
    • Add any new LocalVarSigTokens to the function’s module’s metadata.  (You may not do any other metadata modifications here, though!)
    • Rewrite the IL to your specifications, passing it to ICorProfilerFunctionControl::SetILFunctionBody.
  • You may NOT call ICorProfilerInfo::SetILFunctionBody for a ReJIT!  This API still exists if you want to do classic first-JIT IL rewriting only.
  • Note that GetReJITParameters expresses the function getting compiled in terms of the ModuleID + mdMethodDef pair you previously specified to RequestReJIT, and not in terms of a FunctionID.  As mentioned before, you may not provide instantiation-specific IL!

And then, for all ReJITs (regardless of whether they are for the first generic instantiation or not):

• CLR calls ReJITCompilationStarted
• CLR calls ReJITCompilationFinished

These callbacks express the function getting compiled in terms of FunctionID + ReJITID.  (ReJITID is simply a disambiguating value so that each ReJITted version of a function instantiation can be uniquely identified via FunctionID + ReJITID.)  Your profiler doesn’t need to do anything in the above callbacks if it doesn’t want to.  They just notify you that the ReJIT is occurring, and get called for each generic instantiation (or non-generic) that gets ReJITted.

And of course, for any calls to these functions after they have been ReJITted, there are no further ReJIT compilations or callbacks to your profiler.  This ReJITted version is now the current and only version for all new calls to the function.

Versions

Your profiler is welcome to call RequestReJIT again on these functions, and the cycle starts again.  The next time a call comes in, they’ll get ReJITted again, and you’ll provide instrumented IL at that time, as usual.  At any given time, only the most recently ReJITted version of a function is active and in use for new calls.  But any prior calls still inside previously ReJITted (or original) versions of the function stay in that version until they return.

RequestRevert Time

Eventually your user may turn the dial back down, and request that the original, un-instrumented, version of the function be reinstated.  When this happens, your profiler receives this signal from out-of-proc using your nifty cross-proc communication channel, and your ReJIT Thread calls RequestRevert.

At this time, the CLR sets the original version of the function that it JITted the first time as being the current version for all future calls.  Any prior calls still executing in various ReJITted versions of the function remain where they’re at until they return.  All new calls go into the version originally JITted (from the original IL).

Note that RequestRevert allows you to revert back to the original JITted IL, and not back to some previous ReJITted version of the IL.  If you want to revert back to a previous ReJITted version of the IL, you’ll need to do so manually, by using RequestReJIT instead, and providing that IL explicitly to the CLR.

Errors

If there are any errors with performing the ReJIT, you will be notified by the dedicated callback ICorProfilerCallback4::ReJITError().  Errors can happen at a couple times:

• RequestReJIT Time: These are fundamental errors with the request itself.  This can include bad parameter values, requesting to ReJIT dynamic (Ref.Emit) code, out of memory, etc.  If errors occur here, you’ll get a callback to your implementation of ReJITError(), sandwiched inside your call to RequestReJIT on your ReJIT Thread.
• Actual ReJIT Time: These are errors we don’t encounter until actually trying to ReJIT the function itself.  When these later errors occur, your implementation of ReJITError() is called on whatever CLR thread encountered the error.

You’ll note that ReJITError can provide you not only the ModuleID + mdMethodDef pair that caused the error, but optionally a FunctionID as well.  Depending on the nature of the error occurred, the FunctionID may be available, so that your profiler may know the exact generic instantiation involved with the error.  If FunctionID is null, then the error was fundamental to the generic function itself (and thus occurred for all instantiations).

Ok, that about covers it on how your profiler is expected to use ReJIT.  As you can see, there are several different tasks your profiler needs to do at different times to get everything right.  But I trust you, you’re smart.

Is ReJIT For You?

If you’re writing a monitoring tool, typically run in production, and…

If your tool is always on, always monitoring, but needs a way to fine-tune the amount of instrumentation it does without forcing the monitored application to restart, and…

If your tool instruments potentially everything, including framework assemblies like mscorlib, and you therefore disable the use of NGENd images and are willing to put up with longer startup times as a result, then…

ReJIT may be for you.

The ReJIT we plan to release in .NET 4.5 was designed with this scenario in mind.  As such, there are many potential sub-features of ReJIT that will not be available, because they are not essential for this scenario.

List those Limitations!

ReJIT + Attach? No!

In order to enable ReJIT, your profiler must load at startup, and set an immutable flag in your Initialize method that enables the ReJIT functionality.

ReJIT + NGEN? No!

In order to set that ReJIT flag I just mentioned, you must also set a new flag which completely disables the use of NGENd images.  Kind of similar to the existing COR_PRF_USE_PROFILE_IMAGES flag, except even NGEN /Profile images (should they exist) will be ignored, and everything will be JITted, when you set this new flag.  This includes all framework assemblies like mscorlib.

Metadata changes in ModuleLoadFinished only

If you add any new methods, AssemblyRefs, MemberRefs, etc., your metadata changes must be done during the module’s ModuleLoadFinished callback.  This is not a new limitation, but it could be a surprise to some that it’s still a limitation even with ReJIT.  There is an exception to this.  If you need to create a new LocalVarSigToken, you may do this “late”, at ReJIT time, rather than early at ModuleLoadFinished time.

Memory reclaimed at AppDomain unload, not revert

ReJIT will include the ability to “revert” back to the original IL from the assembly.  Doing so, however, will not reclaim any memory that was allocated to support the ReJIT (e.g., the instrumented IL, the JITted code, internal bookkeeping, etc.)  This memory will be reclaimed when the containing AppDomain is unloaded.  And if the code is owned by the shared domain, then, well, that memory is never reclaimed.

ReJIT inlined functions?  No!

If function A inlines function B, then you cannot ReJIT function B.  Well, technically you can, you just won’t see the effect of that anytime A (or another inlining caller of B) is called.  The reason is that, even if you create your new, instrumented B’, A still inlined the original B.  So every time A is called, the code from the original B will be executed, and your B’ will be ignored.

Since your profiler must be loaded at startup, you can work around this by either turning off inlining altogether, or by monitoring it (via the JITInlining callback), so you know which callers to ReJIT.  In the example above, you’d have to rejit A, and could then request that the rejitted A not inline anyone, so that your new B’ would get called.  Note that you’d have to track the inliners recursively, as there can be arbitrarily many levels of inlining.

ReJIT + managed debugging? No!

While not technically disabled, it is not advised or supported to run a managed debugger against a process that also has a ReJITting profiler enabled.  (Native-only debugging is just fine, though.)

Whatever debugging support there is, is only there for you, the profiler writer, and not for your profiler’s users.  For example, there is no way for the ReJITting profiler to adjust the instrumented IL map for rejitted code (i.e., no equivalent of SetILInstrumentedCodeMap for ReJIT).  And attempting to step into or set breakpoints in rejitted code will have unpredictable results.

However, as a profiler writer attempting to debug your own profiler, you should have a good experience debugging other parts of the process.  For example, if rejitted user code calls into an on-disk profiler IL assembly, you could set breakpoints in your profiler’s IL assembly, and step through that code.

ReJIT dynamic code?  No!

Not a new limitation, but just to be explicit, profilers are not allowed to instrument dynamic code generated via the Reflection.Emit namespace, and that includes ReJIT.

Why so strict?

ReJIT, as originally conceived by the CLR team, involved allowing profilers to attach to running processes and then instrument arbitrary code at any time.  Just that one sentence would eliminate almost all the restrictions mentioned above.  So what happened?

Reality, that’s what.

Stuff takes time.  And in this case a lot of time.  Lifting just about any of the above restrictions may well have increased development or testing time, or general risk, to the point where the entire ReJIT feature might have been jeopardized.  So although it was a painful process, we had to think hard about every sub-feature we wanted to support that had non-trivial cost to implement.  And at the same time, we had to think about actual, real-world, end-to-end scenarios that would be using ReJIT to ensure that we ended up with something that would be useful, if not perfect.

So we picked the real-world use-case of production monitoring tools that use instrumentation to gather data from various servers in a data center.  “Attach” isn’t interesting to many of these tools (which run all the time), but they do want to dynamically change the level of instrumentation to help diagnose problems as they come up, without having to restart the process.  This scenario fit very nicely with the time we had, and so that’s what we shot for.

I can’t comment on what we will or will not do in any releases of the CLR after 4.5, but I like to think that ReJIT in .NET 4.5 might simply be a first step toward a richer instrumentation feature set, such that some of these limitations may eventually get lifted.  We won’t know if that’s true until the time comes, though.

https://channel9.msdn.com/Shows/Going+Deep/CLR-45-David-Broman-Inside-Re-JIT

In the upcoming .NET 4.5, we do indeed plan to include a limited form of ReJIT that will allow diagnostic tools to perform IL Rewriting after the code has been JITted—by instructing the CLR, via the Profiling API, to JIT the (now instrumented) method again (i.e., ReJIT it!).

If you want to play with this feature, it is available in the .NET 4.5 Developer Preview.  You can search for it on the Microsoft Download Center, and you’ll find the .NET 4.5-only link and the full Visual Studio 11 link:

https://www.microsoft.com/download/en/details.aspx?id=27537
https://www.microsoft.com/download/en/details.aspx?displaylang=en&id=27543

It will be tough to try it out now, though, as we don’t have docs or samples yet.  And note that there are several limitations, some of which are touched on in the video.  You’ll need to look through corprof.idl and turn on your mind-reading skills to figure it out.  But I will have a lot more to say about the feature and how you can use it in upcoming posts right here.