The early registration deadline for CGO is tomorrow, February 3, so if you are interested in attending, now’s the time to act.

You can read all about the .NET Framework 4.6 release (which includes RyuJIT) here: https://blogs.msdn.com/b/dotnet/archive/2015/07/20/announcing-net-framework-4-6.aspx.

You can see Soma's blog post on the Visual Studio 2015 and .NET Framework 4.6 releases here: https://blogs.msdn.com/b/somasegar/archive/2015/07/20/visual-studio-2015-and-net-4-6-available-for-download.aspx.

Enjoy these releases, and keep the feedback coming!

First and foremost, RyuJIT is the default x64 JIT for .NET 4.6! As a result, there is no need to use a RyuJIT CTP to try out RyuJIT. If you install the .NET 4.6 Preview (included in Visual Studio 2015 Preview), or later build, such as Visual Studio 2015 RC, you will be using it. See here or the .NET blog for more details.

One thing we need to point out, though: if you have previously configured your machine to use a RyuJIT CTP (using the instruction here), and you update to .NET 4.6, you need to make sure the COMPLUS_AltJit environment variable and the HKLM\SOFTWARE\Microsoft\.NETFramework\AltJit registry value do not exist. If they do exist, every .NET application will fail to run. You don’t want that!

Update: the specific failure you will get is exception number -2146233082 (in hex: 0x80131506), otherwise known as COR_E_EXECUTIONENGINE or "Internal CLR error". This will be the process exit code. For console applications, you can see this by examining %ERRORLEVEL% after the process terminates (if you are working in a CMD.EXE window). You may also see a dialog box stating "<application> has stopped working", and if you expand "View problem details", you will see "Fault Module Name: clr.dll" and "Exception Code: 80131506". You will also see an entry in the Event Viewer, Application log, source ".NET Runtime", listing this error code. Note that there are other reasons this exception number could be generated; a stray AltJit configuration is not the only reason for this exception.

Note that this information has also been published as KB3065367 here.

Secondly, RyuJIT is now open source! See here for details. 

We hope you enjoy the benefits of all the work we’ve put into RyuJIT. If you have any feedback on RyuJIT, send us mail at ryujit@microsoft.com. We want to hear from you!

Correctness

We have spent a lot of time finding and fixing those last few pesky corner-case functional issues in RyuJIT.  Fortunately, we have the luxury of having many internal partners with a significant managed codebase, making it easy to throw as much managed code as we can find at RyuJIT.  While some of the issues we have found are legitimate bugs, others are not so clear cut.  For example, we have found that JIT64 accommodates some illegal IL disallowed by the ECMA spec.  Since backward compatibility is a major concern for us, we evaluate these issues on a case-by-case basis to decide if we should quirk RyuJIT to accommodate the same illegal IL.

Real-World Throughput Wins

In case you have missed the original blog post announcing the first CTP, RyuJIT beats JIT64 handily in terms of throughput while staying very competitive in terms of code quality (CQ).  Recently the Bing team has tried using RyuJIT on top of 4.5.1 in some of their processing, and they see a 25% reduction in startup time in their scenario.  This is the most significant real-world throughput win we have witnessed on RyuJIT thus far.  :)

Code Quality

We didn’t publish any benchmark results with RyuJIT CTP4, so here are some graphs to show that we haven’t regressed CQ in RyuJIT CTP5.  However, since CQ hasn’t been the focus for this CTP, we also haven’t made any significant improvements either.

These graphs follow the same basic format as previous ones.  The higher the bar, the better RyuJIT CTP5 is at that benchmark.  The grey area is the standard deviation, so any benchmark falling in the grey area is just noise. 

What’s New in JIT Support for SIMD types?

RyuJIT CTP5 supports acceleration of the latest version of the Vector APIs available via NuGet here.  This version contains a number of changes that were requested by developers.

One of the most popular requests was to publicly expose the fields of the fixed-size vector types (e.g. Vector2.X).  Why wasn’t this done originally?  The short answer is that it was for performance, but really it was to make it easier for the JIT to handle all the references to these types as intrinsics, and to transform them into the appropriate target instructions.  It’s a tricky business, however, to determine where to allocate a local Vector instance for best efficiency:

• If the instance will be primarily used in Vector intrinsics, putting it in an xmm/ymm register is the best option.
• If the instance will primarily be referenced via its fields, then either putting it in memory, or separately allocating its fields to registers, is the best option.
• If the instance is larger than 8 bytes (i.e. not a Vector2), and it is primarily passed as a method argument, then putting it in memory is the best option.

With CTP5 we have made the JIT a bit smarter about identifying these field accesses, analyzing the usage of the vector instance, and selecting among these options, but there is still room for improvement, so you may find that some SIMD code runs more slowly with this new release.

We’ve also improved register allocation for SIMD types, reducing a number of cases where we had unnecessary copies of vector registers.

Since we are talking about SIMD performance, it wouldn’t be fair to not include any SIMD benchmark results.  We are using the sample code here as our SIMD benchmarks.  (However, note that we are using an updated version of RayTracer which uses our latest Vector APIs.  We’ll update the sample shortly.)

Stay tuned – we are continuing to work on performance for SIMD types, including tuning of inlining heuristics for SIMD methods, and improved dead store elimination.  We’ll also be diving into the usage data from Bing and other internal partners to see how we can improve the performance of RyuJIT even more on both throughput and CQ.

In case you need them again, you can refer to this blog post for the instructions to turn on RyuJIT, and this blog post for instructions on using SIMD.  Note that if you are running on the 4.5.2 version of the .NET Framework, you can use RyuJIT CTP5 on Windows Vista, 7, 8, and 8.1 as well as Windows Server 2008, 2008 R2, 2012, and 2012 R2.  However, RyuJIT CTP5 currently doesn't work on Visual Studio "14" CTP4.  You don't need it anyway, since RyuJIT is enabled by default on Visual Studio "14" CTP4.  :)  (The version of RyuJIT in Visual Studio "14" CTP4 is slightly older than this CTP, but not by much.)

1. Probably the biggest news is that if you install the 4.5.2 runtime (check the .NET blog for details on that), you can use RyuJIT CTP4 on Windows Vista, 7, and 8, as well as Windows Server 2008, 2008 R2, and 2012. In the CTP1 FAQ, I made mention that 4.5.1 on “downlevel” OS’es looked different from a code generation perspective. Well, that’s been addressed in the 4.5.2 update, so we’re happy to support RyuJIT CTP4 across all platforms that support 4.5.2.
2. Nearly all the available Vector<T> types are now accelerated! The only ones missing are Vector<uint> and Vector<ulong>. In addition, there are a handful of other methods that we are now accelerating, including the CopyTo() method, which means any performance you have measured is now completely invalidated! Wait, no I mean, any performance you measured could potentially be faster!
3. The fixed-size vector types are all mutable, now. This was the single biggest piece of feedback we received, so we took it.

There you have it. For now, you can download the CTP4 bits from here. The BCL SIMD NuGet package has also been updated, so update that, and you should be good to go. Same directions as before for how to use the types, enable RyuJIT, all that stuff. As always, send feedback to ryujit@microsoft.com. Happy RyuJIT-ing!

-Kev

First, you should go read Sasha's excellent write-up on SIMD: https://blogs.microsoft.co.il/sasha/2014/04/22/c-vectorization-microsoft-bcl-simd/. He's done an excellent job of talking about how scalar & vector operations compare, and really dug in deep to performance of the generated code.

Okay, now let me explain a few things. Sasha's looking at the code quality coming out of the preview: we know this code quality is less than optimal. Here's what I'm seeing from the same code with a compiler I built about 30 minutes ago (Siva, one of the JIT devs, just got CopyTo recognized as an intrinsic function):

; parameters: RCX = a, RDX = b (as before)
    sub    rsp,28h
    xor    eax,eax                          ;rax = i (init to zero)
    mov    r8d,dword ptr [rcx+8]            ;r8d = a.Length
    test   r8d,r8d
    jle    bail_out                         ;r8d <= 0?
    mov    r9d,dword ptr [rdx+8]            ;r9d = b.Length
loop:
    lea    r10d,[rax+3]                     ;r10d = i + 3
    cmp    r10d,r8d                         ;we still have to range check the read from the array of &a[i+3]
    jae    range_check_fail
    mov    r11,rcx
    movupd xmm0,xmmword ptr [r11+rax*4+10h] ;xmm0 = {a[i], a[i+1], a[i+2], a[i+3]}
    cmp    r10d,r9d                         ;and we have to range check b[i+3], too...
    jae    range_check_fail                 ;this one too :-(
    movupd xmm1,xmmword ptr [rdx+rax*4+10h] ;xmm1 = {b[i], b[i+1], b[i+2], b[i+3]}
    addps  xmm0,xmm1                        ;xmm0 += xmm1
    movupd xmmword ptr [r11+rax*4+10h],xmm0 ;{a[i], a[i+1], a[i+2], a[i+3]} = xmm0
    add    eax,4                            ;i += 1 * sizeof(float)
    cmp    r8d,eax                          ;a.Length > i?
    jg     loop
bail_out:
    add    rsp,28h
    ret
range_check_fail:
    call   clr!JIT_RngChkFail (00007ff9`a6d46590)
    int    3

So the most egregious code quality issue is gone (we're going to try and get an update out quickly: I'm saying mid-May, but things could change :-/). That gives me numbers like this:

Scalar adding numbers: 765msec

SIMD adding numbers: 210msec

MUCH better. But Sasha makes another interesting point: why did we create two versions of the scalar loop, but left those pesky bounds checks in the SIMD loop? Well, for starters, the two functions aren't identical. The SIMD version will actually throw a bounds check failure exception. To fix that, we actually need to change (some might say "fix" :-)) the code:

static void AddPointwiseSimd(float[] a, float[] b)
{
 int simdLength = Vector<float>.Length;
 int i = 0;
 for (i = 0; i < a.Length - simdLength; i += simdLength)
 {
 Vector<float> va = new Vector<float>(a, i);
 Vector<float> vb = new Vector<float>(b, i);
 va += vb;
 va.CopyTo(a, i);
 }
 for (; i < a.Length; ++i)
 {
 a[i] += b[i];
 }
}

And now you might be expecting some great revelation. Nope. Sasha's right: we're missing some fairly straight-forward bound checks here. And we're not dual-versioning the loop, like RyuJIT does for normal scalar loops. Both items are on our to-do list. But the original code Sasha wrote would probably never have those optimizations. Hopefully, we'll get our bounds-check elimination work strengthened enough to make this code truly impressive. Until then, you'll have to limp along with only a 3.6x performance boost.

And quickly, no, I'm not normally a work-aholic: I didn't check e-mail once since leaving work last Friday, until I needed the break from my vacation Wednesday after using a 1 man auger to dig 16 holes in my back yard. So having the engineers that actually make the stuff (and fix the bugs) call me and ask me to sign off & whatnot gave my aging back a nice break from hefting around 55 lb. pier blocks and 80 lb. sacks of gravel.

-Kev

1. Go get RyuJIT CTP3 and install it (requires x64 Windows 8.1 or Window Server 2012R2, same as before)
2. Set the "use RyuJIT" environment variable: set COMPLUS_AltJit=*
• Update: make sure you un-set this when you are done testing RyuJIT! See here for more details.
3. Now pay attention, because things diverge here a bit:
4. Set a new (and temporary) “enable SIMD stuff” environment variable: set COMPLUS_FeatureSIMD=1
5. Add the Microsoft.Bcl.Simd NuGet package to your project (you must select “include Prerelease” or use the –Pre option)
6. Tricky thing necessary until RyuJIT is final: Add a reference to Microsoft.Numerics.Vectors.Vector<T> to a class constructor that will be invoked BEFORE your methods that use the new Vector types. I’d suggest just putting it in your program’s entry class’s constructor. It must occur in the class constructor, not the instance constructor.
7. Make sure your application is actually running on x64. If you don't see protojit.dll loaded in your process (tasklist /M protojit.dll) then you've missed something here.

You can take a gander at some sample code sitting over here. It's pretty well commented, at least the VectorFloat.cs file is in the Mandelbrot demo. I spent almost as much time making sure comments were good as writing those 24 slightly different implementations of the Mandelbrot calculation. Take a look at the Microsoft.Numerics.Vectors namespace: there some stuff in Vector<T>, some stuff in VectorMath, and some stuff in plain Vector. And there are also "concrete types" for 2, 3, and 4 element floats.

One quick detail, for those of you that are trying this stuff out immediately: our plan (and we've already prototyped this) is to have Vector<T> automatically use AVX SIMD types on hardware where the performance of AVX code should be better than SSE2. Doing that properly, however, requires some changes to the .NET runtime. We're only able to support SSE2 for the CTP release because of that restriction (and time wasn't really on our side, either).

I’ll try to answer questions down below. I know this doesn’t have much detail. We’ve been really busy trying to get CTP3 out the door (it was published about 45 seconds before Jay’s slide went up). Many more details are coming. One final piece of info: SIMD isn’t the only thing we’ve been doing over the past few weeks. We also improved our benchmarks a little bit:

The gray bars are standard deviation, so my (really horrible) understanding of statistics means that delta's that are within the gray bars are pretty much meaningless. I'm not sure if we really did regress mono-binarytrees or not, but I'm guessing probably not. I do know that we improved a couple things that should help mono-fasta and mono-knucleotide, but the mono-chameneos-redux delta was unexpected.

Anyway, I hope this helps a little. I'm headed home for the evening, as my adrenaline buzz from trying to get everything done in time for Jay's talk is fading fast, and I want to watch Habib from the comfort of my couch, rather than be in traffic. Okay, I'm exaggerating: my commute is generally about 10 minutes, but I'm tired, and whiny, and there are wolves after me.

-Kev

And, just for fun, let’s include one more (positive numbers indicate CTP2 does better than CTP1):

First things first: there is no rhyme or reason to the order of the benchmarks we're running. They're in a list in a file, so that's the order they run. The methodology used to measure performance is pretty straight forward: each benchmark is run once, as a "warm-up" run (get input files in the cache, etc...), then run 25 times. The average of those subsequent 25 runs is what's reported. The standard deviation is also calculated, in an effort to make it easier to detect noise from actual performance differences. The benchmarks are run on a Core i7 4850HQ, 4G RAM, an SSD, on-board video, running Windows 8.1. Nothing too fancy, just a relatively up-to-date piece of hardware that does a reasonable job of spanning laptop/mobile performance and workstation/server performance. Every benchmark is an IL-only, non-32-bit-preferred binary, so they'll all run with either a 32 or 64 bit runtime.

Now that you have a crystal clear understanding of how we're running the benchmarks, let's talk about them in left-to-right order. I'll warn you before I start: some benchmarks are better than others, and I have spent more time looking at some benchmarks more than others. That will become incredibly obvious as the discussion continues.

Roslyn:
this one is hopefully pretty self-explanatory. We perform a “self-build” of the Roslyn C# compiler: the Roslyn C# compiler reads in the C# source code of Roslyn, and generates the Roslyn compiler binaries. It’s using bits later than what are publicly available, and the source code isn’t publicly available, so this one is pretty hard for other folks to reproduce :-(. The timer is an 'external' timer: the self-build is launched by a process that has a timer, so the time reported includes process launch, JIT time, and code execution time. This one is probably the single largest benchmark we have. The improvement in JIT compile time (Throughput) accounts for more than half of the improved performance. Outside of improved throughput, there are places where we generate better code for Roslyn (code quality: CQ for short) and places where we’re worse. We’re continuing to look into performance of this code. Since the Roslyn team works closely with the .NET Runtime & Framework team, we have lots of experts in that code base nearby. One final note on Roslyn: it may appear at first glance that we did nothing here between CTP1 and CTP2, but the CTP1 build didn’t work properly. So we fixed a few bugs there to get where we are now.

System.XML:
this one should probably not really be included for RyuJIT CTP runs. It's running the XML parser against an input XML file. The reason the data isn't particularly interesting is because the XML parser is in System.XML.dll, which is NGen'ed, which means that it's actually just running code that JIT64 produced, where RyuJIT is only compiling the function that's calling the parser. Internally, we can use RyuJIT for NGen as well, so that's why it's there, but it's not showing anything observable for CTP releases of RyuJIT.

SharpChess:
This is an open source C# chess playing application written by Peter Hughes. You can download the latest version from https://www.sharpchess.com. We’re using version 2.5.2. It includes a very convenient mode for benchmarking which reads in a position, then prints out how many milliseconds it takes to calculate the next move. RyuJIT does a respectable job, keeping pace with JIT64 just fine, here.

V8-*:
these benchmarks are transliterations of JavaScript benchmarks that are part of the V8 JavaScript performance suite. A former JIT developer did the transliteration many years ago, and that’s about the extent of my understanding of these benchmarks, exception that I assume that v8-crypto does something with cryptography. Of the 3, Richards and Crypto have fairly innocuous licenses, so I’ve put them on my CodePlex site for all to enjoy. DeltaBlue due to lincensing restrictions can’t be hosted on CodePlex.

Bio-Mums:
This is a benchmark picked up from Microsoft Research’s “Biological Foundation” system several years ago. Since we grabbed it, they’ve open-sourced the work. Beyond that, I know it has something to do with biology, but just saying that word makes my brain shudder, so you’ll have to poke around on the web for more details.

Fractals:
Matt Grice, one of the RyuJIT Developers, wrote this about a year ago. It calculates the Mandelbrot set and a specific Julia set using a complex number struct type. With Matt’s permission, I’ve put the source code on my CodePlex site, so you can download it and
marvel at its beauty, its genius. This is a reasonable micro-benchmark for floating point basics, as well as abstraction cost. RyuJIT is pretty competitive with JIT64 on floating point basics, but RyuJIT demolishes JIT64 when it comes to abstraction costs, primarily due to Struct Promotion. The graph isn’t accurate for this benchmark, because RyuJIT is just over 100% (2X) faster than JIT64 on this one. But if that value were actually visible, nothing else would look signficant, so I just cropped the Y axis.

BZip2:
This is measuring the time spent zip’ing the IL disassembly of mscorlib.dll. RyuJIT is a few percent slower than JIT64. This benchmark predates the .NET built-in support for ZIP, so it uses the SharpZip library.

Mono-*:
We grabbed these benchmarks after CTP1 shipped. A blog commenter pointed us at them, and they were pretty easy to integrate into our test system. They come from “The Computer Language Benchmarks Game”. When we started measuring them, we were in pretty bad shape, but most have gone to neutral. We do still have a handful of losses, but we’re still working. For details about each benchmark, just hit up the original website. They’re all pretty small applications, generally stressing particular optimization paths, or particular functionality. Back when JIT64 started, it was originally tuned to optimize C++/CLI code, and focused on the Spec2000 set of tests, compiled as managed code. This generally tuned for optimizations that pay off in this set of benchmarks, as well. The only benchmark out of the batch that you won’t find on the debian website is pi-digits. The implementation from the original site used p/invoke to use the GMP high-precision numeric package. I implemented it using .NET’s BigInteger class instead. Again, you can find the
code on my codeplex site.

SciMark:
This one is pretty self-explanatory. SciMark’s been around a long time, and there’s a pretty reasonable C# port done back in Rotor days that now lives here. It’s a set of scientific computations, which means heavy floating point, and heavy array manipulation. Again, we’re doing much better than we were with CTP1, but RyuJIT still lags behind JIT64 by a bit. We’ve got some improvements that didn’t quite make it in for CTP2, but will be included in future updates.

FSharpFFT:
Last, but not least. When we released CTP1, there was some consternation about how F# would perform, because JIT64 did a better job of optimizing tail-calls than JIT32 did. Jon Harrup gave me an F# implementation of a Fast Fourier Transform, which I dumped into our benchmark suite with virtually no understanding of the code at all. We’ve made some headway, and are now beating JIT64 by a reasonable margin. One item worth mentioning for you F# fans out there: We’ve got the F# self-build working with RyuJIT, which was pretty excellent. Maybe we’ll start running that one as a benchmark, too!

There you have it: a quick run through of the benchmarks we run. Having read through it anyone that paid attention should be able to tell that I don’t write code, I just run the team :-). If folks would like more details about anything post questions below, and I’ll see what I can do to get the engineers that actually write the code to answer them.

Happy benchmarking!

-Kev

One thing that's buried in the FAQ: we fixed bugs that people reported, so if you uninstall RyuJIT CTP1, re-download it and reinstall it, you'll get an ever so slightly less buggy version!

If the FAQ doesn't answer your questions, I'll answer other questions over here. Fewer people read this one, so I'll get myself into less trouble than my previous Answers to Questions :-)

-Kev