AlphaGeek https://docs.microsoft.com/archive/blogs/alphageek/feed.xml Deep tech and analysis directly from the engineers of DevDiv Wed, 17 Apr 2019 03:39:47 GMT en-US hourly 1 Significant Garbage Collector Changes in .NET 4.6.2 https://docs.microsoft.com/archive/blogs/alphageek/significant-garbage-collector-changes-in-net-4-6-2 Tue, 24 Jan 2017 15:02:12 GMT https://blogs.msdn.microsoft.com/alphageek/?p=125 Maoni Stephens - Main GC Developer[/caption] In this blog entry, we would like to discuss some significant changes that were made in the .NET 4.6.x garbage collector (GC).  We encourage you to get the latest version, 4.6.2. Our main GC developer, Maoni Stephens wrote up a description of the enhancements that were delivered with the 4.6.2 framework. These changes were made in order to improve performance of the framework and to allow the garbage collector to operator more efficiently.

Make the Pinning Promotion More Efficient

In previous versions of the .NET Framework when an object reported as pinned survived, we couldn’t move that object and the adjacent live objects. Since we tend to leave pins in ephemeral generations (so you can use the free spaces between them sooner), this means we needed to scan them when performing ephemeral collections. If many other objects right around the pinned objects survived collection also then the ephemeral collection time could be higher than desired. Starting with version 4.6.2, this limitation was lifted so we could compact away the adjacent live objects around pinned objects. In testing, we saw dramatic improvements of ephemeral collection time in scenarios where GC was artificially pinning a lot of objects. In the Before pic, the 2 non pinned objects get compacted away but the non pinned object next to the pinned object stays in gen0. Inefficient Pinning Promotion In 4.6.x this limitation was lifted so we could compact away the adjacent live objects around them. We saw dramatic improvements of ephemeral collection time in scenarios where GC was artificially pinning a lot of objects. In the After pic, we can see that the non pinned object next to pinned gets compacted away: Optimized Pinning Promotion When you pin a lot, such as when IO operations last for a long time and you observe long ephemeral GC pauses, you would very likely benefit from this change.

Using Free Space in Gen2 More Efficiently

In prior versions of the framework we used a first fit approach when we were compacting gen1 survivors into gen2 free list, which meant the GC discarded spaces that could not be used. This technique led to wasted spaces in memory. Inefficient use of Gen2 Free Space In the Before pic, we see that F0 disappeared because it was too small to fit S. We were able to compact S into F1 and part of F1 remains (for further compaction from gen1). To improve on this and utilize memory more efficiently, we introduced a bucketed free list where we threaded left over free spaces into their respective bucket. We had to be very careful of the usage in the smallest bucket as there could be many of them. We also worked on a policy of which bucket to try as we didn’t want to lengthen the gen1 collection time much by spending more time in searching for a usable free space. We want to stay doing background GCs as much as possible in order to be more efficient at consuming free spaces. More Efficient use of Gen2 In the After pic F0 remains on the free list and can be used for further compaction from gen1. And we try to fit S into F1 right away. If you observe your gen2 memory increasing, one reason could be that we ran out of items on the free list.  Consequently, promoting gen1 survivors would require an increase in gen2 size.  For scenarios that are helped by this change you would see gen2 size increase much less quickly and we can accommodate more gen1 collections for every gen2 collection. This also means you would see blocking gen2s triggered less frequently because when the heap gets too big which causes the memory load to be too high we would do a blocking gen2 to compact.  In our tests, we have observed first party scenarios where the ratio of gen1 collections to gen2 collections increased from 20 to 1 to more than 200 gen1 collections for every gen2 collection.

Summary

If you were running into situations described above, you should definitely try out .NET 4.6.2 to benefit from these enhancements in the garbage collector.]]> Native Compilation – Why JIT when you can CodeGen! https://docs.microsoft.com/archive/blogs/alphageek/native-compilation-why-jit-when-you-can-codegen Thu, 13 Oct 2016 12:06:44 GMT https://blogs.msdn.microsoft.com/alphageek/?p=77 Typical .NET Compilation, JIT Compilation and Execution Cycle[/caption] For 15 years, .NET apps are created with a compiler that emits intermediate code (IL). That IL is evaluated by a just-in-time compiler (JIT) that performs a native compilation of your application allowing it to run on the destination operating system that hosts the common language runtime (CLR).  Until recently, Microsoft-provided .NET tools only ran on Windows. If we want to shortcut this two-step compilation process and deliver a 100% native application on Windows, Mac, and Linux, we need an alternative to the CLR. The project that is aiming to deliver that solution with an ahead-of-time compilation process is called CoreRT.

CoreRT CodeGen Current and Planned

[caption id="attachment_117" align="alignleft" width="400"] Direct Native-Compilation from .NET Source Code[/caption] The CoreRT compiler currently has the RyuJIT embedded at its heart to allow current environments where .NET is available to compile directly to native code.  You already know some of the places where we have the RyuJIT working nicely:  Windows Desktop, Windows Phone, MacOS, and several Linux Desktop distributions all use RyuJIT.  But what about those operating systems and environments where we don’t currently have .NET developers invested and working?
If I really want to write some C# code and have it “just work” on a new IoT device, I don’t have any options until the RyuJIT is capable of generating machine code that works with that processor and operating system. One strategy we are investigating is the ability to transpile .NET code to C++ code that could be compiled with an appropriate C++ compiler for the target platform.  This strategy opens doors for hardware development because you would no longer need to wait for a Microsoft engineer to extend our compilers to support some new OS or new hardware in order to allow .NET developers to work with your product.

Progress on CodeGen

Code Generation and Transpilation is not an easy task, and our team is working diligently to solve this problem.  In the CoreRT repository on GitHub you can watch the progress the development team is making on the native compilation tools.  In particular, we had an interesting bit of progress over the summer as we completed some tasks that allow our C++ CodeGen to interpret .NET interface usage and properly dispatch it to C++.  We can now support transpiling interface usage in this baseline scenario: You can find the source code commit that supports dispatching this type of interface structure on GitHub at: https://github.com/dotnet/corert/commit/c6a2b271f5cc6d5b4c271e1dd9ecb364b5a2743c   What makes this interesting is the inspection of the .NET dependency graph and object hierarchy to determine which method calls to actually dispatch from C++.

Interface Challenges Addressed

This seems like a simple step that we have taken, but there are several challenges that we faced when examining the code.
  • C++ does not have a 1-to-1 feature mapping to .NET. We need to interpret and determine the appropriate way to map .NET features into C++ calls appropriately.
  • Multiple .NET classes may implement the same interface. The transpiler must be smart enough to know which concrete class to instantiate and pass around as an interface.
    • It accomplishes this by building supporting data-structures (dispatch calls and a dispatch map) in a prescribed format and passes them to the runtime to figure out which method slot to dispatch on given that a particular interface method will likely live in a different slot on the various implementing concrete classes.  In this way, it works the same as the RyuJIT case: we just lay out the structures carefully in the generated C++ so that it is compatible with what the runtime expects.
  • The concrete class may implement virtual methods that belong to another interface or superclass. These interactions need to be preserved in C++

Summary

We believe that investing in C++ code generation is going to allow the .NET ecosystem to grow and support platforms that have not been invented yet.  With this approach, we no longer need to write JIT compilers for every environment, and can rely on the standardized and robust C++ compilers that are commonly available.  To learn more about CoreRT and the CodeGen processes being constructed, follow the CodeGen label on the CoreRT issues repository.  We’ll have more about CoreRT in the future as this framework progresses.]]> Under the Hood - Performance Considerations of a Garbage Collector https://docs.microsoft.com/archive/blogs/alphageek/under-the-hood-performance-considerations-of-a-garbage-collector Tue, 13 Sep 2016 17:51:57 GMT https://blogs.msdn.microsoft.com/alphageek/?p=25 Welcome to the Alpha Geek blog, where we’re going to cover how software gets made at Microsoft.  We’ll share the good and the bad, directly from the engineers that build the tools and frameworks you use.  In this first post to the Alpha Geek blog, we’re going to explore the performance considerations and measurements that the Garbage Collection (GC) team has gone through so that you can have a better understanding of how the GC process affects your applications.

GC performance from a high level

In a system with a garbage collector, the GC acts as a service. It provides memory safety and automatically cleans up objects that are no longer referenced. As a service, it comes with a cost – when GC does its work, it takes away resource from the mutator, ie, the user threads. Naturally, the more heavily you use memory, the more you care about the GC performance. It’s very important to understand that some frameworks have a different memory allocation and survival patterns that make GC performance more or less significant than other frameworks. When you implement something in one framework, they might allocate much fewer and more temporary objects. This means much less pressure on their GC than another framework that allocates more and longer lived objects on that framework’s GC. In this document we are only looking at the performance of a GC, which means we are trying to understand how various kinds of workload perform from GC’s perspective. We are not trying to compare how the allocation and survival patterns differ between different frameworks or different versions of the same framework. Of course even with the same workload, it’s still not an apples to apples comparison as different GCs have different restrictions which can affect the implementation dramatically. For example, the CLR GC needs to handle pinning (since we support pinning as a feature for our framework) which is not the case for some other GCs.

3 performance aspects of a GC

Now we describe the 3 key performance aspects of a GC.
  • Throughput. It determines how fast you can handle requests. It plays a major part in the cost model of running an application (eg, how many cores do you need to pay for a VM you rent on Azure or AWS?). When GC is running, it takes CPU away so less time spent in GC means more time can be spent in the application code.
  • Memory footprint. How much memory your application requires is another extremely important performance metric people look at. You are paying for the memory you need. Lower memory usage means you can pack more sites on a web server; you can run more instances of your processes on one machine or handle more concurrent requests in your process.
  • Pause predictability. Server applications have SLAs to meet. This means customers want the pauses to be predictable. If individual GC pauses are short, generally that means they will have an easier time to cope with these pauses; but if there are long pauses that take up a significant percentage of the total request time, it will cause the application to fail to meet the SLA. So from a practical stand point this means shorter individual pauses are more predictable.
Changing one performance aspect can affect others. For example, if we let GC run less often so we spend less time in GC code, we will increase throughput but the memory footprint will go up. And when we do a GC, that GC might be very long as we’ve accumulated more objects that survive. If we let GC run very often, each individual GC pauses might be very short as we have very few survivors but each request will incur so many GCs that the total GC time add up and the throughput suffers. If we use a pie chart to represent the total performance of a GC, and 3 slices to represent the 3 aspects of the performance and bigger slices represent better the performance, when we make one aspect’s performance better, it usually means we need to make a tradeoff for another aspect. Of course that’s not always the case. If we optimized the implementation of a mechanism we use in a GC, or we added a new flavor of a GC that lifts some inherent limitations of the current GC, we could make one or more aspects so much more performant that the pie itself becomes bigger. For example, if we didn’t have concurrent GC implemented (which means whenever a GC happens it needs to stop all managed threads), let’s say we have the same score for all 3 aspects to begin with. By implementing a concurrent sweeping GC (which is a lot more common than concurrent compacting GCs), we improve the pause predictability, but we have more GC work to do which means we sacrifice throughput. And since we don’t compact, we use more memory, and for memory since we consider it worse perf if we use more memory, it means the Memory area will be smaller. So we might see this transformation in the pie chart:
[caption id="attachment_45" align="aligncenter" width="300"]No Concurrent GC No Concurrent GC[/caption] [caption id="attachment_55" align="aligncenter" width="300"]With Concurrent Sweeping GC With Concurrent Sweeping GC[/caption]

Being competitive in the Cloud space

From the GC team’s point of view, naturally we want to be competitive when we compare ourselves to other GCs. Currently the Cloud space is the one we are focusing on as it’s a very demanding area for GC. This means 2 things: A) We want to make sure our GC meets our current customers’ requirements; B) We want to understand how we compare to GCs in other frameworks in this space so we know how we can attract new customers. CLR has many first party customers that have extremely large and stressful scenarios. We have chosen what to optimize based on their needs. 2 examples from our first party customers: Bing Search: they’ve been optimizing their % time in GC. They’ve gotten it down to 1% or less. Exchange: they run many managed processes (on the order of 60 to 70) on each machine – in order to fit all these processes with the number of concurrent requests they handle, both memory footprint and throughput are very important. Historically, the CLR GC had been concentrating on optimizing for throughput and memory footprint. However, we are considering providing more ways for users to communicate to us what aspect of performance they are trying to optimize for and GC will tune itself accordingly.

Doing performance analysis

There are many dimensions an application can vary its allocation and survival patterns to affect the amount of work a GC has to do. The figure below depicts the most fundamental ones that we use in our performance analysis: Performance Quadrants For server applications we also want to observe long term effect as a server application is expected to stay up for a long time. Often when people do performance analysis, they run the test for a few seconds but for GC analysis, this is not enough at all. We must see the long term of a GC. So we run our benchmarks for a long period of time (~10mins) in order to observe whether any of the key performance aspects regress overtime.

Summary

Our GC team is continuing to work on improving the performance of our garbage collector to find the limits of the pause predictability and determine how best to improve that.  The design, discussion and work on the .NET Core GC is on GitHub if you are interested in learning more. Thanks to Maoni Stephens for helping with this article.  You can read more from Maoni at: https://blogs.msdn.microsoft.com/maoni/ Did we discuss enough technical depth on this topic?  Are there other things about the .NET framework that you would like to learn about?  Reply in the comments below and we will gather the engineering experts to explore these topics on future Alpha Geek blog posts. Truck photo courtesy of bilbobagweed on Flickr]]>