https://www.bradygaster.com/

1.  When to use IO Completion threads and worker threads.

2. The common mistakes the developer do when connecting to other web services / IO bound operation that block the worker threads

3. What actually Min and max Worker Threads count to the thread pool means?

Short good video

There are situations where native thread executing the native code would need to execute managed code. For example, we can consider the case of a com component executing managed code.  In those cases, when the native thread, for the first time, execute the managed code, the native object [not the managed thread object] would be created and the execution would proceed. Here the managed thread object would never get created unless the managed code refers the executing thread using the API like Thread.CurrentThread

 What are Fibers?

Fibers are like light weight threading objects that can be executed in parallel. Multiple fibers can be executed in a single thread. The advantage over here is that the same thread would be serving multiple fibers and the application can take the responsibility of scheduling the fibers. In case of threads, the scheduling responsibility is with the operating system. If we go with fibers, we can control scheduling of the fibers. On the negative end, Fibers are so far not yet supported in Managed code. Fibers do not support synchronization like threads. Hence the overhead fibers create overweighs the light weightiness it provide. CLR team faced some issues before they release .NET v2.0. So they removed the Fiber support from V2.0. Please refer this links for more details.

a.       https://blogs.msdn.com/dinoviehland/archive/2005/09/15/469642.aspx

b.      https://blogs.msdn.com/cbrumme/archive/2003/04/15/51351.aspx

CoRoutine - Yield

We still have native support to Fibers. Only managed support is not there. We can Pinvoke and use fibers whenever we want. Or use Managed Extentions for C++ to implement class that internally used native fibers.  We all might have used yield keyword before the return keyword from the method so as to maintain the state of the method. This link from MSDN Magazine uses the later technique to implement yield functionality. It is very interesting. One may not want to miss it. https://msdn.microsoft.com/en-us/magazine/cc164086.aspx

 Fiber or Thread: Which is best?

Simple. One size does not fit for all. You will have to choose whatever works for you. But before you choose one, why not get an expert idea from this link - https://blogs.msdn.com/cbrumme/archive/2003/04/15/51351.aspx

Tomorrow let us jump into Threadpool and explore.

Executing atomic operation is relatively less expensive In a single processor / core machine, when compared to the multi processor machine. Operating system on single processor machine can simply make  sure the scheduler does not preempt the atomic operation executing thread by raising the IRQL  is sufficient. In fact some of the atomic operations [like XADD] are supported at assembly level.

But in case of multi-core / processor machine, there may be chance that multiple processor are trying to modify the same memory location. This case has to be handled differently.  We all know the processor access the values  from the caches.  The caches are capable of holding multiple values. When one processor modifies one specific cache-line, the corresponding cache-lines accessed by other processors would be invalidated. This is achieved with the help of bus transaction. During the bus transaction all the processors are stopped and corresponding cache-lines are invalidated. Then the modified value would be transferred to the main memory. So when other processors need the value , it would try to access it from the cache. Since the cache-line is invalidated, the value would be fetched from memory. The processor that writes the cache-line owns the owner key and that key would be moved across the processors whoever writes the same cache line. The more number of owner keys need to be transferred across processors involves more number bus transactions. Here I briefed the common technique.  But this implementation vary from one architecture to other. There are around 8 to 9  protocols on how to share values across processors caches. More number of Bus transactions reduces application performance. For more details refer this link - https://cache-www.intel.com/cd/00/00/01/76/17689_w_spinlock.pdf 

Managed Thread and Native Thread

Operating systems do not understand what is managed thread. Os know only native threads. So when you create a managed thread, you are simply creating a managed object. That is not a thread. This object holds fields like delegate method, exception state and name etc. But these information is not sufficient to create a native thread which is actually a real thread . CLR internally maintains another object that  contains all the fields required to create a native thread. So the managed thread object refers to a native object created by CLR and this native object refers to the actual thread. So when a developer creates the managed thread the below happens.

 MSDN says "The volatile modifier is usually used for a field that is accessed by multiple threads without using the lock statement to serialize access. Using the volatile modifier ensures that one thread retrieves the most up-to-date value written by another thread."  And refer C# language specification for volatile field.  Now you might have good idea on what is happening when Volatile read / writes are performed. Yes. You are right. Volatile keyword tells the compiler to not to optimize the field. So that instruction reordering will not happen for reads and writes of volatile field. Volatile fields simply guarantees  that the sequence of writes and reads of values would be performed as specified in the code. JIT optimization does not affect volatile fields.  We can also say volatile always makes the read happening from the RAM

 Interlocked vs Volatile.

Volatile fields only mandate sequence. Not atomicity.  We may not get up-to-date value if we use volatile as they are not atomic. Between volatile read  and volatile write,  the data processing thread may get preempted and the value in the RAM may get updated by other threads [from other processors]. When the thread in question gets the processor time, it would continue process the old value as the value from the RAM is different from the time it is read.

 One should consider using Volatile on following criteria

1.       Value need not to be very recent.

2.       JIT optimization is not required as the instruction reordering may disturb the business logic.

I would suggest you the following link to go through to have good Idea on the difference between Atomic operation and Volatile keyword.

https://forums.msdn.microsoft.com/en-US/clr/thread/ec656080-7521-4f61-8df3-d7168da5cf4c

Memory Barriers

Instructions re-ordering happen at two levels. 1. A the compiler level which can be controlled with Volatile keyword. By including Volatile keyword we instruct the compiler, not to optimize the code by re-ordering the instructions. 2. Second level of reordering happens at the processor level. Memory barrier is the way to tell the underlying hardware to skip any reordering that happens at the hardware level.

 Interestingly Thread.VolatilRead and Thread.VolatileWrite use memory barriers internally. Below is the reflected code snippet for Thread.VolatileRead and VolatileWrite for Int32 overloads

 [MethodImpl(MethodImplOptions.NoInlining)]
public static int VolatileRead(ref int address)
{
    int num = address;
    MemoryBarrier();
    return num;
}

For Thread.VolatileWrite

[MethodImpl(MethodImplOptions.NoInlining)]
public static void VolatileWrite(ref int address, int value)
{
    MemoryBarrier();
    address = value;
}

 Ok. Now I have set of  questions for you.

1.Can you write atomic functions on your own?

2.Can I some how avoid thread pre-emption?

3.Who will preempt windows scheduler?

4.I know about pre-emption. What is interruption?

5.What is Acquire and Release from Memory barrier?

 For answers,  refer below the links:

 https://download.microsoft.com/download/e/b/a/eba1050f-a31d-436b-9281-92cdfeae4b45/Locks.doc

 https://download.microsoft.com/download/e/b/a/eba1050f-a31d-436b-9281-92cdfeae4b45/IRQL_thread.doc

 These are must read links that gives in-depth knowledge on Memory barrier,  Atomicity, Locking and Thread IRQL etc.  

 Processor affinity

On a multi-processor environment, a thread can run on any processor whenever the thread gets switched.  The  OS scheduler decides on assigning different thread to different processor. There is no guarantee that the a particular thread will always run on a particular processor.  We can confine a thread to run on a particular processor. This is what called setting processor affinity to a thread. Unless there is a compelling reason , one should not set processor affinity to a thread. Setting processor affinity to a thread is not good for your program performance.  Setting processor affinity through code may lead to under utilizing some of the processors. ProcessThread class lets you assign affinity to one processor to a thread. Here you have to enumerate through all the threads from the process to get hold of ProcessThread. We cannot create Process Thread as this class does not have any constructor or singleton supporting static methods. If I'm not wrong  we have to use Native Thread ID to match a Thread and ProcessThread.

What is the current thread means on a multi processor environment?

A process running on a multi-processor environment may have more than one thread. In  that case which thread would be represented by Thread.CurrentThread property? This property always return the object of the thread that is currently executing that statement. Do not assume a process will have only one current thread at any point in time  in multi-processor environment.

Lock convoy?

Think about this situation. Many threads are waiting for a lock. But no one owns the lock. This is kind of situation is called lock convoy. Interesting. Read more from the below link. https://blogs.msdn.com/larryosterman/archive/2004/03/29/101329.aspx 

Reads / Writes to the memory

Before we jump in to atomic operations, I would like to give some information on order of reads and writes into the  main memory. The chip makers are under pressure to make the processor faster and faster.  To improve processor performance, the chip makers introduce multiple caches between the processor and the RAM.  Refer the diagram for clear understanding] The reads and writes are not performed the same order as our program does even in case single processor machines. Like single processor counterparts multiple processor machines perform read and writes in a more performing way that order of the reads and writes are not considered. When a value is read from the memory, other values that exist adjacent to the value being read also brought to the caches of the processor. So when the processor wants a value from the RAM, it looks into its local cache first. If the required value is not available in the local cache, then only the processor would try to bring the value from the main memory. If the value is available in the local cache, the processor would not go to RAM to fetch the value.  In a multi processor scenario, there could be chances that the value available in the processor’s local cache may be stale.

 To avoid the issue of reading stale data, we put locks. What does lock do? It instructs the processor to follow some order while reading and writing the data to/from the RAM.  The .NET 2.0 rune time instructs the processor(s) to follow the below protocol when the CLR executes the statement that is inside the lock block.

1.       Read should never go before the time the lock is acquired.

2.       Write should never go beyond the lock is released.

 This also means,  the reads and writes inside the lock block can be of any order. This freedom is given for processors  to improve their performance. When we force the processor to perform reads and writes in certain order, it affects the speed of the processor.  In case of locks we trade off processor performance with synchronization. 

 What are Atomic operations?   Operations that would get executed fully before the thread that executes the operation pre-empted are called atomic operation.   In other words, The thread that executes the atomic operations would not get preempted before completing the atomic operation.  Interlocked class from .NET framework provides atomic functionality.  Atomic operations are faster then lock.  Atomic operations guarantee that the atomic reads are always from the main memory and atomic writes are immediately updated to the main memory immediately.  One can write effective locking with the help of Interlocked. Exchange method.  Though locks are best fit for many places and solves most of the synchronization related issues, performance of the locks are not so great when we compare them with interlocked.

 Locks vs atomic operations

Locks controls instruction reordering where as interlocked operations guarantees reads /writes from/to the processors is immediate and would be available to other processors / threads that does appropriate reads. In fact acquiring lock is an atomic operation. If acquiring lock is not an atomic operation, then more than one thread could acquire same lock. I have seen code where people use locks to just to increment one value.  For these cases interlocked class is of good fit.

Refer below the code snippets:

// Uses Interlocked classes

 static void Main(string[] args)

        {

            int j = -10;

            Stopwatch sw = new Stopwatch();

            sw.Start();

            for (int i = 0; i < Int32.MaxValue; i++)

            {

                j++;

            }

            sw.Stop();

            Console.WriteLine("Total time taken {0} ms", sw.ElapsedMilliseconds);

            Console.ReadLine();

        }

        // Uses Locks

        static void Main(string[] args)

        {

            object lockObj = new object();

            int j = -10;

            Stopwatch sw = new Stopwatch();

            sw.Start();

            for (int i = 0; i < Int32.MaxValue; i++)

            {

                lock (lockObj)

                {

                    j++;

                }

            }

            sw.Stop();

            Console.WriteLine("Total time taken {0} ms", sw.ElapsedMilliseconds);

            Console.ReadLine();

        }  

I have written these code snippets to compare the speed of lock and interlocked statement execution. When I executed above snippets from my laptop [Toshiba Tecra-M5, 2 GHZ, Dual core, 4GB RAM- Win2K3], below are results that I got.

 Interlocked - Total time taken 64326 ms:

Lock - Total time taken 141203 ms:

It is clear that locks take more time.  Whenever possible, try to use methods from interlocked classes.

Atomic operations on Multi-core / Multi processors machines.

Though atomic operations guarantee to reflect the latest modification to the primary memory and in the processor that executes, it the responsibility of the code to use atomic operations wherever required.  Assume the following

1.       You have got two processor machines.

2.       Processor 1 Execute one atomic operation on a variable var1.

3.       At the same time Processor2 access var2 using normal read.

In this case, Processor2 may get stale value as it has no idea that var2 value is changed in the memory.  To make the processor 2 to read new up to-date value, code that reads var1 from processor 2, should do volatile read / make use of lock as the processor 2 may have stale data in its cache which might not have been invalidated.

To constrain processor, the Initial idea was to setting the processor affinity to the application process through task manager or sys internal's process explorer tool. But the idea became invalid as the OS and other services that support our application may be using all other processors as we are setting affinity to our application process only. Our test team members felt that, this case may not simulate the real time environment. So we left this idea.

Then we have found out the tool "Consume.exe" that is shipped with VS.NET 8.0 tools directory. Consume.exe can block the following resources.

1. Physical memory
2. Page file
3. Disk space
4. CPU time
5. Kernel pool

It is a good tool that can eat all the mentioned resources and make the application to use only the remaining resources. Our team members suspected that, with this approach, there could be unknown factors that would influence our application's performance in negative / positive way. This tool can be used to simulate a situation where other applications also runs along with your application. So we had to leave this idea also as our application is server application and it would be running alone in the server.

Finally found out one good approach that was accepted by all of our team members. That was nothing but modifying boot.ini file. So we used the following switches from boot.ini file for our requirement.

/NUMPROC = 1

/MAXMEM = 1024

After modifying and rebooting the system, the OS was showing / utilizing only the mentioned amount of resources. To read more about other switches and its details refer Technet site.