Abstract

Interesting documentation can be found comparing the many options available in WCF to encode web services data. Any encoding strategy may have advantages and disadvantages in different environments and runtime conditions.

For example, binary encoding provides best performance on fast network connections, text encoding may be easier to debug and a custom compressed encoding may be better for big messages and slow network connections.

On the other side, a custom compressed encoding uses much CPU and may not be appropriate when the server is very busy; also, it would be completely useless for small messages or messages with data already compressed (i.e. images in a compressed format like jpg).

Then, an interesting point is:

• Is it possible to make the choice at runtime, and select the most appropriate encoding for each specific condition?

In fact it is, and you may get interesting performance benefits from such approach!

In this article we’ll show how to create a custom DynamicEncoder that extends existing encoders (such as WCF binaryMessageEncoding, textMessageEncoding etc.) selecting the most appropriate at runtime and applies compression selectively to the encoded stream to minimize calls latency and maximize their throughput.

Why adaptive encoding?

In the following link, Dustin Metzger proposes a very interesting analysis on the effect you may obtain applying compression to standard WCF binary encoding.

What seems evident by the numbers is that compression provides such a great benefit on medium and low bandwidth network connections at the cost of an even greater negative impact on high bandwidth network connections.

For example, some measures taken by Dustin show that adding compression to a binary encoded stream may provide significant benefit on a 100 Mbps network where the cost of invocations may be reduced by up to 4 times.

Figure 1: Throughput comparison, from Dustin Metzger article, between WCF binary encoding and compressed binary encoding on a 100 Mbps network, for different message sizes

Understandably, such benefit may grow by far on production environments where it’s common to find network bandwidths close to one megabit.

Dustin graph shows however that compressing small messages doesn’t really pay off the CPU and latency spent for compression. Also, the following graph from Dustin article shows that with high bandwidth networks or generally with CPU bound loads the cost of compressed encoding may outgrow the cost of standard binary encoding up to 5 times.

Figure 2: Throughput comparison, from Dustin Metzger article, between WCF binary encoding and compressed binary encoding on a 1 Gbps network, for different message sizes

Again, in real production environments you may find messages with images, pdf or xps documents that are already compressed so, compressing their stream at the encoder level wouldn’t provide any benefit.

For these reasons we chose to use an adaptive approach and we wrote an encoder (DynamicEncoder) to extend binary encoding that applies compression to the encoded stream selectively, based on few basic rules:

• Compression should be avoided for small messages.
• Compression should be disabled automatically when the server CPU load is too high.
• Compression should be disabled automatically for soap actions providing messages with bad compression ratio.

To avoid compressing small messages we chose to allow the configuration of size thresholds for the different compression levels:

In particular, DynamicEncoder avoids compressing messages with size below the value configured as NoCompressionLimit.

In the example above, the encoder is configured to avoid compression for all messages below 1024 KB and compress other messages with Fast Compression; thresholds for Normal and Maximum compression are set to -1 so they are not considered by the encoder.

To prevent compression on CPU bound load we chose to allow the configuration of an additional threshold:

When the machine CPU load overtakes the CPUDisableLimit DynamicEncoder falls back progressively to standard binary encoding. In the example above, the encoder is configured to disable compression progressively when the server CPU usage outgrows the value of 70%.

To prevent compression of uncompressible messages we chose to allow configuration of a last setting:

According to this setting, our dynamicEncoder will fall back to standard binary encoding on every invocation that shows an insufficient compression ratio. In the example above, the encoder is configured to disable compression for all soap Actions that show a compression ratio above the value of 70%.

The evaluation of the compression ratio for an outgoing message is based on the average ratio produced by the last messages with similar size, in the same soap Action. this means that messages with similar sizes, in the same soap action are assumed to produce similar compression ratios.

So, dynamicEncoder starts compressing messages for all soap Actions. Soap Actions that produce a compression ratio worse than the threshold defined by "CompressionRatioDisableLimit" are then disabled progressively.

This setting is very important because it allows DynamicEncoder disable compression on big complex messages that contain images or documents that are already compressed. Compressing them would waste a lot of CPU Usage and time without producing any benefit.

The following paragraph shows some measures of the performance we obtained with dynamicEncoder. We’ll then delve into few details of dynamicEncoder implementation.

Some measures of DynamicEncoder performance

In this section we compare dynamicEncoder performance to WCF binary encoding with different message types and network conditions.

More specifically, we measure throughput and CPU usage obtained by a sample service returning messages of different sizes and complexities, respectively on a 100 Mbps and a 1.5 Mbps networks.

The performance measures are taken with 3 separate configurations for the service endpoint:

• Binary encoding: here the service is configured with WCF binary encoding; the values in this measure serve as a comparison to understand the impact of the compression strategies.
• Static compression: in this case the service is configured using dynamicEncoder with CompressionRatioDisableLimit=1 and NoCompressionLimit=0. This way compression is applied to all messages returned by the service to the client.
• Adaptive compression: here the service is configured using dynamicEncoder with CompressionRatioDisableLimit=0.7 and NoCompressionLimit=1024. This way compression is not applied to small messages and it is disabled on messages with bad compression ratio.

The different encoding strategies are compared configuring the sample service to return the following message types:

• Simple messages: these are messages with high redundancy; they are compressible by ten times on average (i.e. compression ratio close to 0.1).
• Medium complexity messages: these are messages with average redundancy; they are compressible by a factor close to 0.4.
• Complex messages: these are messages with low redundancy i.e. compressible by a factor between 0.8 and 1.

The measures were taken with random message sizes ranging between 0.6KB to 250KB on a distribution with most messages close to 150KB.

On a 100 Mbps network we observed the following results:

Figure 3: Throughput values produced by adaptive compression, static compression and binary encoding on a 100 Mbps network

Figure 4: CPU usage produced by adaptive compression, static compression and binary encoding on a 100 Mbps network

Figure 3 show that all encoding strategies provide similar throughput for complex messages. Indeed, complex messages produce a bad compression ratio so, only few bytes are saved when compressing them.

Figure 4 shows however that static compression wastes a lot of CPU usage for this case.

Adaptive compression instead, gives up compressing complex messages and saves the precious CPU resource for a better use.

This approach pays well in the other scenarios (and in real production environments) where the CPU is a more precious resource and adaptive compression is able to produce better throughput with lower CPU usage.

You may notice that only the test with simple messages was bound by CPU usage (i.e. CPU usage reached 100%) while all other measures were bound by network usage; Understandably, on a 1.5 Mbps network the bottleneck is always the network usage and the measures produced a similar pattern with much lower CPU usage and throughput values.

It’s great to get this better throughput but what about latencies?

How does the time spent for compressing messages compare to the time saved during transmission?

Given the network speed, the compression speed of a system and the compression ratio obtained on messages it is easy to calculate the time saved for transmission and the time spent for compression:

• TransmissionTimeSaved = messageSize *(1-compressionRatio) / networkSpeed
• CompressionTimeCost = messageSize *2 / compressionSpeed

On our test systems we measured an average compression speed from few KB/sec for smallest messages up to 50 MB/sec for biggest messages, with an average compression ratio of 0.24.

On a 100MB network you may experience benefits in latency only for bigger messages, when the compression speed is higher.

Figure 5: Latency improvement applying compression to messages on 100MBit networks

In this scenario, to avoid latency penalties it may be appropriate to raise the NoCompressionLimit setting and apply compression only to bigger messages.

On a 1.5MB network you get more interesting results with a latency benefit from smaller messages.

Figure 6: Latency improvement applying compression to messages on 1.5MBit networks

Figure 6 shows that, in this scenario, both messages produce a transmission time improvement of 75%, with absolute latency improvement of 2 seconds for the bigger one.

Some measures on DynamicEncoder benefits in a production environment

Luckily enough, we could measure some benefits on our production systems too.  

In our case, messages have the following distribution with sizes between 100B and 500KB.

Figure 7: Message size distribution for our production environment

In such scenario, dynamic encoder ends up to apply compression to about half the messages with an average compression ratio of 0.22. Overall, the network traffic is reduced by a factor 0.31.

Figure 8: messages compression ratio obtained by dynamicEncoder on our production environment

It is important to notice that in this production environment we have tested the compression provided by the dynamicEncoder over the WCF Binary Encoding. Even if Binary Encoding is a quite efficient way the transmit data on the network, the dynamicEncoder has shown a significant decrease in the network traffic.

Clearly the usage of the dynamicEncoder over the WCF Text Encoding, can demonstrate much better results.

For most clients, the available bandwidth to the server is around 2Mbit so the expected latency improvement may range between a few milliseconds for smallest messages up to a couple of seconds for the largest.

WCF Encoders creation process

Writing a WCF encoder is just a matter of writing 4 classes, each one creating the other, the last of which (the encoder) does the job of translating incoming messages into the binary arrays sent over the wire.

Two classes of them, a BindingElementExtension and a MessageEncodingBindingElement class, are used to load the encoder configuration from the WCF endpoint configuration. A third MessageEncoderFactory class is needed to support the encoder creation process, according to the factory pattern. The last MessageEncoder class, which is the actual encoder, contains the code to translate WCF messages into binary arrays.

The figure below shows the classes involved into the encoder creation process and the message encoder working on a WCF method call.

Figure 9: classes involved into a WCF encoder creation

To write our DynamicEncoder we will then follow the following steps:

• Create a DynamicEncoderBindingElementExtension class

DynamicEncoderBindingElementExtension class must derive from BindingElementExtensionElement; it loads the encoder configuration from the binding configuration and participates to the binding elements creation.

• Create a DynamicEncoderMessageBindingElement class

DynamicEncoderMessageBindingElement must derive from MessageEncodingBindingElement and will be invoked by WCF to obtain MessageEncderFactory instances.

• Create a DynamicEncoderFactory class

DynamicEncoderFactory class is used by WCF to generate encoder instances according to the factory pattern. The factory is invoked by WCF to obtain message encoder instances whenever needed.

• Create a DynamicEncoder class to do the encoding.

The message encoder job is to translate the WCF Message object into a buffer to be sent over the wire.

Once written, we’ll be able to activate DynamicEncoder on a WCF endpoint with few configuration steps:

• Declare the DynamicEncoderBindingElementExtension in the extensions node of application configuration file:

Figure 10: dynamicEncoder definition in the application configuration file 

• Add a custom binding configuration to a WCF server endpoint with a reference to our dynamicEncoder binding element extension:

Figure 11: dynamicEncoder binding element extension configured into a server side endpoint

• Add a symmetric configuration to its corresponding WCF client endpoint:

Figura 12: dynamicEncoder binding element extension configured into a client side endpoint

In the configuration examples above dynamicEncoder is configured to work with two inner encodings (BinaryMessageEncoding and TextMessageEncoding) using BinaryMessageEncoding as the default.

This means that any client endpoint with BinaryMessageEncoding will be able to talk to the server without changes; of course, only client endpoints configured with our dynamicEncoder will be able to leverage the performance benefits of adaptive encoding and selective compression.

For this reason we’ll say that DynamicEncoder extends the message encoding specified as the default.

Step 1: Create a DynamicEncoderBindingElementExtension class

The role of a binding element extension is to deserialize a binding element configuration and expose WCF the methods to create its associated binding element.

DynamicEncoderBindingElementExtension inherits from BindingElementExtensionElement as shown below:

Figura 13: DynamicEncoderBindingElementExtension definition

As shown in Figure 11 and 12 we use few properties to allow nesting the configuration of the inner encodings within the configuration of DynamicEncoderBindingElementExtension:

• BinaryMessageEncoding, TextMessageEncoding and MtomMessageEncoding are used to configure the supported inner message encoders
• DefaultMessageEncoding is used to define the one used by default.

Figure 14: DynamicEncoderBindingElementExtension properties to deserialize inner encodings configuration

3 additional properties (Providers, Algorithms and Levels) are used to configure compression providers, algorithms and compression levels allowed for the encoder.

Figure 15: DynamicEncoderBindingElementExtension properties to deserialize compression settings

By default, our DynamicEncoder uses one compression provider (CLR) based on .Net System.Compression libraries.

Step 2: Create a DynamicEncoderMessageBindingElement class

Our DynamicEncoderMessageBindingElement class needs to deliver inner encoders and compression configuration from the binding element extension to the encoder factory and DynamicEncoder instances.

For this reason, we’ll design it with a structure similar to that of DynamicEncoderBindingElementExtension.

Figura 16: DynamicEncoderBindingElement properties to hold inner encodings and compression settings

To get encoder factories from the binding element WCF will use the following entry point: 

Figura 17: Encoder factory creation entry point

Step 3: Create a DynamicEncoderFactory class

The factory class role is to prepare everything possible to allow efficient encoder instantiation, according to the endpoint configuration.

So, we’ll design our factory class to hold factories instances for the inner encoders:

• InnerFactories collection is used to keep the instances of encoder factories necessary to obtain inner encoders.
• DefaultMessageEncoding property used to define the one used by default.

Inner factories and other configuration information is taken from the binding element received at construction time

Figura 18: encoder factory construction time

As usual, few properties are used to hold inner encoder factory instances and compression configuration

Figura 19: inner encoders factory instances and compression configuration into DynamicEncoderFactory

WCF will then be able to get encoder instances by means of Encoder property (for sessionless connections) and CreateSessionEncoder method. 

Figura 20: DynamicEncoderFactory entry points used to deliver encoder instances to WCF

Step 4: Create a DynamicEncoder class to do the encoding

The DynamicEncoder class is the real WCF encoder class. At every service call WCF will make sure that our encoder be loaded and initialized by means of the encoder creation process described up to now.

As every WCF encoder, DyanmicEncoder class inherits from MessageEncoder class.

Not surprisingly, DynamicEncoder class will hold inner encoder instances, the default encoder and configuration settings about compression providers, algorithms and levels configured for the endpoint. 

Any information necessary for operating is received by the encoder at construction time, from the factory object.

DynamicEncoder performs few basic tasks:

1. It chooses what encoding algorithm should be used to encode the message
2. It uses the selected inner encoder to encode the message into a binary stream
3. It chooses whether the encoded stream should be compressed and what algorithm should be applied
4. In case, It applies compression to the binary stream

Such tasks are performed into its WriteMessage overrides, by means of some provider classes that are loaded statically:

One or more compression providers expose compression algorithms to the encoder by means of IDataCompressor interface.  DynamicEncoder supports use of multiple compression provider classes; by default, DynamicEncoder uses DataCompressor class: a very simple compression provider, based on .Net framework System.Compression libraries.

DataCompressor provider supports GZip and Deflate algorithms with only one compression level.  Additional providers supporting different compression levels may be added easily.

One encoder selector class contains all the rules for choosing the inner encoder and the compression settings to apply to the encoded stream.

The encoder selector exposes to the encoder such rules by means of interface IDynamicEncoderSelector.

Initialize() and Clone() are used by DynamicEncoder to initialize selector instances for any WCF channel.

OnSelectEncoder(), OnSelectAlgorithm() are invoked to choose the inner encoding and the compression settings to apply to the compressed stream.

OnEncodeComplete() is invoked after compression to allow gathering information on the encoded call that can be used to adapt encoder choices to the runtime conditions.

OnEncodeComplete()  is used by the default EncoderSelector to gather information about the compression ratio obtained on outgoing messages. This information is used by EncoderSelector to honor CompressionRatioDisableLimit setting.

References

• Dustin Metzgar: Compressing messages in WCF part four - Network performance
  https://blogs.msdn.com/b/dmetzgar/archive/2011/03/22/gzipmessageencoder-part-four.aspx

CalculatorClient.zip

Well, after a while working with WPF I could understand and experience some key reasons why WPF is really worth using, in every business application.

Reason n° 1: Functional Analysts and Graphic designers may define the user interfaces by means of XAML instead of bitmaps.

With older technologies, functional analysis and graphic designs used to be defined with documents and images; It was the developer work to translate all that in code.

WPF allows using XAML since functional analysis and graphic design stages. This way, part of the developer work is already done and many misunderstandings can be avoided.

Reason n° 2: C# code is significantly reduced.

Writing WPF applications, C# code is only necessary to access and manage business data. All the data rendering is managed by the User Interface XAML and WPF Data Binding tags and attributes. C# code no longer needs to manage user controls and their properties as all the rendering logic is into the XAML. So, the application code is generally reduced significantly.

The example below shows a complex form that manages data in a business entity:

Figure 1: a typical form to manage data in a business entity

The whole form only needs very few lines of business code to get the data from the server and save it into some dependency properties. All the data transfer and validation between the controls and the application code behind is entirely managed by XAML tags.

The code snippets below shows the form left and right panels code behind which is limited to few dependency properties and the commands to act on them:

Figure 2: The forms code-behind only contains business data and commands; Rendering and validation is specified at XAML level.

Reason n° 3: User Interface and Business Logic may be separated easily and clearly

As illustrated in reason n° 2, WPF allows keeping a strong separation between the business data and the user interface controls. The data transfer between the business data and the user controls is carried over by WPF data binding so, the business code never needs to access the application user interface controls and their properties explicitly.

As a result, the entire business logic is independent from the controls used to render the data. So, changing the application user interface is easier and, most often, doesn't affect the application code.

Old Windows Forms and other te chnologies used many "custom" frameworks to obtain such separation between the User interface logic and the Business logic, often, at the cost of some overhead for developers.

WPF provides this separation by design and doesn't require any extra code to get it!

Reason n° 4: Routed Commands allow easy command management with automatic support for enabling controls

WPF provides a powerful, built-in, infrastructure for defining commands and attaching them to UI elements. The developer defines typed commands, associates them to user interface elements and provides command implementations within code. Any command, when started from a UI element, is routed through the parent elements chain until a command implementation is found and executed. This allows decoupling the UI elements that generate commands from the command implementations.

The XAML snippet below shows 2 buttons associated to commands by means of the command attribute:

The (Search) command implementation and its enabling condition can be defined in the code behind with a CanExecute() and a Command() methods, as shown below:

The WPF Command Manager routes commands and verifies their conditions; all UI elements associated to commands that cannot be executed are disabled automatically.

The image below shows the two buttons run time: the Command Manager is disabling the Search Button as Search command cannot be executed.

Figure 3: the command manager disables buttons and other UI elements based on Command CanExecute() handlers

Reason n° 5: XAML dynamic loading allows changing the User interface and the application behavior depending on the user profile or user rights.

A very common requirement for business applications Is that the menus and the functions available may depend on contextual information such as the user profile or the user rights. Traditional application architectures usually address such requirement with complex configuration schemas and lots of code to load the required UI elements, at run time.

WPF allows loading entire XAML sections (e.g. contextual menus) dynamically and merging them into the application User Interface with very few instructions. The loaded XAML is automatically bound to the business code by the command routing.

The code snippet below shows how to load dynamically a ribbon control from a resource file:

So, configuration for dynamic menus may now be defined directly with XAML and the code for loading them may be significantly simpler.

Reason n° 6: client side and server side validations are very easy and powerful with Data Binding Validation Architecture and Routed Events.

When defining a data binding, few additional attributes allow the developer to specify validations to ensure the correctness of the data transferred from the control to the underlining properties.

The code snippet below shows a textbox, bound to a "company object" property with two validation rules for the data transferred.

By default, WPF gives evidence to validation failures adding a red border to the control that owns the binding, as shown below:

Figure 4: validation failures are rendered as a red border around the failing control

Styles and routed events may change the visual feedback of failed validations in a way completely transparent to the developer: with, the same XAML code you may generate an entirely different visual feedback for validations, as shown below:

Figure 5: Styles and routed events allows changing the visual feedback of validation failures without affecting the business code

Reason n° 7: The application is more flexible and easy to maintain.

Reasons n° 2, 3 and 4 generally ensure the business code to be simpler and reduced. Changes to the business logic are easier to maintain as the business code is isolated and It is not complicated by the rendering logic. Changes to the rendering logic are easy as they usually involve only changes to the XAML without affecting the application code.

Reasons such as n° 4, 5 and 6 allows having less code also for the application infrastructure (e.g. dynamic menus, command management, validation, authorization etc).

Reason n° 8: XAML, styling and control templating allow rendering complex data very flexibly.

The rendering reflects data changes very efficiently, the connection between the data and the user interface can be read only or read write. Most often, changing the rendering of data is very easy and doesn't affect the application code.

In the example below you can see a dialog box showing information about an application exception, for the Help Desk:

Figure 6: a sample form to render information about an exception

Another Tab of the same dialog shows exactly the same data for the application developer

Figure 7: the same data may be rendered in a radically different way changing the XAML only.

The business data behind the XAML is the same for the two views: the xaml makes the difference rendering the same data in different ways.

Reason n° 9: WPF works with device independent metrics and allows creating interfaces that adapt to different screen resolutions

Many panels such as Grid, DockPanel, StackPanel, etc allow adapting the controls sizes and layout to the various screen resolutions that may be available.

Scaling and rendering to XPS may help you creating your support for printing.

Conclusions

WPF may change significantly the way of developing your applications.

Less and easier code is required to write your traditional applications and addressing more complex scenarios is now easier!

Additional References

https://www.thejoyofcode.com/10_reasons_you_should_consider_WPF_for_your_next_desktop_application.aspx

Technorati Tag: Framework 3,Windows Presentation Foundation,WPF

The following key points:

• Minimal hardware requirements
• Strong runtime support
• Easy and intuitive development model
• An extensible emulation environment

Make it easy to address application development for simple devices used in consumer electronic and industrial automation.

Additional resources about Micro Framework are:

• Microframework WhitePaper
• MSDN Webcast: An Introduction to the .NET Micro Framework
• Micro Framework msdn web site.
• Micro Framework web site
• Alden Linn blog
• Micro Framework newsgroup
• Micro Framework SDK Download page

Attached, I propose the TripComputer Sample and SampleEmulator applications I used in a recent Webcast.

The Trip Computer sample is a very simple application that shows the structure of a Micro Framework application.

SampleEmulator application shows a simple customization where I added a GPIO button to the default emulator to better simulate the real tripcomputer.

The following figure shows the tripcomputer application run with the default emulator and with the customized version of the emulator:

The application starts on the static main() method of a class deriving from Microsoft.SPOT.Application.

Static main method creates an instance of the TripComputer Window and starts a Dispatch loop on it.

Class TripComputerWindow derives from Microsoft.SPOT.Presentation.Window and aggregates some panels and textboxes using the Windows Presentation Foundation (WPF) object model.

Load() Method shows the code to start a timer for updating the clock on the bottom right angle of the trip computer.

The TripComputer application shows some features unique to Micro Framework WPF graphic engine like the linear gradient brush on the background, the controls layout by means of vertical and horizontal stack panels and the interception of the ButtonUpEvent routed event at the main window level.

 In summary, developing embedded applications with Micro Framework is easy and intuitive and relays on the same development model of other .Net applications

Micro Framework Samples.zip

• WCF replaces old web services as WCF Services support HTTP/Soap based communication.

  In particular, WCF now supports WS* standards for secure, reliable communication etc.

  Also, WCF services can be hosted by IIS Services, WAS or they can be self hosted into a custom application.

• WCF replaces old .Net-Remoting communication as WCF Services support binary communication above TCP/IP and named pipes.

  Other channels such as MSMQ and peer messaging channel are supported.

  Also, a flexible extensibility model allows adding new custom protocols and channels.

• WCF replaces enterprise services as WCF Services support transactions by means of OleTx and WS-Transactions etc.

This article contains the following samples:

• SelfHosting sample: WPF calculator client calling a calculator service by SelfHosting.
• MessageInterceptor Sample: IClientMessageInspector, IParameterInspector and IDispatchMessageInspector to perform message and parameter interception.

SelfHosting sample

Selfhosting sample shows a WPF windows application that implements a client for the calculator service.

The calculator client performs the calculations calling the CalculatorService component.

The following diagram shows the CalculatorClient and CalculatorService components.

The CalculatorService is SelfHosted into the client application by means of the following code:

The service configuration below shows that at startup 2 endpoint listeners are loaded do that the service can be connected by means of TCP/IP or by named pipes:

With the following configuration the calculator clients connects the selfhosted service by means of the tcp/ip endpoint:

MessageInterceptor Sample

MessageInterceptor sample shows using the WCF extensibility model to intercept client application calls at class and method level.

This can be done creating a WCF Behaviour class for the service and attaching it to the service by means of the configuration.

Client Interception is performed by means of IClientMessageInspector, IParameterInspector interfaces and Service interception is performed IDispatchMessageInspector, IParameterInspector.

The MessageInterceptorClass shown below implements all the interfaces to provide CalculatorService interception at the client and dispatcher side.

The message interceptor is installed into the client or dispatcher side by means if a Behaviour.

MessageInterceptorBehaviour class below shows how to install the interceptor class at the client side (ApplyClientBehaviour()) or dispatcher side (ApplyDispatchBehaviour()).

WCF Samples.zip

In particular, CertifiedforVistaSamples.zip include:

• Manifestsample: shows how to apply a manifest to your .Net application resources with a postbuild rule.
• ManifestsampleVB6: shows how to apply a manifest to an existing application with MT utility.
• Signaturesample: shows how to apply a certificate to your application resources with a postbuild rule.

  The sample also shows how to generate a certificate with makecert utility.

The zip also include:

• RegistrySample: a sample application that allow writing any registry key and check whether the key is a Windows Resource Protected key.

For more information on Windows Vista Logo Programs see:

• Frequently asked questions on Windows Vista compatibility and Certification Logos
• https://www.microsoft.com/windowsvista/

CertifiedForVistaSamples.zip

Also, only a single image may be applied to different machines.

The steplist described below is taken from Getting Started with the Windows Automated Installation Kit.

Requirements

• Vista DVD: a DVD with Windows Vista.
• WAIK: Windows Vista Automated Installation Kit (get the download here).
• A USB Key: this is needed to contain the answers file for the installation (a little key should be ok).
• Blank CD/DVD RW: this is needed to boot lab machines at installation time.
• Master PC: a Lab machine to setup the Vista master installation.
• Technician PC: a pc with a DVD ROM Drive (RW capable), Windows AIK, and the tools described below.
• Network Connectivity: this is needed to store the OS Image for lab machines (NB The LAB machine image file will contain the lab machine system drive, so it can possibly be very large!!).
• CDBurn/Dvdburn utility: this tool allows burning a DVD from a ISO Image (Windows Server 2003 Resource Kit Tools).

The following steps can be applied to take the image of a single disk lab machine.

Step 1: create a bootable Windows PE RAM CD/DVD

• Follow instructions in "Steps to create a bootable Windows PE RAM CD/DVD"

Step 2: setup the master PC

1. Start the master PC with Vista DVD and setup Vista.
2. Configure Vista and setup all the software you need to the master PC.

• run the "disk clean up wizard" from the disk manager to reduce as much as possible the image size.
• Run sysprep.exe /oobe /generalize /shutdown from folder "c:\windows\system32\sysprep"

Step 3: take an image of the master PC

• Boot the Master PC with the Windows PE RAM CD.
• type d: from "command prompt" to change the current drive from the RAM DRIVE to the DVD.
• Type imagex /compress fast /capture c: c:\ image.wim "Vista Lab Install" /Verify
• Save the image (image.wim) onto a reachable network share (es. \\master\labimage).

Step 4: apply the Image to Lab PCs

• Boot the Master PC with the Windows PE RAM CD.
• Create 2 partitions on the pc with the following diskpart script

  diskpart /s diskpartcleanscript.txt

   

  Where diskpartcleanscript.txt contains the following text:

  select disk 0

  clean

  create partition primary SIZE=60000

  select partition 1

  active

  format quick

  assign letter=C

  create partition primary

  select partition 2

  format quick

  assign letter=Y

  exit

   

• Download Vista Image (image.wim) on drive "y:"

  use the following script:

  net use \\master\labimage /User:master\lab *****

  robocopy \\master\labimage . install.wim

   

• Run imagex /apply of the image to the first partition

  imagex /apply y:\labimage\install.wim 1 c:

Requirements

• WAIK: Windows Vista Automated Installation Kit (get the download here).
• CDBurn/Dvdburn utility: this tool allows burning a DVD from a ISO Image (Windows Server 2003 Resource Kit Tools).
• Technician PC: a pc with a CD/DVD ROM Drive (RW capable) with Windows AIK installed on it.
• A blank CD or DVD.

Steps to create the bootable disk

• Run the following commands from "Windows PE Tools Command Prompt":
  • copype.cmd x86 c:\winpe_x86
• (optional) Copy imagex tool and an imagex exclusion list:
  • Run copy "C:\Program Files\Windows AIK\Tools\x86\imagex.exe" "C:\winpe_x86\ISO"
  • create a wimscript.ini file to "C:\winpe_x86\ISO" with the following exclusions

    [ExclusionList]

    ntfs.log

    hiberfil.sys

    pagefile.sys

    "System Volume Information"

    RECYCLER

    Windows\CSC

    [CompressionExclusionList]

    *.mp3

    *.zip

    *.cab

    \WINDOWS\inf\*.pnf

• Create the iso for the bootable disk:
  • oscdimg -n -bC:\winpe_x86\etfsboot.com C:\winpe_x86\ISO C:\winpe_x86\winpe_x86.iso
• Burn the iso file to a dvd or cdrom
  • "dvdburn d: C:\winpe_x86\winpe_x86.iso" to burn the iso file to a DVD.
  • "cdburn d: C:\winpe_x86\winpe_x86.iso" to burn the iso file to a CD.

Note that step 2 is not required and is only useful to have imagex tool on the bootable disk.

You may add any other tools or batches to your bootable disk in a similar way.

Requirements

• Vista DVD: a DVD with Windows Vista.
• WAIK: Windows Vista Automated Installation Kit (get the download here).
• A USB Key: this is needed to contain the answers file for the installation (a key with very little space is ok).
• Technician PC: a pc with a DVD ROM Drive (RW capable) with Windows AIK installed on it.

Steps to create the answer file

• Copy Install.Wim (2GB) from vista DVD to the technician PC.
• Start "Microsoft Windows AIK - Windows System Image Manager".
• Select the Install.Wim file from the "File - Select Windows Image" menu.

  NB. The command will let you choose a Vista version (es. Ultimate) and create a catalog file.

• Create a new answer file with "File - New Answer File" menu.
• From the "Windows Image" pane add the following settings to the Answer File:
  • Microsoft-Windows-Setup\DiskConfiguration\Disk\CreatePartitions\CreatePartition (1 windowsPE component).
  • Microsoft-Windows-Setup\DiskConfiguration\Disk\ModifyPartitions\ModifyPartition (1 windowsPE component).
  • Microsoft-Windows-Setup\ImageInstall\OSImage\InstallTo (1 windowsPE component).
  • Microsoft-Windows-Setup\UserData (1 windowsPE component).
  • Microsoft-Windows-Shell-Setup\OOBE (7 oobeSystem component).
  • Microsoft-Windows-Shell-Setup\AutoLogon     (7 oobeSystem component).
  • Microsoft-Windows-International-Core-WinPE (1 windowsPE component).
• Into the the "Answers File" pane set the following settings:
  • Microsoft-Windows-Setup (1 windowsPE component).
    • Microsoft-Windows-Setup\DiskConfiguration    WillShowUI = OnError
    • Microsoft-Windows-Setup\DiskConfiguration\Disk    DiskID = 0
    • WillWipeDisk = true
  • Microsoft-Windows-Setup\DiskConfiguration\Disk\CreatePartitions\CreatePartition
    • Order = 1
    • Extend = true (this formats the entire disk)
    • Type = Primary
  • Microsoft-Windows-Setup\DiskConfiguration\Disk\ModifyPartitions\ModifyPartition
    • Active = true
    • Format = NTFS
    • Label = OS_Install
    • Letter = C
    • Order = 1
    • PartitionID = 1
  • Microsoft-Windows-Setup\ImageInstall\OSImage\    WillShowUI = OnError
  • Microsoft-Windows-Setup\ImageInstall\OSImage\InstallTo    DiskID = 0
    • PartitionID = 1
  • Microsoft-Windows-Setup\UserData    AcceptEula = true
  • Microsoft-Windows-Setup\UserData\ProductKey    Key = <product key>
    • WillShowUI = OnError
  • Microsoft-Windows-Shell-Setup\OOBE    HideEULAPage = true
    • ProtectYourPC = 3
    • SkipMachineOOBE = true
    • SkipUserOOBE = true
  • Microsoft-Windows-International-Core-WinPE    InputLocale = <Input Locale> (es 'it-IT')
    • SystemLocale = <System Locale> (es 'it-IT')
    • UILanguage = <UI Language> (es 'en-US')
    • UserLocale = <User Locale> (es 'it-IT')
  • Microsoft-Windows-International-Core-WinPE\SetupUILanguage    UILanguage = <UI Language>
  • Microsoft-Windows-Shell-Setup\AutoLogon    Enabled = true
    • LogonCount = 5
    • Username = Administrator
  • Microsoft-Windows-Shell-Setup\AutoLogon\Password    <strongpassword>
• Save the Answer File into the USB Key.

You are now ready to run your Windows Vista unattended installation with the following command:

setup /unattend: answerfilename

Note that Windows Vista setup program automatically looks for answer files named "AutoUnattend.xml" in several different locations including a previously cached answer file and an answer file at the root of a drive.

I gathered information below from compatibility and logo draft documents and from internet resources that are subject to change.

Official and detailed information about Vista compatibility and Compatibility Logo Programs is available at https://www.microsoft.com/windowsvista/ and https://microsoft.mrmpslc.com/InnovateOnWindowsVista/.

You can access the latest version of this document here.

FAQ on Windows Vista Certification Logos

Question: How does "Certified for Vista" differ from "Works with Vista" logo certification?

Answer: "Certified for Vista" applications must conform to "Windows Vista Software Logo Spec 1.1" and "Certified for Program TestCases" and must be submitted for verification to get the logo. "Works with Vista" is a self-asserted certification where companies just verify compatibility and pledge to support on Vista. "Works with Vista" may be obtained with no charge, apart the cost of the Verisign Certificate needed to create an account on winqal site.

Question: Does "Certified for Vista" logo certification include requirements about accessibility, application graphic layout and rendering behavior (ex. icon sizes, toolbars etc) as it did on the Certified for windows XP logo certifications?

Answer: No, most vista logo requirements only include Install/Uninstall, Reliability and Security Requirements (see. "Windows Vista Software Logo Spec 1.1" and "Certified for Program TestCases").

Question: When are Manifest and Digital signature required by Vista Logos?

Answer: Windows Vista 64-bit editions always require the "Digital Signature" for 64-Bit drivers

• "Certified for vista" software logo requires that all exe and drivers be signed with an authenticode certificate.
• "Certified for vista" software logo also requires that all exe and installers contain an embedded manifest with the declaration of appropriate UAC Execution level.
• "Works with vista" software logo doesn't require that exe and installers contain an embedded manifests or signature.

An authenticode certificate to comply this requirement may be acquired from a vendor like Verisign or it may be generated by means of a custom certificate authority (ex. The certificate authority included in Windows 2003).

Question: what happens if an application uses third party components or drivers?

Answer: if the application uses a third party component that is not signed the ISV can apply for a waiver on that particular file.

If an application cannot sign a kernel mode driver they must contact swlogo@microsoft.com and detail exactly why their driver cannot be signed.

An application can certify without printing functionality if the printing functionality is not part of its primary functionality.

Question: what happens if an application uses third party runtime that is not singed?

Answer: if the application uses a third party runtime that is not signed the ISV can apply for a waiver on that particular file.

This applies, for example, to VB6 applications as vb6 runtime files are not signed.

Question: Do Vista Logo requirements allow including "system components merge modules" into custom application setup programs?

Answer: Yes, as long as the merge modules for the system components are those published by microsoft.

Again, these solutions to these problems can be work out on a case by case.

Question: Do Vista Logos require "Uninstall Programs" to delete application files and shortcuts from existing user profiles?

Answer: logo verification steps require that appropriate clean up is performed on setup failure.

Use of the application may create files and folders on the user profile that are not required to be removed.

Question: Do Vista Logos require "Uninstall Programs" to remove database server and data files?

Answer: Roll back requirement refers to a failed install, not a standard install. The requirement is that if the msi installer fails to successfully install it must rollback and return the system to its previous state before the install began. For uninstallation which is not part of this test case, data can be left behind.

Question: Can VB6 and older Visual Studio applications apply for Windows Vista Logo certification?

Answer: Yes, as long as the application conform to logo requirements. Sometimes older runtime libraries violate security rules (es. Write access to WRP resources) or reliability rules (es. Fail APPVerifier checks). Migration to Framewark 2 is recommended but not required. In such cases, the ISV can contact swlogo@microsoft.com to work out solutions on a case by case basis.

Question: Do Vista Logo requirements allow using "Always Override" logic for a content file that cannot be versioned (es. a text file) installed by a setup program?

Answer: Yes.

Question: How is it possible to obtain certificates for signing exes and Dlls? How much does it cost?

Answer: Certificates can be obtained from third party Certificate Authorities such as Verisign or Geotrust. soft Windows Vista Software Logo Program Participants Only can get a certificate from Verisign at a discounted rate. For additional information see Microsoft Authenticode Code Signing Digital ID

Question: How is it possible to integrate Windows Error Reporting?

Answer: This is possible submitting your application and registering on the winqual site.
This requires using a product identifier tool that maps the applications unique identifier onto the Windows Error Reporting site. Every error submitted through WER is grouped by offending application. An ISV can then log on to the WER site and look at the reports for their applications.

Question: is it possible to certify Smart document applications or Add-ins?

Answer: No, If an application must run on top of another application, similar to a plug in, the application does not qualify as a native Windows Vista application. The application must be a native standalone project to qualify for the Logo program.

Question: how much does it cost submitting an application for Vista Certification?

Answer: testing fee for one application is about 1000 USD; If there is a failure in any of the tests the application will need to be corrected and resubmitted at an extra charge, typically around 500 USD per retest. This is why it is recommended to check that the application verify all Test Cases described in "Certified for Windows Vista Test Cases". Pretesting is available (but not required) at an extra cost through both VeriTest (Certified for Windows Vista) and Wipro (https://www.wipro.com/microsoft/certification/).

FAQ on Windows Vista Compatibility

Question: What areas may cause incompatibility for applications on Vista?

Answer: Most aspects are documented in "The Windows Vista Developer Story Application Compatibility Cookbook".

Important aspects include:

• User Account Control
• Windows Resource Protection
• Internet Explorer Protected Mode
• Session 0 Isolation
• Deprecated components
• Changes in the standard file system standard folders and applications location.

Question: Is there a list of the APIs that require Admin Privilege available?

Answer: No, an API may require Admin Privilege depending on the ACLs on objects it manages.

Question: Can .Net Applications address more than 2GB memory on Windows Vista 64bit?

Answer: .Net Applications can address 16TB memory on 64Bit platfrom (instead of 4GB).
However, there is a 2GB limit on the size of every single object you create.

Question: How is it possible to elevate an application to use Admin Privilege?

Answer: Elevation can be:

• forced by the developer adding a manifest to the application.
• forced by the user with "run as administrator" command.
• forced by the developer for a component using the "Elevation moniker".

         

Question: how is it possible to understand if a user is currently running with the filtered access token?

Answer: Use OpenProcessToken(GetCurrentProcess()…) and GetTokenInformation( … TokenElevationType… ); and check TokenElevationType:

• TokenElevationTypeDefault: The process user is not using a filtered token (i.e. it is a standard user, or UAC disabled for the user).
• TokenElevationTypeFull: The process has a split token and is running elevated.
• TokenElevationTypeLimited: The process has a split token and is running as standard-user.

Question: how is it possible to read all of the access tokens associated to a user (i.e. the filtered access token and the real user token)

Answer: use OpenProcessToken(GetCurrentProcess()…) and GetTokenInformation. To detect if the user has two tokens.

If you have 2 tokens and you are running with the filtered token, use the ShellExecuteEx() or the elevation moniker to run a process with the unfiltered token.

If you have 2 tokens and you are running with the unfiltered token, use the ShellExecuteEx() to run a process with the filtered token.

Question: What is Session 0 Isolation and how may it cause incompatibility to Windows applications and services?

Answer: In Vista, Services are run on an isolated and non-interactive session (Session 0).
In Vista, a new session is created for every user logging on.
So services never run on the same session of user applications

This is different from earlier OSs where the first logged on user is run within Session 0.

Question: How is it possible to understand if an "Access Denied" error message comes from WRP or from another source? 

Answer: You can attach Appverifier to the application and run a Luapriv check.  Look at the logs generated and you are looking for any message with a severity = "error" and that is accessing a WRP resource 

Question: Is the EXE manifest file used to specify UAC Execution Level the same manifest file used for COM side by side execution?

Answer: Yes, the manifest file can be embedded as an exe resource and it can also be used as an external file; having a manifest as an external file may not work for all scenarios.

Question: Is Restart Manager a Windows Vista only technology? Can setup programs using Restart Manager be used on Windows XP also?

Answer: Restart Manager is available for Microsoft Windows Server "Longhorn" and Windows Vista only. To use restart manager all you need is the MsiRMFilesInUse dialog. The presence of the dialog is simply ignored on down-level systems and the default "files in use behavior" is used.

Question: Is Windows Installer 4 Technology only available on Windows Vista? Does using installer 4 technology require that separate installs be created for windows XP?

Answer: Windows Installer 4.0 is available only on "Microsoft Windows Server Longhorn" and "Microsoft Windows Vista" but it doesn't change the msi database format. Using Windows Vista Only technologies such as the restart manager just lights up on Vista and doesn't require building a separate installation for down-level platforms (ex. Windows XP).

Question: How does Vista identify an Executable or application as a setup program?

Answer: Before a 32 bit process is created, the following attributes are checked to determine whether it is an installer:

• Filename includes keywords like "install," "setup," "update," etc.
• Keywords in the following Versioning Resource fields: Vendor, Company Name, Product Name, File Description, Original Filename, Internal Name, and Export Name.
• Keywords in the side-by-side manifest embedded in the executable.
• Keywords in specific StringTable entries linked in the executable.
• Key attributes in the RC data linked in the executable.
• Targeted sequences of bytes within the executable.
• Due to the intricate nature of install detection heuristics, the recommendation is to embed an application manifest in all executables with an appropriate requestedExecutionLevel to declare the desired runtime access. It is not recommend to try to avoid heuristics, as there might be some other matching characteristic of the binary that cannot be easily modified.

See Windows Vista Application Development Requirements for User Account Control Compatibility for additional information.

Question: Is Visual Studio 2003 supported on Vista?

Answer: Changes in Windows vista impact Visual Studio so that Microsoft is unable to provide support for Visual Studio .NET 2002 or Visual Studio .NET 2003 on Windows Vista. see Visual Studio on Windows Vista for additional information.

Question: Are Framework 1.0 and Framework 1.1 supported on Vista?

Answer: Framework 1.1 is a 32bit only technology and it is supported on windows Vista.

Framework 1.0 is not supported on windows Vista and Framework 1.0 assemblies may be run with Framework 1.1 runtime.

Question: How is it possible to integrate custom application data into Windows Vista Search?

Answer: Create an iFilter provider. How to Write a Filter for Use by SharePoint Portal Server 2003 and Other Microsoft Search-Based Products.

Question: How exactly is it possible to apply an application compatibility fix to an EXE and deploy it to corporate computers?

Answer: Generate a new SDB using ACT 5.0 "Compatibility Administrator" then invoke sdbinst.exe to install the custom sdb to machines system32 folder.

Question: how is it possible to embed a certificate into MsiPatchCertificate table.

Answer:see Understanding and Configuring User Account Control in Windows Vista.

Question: how is it possible to embed a certificate into an executable.

Answer:see Windows Vista Application Development Requirements for User Account Control Compatibility.

Thread pooling is important for scalability.
Pooling algorithms generally include complex logic for self tuning, queuing incoming requests, etc.
These are some of the reasons why I'd rather not bother writing thread pooling code in my services and leverage existing technologies instead.

In this article I provide a sample code for a service to receive messages from a MSMQ queue and process them by means of the ASPNet Thread Pool.
Similar code can be written to receive commands from a socket or some other source.

The code is divided into 3 sections:

• WAITING SECTION (PeekWorkItem() method): this section is used to enter wait mode ('peek mode') on the resource (the queue). Only one thread is allowed to enter 'peek mode' to avoid too many callbacks when a new message arrives. 'peek mode' is represented by flag _nIsPeekActive which is protected against concurrency.
  In case of exceptions (es. because of MSMQ service down) a new PeekWorkItem() is submitted with a delay to avoid 100% CPU loops on error conditions.
• WAKE UP SECTION (PeekCompleted() method): this section is run when a new message arrives on the queue. In this section a number of requests are submitted to the thread pool to read messages from the queue and process them.
  The code keeps count of the number of requests submitted to the thread pool (into variable _nReceiveCount). It is the thread pool choice to really process those requests concurrently or to serve them in a more scattered way. In case of exceptions (es. because of MSMQ service down) a new PeekWorkItem() is submitted with a delay to avoid 100% CPU loops on error conditions.
• RECEIVE SECTION (ReceiveWorkItem() method): this section is the work item run by the thread pool that reads messages from the queue and process them.
  Code sections run on a thread pool should never be 'too short' or 'too long' as that may waste resources. So, ReceiveWorkItem() is designed to read and process as many messages as possible up to a configurable timeout (or the queue becomes empty).

Figure 1: MSMQ Custom Listener engine

WAITING SECTION (PeekWorkItem() method)

IO mechanisms generally provide asynchronous ways of receiving data.
This saves the need of creating threads just to wait on them.
In facts, there are cheaper ways to wait on empty resources, which are usually exposed as Asynchronous APIs.
.Net MSMQ asynchronous API to read messages from a queue is Queue.BeginPeek() method.

PeekWorkItem() is the code section where my service calls _queue.BeginPeek() method to look for messages into the MSMQ queue.
_queue variable refers to the queue the service is looking for messages from.

Before calling _queue.BeginPeek(), PeekWorkItem() sets flag _nIsPeekActive to 1 (i.e. it enters 'Peek mode').

_nIsPeekActive flag keeps track that somebody is waiting on the queue and avoids other threads call BeginPeek() on it.
This reduces the number of callbacks that are processed when new messages arrive on the queue.
 

If _queue is empty, _queue.BeginPeek() doesn't block the calling thread.
So, no thread objects are wasted waiting on empty queues and no threads are kept blocked for long time.
That's why, PeekWorkItem() may safely be run on thread pool threads.
 

#region PeekWorkItem

/// <summary>submits a BeginPeek() to the queue.</summary>

/// <remarks>only one BeginPeek() may be active on the queue at any moment.

/// BeginPeek() invocation is protected by the synchronized flag 'IsPeekActive' which is set just before calling it and reset at the PeekCompleted() callback.

/// In case BeginPeek fails (ex. because of invalid access or any other reason) 'IsPeekActive' is also reset and a delayed peek is submit to check whether it may succeed at a later time.</remarks>

public void PeekWorkItem() {

try {

// Try a BeginPeek() on the queue; if _nIsPeekActive==1 somebody else is in peek mode, so, PeekWorkItem does nothing...

if (Interlocked.CompareExchange(ref _nIsPeekActive, 1, 0) == 0) {

// Entering Peekmode: Only one thread is allowed to call BeginPeek() to prevent too many callbacks when a new message arrives

_queue.BeginPeek(MessageQueue.InfiniteTimeout, null, new AsyncCallback(PeekCompleted));

}

} catch (Exception ex) {

// Submit a delayed PeekWorkItem() and ignore the exception.

// BeginPeek may fail when there are problems with the MSMQ service.

// The new PeekWorkItem() is delayed to avoid an loop with 100% CPU in such case.

ICall call = new DelayedCall(new DelegateCall(new VoidEmptyArgsDelegate(PeekWorkItem)), _nPeekDelayOnException);

// Release _nIsPeekActive flag: the next PeekWorkItem() will be allows to enter peekmode!

Interlocked.Exchange(ref _nIsPeekActive, 0);

// Start the delayed PeekWorkItem() and save the call object into '_call member'.

// if '_call member' already contains a delayed call object dispose it to have only one active call object at a time

call.Invoke();

IDisposable oldCall = Interlocked.Exchange<ICall>(ref _call, call) as IDisposable;

if (oldCall != null) { oldCall.Dispose(); }

}

}

#endregion

WAKE UP SECTION (PeekCompleted() method)

The wake up section is run when new messages arrive on the queue (or, exceptional conditions happen waiting on it).
In facts PeekCompleted() method was registered as a callback on the queue, in the 'waiting section', when calling BeginPeek() method.
Remember that only one thread is allowed to set _nIsPeekActive flag and call _queue.BeginPeek().
So, only one callback is run when messages arrive on the queue.

Just after wake up, PeekCompleted() calls _queue.EndPeek() to check whether new messages are available on the queue.

If _queue.EndPeek() succeeds, there are messages available so, PeekCompleted() submits some requests for processing them (i.e. delegates to ReceiveWorkItem()), to the thread pool.
PeekCompleted() doesn't use the message returned by _queue.EndPeek(); in facts, it is the Receive section that chooses how to receive messages, whether to share the receive transaction with the message processing etc.

PeekCompleted() keeps count of the requests currently submitted to the thread pool by means of variable _nReceiveCount.
In facts, PeekCompleted() increments _nReceiveCount every time it submits a request for processing to the thread pool and, in turn, _nReceiveCount is decremented every time a thread pool request completes.    

PeekCompleted() makes sure that requests for message processing are sent to the thread pool with _nReceiveCount up to a configurable limit (held in variable _nMaxConcurrency).

PeekCompleted() releases always _nIsPeekActive flag: it does that before sending the last request for processing or in the finally block.
This allows a new thread enter 'peek mode', when the queue becomes empty or some receive work item complete for other reasons (e.g. by timeout).

In case of exceptions there might be problems on the queue so (after releasing _nIsPeekActive flag) PeekCompleted() submits a delayed request to PeekWorkItem() (the waiting section).
The delay is necessary to prevent a CPU overhead on the PeekWorkItem()/PeekCompleted() loop during the exceptional condition.

#region PeekCompleted

/// <summary>PeekCompleted is triggered when a BeginPeek() is active and any message is received by the queue.</summary>

/// <remarks>PeekCompleted resets 'IsPeekActive' and submits an appropriate number of ReceiveWorkItems to the thread pool for receiving messages.

/// ReceiveWorkItems are submitted according to the concurrency configuration setting for the queue.

/// currently active ReceiveWorkItems can be monitored by means of 'Requests Current' performance counter.</remarks>

public void PeekCompleted(IAsyncResult asyncResult) {

bool bPeekActiveReset = false;

try {

// any BeginPeek() must be matched with an EndPeek()...

Message msg = this._queue.EndPeek(asyncResult);

// "Submits receive requests up to the maximun concurrency allowed; current ReceiveCount '_nReceiveCount', Max Concurrency '_nMaxConcurrency'."

int nReceiveCount = Interlocked.CompareExchange(ref _nReceiveCount, -1, -1); // access to _nReceiveCount is sinchronized to allow

for (; nReceiveCount < _nMaxConcurrency; ) {

nReceiveCount = Interlocked.Increment(ref _nReceiveCount);

if (nReceiveCount <= _nMaxConcurrency) {

// "Releases the peekactive mode before sending the last ReceiveWorkItem()"

if (nReceiveCount == _nMaxConcurrency) { Interlocked.Exchange(ref _nIsPeekActive, 0); bPeekActiveReset = true; }

ThreadPool.QueueUserWorkItem(new WaitCallback(ReceiveWorkItem), this);

}

}

} catch (Exception ex) {

// trace a warning and submits a delayed PeekWorkItem() in the finally block.

} finally {

if (bPeekActiveReset == false) { // Submits a delayed PeekWorkItem()

ICall call = new DelayedCall(new DelegateCall(new VoidEmptyArgsDelegate(PeekWorkItem)), _nPeekDelayOnException);

Interlocked.Exchange(ref _nIsPeekActive, 0);

call.Invoke();

IDisposable oldCall = Interlocked.Exchange<ICall>(ref _call, call) as IDisposable;

if (oldCall != null) { oldCall.Dispose(); }

}

}

}

#endregion

RECEIVE SECTION (ReceiveWorkItem() method)

ReceiveWorkItem() is run on the thread pool to receive messages from the queue and process them.
Code sections run on a thread pool should never be 'too short' or 'too long' as that may waste resources.
For this reason ReceiveWorkItem() is designed to process multiple messages up to a configurable time interval (_nWorkItemTimeLimit variable) or the queue becomes empty.

if the queue becomes empty, every running ReceiveWorkItem() completes and (only) the last one processes a PeekWorkItem() to enter 'Peek Mode' on the queue.
Also, PeekWorkItem() is processed when a ReceiveWorkItem() completes because of _nWorkItemTimeLimit expiration.
if a ReceiveWorkItem() completes because of an exception receiving or processing a message, the call to PeekWorkItem() is delayed to limit CPU overhead in exceptional conditions.

In case of failures processing a message, ReceiveWorkItem() tries to save the failed message to a '_failedmessages' queue by means of hlpSaveDeadMessage() method.

#region ReceiveWorkItem

/// <summary>ReceiveWorkItem is processed on the thread pool to receive messages from the queue and process them.</summary>

/// <remarks>ReceiveWorkItem retrieves messages according to the configuration and process them.

/// in case of a processing failure a message, ReceiveWorkItem() tries to save the failed message to the '_failedmessages' queue.

/// the move of the message to the failedmessages queue is performed with the same transactional settings used for retrieval from the original queue. in case of failure saving the message to the failedmessages queue the original message is restored to the original queue ONLY if the queue is configured for use of external transactions.</remarks>

public void ReceiveWorkItem(object oThis) {

int nReceiveCount = 0; bool bIsEmpty = false; bool bDelayPeekWorkItem = false; string sMessageId = null;

try {

// Receives a message from the queue '_queue' with transaction mode '_transactionType' (ReceiveCount: '_nReceiveCount')

DateTime dtExpiration = DateTime.MinValue; if (_nWorkItemTimeLimit >= 0) { dtExpiration = DateTime.Now.AddMilliseconds(_nWorkItemTimeLimit); }

for (bool bTimeLimitExceeded = false; bIsEmpty == false && bTimeLimitExceeded == false; sMessageId = null) {

Message oMessage = null; bool bIsReceiveCommitted = false;

try {

// creates a transaction scope for use of external transactions. A null transaction scope is used otherwise");

TransactionScope oTransactionScope = (_transactionType == MessageQueueTransactionType.Automatic) ? new TransactionScope(TransactionScopeOption.RequiresNew) : null;

using (oTransactionScope) {

try { oMessage = _queue.Receive(TimeSpan.Zero, _transactionType); }

catch (MessageQueueException tex) {

if (tex.MessageQueueErrorCode == MessageQueueErrorCode.IOTimeout) { bIsEmpty = true; }

else { throw; }

}

if (_transactionType != MessageQueueTransactionType.Automatic) { bIsReceiveCommitted = true; }

if (oMessage != null) {

// Processing message 'oMessage.Id'

sMessageId = oMessage.Id;

if (_formatter != null) { oMessage.Formatter = _formatter; }

ServiceMessage oServiceMessage = (ServiceMessage)oMessage.Body;

ServiceMessage oRetMessage = hlpProcessMessage(oServiceMessage);

}

if (_transactionType == MessageQueueTransactionType.Automatic) { oTransactionScope.Complete(); }

}

} catch (Exception ex) {

if (sMessageId == null) { throw new CommunicationEngineTargetException(string.Format("Exception '{0}' - '{1}' occurred receiving a message from queue '{2}' and the message ID could not be retrieved; the message cannot be saved to the deadqueue. If the queue is not configured with automatic transactions the message may be lost.", ex.GetType().Name, ex.Message, _queueConf.FormatName)); }

bIsEmpty = hlpSaveDeadMessage(sMessageId, oMessage, ref bIsReceiveCommitted);

if (bIsReceiveCommitted == false) { bDelayPeekWorkItem = true; return; }

}

if (_nWorkItemTimeLimit >= 0 && dtExpiration <= DateTime.Now) { bTimeLimitExceeded = true; }

}

} catch (Exception ex) {

// "Exception occurred receiving a message from queue '_queue'; message id is 'sMessageId'

bDelayPeekWorkItem = true; // "Ignores the exception. and run finally block

} finally {

nReceiveCount = Interlocked.Decrement(ref _nReceiveCount);

PublishEndRequestCounters(_queueConf);

if (!bIsEmpty || nReceiveCount == 0) {

if (!bDelayPeekWorkItem) {

// Submits a PeekWorkItem()

PeekWorkItem();

} else if (nReceiveCount == 0) {

// Submits a delayed PeekWorkItem()

ICall call = new DelayedCall(new DelegateCall(new VoidEmptyArgsDelegate(PeekWorkItem)), _nPeekDelayOnException);

call.Invoke();

IDisposable oldCall = Interlocked.Exchange<ICall>(ref _call, call) as IDisposable;

if (oldCall != null) { oldCall.Dispose(); }

}

}

}

}

#endregion

20060908 Write your services leveraging existing thread pool technologies Figure 1.JPG