MSDN Utopia

Distributed Cache Capabilities - Notifications

SalvaPatuel — Sat, 03 Dec 2011 07:26:16 GMT

A notification system allows the cache to interact with external applications. This attribute can be useful on scenarios where certain activities, for example workflows, need to know the state of an item.

Capability	Description	Weight
Cache item notification	A cache item notification system will allow clients to subscribe to any event associated with a specific cache item. These notifications are usually handled by a disconnected proxy that allows subscribers to hold an open channel with the notification service. The cache item is usually decorated with a reference to the notification tables and signals the entry when the item is modified, accessed or deleted.	2
Guideline	This feature can be useful in scenarios where the cache is used as a pub/sub storage mechanism. But only use this feature if the amount of clients is below 25 as there is a performance penalty when using a notification system that can drag the update times to unacceptable levels. If this model is required consider using cache regions to minimize the overhead.
Capability	Description	Weight
Group Notification	Group notifications are an advanced version of cache item notification. This model allows grouping several items into a single notification model or to attach a notification to the cache region or cache segment (similar to a table or database notification in the database world).	2
Guideline	This feature can be useful in scenarios where the cache is used as a pub/sub storage mechanism. But only use this feature if the amount of clients is below 25 as there is a performance penalty when using a notification system that can drag the update times to unacceptable levels. If this model is required consider using cache regions to minimize the overhead.
Capability	Description	Weight
Custom notifications	Some caches provide a custom notification extensibility model where system developers can create custom notifications. This capability is usually implemented with a transparent proxy from the notification system to the external plug-in (usually implemented through inversion of control).	3
Guideline	This capability will allow developers to create their own notifications, this can involve external systems. It is recommended to use async operations when notifying if the cache uses the notifications model synchronically.

Distributed Cache Capabilities - Data Integrity

SalvaPatuel — Sat, 03 Dec 2011 07:17:00 GMT

Data expiration contains elements that make the cache allocation more efficient. The feature removes items that are not current from the cache, freeing the memory and the resources associated.

Capability	Description	Weight
Transaction support	A cache that supports transactions will allow grouping actions and committing them at the same time. Usually the cache can be enlisted as a distributed transaction under technologies like DTC. The cache will provide BEGIN, COMMIT and ROLLBACK functionality. Usually this feature is implemented using a separate transaction log.	3
Guideline	The designer needs to consider if this feature is needed, first, this feature is usually quite expensive, not only on cache price tag but in performance. A designer should ask itself if a database will not be able to provide the functionality that requires, as they are transaction aware by design. If this feature is needed then try to avoid deadlock situations as cache servers are not very good detecting them when working in a very high distributed environment. We will recommend to analyze other options and if a transactional cache is needed do perform exhaustive testing.

Capability	Description	Weight
Multi-threading support	A multi thread environment aware cache will allow separate threads to access the same cache item without entering in race conditions while writing the item. Usually this capability implements an internal lock over the item only (note that some caches may use a table or region lock). Certain caches will also offer a capability to lock several cache items at the same time.	5
Guideline	This is simple, if a cache does not provide proper multithreading support just don’t buy it. Is within the nature of distributed caches to support multiple connections accessing the same information and if it does not support it then is not worth the investment.

Capability	Description	Weight
External dependencies	This capability allows cache items to have external cache items dependencies, for example files. This allows cache items to be either backed up or extended through a connection to an external source. Quick recovery caches use this technique to recover from faults.	2
Guideline	This feature has been very useful in the past, when 32 bits processes where running out memory pretty fast. Designers have implemented a secondary storage, for example a file to optimize the cache (using the cache as an index entry). With the new 64 bits cache the memory can be plentiful but still consider this feature if you need to allocate large amount of data in the cache (for example photos, maps, videos).
Capability	Description	Weight
SQL Server synchronization	A cache may allow a cache item, cache region or table to be synchronized with a SQL Server database. This will mirror each entry to a database entry. This back up repository is usually accessed when a fault occurs or when a database item has changed and the cache needs to reflect the changes (instead of evicting the item).	3
Guideline	Losing a cache can be a big event in certain scenarios or if the system may be accessed by other systems that are not cache aware then a SQL dependency can be very helpful. We have seem dozen of scenarios where the data is replicated from a another location, i.e. a call center, and the data in the cache needs to be modified to stay current. There is a performance implication but experience says that is a very useful feature if those scenarios are considered.
Capability	Description	Weight
Oracle Synchronization	A cache may allow a cache item, cache region or table to be synchronized with an Oracle database. This will mirror each entry to a database entry. This back up repository is usually accessed when a fault occurs or when a database item has changed and the cache needs to reflect the changes (instead of evicting the item).	3
Guideline	Same guideline as above, in this case if your data is going to be supported by Oracle.
Capability	Description	Weight
Other DB synchronization	A cache may allow a cache item, cache region or table to be synchronized with other vendors database (i.e. Informix, Ingress, DB2 no name a few). This will mirror each entry to a database entry. This back up repository is usually accessed when a fault occurs or when a database item has changed and the cache needs to reflect the changes (instead of evicting the item).	3
Guideline	Same guideline as the SQL dependency but in this case consider caches with heterogeneous data models if you have scenarios with multiple database technologies utilization, therefore use it only when needed. Having individual gets on the hand will require a separate call per retrieval, affecting the network and cache performance.

Distributed Cache Capabilities - Data Expiration

SalvaPatuel — Mon, 21 Feb 2011 07:34:12 GMT

Data expiration contains elements that makes the cache allocation more efficient. The feature removes items that are not current from the cache, freeing the memory and the resources associated. The capabilities list looks like this:

Capability	Description	Weight
Absolute time expiration	If the cache item has a property to include a specific date and time where the item will expire this will be removed automatically from the cache once the time has arrived. The cache will probably have a secondary monitor that scans each entry and in certain occasions, if the cache is advanced enough will just check the timespan dictionary.	4
Guideline	This feature is quite important, when the client stores an item on the cache it will probably know for how long the object is current. Having some kind of expiration will allow architects and developers to keep a healthy cache. If this feature is not present the system can run easily into resource starvation, as the cache will grow without control.

Capability	Description	Weight
Idle time expiration	When a cache has this capability the cache item will be removed from the cache if the item hasn’t been accessed in a period of time (usually configurable). This model implements a frequently access cache architecture.	2
Guideline	This feature is useful when you have a very dynamic cache, for example when you want to store only the most accessed objects. The cache will only keep alive objects that are really shared and frequently used. In certain scenarios this type of behavior is tracked as it can give the developer information about access patterns.

Capability	Description	Weight
Cache item evictions	Cache evictions occur when the cache removes item arbitrarily, this can happen when the cache is under memory pressure or other conditions that affect the storage stability. A cache with this feature will usually provide the developers the possibility of configuring the eviction rules.	4
Guideline	If the cache does not have expiration at least it should have evictions. If neither feature is present is probably a bad cache choice therefore pay special attention to the combination of these features. A growing cache will hit a resource limit sooner or later, with 64bits the memory barrier has been moved quite far but you may still have certain exponential search time (annexed tables, relationships or any other index that cannot be hashed) that will consume CPU. Some kind of eviction is usually recommended.

Capability	Description	Weight
Priority evictions	If the cache implements priority eviction rules it will allow developers to assign a priority (usually a range) to an item, this will protect the item from arbitrary evictions when the cache is under pressure. Note that a priority eviction will not prevent the cache to remove the item if necessary, it will just follow an order.	2
Guideline	If the cache has an eviction model and the data that the system needs to store is critical then an eviction rule like priority can help the designers to have some kind of protection against random evictions. Consider this feature if the system really has different cache items priority mode.

Capability	Description	Weight
LRU/LFU eviction	This is an advanced mode of eviction that combines the standard eviction with idle time expiration. The eviction can be configured using the least recently used (LRU) or least frequently used (LFU) in order to prevent useful cache items from being evicted from the cache.	2
Guideline	This is a more sophisticated eviction model and should be used if eviction is needed as it will keep hot cache items loaded when under pressure. Note that this logic will usually incur into performance penalties as the cache needs to track the history of the cache item.

Capability	Description	Weight
Eviction policy	This feature is similar to the priority eviction but with an extra feature, marking cache items as “protected”. This means that the cache will not remove under any circumstance the stored item, even if is under pressure.	2
Guideline	Eviction protection surplus the priority eviction, this feature is even better for critical data storage. The cache can guarantee that the item will not be evicted.

The average weight of this category is 2.67

Distributed Cache Capabilities - Data Organization

SalvaPatuel — Mon, 21 Feb 2011 07:27:50 GMT

Here is the third installment, this time focusing on the data organization.

Capability	Description	Weight
Cache Groups	This capability allows caches to be partitioned in groups; this allows cache subdivision that can help the designers to organize the cache allocations across multiple devices. Some caches allows further grouping using group trees.	5
Guideline	Cache groups can be utilized as “tables”. They are very useful to organize the data allocated in the caches. For example, you can use a cache for “Customers” and another one for “Orders”. This type of organization is essential for large caches and should be only ignored if the scenario stores a single type of object (i.e. web session information).

Capability	Description	Weight
Tags support	Supporting tags allows designers to add metadata to the cache items, this feature is usually implemented in conjunction with query trees in order to filter cache items. A cache can allow one or multiple tags per item.	3
Guideline	Adding metadata to a cache item is an interesting feature when the scenario requires rich retrieval models. This is required when you need to use query languages to retrieve items. We recommend limiting the size of the tags and favoring blittable values as they will be indexed. There is an extra hop when reading and writing that can affect performance therefore only used when required.

Capability	Description	Weight
Cache items dependencies	The cache items can be completely independent or in certain technologies can have a dependency on other ones. This is not a relational model as described on other capability but is rather a content dependency that is usually implemented by eviction or notification policies. A cache item with a dependency will relay on the other item to provide the information, it can not leave with the other part.	1
Guideline	The cache item dependency can be very useful when an item is not useful without the other one, for example if you split the orders across single entries, all of them can be related to create an “order”. Do not confuse with relationship models as this applies to completeness rather than relation. This is a faster alternative to relationship models and are usually very fast , as they use tail reference to locate the items.

Capability	Description	Weight
Close-to-caller capability	This feature allows cache to organize data based on taxonomies that can be allocated on different physical repositories. This means that a cache can store certain items on devices near the consumers that need that specific category of items. This allows fast access when distributing across large networks.	2
Guideline	A cache with the ability to configure where the items can be stored (physically) can be an interesting feature if you use multiple geographies. For example if you use session state objects you may want them closer to the web servers but maybe information about products or customer details may be located in a different server array. In certain cache technologies, this will be implemented using local caches on each client. This capability is useful only on large cache deployments.

Capability	Description	Weight
Cache items versioning	This capability allows cache items to be versioned, this means that the cache will store historic information about an item. The cache item is commonly decorated with a version number that increases on each update. There is usually a retention policy and the longer the policy the more space required.	2
Guideline	There are only few scenarios where this capability is useful, one of them is when you need to keep the change history of an item, maybe when storing documents or other type of complex types that requires rollback capabilities. Another scenario is when you need to generate a comparison to analyze what has changed from one version to the other one. Keep in mind that this model has an exponential growth model and can consume a lot of memory, therefore make sure that you run 64bits version of the cache service before using this capability.

The average weight for this category is 2.60

Distributed Cache Capabilities - Data Manipulation

SalvaPatuel — Tue, 01 Feb 2011 06:50:57 GMT

This is the second entry, data manipulation. Each one of them have a description and a generic guideline based on the experience of our implementations. I have added a weight, may look arbitrary but is based on the average importance on our projects.

Capability	Description	Weight
Atomic operations	Usually implemented through Get/Set. Allows clients to read and write operations in a single and atomic operation preventing dirty reads/writes safety, without getting into race conditions under multi-threaded environments.	5
Guideline	This feature is highly recommended, we would advise to avoid any cache that does not contain atomic operations as is a fundamental part of a multi connection environment. The only scenario where this can be acceptable is if you are not concern about dirty reads.

Capability	Description	Weight
Multi Gets	Multi Gets allows clients to retrieve multiple cache items within the same operation. The return type is usually an array of objects rather than complex types (i.e List, Hashtables)	3
Guideline	If you are planning to retrieve multiple items at the same time then this is an interesting feature as reduces the network traffic as well as the thread utilization levels, but will increase the CPU utilization, therefore use it only when needed. Having individual gets on the hand will require a separate call per retrieval, affecting the network and cache performance.

Capability	Description	Weight
Multi Writes	Multiple writes allow clients to send multiple cache items to the cache system at the same time. The cache then will navigate through the items and update accordingly. During this operation, the order is usually not guaranteed.	3
Guideline	Do not confuse this with bulk insertion; the idea is to use the same call to send an array of items. This can be useful on scenarios that require multiple cache entries. For example, this won’t be required if you just store a single model per operation but very useful if you store the changes on the client and then update them at the same time once the full process has finished. This can help reducing the network traffic and thread utilization but usually requires extra CPU time and may produce queues.

Capability	Description	Weight
GetAndSet/SetAndGet operation	These operations are usually known as “double-shooters”, the cache receives a request to perform two operations at the same time (i.e increment and read result), usually involving a longer lock. These operations avoid race conditions on a multi-threaded environment.	2
Guideline	If the cache allows different operations like increment, decrement or bitwise operations it can help to solve certain scenarios where you need to maintain unique values across all your client instances (i.e. the amount of clients connected). If the value is not shared across clients then is preferable to calculate first on the client and then send a standard “set” to the cache, as it can affect concurrency.

Capability	Description	Weight
Relationship/Union capability	This capability allows cache items to be related to other cache items or even cache groups. For example an item that contains an order may be related to other cache item that stores the customer information. This relationship is usually maintained in a different cache table and is subjective to multi-locks, as the items need to be queried from the relationship table rather than a hash entry.	2
Guideline	Use this feature with care, the relationship model will emulate a relational database and only should be used if you have requirements to do so. The main reason is the extra hop that the items need to go through every time they are accessed. The relationship table will use a lock as well that can affect concurrency.

Capability	Description	Weight
Ordering capability	This capability offers the clients the ability to get results in a specific order, this also applies to writes. The items are usually adorned with a correlation token that will be used to process the items in a deterministic order.	1
Guideline	This feature is usually interesting when you need to save items in a particular order. Having said that this can be easily emulated by the clients as they can send the items in a specific order if there is any relationship involved, on the reading side, you can order on the client using technologies like LINQ. We recommend that the cache items should be independent from each other and this feature should be avoided.

Capability	Description	Weight
Async operations	Asynchronic operations will disconnect the relationship between the client and the server. This allows client to fire a request and continue working rather than waiting for a response from the cache. This usually involves two modalities, “fire and forget” where the client does not receive any notification or an “async notification”, where the client will receive an event once the operation has been completed.	4
Guideline	This feature is very useful if you are working on real time scenarios, where the fire and forget mode comes very handy. Caches should never be treated as the “only” source but rather a performance enhancement for the application therefore reading and writing from the cache helps but is not critical, this means that a fire and forget model works very well on those scenarios.

Capability	Description	Weight
Bulk operations	Bulk operations allow caches to load massive chunks of data at the same time (in certain cache technologies this also includes dumping the cache in a single operation). Usually this supports fast cache recoveries from faults, when other media is used to back up the cache.	3
Guideline	If you are treating the cache as critical then you will need a backup support, for example a file or a database. The bulk operations will allow you to “fast-load” your cache in a recovery event from the last good known state. This will also allow you to warm up the cache, making the cache more efficient.

Capability	Description	Weight
Object Query Language support	Object query language (OQL) is a SQL style language that allows you to search cache items in a database similar fashion, using SELECT, FROM, WHERE you can filter your cache items before retrieving them.	2
Guideline	A query language can help you to quickly filter cache items when you need several items at the same time. This model can search usually on cache items tags or attributes. This feature is useful if you store items individually rather than grouped (i.e. if you store all the orders on different slots instead of storing an “order” object with all the orders embedded). Note that you may need to know the OQL language beforehand or have some SQL syntax skills.

Capability	Description	Weight
LINQ support	LINQ is a Microsoft .NET technology that allows you to query objects based on a projection tree. This allows you to search cache items in a database similar fashion, using SELECT, FROM, JOIN, WHERE, ORDERBY you can filter your cache items before retrieving them.	2
Guideline	A query language can help you to quickly filter cache items when you need several items at the same time. This model can search usually on cache items tags or attributes and works very well with .NET solutions. This feature is useful if you store items individually rather than grouped (i.e. if you store all the orders on different slots instead of storing an “order” object with all the orders embedded). Note that you may need to know the LINQ language beforehand or have some SQL syntax skills.

Capability	Description	Weight
Parallelism support	Parallelism algorithms will allow the cache to take the most from multiple CPU cores, this can make retrieval operations faster as the cache can schedule filters on different thread queues. In certain caches, this feature is also used during write operations, when multiple writes are allowed or bulk inserts are processed.	4
Guideline	If the scenario involves queries (OQL, LINQ) or relationship retrievals then the parallelism capability becomes really important. As most servers will have more than one core this feature should be a must as will allow systems to scale up better. Note that it can also add complexity to the cache and as mentioned on the memory management this can be tricky for certain cache vendors unless they use proven frameworks like the PFX or OpenMP.

Capability	Description	Weight
Streaming	Streaming allows caches to send the cache items in chunks. This is usually available for large cache items that are not suitable for bulk transition (i.e. a video). The cache opens a channel with the client and sends the data in blocks, allowing other connections to process the requests without blocking the thread pool.	2
Guideline	This feature is only useful on specific scenarios where large data will be used, for example, music, videos or large images. The streaming will allow the client to receive data blocks that needs to be processed by the client. Only use streaming if you really have objects larger than 10Mb.

The global weight for this category is 2.75

Distributed Cache Capabilities - Data Storage

SalvaPatuel — Mon, 31 Jan 2011 08:42:51 GMT

In this first installment we are going to share the data storage attribute and its capabilities. Each one of them have a description and a generic guideline based on the experience of our implementations. I have added a weight, may look arbitrary but is based on the average importance on our projects.

Capability	Description	Weight
.NET Types support	Does the cache support .NET types natively? Some caches may accept any type of data types but if they are not .NET aware they will incur in a performance penalty during the serialization process.	5
Guideline	If your clients are going to be mainly .NET you must consider using a cache that supports .NET natively. Even if 50% of your clients use .NET we recommend to select a cache that can support it as the .NET transition is the most expensive if we compare it against native objects or java objects.

Capability	Description	Weight
.NET Session state support	Some caches have native support for web session states, they may provider also a native provider that you can link up to your web application, saving development time.	4
Guideline	This is an extremely useful feature if the cache is going to be used as distributed state management storage as it can provide an API to reference in your project saving a lot of plumbing code. Distributing the session state will remove the dependency to load balancer affinity models for web applications.

Capability	Description	Weight
.NET View state support	Web pages view state can take extra processing time, some caches address this issue integrating a view state provider that handles, caches and distribute this raw types.	1
Guideline	Although it can be useful in certain scenarios, this can be simulated with a standard string entry on the cache. Do not weight your selection decision on this factor as is a replaceable feature that won’t require a lot of code.

Capability	Description	Weight
Raw types (native)	This capability provides support for raw types, usually binary arrays that can store any type of content. Some caches have limitations on the storage and will not allow this type of writing.	2
Guideline	Consider native storage when you need to store heterogeneous data types, for example mixing .NET objects, Java objects, pictures, videos or circular arrays.

Capability	Description	Weight
Java types support	Some caches support native java objects, this will allow clients to access the objects without intermediate transformation, increasing the read/write performance.	5
Guideline	If your clients are going to be mainly java then considering a cache with this capability can make a real difference. Java objects are very lightweight but there is still some processing involved when using a native storage rather than a java-aware one.

Capability	Description	Weight
Maximum storage size	Each cache item may have a maximum storage size, this can limit the type of information that can be stored on it. There is another aspect of maximum storage size and is the total cache size, usually related to the CPU bitness.	4
Guideline	Some caches offer ultra-fast read/write capabilities, this is usually related to the size constrains. If you are expecting to store less than 1Mb per item then go for the fast option, as the allocation is very efficient, but note that this type of cache is usually implementing a native storage, this can affect the performance if you use .NET/Java objects. On the other hand, if you need to store large objects make sure that the cache can support it.

Capability	Description	Weight
Managed memory storage	Usually caches that are written in .NET will use a managed heap to store the cache items. These objects will be governed by a garbage collection that can affect the total cache size.	2
Guideline	This can be a good capability if you are using .NET clients and .NET types, the garbage collection is designed to understand these objects and is very efficient doing so. If you are going to use other types of objects then the collector will use the pinning model, but relays on the cache to apply memory pressure to those objects (this can lead to fragmentation).

Capability	Description	Weight
Native memory storage	Caches that are written using native code will use the default heap or the C runtime heap. The cache will manage its own memory and depends on its ability to properly avoid fragmentation.	2
Guideline	If your clients are going to be native or there is a scenario with mixed technologies this can improve performance, but keep in mind that the cache vendor has to manage its own memory, this usually requires different skills that some companies have not master. Using managed memory managers like the .NET or Java is a good alternative as both technologies are well mature in this area.

Capability	Description	Weight
Load balancing	Some caches will load balance the storage automatically between several cache instances, this is usually achieved using a balancer in conjunction with a router. When a request arrives the router will point to the right cache, where the data is located.	3
Guideline	Load balancing technologies can be very good to reduce the cache load in certain scenarios, usually is implemented in conjunction with data partitioning, which can achieve similar results. If you are going to use a partitioning topology then this feature is not really important, only consider it if you are planning to use replication or mirrored topologies.

The average weight for this category is 3.11

Distributed Cache Capabilities guideline

SalvaPatuel — Mon, 31 Jan 2011 07:50:00 GMT

In this series, I am going to publish a series of guidelines that can be used when selecting a distributed cache technology. The series will be divided in the main caching attributes, each of them containing a list of capabilities. The main categories that I will present are the following ones:

Data Storage: This attribute compares the cache based on their storage capabilities, limitations and format supported. This item is beneficial to companies exploring specific scenarios where these caches may be useful.

Data Manipulation: This attribute describes the data manipulation operations available for each cache. The capabilities enlist different techniques that may be available for each solution that can be aligned with specific scenarios.

Data Organization: This attribute describes how the data can be organized inside the cache. Organizing data on different groups or nodes can improve the data taxonomy architecture, allowing allocation strategies that can not only improve the readability but the performance of the cache.

Data Expiration: Data expiration contains elements that makes the cache allocation more efficient. The feature removes items that are not current from the cache, freeing the memory and the resources associated.

Data Integrity: Data expiration contains elements that make the cache allocation more efficient. The feature removes items that are not current from the cache, freeing the memory and the resources associated.

Notifications: A notificacion system allows the cache to interact with external applications. This attribute can be useful on scenarios where certain activities, for example workflows, need to know the state of an item.

Cache Topology: The cache topology describes the way that a cache can be implemented across different processes. This attribute is useful in order to understand how the cache can be implemented within the system architecture and to calculate the cost associated to it.

Security: Security can be a critical factor if the cache is shared across a mixed security platform. This attribute analyzes the security and privacy capabilities that the cache provides by default.

Cache Integration: This attribute analyzes the different capabilities related to 3rd party integration. Integrating natively with other software or architecture can improve the cache performance and will reduce the risk of using custom adapters.

Platforms and protocols: This attribute describes the cache platform supportabilty, native technologies used internally and the available protocols to access the data. This item should be considered in scenarios where you may have mixed environments.

Cache Infrastructure: This area explores the different cache infrastructure deployment models. This section is usually analyzed by the infrastructure team in order to understand what kind of kit is necessary to run the distributed cache system.

Management tools: Understading what is inside the cache is a useful feature when memory pressure appears on the distributed cache system. External management tools can help to diagnose what the cache is storing, offering online solutions that can reduce downtime.

All the items will be weighted based on importance, this guide can be used to build comparison matrices, we have done this exercise several times, specially comparing AppFabric Cache, NCache, ScaleOut and MemCached.

.NET Hidden Gems - Memory fail points

SalvaPatuel — Tue, 20 Jul 2010 02:59:00 GMT

I am embarking in a series of mini post called .NET hidden gems, not necessarily new bits but usually unknown. They are dedicated to those small features and components that are not commonly used but can deliver interesting value in certain scenarios.

Today I am going to show you some functions that can help you to understand the memory state of your application when you need to store large objects.
The first hidden gem is the MemoryFailPoint object, this can be accessed through the System.Runtime namespace.
The main function of it is validating that there is enough memory to perform an operation.

[SecurityCritical]
public MemoryFailPoint(int sizeInMegabytes);

It is very easy to use it, just create an instance when you need the memory:

//
// Do I have enough memory to run this method?
//
System.Runtime.MemoryFailPoint CheckForMemory = new System.Runtime.MemoryFailPoint(1000);

Internally, it checks the global memory status of the process; if it does not have enough it will perform a full collection.
If it has enough it will execute a virtual allocation for that amount of bytes. This is really important as the MemoryFailPoint will reserve the memory for your application and it won’t be available for other resources until is freed using the Dispose() function, so use it with care.

If you want to know more about Memory managent in .NET please check my other post here

How to configure WinDbg to run other versions of the .NET

SalvaPatuel — Tue, 09 Mar 2010 00:39:00 GMT

I usually receive a very common question when customers jump into the WinDbg adventure, and is related to the loading of SOS when the version of the .NET mismatches the version installed on their system.

The scenario is quite common, you have a customer that is running another service pack or even some random hotfix on its machine, produces a memory dump and when you try to load it on your machine you will receive this message:

Unable to load image

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll, Win32 error 2
*** WARNING: Unable to verify timestamp for mscorwks.dll
*** ERROR: Module load completed but symbols could not be loaded for mscorwks.dll
PDB symbol for mscorwks.dll not loaded

Failed to load data access DLL, 0x80004005
Verify that

       1) you have a recent build of the debugger (6.2.14 or newer)
       2) the file mscordacwks.dll that matches your version of mscorwks.dll is
       in the version directory
       3) or, if you are debugging a dump file, verify that the file
       mscordacwks___.dll is on your symbol path.

4) you are debugging on the same architecture as the dump file.

What is happening is that the mscordacwks.dll is not matching the one loaded on the image and SOS can not be loaded.

How to solve the issue:

1) You need to ask the customer or the person that sent you the memory dump the file mscordacwks.dll localted on \Microsoft.Net\Framework\ from the machine where the mem dump was produced. Note: Remember that if the customer is running 3.0 or 3.5 is still version 2 of the CLR !

2) Put that file on a separate folder (do not overwrite your one!)

3) Load the windbg and the memory dump

4) Go to File >> Symbol File Path

5) Add the folder where you copy the mscordacwks.dll file

6) Load sos.dll using the .load command

7) use !threads to see if you see the threads

What to do if the above steps do not work

The first one and probably the one that you are experiencing is that the sos is already using a different path and seems that is not using the new path. The reason is WinDbg will usually save the configuration for that memory dump, therefore it does not matter if you kill WinDbg and open it again, it will use the saved parameters. What we need to do here is to unload it and change the path. Load the dump again, and make sure you have done the steps above, then do:

1) .cordll -ve -u -l

2) You have unloaded the CLR and we are enabling the verbose mode

3) Change the CLR path using: .cordll -lp

4) load the SOS again

5) run !threads

There are certain scenarios where this will not work either, one can be the name of the mscordacwks.dll. Depending on the image, there may be a need to change the file name to the full version one (mscordacwks__.dll). But which one to use?

As you have enabled the verbose mode you should be able to see the full name of the mscordacwks.dll that is trying to locate, for example (mscordacwks_x86_x86_2.0.50727.3603.dll). Go back to your folder where you have put the mscordacwks.dll and rename it as indicated on the verbose output.

Repeat all the process above and it should work.

Now, if is still not working it seems that you are messing around with the steps or you have been trying a lot of things and everything is a mess. What I will recommend is the following steps to try a brand new process:

1) Close WinDBG

2) Copy the memory dump to a new location

3) Open the memory dump

4) Make sure that you have the new mscordacwks.dll (with the new name) on the new folder (last step), change the symbol paths (first step) and try again. It should work.

I hope this helps.

Accessing TFS 2010 from VS2008

SalvaPatuel — Fri, 24 Jul 2009 05:20:00 GMT

In order to access the new Team Foundation Server 2010 from your current Team Explorer 2008 you will need to install the Visual Studio 2008 Service Pack 1. Several customers were complaining that even with the service pack installed they couldn't manage to connect.

The problem is the order of the installation. For many customers, the orders was:

1) Visual Studio SP1
2) Team Explorer 2008

but if you have installed Team Explorer 2008 after the SP1 the changes are not in place. In order to sort out this just reinstall the SP1 and it will work.