ETW Components

ETW works as follows: a controller tool is used to configure sessions, telling ETW what events should be routed to a session and how to store the events that have been routed to a session; an event provider generates events; the ETW runtime routes events to the appropriate sessions; the sessions record the event; a decoder tool extracts information from the events; an analysis tool makes use of the information from the event.

Microsoft provides several controller tools such as XPerf, LogMan, and TraceLog. You can use these tools to start a session, to configure a session (controlling which events should be saved and the location to which they should be saved), and to stop a session. 3rd parties can make use of the ETW APIs (StartTrace, ControlTrace, TraceSetInformation, EnableTraceEx2),  to create their own controller tools. In addition, the Windows OS itself controls several sessions that it uses to monitor its own performance and reliability. ETW supports global sessions (receive events from any provider on the system) and private sessions (receive events only from providers in the same process). The system is limited to 64 global sessions, and each process is limited to 4 private sessions. Special privileges (e.g. membership in Administrator or Performance Log Users) are required to control a global session. You can use "TraceLog -l" to get a list of active global sessions.

ETW sessions can be configured to save events in various ways:

• The session can be configured to write events to a file. In addition, ETW can treat the file as unlimited, as a circular buffer (overwriting old events once the file has reached a certain size), starting a new file when the file has reached a certain size, or stopping the trace when the file has reached the size limit.
• The session can be configured to write events to a reserved block of memory. These events will be lost if the system is turned off. However, the data in memory can be flushed to disk on-demand. The session will use the memory as a circular buffer (overwriting old events once the block is full).
• The session can be configured to send events to a consumer in real-time (via a callback). The consumer can then decide what to do with each event.

3rd parties can make use of the ETW APIs (e.g. EventWrite) to create providers. Many Microsoft-provided components also act as providers. The Windows OS kernel publishes many events regarding the operation of the OS, and many Microsoft-provided drivers and applications publish events. Many ETW sessions are always running on the system, started and controlled by the OS itself. For example, the Windows Event Log uses ETW sessions to receive events, so the Event Log sessions are always running to enable this functionality. Other ETW sessions are started as needed. For example, Visual Studio will start a special ETW session when you ask it to do performance profiling of your application. Visual Studio will configure the session to receive events from the OS about the performance of your application, which it will then analyze to determine where your application is spending its time and memory.

ETW itself treats the data as binary data, and any kind of data can be encoded using any system. If the data is encoded using a custom scheme, you must use a custom tool to decode the data. However, there are several Microsoft-supported systems for encoding ETW data, and corresponding tools that can decode the ETW events generated by these systems. For example:

• WPP is a system for encoding diagnostic trace information and writing it to ETW. If you use WPP to generate ETW events, you can decode them using tools such as TraceFmt or TraceRpt. Note that in order to decode these traces, you must have the TMF or PDB files with information about your events.
• MOF events are rarely used in new code, but many MOF-decodable events are generated by the Windows kernel. TraceRpt and other TDH-based decoders can decode MOF-decodable events as long as the appropriate event schema has been registered with WMI. The XPerf and WPA tools cannot access the WMI-registered data, so they cannot decode 3rd-party MOF-decodable events, but they contain hard-coded decoders for many common events generated by the Windows kernel. In particular, WPA makes heavy use of the kernel-generated events to analyze system behavior.
• Manifest-based ETW is a general-purpose system for encoding and decoding ETW data using XML manifests. If you use manifest-based ETW, you can decode the events using TraceRpt or XPerf. Note that in order to decode these traces, the data from the manifest must be available to the decoding tool. This can be provided directly (e.g. by providing the path to the XML manifest on the TraceRpt command line) or indirectly by compiling the manifest into a binary format, including the binary data in the resources of your component, and registering the component on the system so that the decoding tool can find it. (You can use "TraceLog -enumguid" to get a list of components that have registered their decoding data on a system.)
• TraceLogging is a general-purpose system for encoding and decoding ETW data without using separate decoding information. TraceLogging events can be decoded using the Windows 10 versions of TraceRpt or XPerf. Each event contains its own decoding information. This means each event will be larger, but it also means that the decoding information is always available and no separate XML, MOF, PDB, or TMF files need to be tracked.

3rd parties can make use of the ETW APIs (ProcessTrace, EVENT_RECORD) to get the data from an ETL file, and can use the TDH APIs (TdhGetEventInformation) to decode events that use the Microsoft-supported encoding systems.

ETW was introduced as part of WMI in Windows 2000. Events written using the Windows 2000 APIs (e.g. TraceEvent) are sometimes called "Classic" events. The WPP and MOF frameworks for writing events use the Windows 2000 APIs, so they generate classic events. Providers that use the Windows 2000 APIs have a significant limitation -- they can only be associated with one session at a time. If multiple sessions try to subscribe to events from the same classic provider, only the last session will actually receive events.

ETW was significantly updated for Windows Vista. Events written using the Windows Vista APIs (e.g. EventWrite) are sometimes called "Crimson" events. The XML manifest and TraceLogging frameworks use the Windows Vista APIs, so they generate crimson events. Providers that use the Windows Vista APIs can be associated with up to 8 sessions at a time.

Event Content

Each ETW event contains the following information:

• Required user-supplied information, such as the event provider GUID, the event ID, the event's "severity level", and the event's keywords.
• Optional user-supplied information, such as event payload (binary data), activity GUID, and related activity GUID.
• Always-present ETW-supplied information, such as the event's timestamp and the ID of the thread logging the event.
• Optional ETW-supplied information, such as call stacks. The session controller configures ETW to include or exclude this data.

The Event Provider GUID is supplied in the call to EventRegister. This GUID is used when routing and decoding events. For example, an event controller might configure a session to include all events with a particular provider GUID. When decoding MOF-based and manifest-based events, the provider GUID is used to look up the data that explains how to decode events. Note that multiple components can use the same provider GUID (even if they run at the same time) as long as you intend to enable/disable/route the events from multiple components as a unit, and in the case of manifest-based and MOF-based events as long as they share the same decoding information. Note also that there should be a strong 1-to-1 relationship between a provider name and the provider GUID. In order to maintain this relationship, you might want to base the provider GUID directly on the provider name using a hash. (A tool that performs this hash is provided in an earlier blog post.)

Each event has an event ID. The event ID is used when routing and decoding events. For example, an event controller might configure a session to include only events 1, 2, and 5 from a particular provider. When decoding MOF and manifest-based events, the event ID + event Version should uniquely identify the event within the provider (e.g. given a provider GUID, an event ID, and an event Version, you should be able to look up the event's binary layout, and the binary layout should never change for the GUID + Id + Version combination; every event with the given GUID + Id + Version should have the same fields with the same types in the same order).

Each event has an event version. This is used when you need to change the fields in an event. When the binary layout of an event is changed, you'll typically duplicate the event in the manifest or MOF file, increment the event version in one copy, and make the necessary changes in the copy. That way, you can still decode events in the old format, and you can move forward using the new format.

Each event has an event level. This is an indicator of the event's severity or importance, and is used in event filtering. For example, an event controller might configure a session to include only events with severity "error" from a particular provider, or an event analysis tool might filter out events with severity below "warning". The level can be any value from 0 to 255. Levels 1-5 have been given names by Microsoft: 1 is critical, 2 is error, 3 is warning, 4 is info, and 5 is verbose. Typically, events should default to level 5 (verbose). The level 0 is special -- it means that the event does not specify a particular level, and will be enabled regardless of any level-based filtering done for the session. A lower numeric value means a more-severe event.

Each event has a keyword field. (In classic ETW, the keyword field was called "flags".) Each bit in the keyword corresponds to a category, and if the bit is set in the keyword for an event, it indicates that the event belongs to the specified category. The low 48 bits of the keyword are user-defined, while the upper bits must only be set for specific scenarios as defined by Microsoft. For example, the provider might define bit 0x1 as "Networking event", bit 0x2 as "I/O event", and bit 0x4 as "UI event", and a particular event might set its keyword to 0x5 indicating that the event is related to both networking and UI. An event controller might configure a session to only include events with certain keywords. If the keyword value is 0 (no bits set), that means the event does not specify any keyword and will be enabled regardless of any keyword-based filtering. (Note that starting with Windows 10, a controller can configure a session to ignore events with keyword set to 0.)

Each event has an opcode field. (In classic ETW, the opcode field was called "class".) The opcode is used to mark events, but does not affect ETW routing. A few opcodes have well-defined semantics recognized by analysis tools. Other opcodes can be defined by the user for any purpose. General-purpose analysis tools will only use the opcode to label or group the events. The default opcode is 0 = Info. The most commonly-used opcodes are Start (indicating the beginning of an activity) and Stop (indicating the end of an activity). Other commonly-used opcodes include DC_START (typically indicating that the associated event contains a dump of the provider's state, triggered by a notification that a session has started collecting data from the provider) and DC_END (similarly indicating that the associated event contains a state dump triggered by the end of data collection). The remaining opcodes can be used or defined at the discretion of the user.

Each crimson event has a task field. The task is a user-defined category that can be used to mark events, but does not affect ETW routing. Tasks can be used to group events into categories. It is also common to use tasks to give a name to an event, since manifest-based ETW does not have a first-class concept of an "event name" but it does have a concept of "task name". This is done by assigning the same numeric value for Event ID and Task ID, then setting the Task Name to the desired Event Name. (TraceLogging-based ETW is the reverse. TraceLogging supports Task ID, but does not support Task Name. On the other hand, TraceLogging does support Event Name, and uses the Task Name field for the Event Name.)

Each crimson event has a channel field. The channel can cause an event to be given special treatment by the ETW runtime or by an event consumer. The default channel 0 means no special treatment is requested. Channels 1-15 are reserved for definition by Microsoft and may lead to special treatment for the event by the ETW runtime or other built-in Windows OS components. Other channel values are user-defined and can be used to request special treatment from a session. The kind of special treatment depends on the kind of session that receives the event. For example, a real-time session might interpret channel 25 as indicating a high-priority event. Channels are most commonly used when interacting with the Windows Event Log. The Event Log's session listens to certain event providers that have registered themselves with the service. The Windows Event Log service uses the event's channel to route the event to the correct log. For example, a particular provider might register itself with Event Log and configure channel 16 to mean the Operational log.

Activities

The Windows Vista (crimson) ETW APIs support the concept of an activity. An ETW activity is a set of related events. For example, there might be an activity for reading the data from a file. The activity might start with an event from opening the file, continue with an event each time a block of data is read, and end with an event from closing the file.

At an ETW level, an activity is simply a set of events that all have their ActivityId field set to the same nonzero value. The first event in the activity is the Start event, and it should have its OpCode set to START. The last event in the activity is the Stop event, and it should have its Opcode set to STOP. Activities can be logically nested by associating one activity with a "related" activity. This association is recorded by setting the RelatedActivityId field of the Start event to the ActivityId of the parent event.

The ActivityId is a 128-bit value that is intended to be unique within the scope of a given ETW trace. The ActivityId can be a GUID. However, GUID generation is relatively slow, and ActivityIds do not need to be globally unique, so ETW provides a more efficient API that generates locally-unique ActivityId values (EventActivityIdControl).

Every crimson event has an ActivityId field. The value for the field can be set using the EventWriteTransfer or EventWriteEx APIs. If the user does not explicitly specify the value of the ActivityId field (e.g. if the EventWrite API is used, or if the user passes an activity ID of NULL to the EventWriteTransfer or EventWriteEx API), ETW will use the value of a thread-local ActivityID variable.

Every crimson event can optionally have a RelatedActivityId field. The value for the field can be provided using the EventWriteTransfer or EventWriteEx APIs. If the user does not explicitly specify the value of the RelatedActivityId field (e.g. if the EventWrite API is used, or if the user passes a related activity ID of NULL to the EventWriteTransfer or EventWriteEx API), ETW not include a RelatedActivityId in the event.

The value of the thread-local ActivityID variable can be read and written using the EventActivityIdControl API. Setting this variable will make all subsequent events from the same thread default to the specified activity ID value. This allows for an the following pattern for creating activities:

1. Save the original value of the thread-local ActivityID variable.
2. Enter the scope of the activity.
3. Generate a new activity ID value, and set the thread-local ActivityID variable to this value.
4. Write a start event. Set the event's OpCode to START. Optionally set the event's RelatedActivityId to the saved original value of the thread-local ActivityID variable. Note that the start event will implicitly use the thread-local ActivityID value, so an explicit Activity ID does not need to be provided.
5. Write any number of events. As long as no code overwrites the thread-local ActivityID, all of the events on this thread will default to using the thread-local ActivityID value, implicitly making them part of the activity. (Events can exclude themselves from the activity by explicitly providing a different activity ID using the EventWriteTransfer or EventWriteEx APIs.)
6. Write a stop event. Set the event's OpCode to STOP. Note that the start event will implicitly use the thread-local ActivityID value, so an explicit Activity ID does not need to be provided.
7. On scope exit, restore the original value of the thread-local ActivityID variable.

• This pattern only works for single-thread synchronous activities. It does not work for asynchronous or multi-threaded activities.
• This pattern depends on the good behavior of all other code that runs on the thread. If you call into a function that sets the thread-local ActivityID and does not restore the value on exit, the events after that function returns will not be associated with the correct activity. Note that while it is good practice to restore the activity ID on scope exit (even on exception or other error), there is no enforcement of this rule. In addition, it is possible for a call to a Windows OS API to overwrite the current thread's Activity ID, e.g. if the OS API calls into a driver that sets the thread-local activity ID and does not restore the original value.

Event Filtering and Routing

ETW is responsible for routing each event to the correct set of sessions. Classic event providers (providers using the Windows 2000 APIs such as TraceEvent) can be routed to at most one session. Crimson event providers (providers using the Windows Vista APIs such as EventWrite) can be routed to up to 8 sessions.

Most events are not routed at all. If no session has asked for any events from a particular provider, the provider is considered disabled, and any events from that provider are ignored. When a disabled provider calls EventWrite, ETW will determine that the provider is disabled and will immediately return success, indicating that the event was delivered to all interested providers (all 0 of them). However, this is relatively inefficient -- it takes ~100 CPU cycles for EventWrite to validate its parameters and look up the provider's state to determine that the provider is disabled. (Note that using APIs such as EventEnabled or EventProviderEnabled will not help, since these APIs also require about the same amount of time to look up the provider's state. The only difference is that EventEnabled can be called before the event payload is set up.) The implementation of EventWrite looks something like this:

ULONG EventWrite(
    REGHANDLE handle,
    PCEVENT_DESCRIPTOR descriptor,
    ULONG count,
    PEVENT_DATA_DESCRIPTOR data)
{
    if (!ValidateParameters(handle, descriptor, count, data))
    {
        return ERROR_INVALID_PARAMETER;
    }

    ULONG errorCode = ERROR_SUCCESS;
    PROVIDER_INFO info = LookupProviderInfo(handle);
    if (info.IsEnabled &&
        info.IsLevelOk(descriptor.Level) &&
        info.IsKeywordOk(descriptor.Keyword))
    {
        errorCode = info.SendEventToSessions(descriptor, count, data);
    }

    return errorCode;
}

Providers can improve their performance by providing a callback function when they register with ETW (see the EventRegister API). ETW will invoke the callback whenever a session enables or disables the provider. This allows the provider to keep track of its own state, including whether it is enabled or disabled, what levels are being filtered out, and what keywords are being filtered out. The provider can check its own state much more efficiently than EventWrite, allowing it to run much more efficiently when the provider is disabled. A provider might store its state in global variables and might then write events using code like this:

if (g_providerIsEnabled &&
    g_providerLevel > ThisEventDescriptor.Level &&
    IsKeywordEnabled(ThisEventDescriptor.Keyword))
{
    EVENT_DATA_DESCRIPTOR data[4];
    // Prepare data for event ...
    EventWrite(g_providerHandle, &ThisEventDescriptor, 4, data);
}

In the case where the provider is disabled, this will skip the event-write code using only 1 or 2 CPU cycles. In the case where the provider is enabled, but the event is filtered out by level or by keyword, this will skip the event-write code using only 5 or 10 CPU cycles. The code to prepare the data and call into ETW will only run if the provider is enabled for events with the given level or keyword.

The code for maintaining the callback and for checking the state of the provider for each event can be cumbersome. Various frameworks exist to hide this complexity from the user. WPP and manifest-based ETW generate header files with code that looks similar to that shown above, implementing very efficient filtering with little effort on the part of the developer. The TraceLoggingProvider.h header implements similar filtering in the TraceLoggingWrite macro. For example:

TraceLoggingWrite(
    g_hProvider,
    "EventName",
    TraceLoggingValue(GetMyValue(), "MyValue"));

In this code, the GetMyValue() function will only be invoked if the provider is enabled.

The .NET EventSource class implements efficient filtering within the WriteEvent, WriteEventCore, and Write methods, so these methods return very quickly if the provider is disabled or if the event is filtered by level or keyword. Similarly, the Windows Runtime LoggingChannel class implements efficient filtering so that the LogEvent, LogMessage, and LogValuePair methods return very quickly if the event does not need to be written. However, in these cases, the automatic filtering does not eliminate the time spent preparing the data for an event. For example, with .NET EventSource,

myEventSource.WriteMyValue(GetMyValue());

In this code, the GetMyValue() function will be invoked even if the provider is disabled. If GetMyValue() returns quickly, this might be ok. On the other hand, if the GetMyValue() function might be expensive, it would be more efficient to use code like this:

if (myEventSource.IsEnabled())
{
    myEventSource.WriteMyValue(GetMyValue());
}

Note that the above check only tests the overall provider state. This might be sufficient, or it might be better to use another overload of IsEnabled to also check for level-based or keyword-based filters.

The same applies to TraceLoggingProvider.h -- the automatic filtering does not skip code outside the TraceLoggingWrite macro. For these cases, a manual check might be needed for optimal code. Each of these frameworks provides methods for determining whether the provider is enabled without sending an event. For example, if preparing the data is more complex than a single GetMyValue() call, the above code might be written as follows:

if (TraceLoggingProviderEnabled(g_hProvider, 0, 0))
{
    int myValue;
    GetMyValue(&myValue);
    TraceLoggingWrite(...);
}

ETW supports many other kinds of filters that are difficult or impossible to replicate within the provider code. The provider-side filtering will not exactly match the complete filtering and routing done by ETW, and some events will pass the provider-side filtering only to be dropped by the more complete filtering done by the ETW runtime. This is generally ok -- the provider-side filtering is usually successful at eliminating the vast majority of unnecessary events in only one or two CPU cycles. However, for this simple provider-side filtering to be most effective, events and providers should be organized in a way that allows event scenarios to be defined by a combination of provider GUID, level, and keyword.

If scenarios are well-defined using filters understood by the provider (e.g. filters based on level and keyword), the scenario will have minimal impact on the system when the corresponding session is created. For example, if a "networking" keyword is defined, and the "error" level is used consistently, it is very easy to diagnose a networking error by setting up a session to collect events where level <= error and keyword mask = networking. On the other hand, if no keywords are defined for your events, and levels are not used consistently, the same scenario might not allow for any meaningful filtering at the level or keyword layer. All events in the provider might need to be enabled and collected, leading to a performance impact while the session is enabled.

Each session can be configured to receive events from any number of provider GUIDs. For each provider GUID, various filters can be established, such as filtering by level, keyword, or event ID. The session can also request additional information (e.g. callstacks) for certain events.

When ETW receives an event from a provider, it determines the set of sessions that are receiving events from that provider. It then checks the filters for each session, removing the session from the set if the event does not meet the session filters. If the set of sessions is non-empty, it reserves space for the event in each of the session's buffers. After it has reserved space, it copies the event into the reserved space for each session.

If ETW fails to reserve space in all sessions, its default behavior is to drop the event (i.e. if ETW cannot deliver the event to all interested sessions, it does not deliver the event at all). This policy was intended for consistency, i.e. so that the different sessions will all record a consistent story. However, this behavior is not always desirable. A session can opt out of this policy by setting the EVENT_TRACE_INDEPENDENT_SESSION_MODE flag. (Newer versions of the XPerf tool will set this flag for the sessions it creates.)

Each ETW session has a worker thread that handles tasks for the session. For file-based sessions, the worker thread is responsible for noticing when a buffer is full, writing the full buffer to disk, marking the buffer as empty, and allowing it to be reused for future events. For real-time sessions, the worker thread is responsible for sending full buffers to the real-time consumer process.

While it is possible to write custom code to pack and unpack each event, this tends to be hard to manage. Developers don't want to spend time writing custom packing and unpacking code. Worse, if each event has a custom packing/unpacking format, there can be no general-purpose decoding tools. To avoid this situation, several event encoding/decoding protocols have been built over the general ETW infrastructure.

• MOF - description of the event encoding provided in MOF (WMI) files. (Old technology, generally not used for new ETW providers. Covers mostly the same scenarios as manifest-based ETW, but with less flexibility, so it won't be discussed further in this post.)
• WPP - description of the event encoding provided in C/C++ code. A special preprocessor (tracewpp) generates code for packing the event data and generates TMF files that can be used by event decoding tools to decode events. The preprocessor also adds attributes to the code that allows the TMF files to be reconstructed from the PDB. (Intended for logging diagnostic information for use by developers.)
• Manifest - description of the event encoding provided in MAN (XML) files. A special preprocessor (mc) generates a binary representation of the manifest file and optionally generates C/C++ code for packing the event. The binary version of the manifest file is added to the resources of a DLL and registered on a system so that decoding tools can find it when they need to decode events. (Intended for general purpose logging, e.g. for the Event Log or for diagnostic purposes.)

One of the major challenges of ETW is managing the decoding information (the "metadata") for events. Somebody has to author the event description in a way that allows both machines (encoder and decoder) and humans (developer writing the event and whoever wants to read the decoded version of the event) to understand the event's content. This event description has to somehow be made available to the tools that are used to process and decode the event.

WPP is popular because in straightforward cases, it is very easy to use. A developer adds a few macros to the code, sets up the build system to enable the WPP preprocessor, and writes familiar printf-style code. Decoding typically requires TMF or PDB files and an ETL file turns into a list of timestamped messages that is easy to look through. However, it becomes hard to manage in more complex scenarios - the macros are very tricky to customize, the preprocessor can interfere with the build process, the flexibility of the printf-style formatting means the data is not in a standard format that is easy to analyze, and the decoding information (PDB or TMF) is specific to a particular build of the DLL, so it can be hard to decode the WPP data from a particular ETL file (you have to figure out what version of the DLL was used, then locate the TMF or PDB file for that specific version). In addition, WPP is very tightly-coupled to C/C++ and won't work for other languages.

Manifests work well in other scenarios where the centralized manifest provides benefit. Developers maintain a manifest file, and the binary representation of the manifest is included with the product (embedded in the DLL resources). Maintaining the manifest is more work, but it provides benefits like stable event definitions (events from the old product can generally be decoded using the new manifest) and localizable event messages. The data decodes into name-value pairs (where the value has a strong type like "uint32") that allow for structured analysis if necessary. On the other hand, maintaining the manifest is more work, and can be completely infeasible in some scenarios (e.g. scripted environments). Proper decoding requires global registration of the binary during component installation, which can also be problematic (e.g. for Windows store apps, or xcopy-deployed apps).

Neither of these technologies is ideal for all situations. During development of Windows 10, the ETW team looked for ways to improve the metadata management story. We came to the following conclusions

1. There is no way to make a single metadata handling system that meets all needs. A centralized design (a single point where all events in a component are defined, e.g. a manifest) is a requirement for some scenarios but adds unnecessary complexity for other scenarios.
2. Manifest-based ETW is a reasonable solution for centralized scenarios.
3. WPP-based ETW is a reasonable solution for some distributed scenarios.
4. There are many scenarios for which neither manifest-based nor WPP-based ETW is a good fit.

We designed a new ETW metadata system to address scenarios that weren't met with either manifest-based or WPP-based ETW. The new system is called "TraceLogging" or "manifest-free ETW". The new system was designed with the following goals:

1. Usable on any development platform, including C/C++, .NET, and scripting languages.
2. Automatic management of metadata. The developer shouldn't have to worry about how the decoder gets the decoding information. ETW should Just Work, and ETL files should Just Decode.
3. Permit distributed management of decoding metadata. The developer should be able to write the event description in code.
4. Provide strongly-typed and structured data. Data is organized as name-value pairs. Values are strongly-typed as uint32, string, datetime, etc. Values can have complex types such as structure, array, array of structure. Structures can nest.
5. Providers should be able to use this technology on Vista or later.

At its core, TraceLogging is simply a convention on how to encode data and how to describe the encoded data. A TraceLogging ETW event is simply a normal ETW event with a payload packed using the TraceLogging encoding rules and a flag set that tells the decoder to unpack using TraceLogging decoding rules. The implementation of this technology includes helpers for packing the TraceLogging event and libraries for decoding TraceLogging events.

The primary difference between TraceLogging and the other systems is that each TraceLogging event includes its own decoding information. In WPP, MOF, or manifest-based ETW, the decoder must locate a file with the appropriate decoding information, but each TraceLogging event is self-contained, allowing the decoder to interpret the event content without access to any external information. In some ways this is similar to the way a JSON object or XML document can be parsed (and potentially even analyzed) without access to any external schema reference.

The benefits to this approach include the following:

• Decoding information is always available. There is no need to track down a manifest, MOF, PDB, or TMF file. There is no need to register your component's metadata. The decoding information is never out of date.
• Events can be generated dynamically. This makes ETW more easily usable from scripting languages like JavaScript or Python.
• Event definitions can be distributed and can be written directly in code with no need for an external tool or preprocessor.

As mentioned above, there is no way to make a single ETW solution that meets all needs. The following issues must be considered when discussing the new TraceLogging technology:

• Each TraceLogging event contains its own metadata, so TraceLogging events are always larger than the equivalent manifest-based or WPP-based event. While the metadata encoding is relatively compact, it can add several hundred bytes per event. In cases where reducing the event size is a high priority (e.g. when generating millions of events), TraceLogging is not ideal.
• TraceLogging supports distributed event authoring, so there is no single place to go to find a list of all of a component's events.
• To write a TraceLogging event, the metadata must be available (and in memory) at runtime. The design of WPP and manifest-based ETW support installing a provider without also installing the decoding information, potentially saving disk space. In addition, neither WPP nor manifest-based ETW need to load the decoding information into memory until the event needs to be decoded.
• TraceLogging events do not support localization. The provider name, event name, and field names are hard-coded into the event, and there is no built-in support for localized stringtables to map these identifiers to an alternate language.
• TraceLogging does not have built-in support for enumerated types. The existing TraceLogging APIs simply convert the enumerated value to an integer. There is no built-in support for associating friendly names with each enumeration value.

We've implemented support for TraceLogging in C/C++, .NET, and Windows Runtime.

• For C/C++ code (kernel-mode or user-mode), use the Windows 10 SDK and include the <TraceLoggingProvider.h> header. Note that while this header is new to the Windows 10 SDK, you can use this header in projects that target Vista or later.
• For .NET code, we've updated System.Diagnostics.Tracing.EventSource in .NET 4.6 to add extensive new capabilities based on the flexibility provided by TraceLogging. If you want to use this functionality with an earlier .NET runtime, you can use the EventSource NuGet redistributable from nuget.org.
• For Windows Runtime code, we've updated the Windows 10 version of LoggingChannel to support writing structured events based on the new TraceLogging encoding.

The provider side of the TraceLogging technology stack does not require any new OS features, and works correctly on Vista or later (with a few limitations). You can write a program that generates TraceLogging events, run the program on Vista, and capture the events into an ETL file. The TraceLogging provider library will pack the payload using TraceLogging rules, and will set the event's channel = 11 to flag the event as being TraceLogging-encoded. It will then use the Vista-compatible EventWriteTransfer API to send the encoded event to ETW, which will treat it like any other event. However, you will need an OS update for the following functionality:

• Using the TDH APIs to decode TraceLogging events requires new TDH functionality. This functionality is built-into Windows 10, and is available as a Windows Update for Windows 7 SP1 and for Windows 8.1.
• Writing an event with a channel other than 11 requires new functionality in the core ETW runtime. Again, this support is built-into Windows 10 and is available as a Windows Update for Windows 7 SP1 and for Windows 8.1.

xperf -start MySessionName -f MySession.etl -on 2e0582f3-d1b6-516a-9de3-9fd79ef952f8

GUIDs are ugly, so the systems layered above core ETW have established conventions for friendly provider names. For example, ETW manifests include the provider GUID and the provider name. When you compile the manifest and register the resulting decoding information, both the GUID and the name are added to the registry, allowing the decoding information to be found either by GUID or by name.

To avoid problems with this system:

• The association between GUID and Name is one-to-one. Once you've published the Name and GUID for a provider, the association between Name and GUID is permanent. If you change the GUID, you must change the Name, and if you change the Name, you must change the GUID.
• Searching for providers by Name is generally not case-sensitive. However, some reporting tools may be confused if they see two names that differ in case. Because of this, do not change the uppercase/lowercase of any letters in the name once you've published the Name of a provider.
• The provider Name is not localized. It is a symbol, like a variable name or file name.
• The provider Name is global, so it should be properly namespaced. Manifest providers often use the convention CompanyName-ProductName-ComponentName (with additional parts as needed for SubCompontName), e.g. "Microsoft-Windows-Globalization". It is also common to use periods instead of hyphens, e.g. "Microsoft.Windows.DHCP".

The .NET EventSource class developed a convention for generating the provider GUID by hashing the provider Name (algorithm based on RFC 4122). This has worked well in practice -- the developer provides the human-friendly provider name, and EventSource internally manages turning that into a machine-friendly GUID. If the provider Name is changed, the provider GUID will also change (as long as the change involves more than just uppercase/lowercase).

Support for this convention has been added to WPR. Normally, you provide WPR with the provider's GUID, but if you generated the provider GUID using the EventSource algorithm, you can instead provide the friendly name to WPR prefixed with an asterisk, e.g. "*MyProviderName".

This convention has also been adopted by the Windows Framework LoggingChannel class. The provider GUID is an optional parameter, and if omitted, the provider GUID will be generated using the EventSource algorithm.

This convention can also be used with manifest-based ETW or with TraceLoggingProvider.h, but the tools will not automatically generate the GUID for you. Instead, you can use the attached EtwGuid tool (public domain C# sample code). The tool treats each command-line argument as a provider Name. For each argument, it outputs a code snippet with the corresponding provider GUID. The resulting code snippet is formatted for use with TraceLoggingProvider.h, but the GUID can also be used in a manifest.

Example usage:

D:\Stuff\Projects\EtwGuid\bin\Debug>EtwGuid.exe SampleProvider1 SampleProvider2

TRACELOGGING_DEFINE_PROVIDER(
    g_hMyProvider,
    "SampleProvider1",
    // {69aed5be-95eb-5b26-ed54-cf841b064fed}
    (0x69aed5be,0x95eb,0x5b26,0xed,0x54,0xcf,0x84,0x1b,0x06,0x4f,0xed));

TRACELOGGING_DEFINE_PROVIDER(
    g_hMyProvider,
    "SampleProvider2",
    // {7880d1eb-f225-56df-f429-3e3a24c6f149}
    (0x7880d1eb,0xf225,0x56df,0xf4,0x29,0x3e,0x3a,0x24,0xc6,0xf1,0x49));

D:\Stuff\Projects\EtwGuid\bin\Debug>

EtwGuid.zip

I'm on the Core ETW team now, and there are some things I want to share about ETW, so expect a few postings in the near future.

Let me know if there are things you'd like to have explained about ETW. I'm hoping to do some posts that go over some of the details and explain how ETW works.

The first point is that the SD card should be thought of as if it were your phone’s hard drive, not as removable storage. Not all Windows Phone 7 devices support replacing the SD card, and if the device does support replacing the SD card, you’ll have to perform a factory reset (losing all of your settings and saved files) as part of the replacement process. The phone won’t work correctly if you remove the SD card. Replacing the SD card on a Windows Phone 7 device is like replacing the hard drive on a desktop PC.

It follows that a slow SD card will have a big impact on the overall performance of Windows Phone 7. Adding a slow SD card to the device will slow down the whole phone. Unfortunately, determining whether an SD card is “fast” or “slow” requires a lot of testing, and the advertised “class” of an SD card has almost nothing to do with how well it will perform in a phone. SD card “class” measures how quickly an SD card can save a single large file such as a photo (think of this like you would think about a car’s top speed). Windows Phone’s performance is more affected by how quickly the SD card can load and save a whole bunch of tiny files (think of this like you would think about a car’s performance in an obstacle course). Don’t fool yourself into thinking that any old SD card will work or that a higher "class" of SD card will make your phone work better. Microsoft tested a whole bunch of different SD cards. Very few performed well enough to be approved for use in the phone. SD cards that are not "approved" may work correctly, and may even work well, but they may also work poorly or not at all. Unfortunately, the SD card’s “class” won’t be very helpful in predicting how the card will perform in the phone.

Finally, the SD card is locked to your phone, and data on the SD card is encrypted. The data on the SD card can only be read by the phone, and cannot be read by other computers or phones. The encryption key is set when the phone first boots after a factory reset. Once the SD card has been locked, there are two ways to unlock it. The first is to provide the encryption key, which only your phone can do. The second is to perform a low-level reset on SD card, which erases all of the SD card's data. The phone will perform a low-level reset on the SD card during its first boot after a factory reset, and will then lock the card to itself. Unfortunately, I don’t know how to do a low-level reset on an SD from a desktop computer. This means that once your SD card has been locked to a Windows Phone, it won't work anywhere else. (In the future, tools might become available to reset the SD card, but for now, you should assume that the SD card is no longer usable.)

Official Microsoft KB article: https://support.microsoft.com/kb/2450831

Hope this helps!

2010/11/10: Minor editing, added link to KB article.

Back in the days of XP, this wasn't actually a big problem. All of the decent AGP video cards supported two monitors, so I just had to dig up any old PCI video card to complete the trinity. The third monitor would be a bit laggy, but it wasn't a big problem.

Unfortunately, the Vista display driver model made this solution more difficult. Vista's display architecture took several steps forward and enabled a lot of new capabilities, but to make it possible, a few rarely-used capabilities of the older architecture were left behind. One of these capabilities was the ability for Windows to work with more than one display driver at once.

Being a creative genius, I was able to resolve this issue by getting two new nVidia graphics cards. One was AGP, the other was PCI, but otherwise they were identical (same chipset and GPU). This worked, but everything on the PCI display was slow. This was especially noticable when dragging windows from one screen to the other, and also when switching from one window station to another (as required for Ctrl+Alt+Del and UAC prompts).

I later upgraded to a PCI-express system. Unfortunately, it only had one x16 PCI Express slot, but I still managed to get the triple-monitor configuration working with two ATI video cards (one x16, the other x1). Again, the slower card was definitely noticable, but I couldn't figure out a good solution that didn't involve a new motherboard, so I just tried to live with it.

At one point, I got a video card with three outputs (VGA, DVI, and HDMI), only to discover that only two of the outputs could be active at once. Apparently the video card cabal has a rule: thou shalt not create a video card that can support more than two monitors.

 Recently, I found out that ATI had released a new line of video cards with "Eyefinity" technology. In theory, it supports three monitors, but only if one of them uses DisplayPort. None of my existing monitors supports DisplayPort, but minor technical issues like that have never stopped me before, so I immediately ordered a 5770 (I would have gotten a 5750, but my case didn't have room for a two slot card, and the only single-slot card in stock at NewEgg was a 5770) and a DisplayPort-to-DVI adapter.

To make a long story short, after some work with a pair of tin snips (the single slot card fits perfectly, but only for certain definitions of "perfectly") and a bigger power supply (they're not kidding when they say you need a 450 watt power supply - the card was not reliable with just 350 watts), I did get the card installed and running on my desktop. But it still only worked with two monitors at once. Any two, but only two.

Apparently, all DisplayPort adapters are created equal, but some are more equal than others. ATI isn't particularly forthcoming about this, but the "passive" DisplayPort adapters (read: anything less than $50) apparently don't count as a DisplayPort monitor. The video card only has enough circuitry to handle two VGA/DVI/HDMI ports at once, and a passive DisplayPort-to-DVI adapter counts as a DVI port.

So I've ordered a second DisplayPort adapter. This time, it's an "active" DisplayPort-to-VGA adapter (one of my monitors is VGA only, and I suspect that by the time it is replaced, the replacement will have a DisplayPort input), so hopefully it will work. If it doesn't, I'll probably rip my hair out. Tune in next week for the exciting conclusion!

1. CryptAcquireContext(out providerContext, randomContainerName, null, PROV_RSA_FULL, CRYPT_NEWKEYSET)
2. CryptGenKey(providerContext, AT_KEYEXCHANGE, CRYPT_EXPORTABLE, out cryptKey)
3. CertStrToName(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, name, CERT_X500_NAME_STR, 0, dataBuffer, ref dataLength, 0)
4. cert = CreateSelfSignCertificate(providerContext, blob(dataBuffer, dataLength), 0, KeyProviderInfo(randomContainerName, PROV_RSA_FULL, AT_KEYEXCHANGE), 0, startTime, endTime, 0)
5. certificateStore = CertOpenStore("Memory", 0, 0, CERT_STORE_CREATE_NEW_FLAG, 0)
6. CertAddCertificateContextToStore(certificateStore, cert, CERT_STORE_ADD_NEW, out storeCert)
7. CertSetCertificateContextProperty(storeCert, CERT_KEY_PROV_INFO_PROP_ID, 0, KeyProviderInfo(randomContainerName, PROV_RSA_FULL, AT_KEYEXCHANGE))
8. PFXExportCertStoreEx(certificateStore, pfxBlob, password, 0, EXPORT_PRIVATE_KEYS)
9. Free everything.

In case anybody is interested, source code is attached and is free for use by anybody as long as you don't hold me or Microsoft liable for it -- I have no idea whether this is actually the right or best way to do this. Give it the X500 distinguished name, validity start and end dates, and an optional password for encrypting the key data, and it will give you the PFX file data. Let me know if you find any bugs or have any suggestions.

Certificate.cs

I don't really know how it happened, but somehow about 5 years ago, through no fault of my own, I ended up owning the PB "catalog". For those who don't know, the catalog is a repository for information about the CE operating system, and PB uses this information to help developers customize their CE builds. For example, if you have the environment variable "SYSGEN_AUDIO" set when you build a CE operating system, your operating system will include sound system support. The catalog has a list of these "features", along with a friendly title, description, and some metadata that helps PB ensure that the environment variable is properly exposed to the developer.

There were some significant issues with the catalog's design, and maintaining the catalog data was not a fun experience. CEC files were not  Maintaining the code that worked with the catalog was also problematic. For PB 6.0, I finally had the opportunity to start from scratch with a new catalog format. While a certain amount of backwards compatibility was required (the new catalog needed to essentially provide the same data as the old one, and there would need to be an upgrade path). So I designed a new catalog system, written in C# and based on XML files. While it isn't perfect, I think it is an improvement.

From a development perspective, one thing that is nice about the new catalog is that it has a pretty clean object model (an interface for developers who want to use the catalog). While we've never published the interface for the object model, we don't support it for external use, and everything I say here comes without warrantee or guarrantee of future compatibility, it isn't too hard to get started with basic tasks using the catalog object model. The reflection and metadata included in every managed assembly make this possible.

Before I go too far, I want to be clear that the design for the catalog object model emphasized the most common use of the catalog as a read-only repository. Loading and reading data from the catalog is (hopefully) reasonably simple and efficient. Using the object model to create or edit catalog files is a bit more of a challenge and not quite as efficient.

Loading a catalog means parsing about 2 megabytes of XML and loading the data into memory. While this happens in less than 1/10 of a second, we don't want to do this too often. In addition, we don't usually need (or want) multiple copies of the catalog in memory. To help with this, the catalog object model provides a cache of read-only loaded catalogs. When loading a catalog via the object model, you can choose to get your own copy of the catalog or you can ask for a read-only copy, which may come from the cache. (The cache is based on the .NET garbage collector via weak references. As long as anybody is using a catalog, it won't be evicted from the cache. As soon as nobody is using it, it becomes eligible for garbage collection. If it re-enters use before it is collected, it is no longer eligible for collection.)

The first step in using the catalog from your own code is to reference the catalog library. This code is in the Microsoft.PlatformBuilder.Catalog assembly, which (assuming you've installed Platform Builder 6 into C:\Program Files) can be found in C:\Program Files\Microsoft Platform Builder\6.00\cepb\idevs\Microsoft.PlatformBuilder.Catalog.dll. In Visual Studio, add a reference to this file. For convenience, you can add a "using" declaration to the top of your code to import the Catalog's namespace:

Next, to load a catalog, you use the CatalogManager class. The CatalogManager is a collection of catalog files. It handles searching through a winceroot for catalog files to load, loading all of the catalog files into memory, and maintaining the organization of the data in the catalog. It "flattens" the contents of the files, merging the data from all of the catalog files into a single collection of catalog elements. It also groups the data by type (so I can get a list of BSPs without going through all elements in the whole catalog), and generates indexes (so I can quickly look up a BSP by name).

When creating a CatalogManager, you can either create your own (writable) copy or you can load a read-only cached copy that might be shared by other components in the same process. To create a writable copy, just use the CatalogManager's constructor:

The default constructor for the CatalogManager will load only the "Global" catalog files. These are the catalog files in C:\Program Files\Microsoft Platform Builder\6.00\Catalog, and need to be available to PB whether or not it is working with a CE OS tree. To load the catalog files from an OS tree as well as the global catalog files, just add the winceroot path to the constructor call:

For me, Vista definitely glitches a LOT more than previous versions of Windows. As a fairly experienced developer, I think I understand the reasons pretty well, so I can explain it away. But as a user, when my laptop audio is glitchy, I want to find the developer responsible and (censored for mild descriptions of hypothetical violence).

I've read a lot of comments raising various theories, some that call into question the sanity of the Windows developers. I can't say I blame them. There is definitely some room for improvement in the way things work. However, in the interest of fairness and progress, the attention should be focused where it will do the most good. That means we shouldn't simply blame the Windows developers unless there is really something they can do about the problem. And that means that before we start placing blame, we probably ought to figure out where the problem really lies.

The first complaints always mention something about the "lame" and "brain-dead" Windows NT scheduler. However, I'm pretty convinced that this is not the problem. In fact, my audio sounds BETTER when my system is under load (more on this later). I agree with the statement that audio glitches under CPU load are usually the fault of the OS and the scheduler, but I've seen very little correlation between CPU load and audio glitching. I haven't seen any evidence that the CPU scheduler at fault.

Hard disk load is occasionally an issue, but that is generally fairly obvious and easy to fix at the application level. The application simply didn't buffer enough sound samples and ran out of music to play while waiting for the hard drive to load the next bit of music. Either tweak the buffering algorithm of the application or get a faster hard drive (or network).

Memory can also be an issue. Some buffer or code needed to play the music might be paged out because you're running low on free memory, and it didn't get paged in quickly enough once the application tried to access it. This could possibly be blamed on the OS if the OS is too aggressive in trimming the working set. If an app pre-loads 60 seconds of audio, that means it doesn't won't touch the last page of the buffer for 60 seconds, which might be long enough for the OS to page out the buffer. Here, you would probably get better results by buffering only 5 seconds worth of music. In any case, I haven't had any significant trouble with this on Vista (except once in a while when I let Firefox run too long, it eats up 1.5 GB of RAM, and my system goes into memory panic mode).

Drivers are a much more significant issue. Traditional OSes (XP, Vista, Linux) can't really schedule a driver's activity. Once the driver starts doing something it thinks is important, it can only be interrupted by a higher-priority driver. On a single-CPU system, if a driver takes over, no new audio can be buffered until the driver returns. Many drivers written for XP or earlier systems (where the audio buffer was somewhat more forgiving -- more on this later) cause trouble by doing too much processing at once. In testing (under XP), the driver's latency wasn't a problem, but on Vista, the latency requirements are much less lenient. I've seen significant Vista audio glitch issues go away after upgrading from an XP-era driver to a newly released Vista-compiant driver.

Even if the driver only does 1 millisecond of work at a time (or whatever the Vista latency recommendations are), if it has to do this 1000 times a second, it will still use up all available CPU time. Drivers have priority over all applications, so on a single-CPU system, this leaves no CPU for the audio application and mixer. On a multi-CPU system, this can still be a problem if the driver holds certain locks that are needed for audio processing. This is why Vista throttles network activity when the audio channel is open -- network packet bursts can easily use enough CPU to cause audio glitches. Probably a good idea overall, though it seems that the throttling algorithm is a bit too aggressive and has some room for improvement.

Another issue is power management. This turned out to be the major problem on my laptop. My laptop's motherboard (CPU and chipset) goes into sleep mode whenever it detects that it is "idle". That's actually a pretty good thing because it means I can get 2 or 3 hours of use out of the battery instead of 20 minutes. This happens hundreds of times per second -- it sleeps for 2 milliseconds, wakes up to handle a keystroke, sleeps for another 2 milliseconds, wakes up to handle the calculations for an animation, sleeps a bit more, wakes to fill an audio buffer, etc. But it sometimes doesn't wake up quickly enough to buffer the next bit of audio. If it ever takes longer than 9 ms to wake up, there will be a glitch. This was a real problem when my laptop was new. Recent drivers have improved this a lot, but there's still a bit of scritch-scratch during some games or media.

As an experiment, I wrote a very simple application to prevent the motherboard from going to sleep. It starts a low-priority thread that does a simple busy wait in a low priority loop. A Sleep(1) loop didn't help -- it gave the motherboard a chance to go to sleep. While the busy wait makes my laptop get very hot, it also completely stops the glitching.

#include <windows.h>

#include <stdio.h>

 

// This probably doesn't really do anything. At such a low priority, the

// process usually terminates before the thread exits. But this is an easy

// way to avoid certain compiler warnings. Without it, some compilers warn

// that the "return 0" below is unreachable. If I remove the "return 0",

// other compilers warn that I don't return a value from DoNothingQuickly.

volatile BOOL g_stopNow = FALSE;

 

DWORD WINAPI DoNothingQuickly(LPVOID /* unused */)

{

      while (!g_stopNow)

      {

            // Sleep(1) didn't fix the glitching. Sleep(0) just spends all

            // the time context switching, which is probably as bad or

            // worse than a busy wait in terms of impact on the rest of

            // the system. So I'll just do a busy wait.

      }

      return 0; // Usually never reached.

}

 

int main()

{

      int returnCode;

      DWORD dwThreadId;

      HANDLE hThread;

 

      hThread = CreateThread(

            NULL,

            0,

            DoNothingQuickly,

            NULL,

            CREATE_SUSPENDED,

            &dwThreadId);

 

      if (hThread != NULL)

      {

            // We want to keep one CPU wide-awake and leave any other

            // CPU(s) idle.

            SetThreadAffinityMask(hThread, 1);

 

            // We don't want to get in the way of any useful work.

            SetThreadPriority(hThread, THREAD_BASE_PRIORITY_IDLE);

 

            ResumeThread(hThread);

            CloseHandle(hThread);

            printf("Caffeine: Now running. Press <Enter> to quit...");

 

            getchar();

            g_stopNow = TRUE; // Probably useless, but might as well...

            returnCode = 0;

      }

      else

      {

            printf("Caffeine: Unable to create thread. Exiting.\n");

            returnCode = GetLastError();

      }

 

      return returnCode;

}

Drivers seem to be a big part of the problem here -- they either spend too much time working, or they take too long to wake up after going into a sleep state. Hopefully this means that audio problems will go away as the drivers improve. Computer retailers like Dell and HP will probably ensure that their new hardware meets the Vista latency requirements before putting it on the market. Unfortunately, owners of older hardware might be out of luck.

Hindsight is 20/20. I've seen how these kinds of issues come up, and I've been involved in some mistakes myself, so I don't want to sound like I'm smarter than anybody on the Windows audio team. However, there is certainly some room for improvement in they way this issue has played out. While the changes in the audio stack are technically admirable and the problems can generally be blamed on drivers, that's little comfort to those enduring static on their speakers. Things work in XP and don't work in Vista. That sounds like a regression, not an improvement. It's getting better, but there really shouldn't have been a problem in the first place.

What's wrong? Well, Vista aimed for a technically superior audio experience. Latency has been significantly reduced in Vista -- when you fire your machine gun in your favorite game, you'll hear the sound effect a bit more quickly. For gamers and audio professionals, this reduction in latency can make a big difference. For people trying to listen to their MP3s, this probably doesn't matter much. The downside to reducing latency is that it reduces the margin for error. Vista cannot tolerate any delays longer than 5 or 10 milliseconds without glitching, while XP could usually tolerate a much longer delay with no problem. Assuming all of the drivers do their part, modern hardware actually has no problem meeting the deadlines. But if anything goes wrong on Vista, you can hear it.

What was the mistake? The core of the issue is that a change was made that was detrimental to some customers. In the long term, the change is probably a step (or two) in the right direction, but for many people, the change causes trouble and offers no immediate benefit. Obviously the severity of the problem was underestimated (contrary to popular belief, Windows developers do care about their customers and would never have done this had they known the outcome). I've never been a big fan of taking away without giving something in return.

What would have prevented the problem? I don't know how hard this would have been to implement, but I would love to have some kind of adjustment knob to control the amount of latency I want on my system. That would have sidestepped the whole issue by allowing the customer to pick their priorities, i.e. lower latency on my desktop where there aren't power issues and where I want my games to sound great, higher latency on my laptop where I want additional power savings, the drivers aren't as good (and are also optimized for power savings), and where I'm never using the audio stack for anything other than music or videos anyway. This would also have been a great mitigation for driver issues during the transition period.

 This is probably a good lesson for developers in general -- be sure to consider the transition period between the old system and the new system when designing the new system.

In the meantime, I've finally gotten my audio problems worked out. After the latest motherboard chipset upgrade, I no longer have to run caffeine.exe anymore, and I only run into minor static when running a few specific programs. Hopefully things are improving for everybody else, too.

The OS Design View tab is one of the places where the old PB shell code (VS 6.0 vintage 1998) meets up with the new PB object model (totally re-written and designed for integration with the VS 2005 shell). So this is more evidence that you can't put new wine in old bottles -- you'll almost always end up with leaks.

In the case of the OS Design View, most of the issues I've resolved were due to an exception leaking. Exception handling is a great thing, but it has the good/bad quality that it is often easy to forget that you're calling a function that can throw an exception (good when you don't need to handle it, bad when you do...).

Ideally, the exception will be caught by PB and PB will properly recover and/or display a useful error message, but this doesn't always happen, in which case PB just silently fails or crashes. Luckily, the exception type we use does a little bit of logging. The exception constructor will send a quick message to the system debug stream. Sometimes this message is useless, but every once in a while, the exception has enough information to help you track down the problem.

There are several utilities that can monitor the debug stream. DBMon.exe is a simple console tool that is part of the Microsoft Platform SDK. DebugView can be downloaded from the SysInternals website. If you run cepb.exe under a debugger such as WinDbg or Visual Studio, you'll also see the messages from the debug stream.

In the case of Missing OS Design View tabs, I have occasionally been able to track down the problem by monitoring the system debug stream while opening an OS Design (pbxml). For example, I saw an "Access Denied" exception that occurred while trying to open a file in an OS Design. The exception message included the file name. Removing the "Read Only" attribute from the file allowed me to successfully open the OS Design.

Obviously, this is a problem with PB. PB should probably be able to do its job with read-only access to the file. And if it fails, it should at least pop up a useful error message indicating what the problem was. Hopefully, we'll do better in the future. In the meantime, you can try to identify the file that is causing trouble for PB and remove the read-only attribute. While this might cause trouble for your source control system, you'll hopefully at least be able to get back to work.

1. Start a debug stream monitor utility such as DbMon.exe or DebugView, then start cepb.exe. Alternatively, launch cepb.exe under a debugger such as Visual Studio or WinDbg.
2. Recreate the scenario that is causing trouble. For example, if the OS Design View tab is missing when you open a particular OS Design, open that OS Design.
3. Examine the messages logged to the debug output stream.
4. If you find any messages that look interesting, try to resolve the problem they indicate. For example, if you see that one of the messages refers to an access denied failure when opening a particular file, verify that the file exists and that you can access the file. If the file is read-only, try removing the read-only attribute from the file using Explorer (right-click on the file and select "Properties") or the DOS Attrib command.
5. Leave a comment below or contact me to let me know if this does or does not work for you!