DRAFT: Take more than your usual care.


Right from its introduction for Windows 2000, before it even had its current name, Event Tracing for Windows (ETW) has parallel implementations in kernel and user modes. The practical difference is not just with where the code is but with where the trace buffers are managed. One aim of ETW is that events are written quickly into trace buffers which are then serviced independently from a logger thread, e.g., to flush full buffers to an Event Trace Log (ETL) file. A tracing session that enables only event providers that execute in the same user-mode process can choose the user-mode ETW implementation so that the trace buffers are managed in user mode and writing an event (to a trace buffer) avoids the delay of going to and from kernel mode. The ETW_UM_LOGGER_CONTEXT structure is what NTDLL—and before it, ADVAPI32—keeps for managing an event logger, known more formally as an event tracing session, that has user-mode trace buffers.

Documentation Status

Microsoft seems to have been unusually careful about publicly disclosing even a name for this structure. This article originally resorted to naming the structure only as the user-mode WMI_LOGGER_CONTEXT for its correspondence with a kernel-mode structure for which Microsoft’s names (not only of the structure but also of its members) are known from public symbol files, starting with Windows XP SP3 and Windows Server 2003 SP2. Work backwards through the kernel binaries to the kernel-mode structure’s origin in Windows 2000, and it is abundantly plain that whatever the user-mode structure that’s described in this article has ever been named, it and the kernel-mode WMI_LOGGER_CONTEXT started with a shared definition: what members they have in common are laid out in the same order. Though the layouts have since diverged, such that they seem unlikely to be differentiated just by preprocessor directives for conditional compilation, they surely still are developed each with one eye on the other.

The wonder is that Microsoft’s names and types for this structure haven’t slipped out long before. After all, its kernel-mode correspondent seems to be no more secret than many another internal detail, if only to be disclosed as an aid to kernel-mode debugging. Perhaps I’ve just missed the publication or not thought what obvious names to search for.

Or not. User-mode ETW has much about it that Microsoft demonstrably doesn’t document. This applies especially to functionality that is implemented for user-mode loggers but not (yet) for tracing through the kernel—and happens often enough to suggest a pattern of meaning very much not to document it. Consider, for instance, that Windows 8 and higher provide for user-mode loggers to compress what they flush to ETL files, but this useful trick isn't implemented for kernel-mode tracing until later releases of Windows 10. Any programmer or administrator who has ever worried how big their user-mode tracing session’s ETL files may get will plausibly think that the EVENT_TRACE_COMPRESSED_MODE flag for the logging mode might be a helpful thing to know. Microsoft surely agrees, else why does the Start-EtwTraceSession command in PowerShell have a -Compress switch? Yet, if only according to Google, today, 18th December 2018, the whole of Microsoft’s website has just one page that mentions this flag that programmers would need to know for starting an event tracing session that compresses its own ETL file—and this page is not programmer documentation but PowerShell user documentation. For programmers, EVENT_TRACE_COMPRESSED_MODE didn’t even get a C-language definition in EVNTRACE.H until the Software Development Kit (SDK) for the 1607 edition of Windows 10 and it still isn’t listed among the Logging Mode Constants. Such lack of disclosure is not credibly by accident or oversight. The more likely role of oversight is that one mention was allowed to slip out in PowerShell documentation.

Thus does it not surprise that Microsoft’s name for the ETW_UM_LOGGER_CONTEXT has slipped into public view but in a very particular way that confirms Microsoft’s intention to treat the structure as much more for the private use of Microsoft’s own programmers than are many another internal detail. The only known public source of Microsoft’s name ETW_UM_LOGGER_CONTEXT is in the Debugging Tools for Windows package, starting from the Windows Driver Kit (WDK) for Windows Server 2008. This package has long included a debugger extension named WMITRACE.DLL which is specialised for helping with ETW. It is written to do very much more if it has the use of private symbol files, which Microsoft presumably keeps to itself, rather than just the public symbol files that are readily distributed as debugging aids. It expects that private symbol files for NTDLL have type information for ETW_UM_LOGGER_CONTEXT. The text it would use to form its queries when private symbol files are available is in the executable—in something like plain sight—even when private symbol files are not available.


Curiously, the ETW_UM_LOGGER_CONTEXT has been far more stable than its kernel-mode counterpart. It has grown, of course, and members have come and gone, but there has been only one large-scale rearrangement (for version 6.1) and the move from ADVAPI32 to NTDLL in version 5.2 didn’t change the structure at all. The following changes of size are known:

Versions Size (x86) Size (x64)
5.0 0xD0  
5.1 to 5.2 0xD8 0x0120
6.0 0xF8 0x0150
6.1 0x0120 0x01A0
6.2 0x0160 0x0210
6.3 to 10.0 0x0170 0x0220

These sizes, and the offsets, types and names in the detailed layout below, come from inspection of the binaries for ADVAPI32 (in its versions 5.0 and 5.1) and NTDLL (in its versions 5.2 and higher), and comparison both with similar code in contemporaneous versions of the kernel and with the offsets, types and names that are known for the kernel-mode structure from public symbol files for the kernel. Where correspondence seems close, it seems reasonable to assume that types and names are the same for both the kernel-mode and user-mode structures. Such analysis is inevitably inexact and prone to oversight. Editorial decisions for cases where correspondence is not so close are explained after the table.

Offset (x86) Offset (x64) Definition Versions Remarks
0x00 (5.0 to 6.0) 0x00 (5.2 to 6.0)
5.0 to 6.0  
0x00 0x00
ETW_REF_CLOCK ReferenceTime;
6.1 and higher  
0x10 0x10
ULONG ClockType;
6.1 and higher previously at 0x94 and 0xD4
0x08 (5.0 to 6.0) 0x08 (5.2 to 6.0)
HANDLE LogFileHandle;
5.0 to 6.0 next at 0x5C and 0x80
0x0C (5.0 to 6.0) 0x10 (5.2 to 6.0) unaccounted four or eight bytes 5.0 to 6.0  
0x10 (5.0 to 6.0);
0x18 (5.2 to 6.0);
ULONG LoggerId;
5.0 and higher  
0x14 (5.1 to 6.0) 0x1C (5.2 to 6.0)
ULONG LocalSequence;
5.1 to 6.0 next at 0x0108 and 0x0180
0x14 (5.0);
0x18 (5.1 to 6.0)
0x20 (5.2 to 6.0)
HANDLE LoggerSemaphore;
5.0 to 6.0  
0x18 (5.0);
0x1C (5.1 to 6.0);
0x28 (5.2 to 6.0);
HANDLE LoggerThreadId;
5.0 and higher  
0x1C (5.0);
0x20 (5.1 to 5.2);
not in 6.0;
0x30 (5.2);
not in 6.0;
HANDLE LoggerThread;
5.0 to 5.2;
not in 6.0;
6.1 and higher
0x20 (5.0);
0x24 (5.1 to 5.2);
0x20 (6.0)
0x38 (5.2);
0x30 (6.0)
HANDLE LoggerEvent;
5.0 to 6.0 next at 0x58 and 0x78
0x24 (5.0);
0x28 (5.1 to 5.2);
0x24 (6.0);
0x40 (5.2);
0x38 (6.0);
DWORD LoggerStatus;
5.0 and higher  
0x24 0x2C
ULONG EventMarker [1];
6.2 and higher  
0x28 0x30
ULONG ErrorMarker;
6.2 and higher  
0x2C 0x34
ULONG SizeMask;
6.2 and higher  
0x24 (6.1);
GUID InstanceId;
6.1 and higher previously at 0xC4 and 0x0108
0x34 (6.1);
ULONG InstanceCount;
6.1 and higher  
0x38 (6.1);
0x40 (6.1);
unknown CONDITION_VARIABLE 6.1 and higher  
0x3C (6.1);
0x48 (6.1);
unknown CRITICAL_SECTION 6.1 and higher previously at 0xD8 and 0x0120
0x54 (6.1);
0x70 (6.1);
HANDLE FlushEvent;
6.1 and higher  
0x58 (6.1);
0x78 (6.1);
HANDLE LoggerEvent;
6.1 and higher previously at 0x20 and 0x30
0x5C (6.1);
HANDLE LogFileHandle;
6.1 and higher previously at 0x08
0x28 (5.0);
0x2C (5.1 to 5.2);
0x28 (6.0)
0x44 (5.2);
0x3C (6.0)
LONG BuffersAvailable;
5.0 to 6.0 next at 0x98 and 0xD8
0x2C (5.0);
0x30 (5.1 to 5.2);
0x2C (6.0)
0x48 (5.2);
0x40 (6.0)
ULONG NumberOfProcessors;
5.0 to 6.0 next at 0x7C and 0xBC
0x30 (5.0);
0x34 (5.1 to 5.2);
0x30 (6.0)
0x4C (5.2);
0x44 (6.0)
unknown 32-bit page-aligned buffer size 5.0 to 6.0 next at 0x88 and
0x34 (5.0);
0x38 (5.1 to 5.2);
0x34 (6.0)
0x50 (5.2);
0x48 (6.0)
5.0 to 6.0  
0x3C (5.0);
0x40 (5.1 to 5.2);
0x3C (6.0)
0x60 (5.2);
0x58 (6.0)
LIST_ENTRY GlobalList;
5.0 to 6.0 next at 0x9C and 0xE0
0x44 (5.0);
0x48 (5.1 to 5.2);
0x44 (6.0)
0x70 (5.2);
0x68 (6.0)
unknown pointer 5.0 to 6.0  
0x48 (5.0);
0x4C (5.1 to 5.2);
0x48 (6.0)
0x78 (5.2);
0x70 (6.0)
WMI_BUFFER_HEADER **ProcessorBuffers;
5.0 to 6.0  
0x4C (5.0);
0x50 (5.1);
0x4C (6.0);
0x60 (6.1);
0x80 (5.2);
0x78 (6.0);
0x88 (6.1);
5.0 and higher  
0x54 (5.0);
0x58 (5.1);
0x54 (6.0);
0x68 (6.1);
0x90 (5.2);
0x88 (6.0);
5.0 and higher  
0x70 (6.1);
6.1 and higher  
0x78 (6.1);
LONG FileCounter;
6.1 and higher  
0x7C (6.1);
ULONG NumberOfProcessors;
6.1 and higher previously at 0x2C and 0x40
0x80 (6.1);
0xC0 (6.1);
ULONG BufferSize;
6.1 and higher previously at 0xA0 and 0xE0
0x84 (6.1);
ULONG MaximumEventSize;
6.1 and higher  
0x88 (6.1);
  unknown 32-bit page-aligned buffer size 6.1 and higher previously at 0x30 and 0x44
0x8C (6.1);
0xCC (6.1);
ULONG MaximumBuffers;
6.1 and higher previously at 0xA8 and 0xE8
0x90 (6.1);
0xD0 (6.1);
ULONG MinimumBuffers;
6.1 and higher previously at 0xAC and 0xEC
0x94 (6.1);
0xD4 (6.1);
ULONG NumberOfBuffers;
6.1 and higher previously at 0xA4 and 0xE4
0x98 (6.1);
0xD8 (6.1);
LONG BuffersAvailable;
6.1 and higher previously at 0x28 and 0x3C
0x9C (6.1);
0xE0 (6.1);
LIST_ENTRY GlobalList;
6.1 and higher previously at 0x3C and 0x58
0xA4 (6.1);
0xF0 (6.1);
6.1 and higher  
0xB0 (6.1);
0x0108 (6.1);
unknown ETW_BUFFER_QUEUE 6.1 and higher  
0xBC (6.1);
0x0120 (6.1);
6.1 and higher  
0x5C (5.0);
0x60 (5.1 to 5.2);
0x5C (6.0)
0xA0 (5.2);
0x98 (6.0)
LONG CollectionOn;
5.0 to 6.0 next at 0xD0 and 0x0140
0x60 (5.0);
0x64 (5.1 to 5.2)
0xA4 (5.2) unaccounted eight bytes 5.0 to 5.2  
0x60 (6.0) 0x9C (6.0)
ULONG RequestFlag;
6.0 only next at 0xD4 and 0x0144
0x68 (5.0);
0x6C (5.1 to 5.2);
0x64 (6.0);
0xC8 (6.1);
0xAC (5.2);
0xA0 (6.0);
0x0138 (6.1);
ULONG MaximumFileSize;
5.0 and higher  
0x6C (5.0);
0x70 (5.1 to 5.2);
0x68 (6.0);
0xCC (6.1);
0xB0 (5.2);
0xA4 (6.0);
0x013C (6.1);
ULONG LoggerMode; 
5.0 and higher  
0xD0 (6.1);
LONG CollectionOn;
6.1 and higher previously at 0x5C and 0x98
0xD4 (6.1);
union {
    ULONG RequestFlag;
    struct {
        /*  changing bit fields  */
6.1 and higher previously at 0x60 and 0x9C
0x70 (5.0);
0x74 (5.1 to 5.2);
0x6C (6.0);
0xD8 (6.1);
0xB4 (5.2);
0xA8 (6.0);
ULONG LastFlushedBuffer;
5.0 and higher  
0x78 (5.0 to 5.2);
0x70 (6.0);
0xE0 (6.1);
0xB8 (5.2);
0xB0 (6.0);
0x0150 (6.1);
5.0 and higher  
0x80 (5.0 to 5.2);
0x78 (6.0);
0xC0 (5.2);
0xB8 (6.0);
LARGE_INTEGER FirstBufferOffset;
5.0 and higher  
0x88 (5.0 to 5.2);
0x80 (6.0);
0xF0 (6.1);
0xC8 (5.2);
0xC0 (6.0);
5.0 and higher  
0x90 (5.0 to 5.2);
0x88 (6.0)
0xD0 (5.2);
0xC8 (6.0)
5.0 to 6.0  
0x98 (5.1 to 5.2);
0x90 (6.0)
0xD8 (5.2);
0xD0 (6.0)
unaccounted four bytes 5.1 to 6.0  
0x9C (5.1 to 5.2):
0x94 (6.0)
0xDC (5.2);
0xD4 (6.0)
ULONG ClockType;
5.1 to 6.0 next at 0x10
0x98 (6.0) 0xD8 (6.0)
LARGE_INTEGER ReferenceTimeStamp;
6.0 only  
0x98 (5.0);
0xA0 (5.1 to 6.0)
0xE0 (5.2 to 6.0)
ULONG BufferSize;
5.0 to 6.0 next at 0x80 and 0xC0
0x9C (5.0);
0xA4 (5.1 to 6.0)
0xE4 (5.2 to 6.0)
ULONG NumberOfBuffers;
5.0 to 6.0 next at 0x94 and 0xD4
0xA0 (5.0);
0xA8 (5.1 to 6.0)
0xE8 (5.2 to 6.0)
ULONG MaximumBuffers;
5.0 to 6.0 next at 0x8C and 0xCC
0xA4 (5.0);
0xAC (5.1 to 6.0)
0xEC (5.2 to 6.0)
ULONG MinimumBuffers;
5.0 to 6.0 next at 0x90 and 0xD0
0x0100   unaccounted eight bytes 6.2  
0x0100 0x0178
ULONG FlushThreshold;
6.3 and higher  
0xA8 (5.0);
0xB0 (5.1 to 6.0);
0xF8 (6.1);
0x0108 (6.2);
0xF0 (5.2 to 6.0);
0x0168 (6.1);
0x0180 (6.2);
ULONG EventsLost;
5.0 and higher  
0xFC (6.1);
0x010C (6.2);
0x016C (6.1);
0x0184 (6.2);
ULONG LogBuffersLost;
6.1 and higher previously at 0xB8 and 0xF8
0xAC (5.0);
0xB4 (5.1 to 6.0);
0x0100 (6.1);
0x0110 (6.2);
0xF4 (5.2 to 6.0);
0x0170 (6.1);
0x0188 (6.2);
ULONG BuffersWritten;
5.0 and higher  
0xB0 (5.0);
0xB8 (5.1 to 6.0)
0xF8 (5.2 to 6.0)
ULONG LogBuffersLost;
5.0 to 6.0 next at 0xFC and 0x016C
0xB4 (5.0);
0xBC (5.1 to 6.0)
0xFC (5.2 to 6.0)
ULONG RealTimeBuffersLost;
5.0 to 6.0  
0xC0 (5.1 to 6.0);
0x0104 (6.1);
0x0114 (6.2);
0x0100 (5.2 to 6.0);
LONG *SequencePtr;
5.1 and higher  
0x0108 (6.1);
0x0118 (6.2);
ULONG LocalSequence;
6.1 and higher previously at 0x14 and 0x1C
0xB8 (5.0);
0xC4 (5.1 to 6.0)
0x0108 (5.2 to 6.0)
GUID InstanceId;
5.0 to 6.0 next at 0x24 and 0x2C
0xD4 (6.0) 0x0118 (6.0) unknown dword 6.0 only  
0xD8 (6.0) 0x0120 (6.0) unknown CRITICAL_SECTION 6.0 only next at 0x3C and 0x48
0x0110 (6.1);
0x0120 (6.2);
LONGLONG BufferSequenceNumber;
6.1 and higher  
0xC8 (5.0);
0xD4 (5.1 to 5.2);
0xF0 (6.0);
0x0118 (6.1);
0x0128 (6.2);
0x0118 (5.2);
0x0148 (6.0);
unknown pointer to trace buffers 5.0 and higher last member in 5.0 to 6.0

There is something to write here!

Offset (x86) Offset (x64) Definition Versions
0x012C (6.2);
0x01B0 (6.2);
PVOID CompressionWorkspace;
6.2 and higher
0x0130 (6.2);
0x01B8 (6.2);
buffer for compressed data 6.2 and higher
0x0134 (6.2);
0x01C0 (6.2);
32-bit size of compression buffer 6.2 and higher
0x0138 (6.2);
0x01C4 (6.2);
32-bit size of trace buffers in compression buffer 6.2 and higher
0x0144 0x01D0 32-bit count of trace buffers in compression buffer 6.3 and higher
0x0148 0x01D4 32-bit size of partial trace buffer in compression buffer 6.3 and higher

As noted in the introduction, user-mode loggers can compress event data in version 6.2 and higher but there is no kernel-mode correspondent until the 1607 release of Windows 10. The implementations are very different. The only known name that seems safe to take as common to both implementations is that of the buffer that’s used for the algorithm’s workspace.

A user-mode logger compresses successive trace buffers ever deeper into a compression buffer that’s the size of two trace buffers. The first is flushed whenever it fills. Any overflow is then moved down to become a partial trace buffer at the start of the compression buffer. See that any one flush from the compression buffer is of one trace buffer’s worth of data that has been compressed from potentially many trace buffers. Version 6.2 counts them altogether as one buffer written or lost. Version 6.3 tracks their number. If flushing fails, version 6.2 does not reset its compression buffer. In later versions, all whole trace buffers that were in the compression buffer are explicitly lost. If the compression buffer began with the overflow data of a partly flushed trace buffer, then this overflow is retained in the compression buffer so that any subsequent flush that does succeed will complete the partial trace buffer that is already in the ETL file.

Offset (x86) Offset (x64) Definition Versions
0x013C (6.2);
0x01C8 (6.2);
LIST_ENTRY ProviderBinaryList;
6.2 and higher
0x0144 (6.2);
0x01D8 (6.2);
LIST_ENTRY WinRtProviderBinaryList;
6.2 and higher

It seems strange now, but not until version 6.2 do ETL files record which providers are enabled for the session.

Offset (x86) Offset (x64) Definition Versions
0x014C (6.2);
0x01F8 unknown dword 6.2 and higher
0x0150 (6.2);
0x0200 unknown pointer to array of pointers 6.2 and higher
0x0154 (6.2);
0x0208 unknown pointer to array of structures 6.2 and higher
0x0158 (6.2);
0x0200 (6.2);
unknown pointer to processor stream index map 6.2 and higher

There is something to write here!

Offset (x86) Offset (x64) Definition Versions Remarks
0x011C (6.1);
0x015C (6.2);
0x0198 (6.1);
0x0208 (6.2);
6.1 and higher last member in 6.1 and higher

Where the kernel-mode WMI_LOGGER_CONTEXT has objects such as semaphores and events, and pointers to objects such as threads, a user-mode structure can only have handles. Although the applicable members must change type, they needn’t change names and it is assumed above that they don’t.

Each logger creates its own thread for flushing trace buffers. All versions keep the thread ID and disclose it, when queried, by copying it to the LoggerThreadId member of a WMI_LOGGER_INFORMATION. It is here proposed that the thread ID as kept in the logger context likely has the same name (and type). The early versions, implemented in ADVAPI32, also keep a handle to the logger thread. This stops with version 5.2, but it is assumed above that the LoggerThread remained defined (no reuse of its space being known). Version 6.0 seems then to have removed it, only to have version 6.1 go back to keeping the handle.

The unknown CONDITION_VARIABLE supports EVENT_TRACE_BLOCKING_MODE. If tracing an event would exhaust its processor’s current trace buffer but all other trace buffers are in use (including because they are yet to be flushed) and no more can be created, then the event would ordinarily be lost. Waiting for a trace buffer to become available isn’t much of an option. After all, much of the point to tracing an event is that it is done without disturbing whatever activity is being traced. For some loggers, however, the intended activity may be relatively infrequent and take time anyway, such that time spent waiting for a buffer to trace an event to is less a penalty than would be the event’s loss. Blocking mode allows the wait. It’s another ETW feature that’s particular to user-mode tracing sessions (and is not documented).

See that in all versions, a user-mode logger reserves address space sufficient for whatever it ends up adopting as MaximumBuffers. It immediately commits memory for whatever it ends up adopting as MinimumBuffers, and thereafter commits more only when more buffers actually are needed for whatever flow of events are received. For this purpose of reserving and committing, each trace buffer is a whole number of pages, rounded up from BufferSize bytes. Neither the page-aligned buffer size nor the address of the reservation has any correspondent in the kernel-mode structure.

Versions 6.1 and higher end the structure with a variable-size array of pointers, one per processor, each to the trace buffer that is currently in use for that processor. This too has no correspondence with the kernel-mode structure. It does, however, supersede what had been a pointer to just such an array, and in such a way that keeping the name would require no change to the source code. The pointer’s name, ProcessorBuffers, is known from correspondence with the kernel-mode structure.