From Metadata to Event block in nettrace format

The previous episodes started the parsing of the “nettrace” format used when contacting the .NET Diagnostics IPC server, initiate the protocol to receive CLR events and start to parse stacks. This last episode covers the Metadata and Event blocks.

In terms of format, both Metadata and Event blocks share the same memory layout:

The common EventBlockHeader starts the block:

The timestamp fields give the time of the first and last event in the block. The HeaderSize fields is important because additional information can be stored in the header. Since I have no idea what could be stored there, I simply skip it:

The important piece of information to figure out how to unpack the rest of the block is kept in the Flags field. If the lowest bit is set, it means that the blobs header will be compressed:

The rest of the code iterates on each blob:

Here is the tricky part: to gain space, each blob starts with a header that could be “compressed”. The compression mechanism is simple: the first byte is a bitfield value that indicates which fields are present (i.e. their value should be read from the memory block) or skipped (i.e. their value is the same as the previous blob header). Therefore, an EventBlobHeader is passed by reference to the OnParseBlob function. My MetadataParser and EventParser implementations of OnParseBlob both starts with the same code to read the header:

The implementation to read compressed and uncompressed version of the header is a direct translation of the TraceEvent C# code into C++.

The EventBlobHeader contains details of events:

  • The “identity” of an event is given by the MetadataId field that refers to information defined in Metadata “object“ (for which MetadataId is 0).
  • The SequenceNumber field is incremented on a per thread basis each time an event is emitted. This could be used to detect if some events have been dropped (for a given CaptureThreadId, two consecutive events have a SequenceNumber incremented by more than 1 — more on dropped events in the forthcoming SequencePoint “object” description). Its value is 0 for a metadata “object”
  • The ThreadId and CaptureThreadId field have always the same value for Event “object”; CaptureThreadId is 0 for Metadata “object”.
  • In case of Event “object”, the StackId field refers to one of the stacks extracted from a Stack “object”. Its value is 0 for Metadata “object”.

The Metadata “object”

As the documentation states, each MetadataBlock holds a set of metadata records. Each metadata record has an ID and it describes one type of event. Each event has a metadataId field which will indicate the ID of the metadata record which describes that event.

The resulting mapping is stored in EventPipeSession class:

However, the rest of the documentation is partially right in the case of nettrace stream received through EventPipe: Metadata includes an event name, provider name, and the layout of fields that are encoded in the event’s payload section.

First, the fields layout is simply not there. In addition, for some providers (dotnet runtime, private and rundown), the event names are empty strings. So, the data structure filled from the MetadataBlock will most of the time have an empty EventName field. Note that the “Microsoft-DotNETCore-EventPipe” provider (i.e. command events for that specific provider) and EventSource-derived classes written in C# provide the events name:

In addition to the provider’s name serialized as a UTF16 string (including last ‘\0’ wide character), the EventId field is the key used to identify an event.

After these details, you will find a 4 bytes value corresponding to the number of fields in the event payload. As already mentioned, this value is always 0 so my code is skipping the rest of the metadata block payload.

The Event “object”

And at last, here comes the time to parse Event “object” payload! The MetadataId field of the EventBlobHeader is used to find the provider’s name and event id:

So, the rest of the function reads the payload based on the expected event id:

The format of each event payload is usually given by the Microsoft documentation. If not, you should look into the file where the payload of ALL events are defined. For example, the AllocationTick event payload provides the name of the last allocated type to reach the 100 KB threshold (read this blog post for more details about how to use this event):

Based on this fields definition, the EventParser::OnAllocationTick function is reading each field after the other thanks to the ReadWord, ReadDWord, ReadLong and ReadWString :

The bitness of the monitored application is important when “pointers” need to be read from the payload: use ReadDWord for 32-bit and ReadLong for 64-bit:

And if you don’t need the rest of the payload, SkipBytes is your friend:

I had some issues when dealing with the ExceptionThrown event payload:

In case of an empty message, the field itself was not even there! Not even 0 for a ‘\0’ wide character… In fact, there is a bug in the serialization code that skips the field in that case. This has been fixed in .NET 6 by storing “NULL” as the serialized string: I would have preferred ‘\0’ but it seems to be compatible with the ETW implementation.

To support .NET Core 3+ and .NET 5, my code is comparing the size of the remaining of the payload after reading the exception type with the expected size of the 4 remaining fields after the exception message. If it is greater then it means that there is a string for the message. If not, I know that the message is empty:

The SequencePointBlock “object”

The last “object” type is the sequence point block that contains the following fields:

In addition to these fields, it also implicitly tells you that new stack “object” will be received (with stack id restarting from 1) to match next Event “objects”. For example, the following trace shows how a sequence point block resets the stacks by restarting at 1:

So, the stacks you might have cached based on the already received stack “objects” should now be invalidated like what I’m doing in SequencePointParser::OnParse:

You now have all the elements you need to listen to CLR events on Windows and Linux for .NET Core 3+ and .NET 5+. If you are still running applications with .NET Framework, you will need to use ETW but this is another story.


  • Episode 1Digging into the CLR Diagnostics IPC Protocol in C#
  • Episode 2.NET Diagnostic IPC protocol: the C++ way
  • Episode 3 CLR events: go for the nettrace file format!
  • Episode 4 Parsing the “nettrace” steam
  • Episode 5 Reading “object” in memory — starting with stacks
  • Source code for the C++ implementation of CLR events listener
  • Diagnostics IPC protocol documentation



Loves to understand how things work (MVP Developer Technologies)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store