Parsing the “nettrace” stream of (not only) events

The previous episodes explained how to contact the .NET Diagnostics IPC server and initiate the protocol to receive CLR events. It is now time to dig into the “nettrace” stream format!

As the IPC command documentation states, the response to the CollectTracing command is followed by an Optional Continuation of a nettrace format stream of events. In fact, before .NET Core 3, the netperf format was used but I will focus on the nettrace format also used in .NET 5+.

From a high-level view, it is a header followed by a stream of “objects”; each described by a header and ending with a byte with 6 as value:

Let’s start with the nettrace header:

It can be used to check the format and version of the received data (in case of format evolution over time):

In memory, the “strings” are stored as an array of UTF8 characters without trailing ‘\0’

This drives the implementation of the comparison helper:

Everything is an “object”

After the header, data is represented as “objects” whose description is stored in an ObjectHeader:

followed by the name of the object type in UTF8. Note that, like for “!FastSerialization.1” in NettraceHeader, its length is provided in the NameLength field of the ObjectHeader. For example, here is how the initial TraceObject header is stored in memory:

and its equivalent in code:

Note that after the “object” type name, an “end of object” byte (i.e. = 6) appears before the payload.

So, each kind of “object” shares the same ObjectHeader followed by a UTF8 type name:

• “EventBlock” : contains one or more events
• “MetadataBlock” : contains partial description of events (no name nor payload fields)
• “StackBlock” : contains call stacks (i.e. arrays of instruction pointers)
• “SPBlock” : contains check point inside the stream inside the stream (used for drop message detection and callstack cache invalidation)

It means that you need to compare the strings to figure out the “object” type:

The first object that appears in the stream contains details about the whole stream:

After the header, some fields follow as a payload:

Beyond the timestamp information and the pointer size (required when call stack instruction pointers will be read as 8 (64-bit) or 4 (32-bit) bytes addresses) the other fields are not really interesting.

Let’s go back to a higher-level view

Here is the code to listen to the nettrace stream:

I will come back to the different _XXXparser fields soon.

The ReadNextObject helper is responsible for reading the expected ObjectHeader and the string that follows to figure out what is the type of this “object” and what payload to expect:

The GetObjectType function checks the header validity and extracts the “object” type name:

The same IsSameAsString helper is used to compare the read “object” name with the known types.

The end of ReadNextObject simply parses the “object” payload as a memory block based on its type:

The next step will be to look into each “object” type payload.

Resources

• Episode 1 — Digging into the CLR Diagnostics IPC Protocol in C#
• Episode 2 — .NET Diagnostic IPC protocol: the C++ way
• Episode 3 — CLR events: go for the nettrace file format!
• Source code for the C++ implementation of CLR events listener
• Diagnostics IPC protocol documentation