Collecting Statistics

This package provides classes for basic operations with the samples collected by Austin.

Overview

Raw samples collected by Austin can be parsed into its many components with the austin.stats.Sample.parse() static method, which returns a austin.stats.Sample object. This is a data class with fields such as pid, thread, metrics and frames, representing the different parts of an Austin sample.

You can use austin.stats.AustinStats objects to aggregate profiling metrics, like own and total time, by providing it with austin.stats.Sample objects via the austin.stats.AustinStats.update() method. The following is a simple example of a on_sample_received callback implemented on an Austin wrapper object that parses the incoming samples and pass them to an austin.stats.AustinStats for aggregation:

from datetime import timedelta as td

from austin.simple import SimpleAustin
from austin.stats import AustinStats, InvalidSample, Sample


class MySimpleAustin(SimpleAustin):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self._stats = AustinStats()
        self._sample_count = 0
        self._error_count = 0

    def on_sample_received(self, text):
        try:
            self._stats.update(Sample.parse(text))
            self._sample_count += 1
        except InvalidSample:
            self._error_count += 1

    def on_terminate(self, *args, **kwargs):

        def print_frames(framestats):
            print("    {own: >16}  {total: >16}  {frame}".format(
                own=str(td(seconds=framestats.own.time/1e6)),
                total=str(td(seconds=framestats.total.time/1e6)),
                frame=framestats.label
            ))
            for _, child in framestats.children.items():
                print_frames(child)

        for pid, process in self._stats.processes.items():
            print(f"Process {pid or self.proc.pid}")
            for tid, thread in process.threads.items():
                print(f"  {tid}")
                print(f"    {'OWN':^16}  {'TOTAL':^16}  {'FRAME'}")
                for _, framestats in thread.children.items():
                    print_frames(framestats)


MySimpleAustin().start(["python3", "-c", "for _ in range(10000): print(_)"])

You can then collect the aggregated figures from the austin.stats.AustinStats object by navigating the processes and threads that have been profiled. You start by selecting or looping over the observed processes via the processes dictionary. This maps PIDs to austin.stats.ProcessStats objects holding the process statistics. Each austin.stats.ProcessStats object has a dictionary threads of all the observed threads within that process. This maps thread names to austin.stats.ThreadStats objects. The label attribute is just the thread name, while the frame stack can be navigated using the children attribute. This maps austin.stats.Frame objects to austin.stats.FrameStats objects holding the aggregated statistics of each observed frame within the thread.

The example above shows you how to use the austin.BaseAustin.on_terminate() callback to navigate the collected statistics and print a simple report once Austin has terminated.

The austin.stats sub-module

class austin.stats.AustinFileReader(file)

Bases: object

Austin file reader.

Conveniently read an Austin sample file by also parsing any header and footer metadata.

class austin.stats.AustinStats(stats_type: austin.stats.AustinStatsType, processes: Dict[int, austin.stats.ProcessStats] = <factory>, _lock: _thread.allocate_lock = <factory>)

Bases: object

Austin statistics.

This class is used to collect all the statistics about own and total time and/or memory generated by a run of Austin. The update() method is used to pass a new Sample so that the statistics can be updated accordingly.

dump(stream)

Dump the statistics to the given text stream.

Return type

None

get_process(pid)

Get process statistics for the given PID.

Return type

ProcessStats

classmethod load(stream)

Load statistics from the given text stream.

Return type

Dict[AustinStatsType, AustinStats]

update(sample)

Update the statistics with a new sample.

Normally, you would what to generate a new instance of Sample by using Sample.parse() on a sample string passed by Austin to the sample callback.

Return type

None

class austin.stats.AustinStatsType

Bases: enum.Enum

Austin stats type.

class austin.stats.Frame(function: str, filename: str, line: int = 0)

Bases: object

Python frame.

static parse(frame)

Parse the given string as a frame.

A string representing a frame has the structure

[frame] := <module>:<function>:<line number>

This static method attempts to parse the given string in order to identify the parts of the frame and returns an instance of the Frame dataclass with the corresponding fields filled in.

Return type

Frame

class austin.stats.FrameStats(label: austin.stats.Frame, own: austin.stats.Metric, total: austin.stats.Metric, children: Dict[austin.stats.Frame, FrameStats] = <factory>, height: int = 0)

Bases: austin.stats.HierarchicalStats

Frame statistics.

class austin.stats.HierarchicalStats(label: Any, own: austin.stats.Metric, total: austin.stats.Metric, children: Dict[Any, HierarchicalStats] = <factory>)

Bases: object

Base dataclass for representing hierarchical statistics.

The statistics of a frame stack can be thought of as a rooted tree. Hence the hierarchy is established by the parent/child relation between the nodes in this tree. An instance of this class represents a node, and a leaf is given by those instances with an empty children attribute.

The label attribute is used for indexing reasons and therefore should be of a hashable type.

This class overrides the default add operator so that one can perform operations like stats1 + stats2. Note, however, that instances of this class are not assumed to be immutable and indeed this operation will modify and return stats1 with the outcome of the addition.

collapse(prefix='')

Collapse the hierarchical statistics.

Return type

List[str]

get_child(label)

Get a child from the children collection.

Return type

HierarchicalStats

exception austin.stats.InvalidFrame

Bases: austin.AustinError

Invalid frame.

Thrown when attempting to parse a string that is supposed to represent a frame, but has the wrong structure.

exception austin.stats.InvalidSample

Bases: austin.AustinError

Invalid sample.

Thrown when attempting to parse a string that is supposed to represent a sample, but has the wrong structure.

class austin.stats.Metadata

Bases: dict

Austin Metadata.

add(line)

Add a metadata line.

Return type

None

class austin.stats.Metric(type: austin.stats.MetricType, value: int = 0)

Bases: object

Austin metrics.

copy()

Make a copy of this object.

Return type

Metric

static parse(metrics, metric_type=None)

Parse the metrics from a sample.

Returns a tuple containing the parsed metrics and the head of the sample for further processing.

Return type

List[Metric]

class austin.stats.MetricType

Bases: enum.Enum

Sample metric type.

class austin.stats.ProcessStats(pid: int, threads: Dict[str, austin.stats.ThreadStats] = <factory>)

Bases: object

Process statistics.

collapse()

Collapse the hierarchical statistics.

Return type

List[str]

get_thread(thread_name)

Get thread statistics from this process by name.

If the given thread name is not registered with this current process statistics, then None is returned.

Return type

Optional[ThreadStats]

class austin.stats.Sample(pid: int, thread: str, metric: austin.stats.Metric, frames: List[austin.stats.Frame] = <factory>)

Bases: object

Austin sample.

static is_full(sample)

Determine whether the sample has full metrics.

Return type

bool

static parse(sample, metric_type=None)

Parse the given string as a frame.

A string representing a sample has the structure

P<pid>;T<tid>[;[frame]]* [metric][,[metric]]*

This static method attempts to parse the given string in order to identify the parts of the sample and returns an instance of the Sample dataclass with the corresponding fields filled in.

Return type

List[Sample]

class austin.stats.ThreadStats(label: Any, own: austin.stats.Metric, total: austin.stats.Metric, children: Dict[Any, HierarchicalStats] = <factory>)

Bases: austin.stats.HierarchicalStats

Thread statistics.

collapse(prefix='T')

Collapse the hierarchical statistics.

Return type

List[str]