GitXplorerGitXplorer
s

metrix

public
5 stars
0 forks
0 issues

Commits

List of commits on branch master.
Unverified
286998edf03088f749f3e6be243d34e4b5f7307c

2.0.0

sstevegury committed 8 years ago
Unverified
459097d8eacf8c9b9bd7f4048eded306fc0844a2

Refactor Gauges

sstevegury committed 8 years ago
Unverified
a92ad610572bb7b46bf3a9eeefa945dede1b6de7

1.3.0

sstevegury committed 8 years ago
Unverified
d13664e97d043c9d7a34f462c3d1e1614931156a

Introduce new clearing API (1 per metric type)

sstevegury committed 8 years ago
Unverified
ad3e86ef9420bdb00a97003e612bbb85cc277f85

1.2.1

sstevegury committed 8 years ago
Unverified
87489d3fbc1bc6dc4e37e57548c4632f1b4f8fa6

1.2.0

sstevegury committed 8 years ago

README

The README file for this repository.

Metrix

Architecture

The library is composed of two central objects: the Recorder and the Aggregator.

You use the Recorder to create Counters, Timers and Histogram, then the Counters, Timers and Histogram are used to send events to the Recorder.

On the other end, the Aggregator listen to events emitted by the Counters / Timers / Histogram and aggregate them. The default Aggregator aggregates counter events as a single value (the latest) and timer event into an histogram. But nothing is preventing you for having different aggregating strategy, for instance you may want to keep a min/avg/latest/max structure for every counters.

The library provides two histograms implementations, one based on the classical bucketed algorithm and one based on the more inovative frugal streaming estimation. The first one is as precise as you want with an impact on speed and memory footprint, the second one is extremly fast with almost no memory impact but is less precise.

Usage

Initializing the library simply like this:

var recorder = new Recorder();
var aggregator = new Aggregator(recorder);

Creating/using counter/timer:

const myCounter = recorder.counter('my_counter');
myCounter.incr();
myCounter.incr();

const myHisto = recorder.histogram('my_histo');
myHisto.add(12);
myHisto.add(34);

const latencyTimer = recorder.timer('latency');
const id = latencyTimer.start();
// do something...
latencyTimer.stop(id);

Creating a report:

aggregator.report();
aggregator.clear();

This will generate a report like this:

{
    counters: {
        my_counter: 2
    },
    histograms: {
        latency: {
            min: 1,
            max: 56,
            p50: 13.4,
            p90: 34.1,
            p99: 53.2
        }
    }
}

Scoping

The Recorder has a scope method, which will return a new Recorder, every new Counter/Timer created with this new recorder will have their name prefixed by the scope value you provided during scoping.

Scoping is encouraged for categorizing metrics into different namespaces.

Tags

You can add tags to a specific counter / timer, the aggregator is free to do anything with the tags.

aggregator.test.js has a test that demonstrate how to use tags for creating histograms per request url, despite only measuring the latency once.

Configuration

The configuration is pretty flexible and you can control how you send event at both recording-time and aggregation-time.

The config Object contains two entries recorder and aggregator. recorder contains two entries counter, timer and histogram, which are functions used for creating a new Counter / Timer / Histogram. This function can be used to decide how to record metrics event (e.g. disable specific event recording). aggregator contains three entries counter, timer and composites. The first two are used for controlling how to aggregate events, they can create new histogram / counter at will based on the events it receives.

In the following example, I disabled all timers execpt request_latency_ms, I also swaped out the default histogram for a more precise one with more quantiles, and also created a new metrics connections/current out of two existing counters (connections/add and connections/remove).

var config = {
    recorder: {
        timer: function (recorder, name, tags) {
            if (name === 'request_latency_ms') {
                return DefaultTimer(recorder, name, tags);
            } else {
                return NullTimer;
            }

        }
    },
    aggregator: {
        timer: function (event, histograms, counters) {
            if (!histograms[event.name]) {
                histograms[event.name] = new BucketedHistogram({
                    max: 60 * 1000,                    // 1 minute
                    error: 2 / 100,                    // 2% precision
                    quantiles: [0.5, 0.9, 0.99, 0.999] // default quantiles
                });
            }
            var duration = event.stopTs - event.startTs;
            histograms[event.name].add(duration);
        },
        composites: function (counters, histograms) {
            var current = counters['connections/add'] - counters['connections/remove'];
            return ['connections/current', current];
        },
    }
};

var recorder = new Recorder(config.recorder);
var aggregator = new Aggregator(recorder, config.aggregator);

Convention

It's always better to adopt similar convention in a project, here are the conventions we picked:

Stats names

Counters and Timers names are snake_case and should be postfixed by the unit they represent, e.g request_latency_ms.

Passing Recorder

Recorder has to be passed from Object to Object, the convention is that you never scope the Recorder before passing it, you let the receiving class scope it with its desired namespace.

e.g.

var conn = new Connection(recorder)

// ...

function Connection(_recorder) {
    var recorder = _recorder.scope('connection');
    this.add = recorder.counter('add');
}