GitXplorerGitXplorer
r

molecule

public
410 stars
19 forks
1 issues

Commits

List of commits on branch master.
Verified
5557c9500595b1fbfa52f29907c4c60ca4ded5ea

Merge pull request #23 from richardartoul/richardartoul-patch-1

rrichardartoul committed 8 months ago
Verified
998a14d1703c37151af50de29fd70efe03addfbb

Upgrade Go in CI

rrichardartoul committed 8 months ago
Verified
429ac4f47e7015e8d54b8daaa7a6ceecc0028f4e

Merge pull request #22 from evanj/evan.jones/rm-unused-function

rrichardartoul committed 8 months ago
Verified
8db2523e5bef31273f524a7024fca7c768c71042

stream.go: Use protowire package; Remove zigzag32/64 functions

eevanj committed 8 months ago
Verified
4b90334d5d7c86720bbc771a8c7df72d9049d130

go.sum: go mod tidy

ddarccio committed 8 months ago
Verified
d2c048a2f4c2bc58e7a59d7646e6e92779744901

go.mod: upgrade google.golang.org/protobuf to v1.33.0

ddarccio committed 8 months ago

README

The README file for this repository.

GoDoc C.I

Molecule

Molecule is a Go library for parsing protobufs in an efficient and zero-allocation manner. The API is loosely based on this excellent Go JSON parsing library.

This library is in alpha and the API could change. The current APIs are fairly low level, but additional helpers may be added in the future to make certain operations more ergonomic.

Rationale

The standard Unmarshal protobuf interface in Go makes it difficult to manually control allocations when parsing protobufs. In addition, its common to only require access to a subset of an individual protobuf's fields. These issues make it hard to use protobuf in performance critical paths.

This library attempts to solve those problems by introducing a streaming, zero-allocation interface that allows users to have complete control over which fields are parsed, and how/when objects are allocated.

The downside, of course, is that molecule is more difficult to use (and easier to misuse) than the standard protobuf libraries so its recommended that it only be used in situations where performance is important. It is not a general purpose replacement for proto.Unmarshal(). It is recommended that users familiarize themselves with the proto3 encoding before attempting to use this library.

Features

  1. Unmarshal all protobuf primitive types with a streaming, zero-allocation API.
  2. Support for iterating through protobuf messages in a streaming fashion.
  3. Support for iterating through packed protobuf repeated fields (arrays) in a streaming fashion.

Not Supported

  1. Proto2 syntax (some things will probably work, but nothing is tested).
  2. Repeated fields encoded not using the "packed" encoding (although in theory they can be parsed using this library, there just aren't any special helpers).
  3. Map fields. It should be possible to parse maps using this library's API, but it would be a bid tedious. I plan on adding better support for this once I settle on a reasonable API.
  4. Probably lots of other things.

Examples

The godocs have numerous runnable examples.

Attributions

This library is mostly a thin wrapper around other people's work:

  1. The interface was inspired by this jsonparser library.
  2. The codec for interacting with protobuf streams was lifted from this protobuf reflection library. The code was manually vendored instead of imported to reduce dependencies.

Dependencies

The core molecule library has zero external dependencies. The go.sum file does contain some dependencies introduced from the tests package, however, those should not be included transitively when using this library.