What the hell is protobuf?
It’s speed. It’s modern. Or is it?
What’s protocol buffers?
Protobuf is a data serializing protocol like a JSON or XML. But unlike them, the protobuf is not for humans, serialized data is compiled bytes and hard for the human reading.
It's description from Google official page:Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.
Why do we need another format for data serialization?
Modern server architectures is built on the constant communication of services. It can be REST API, GraphQl, RPC, Queues, etc. Services generate thousands of messages to each other, load the network and require a lot of resources. We need a fast way to serialize for transferring compact data between services.
A buffer can save us money and resources in the clouds like aws or gcloud.
Benchmarks (as we like it)
Encode structure by transfer bytes with json and protobuf comparing:
BenchmarkSerializeToJson-8 3000000 527 ns/op 112 B/op 1 allocs/op
BenchmarkSerializeToJsonStream-8 3000000 463 ns/op 0 B/op 0 allocs/op
BenchmarkSerializeToProtobuf-8 10000000 197 ns/op 80 B/op 1 allocs/opProtobuf encoding is faster than json stream 2.3 times and json 2.7 times.
Decode structure by transfer bytes with json and protobuf comparison:
BenchmarkUnserializeToJson-8 1000000 2183 ns/op 448 B/op 11 allocs/op
BenchmarkUnserializeToJsonStream-8 500000 2500 ns/op 1216 B/op 15 allocs/op
BenchmarkUnserializeToProtobuf-8 3000000 461 ns/op 272 B/op 9 allocs/opProtobuf decoding is faster than json stream 5.4 times and json 4.7 times.
The problem
Why is it not so popular yet? Because protobuf is not so as simple as google says. Protobuf must preprocess from proto files to the sources of you programming language. Unfortunately, for some platforms, the protoc generator produces a very impractical code and it’s too hard to debug it. Also protobuf is hard to develop when services are so many and we have more than one team in development, because it’s not a human reading standard.
Conclusion
Protobuf is fast and modern technology. It can make you happy. But you need to be careful with the choice of new technologies, because they can give you problems.
Github repository with benchmarks are here.
📝 Read this story later in Journal.
👩💻 Wake up every Sunday morning to the week’s most noteworthy stories in Tech waiting in your inbox. Read the Noteworthy in Tech newsletter.
