Tag Archives: activemq protobuf

[repost]thrift-protobuf-compare

We moved to a new home

The wiki moved to http://wiki.github.com/eishay/jvm-serializers/

For discussions please use http://groups.google.com/group/java-serialization-benchmarking

Intro

Started with few blog posts and with the help of many contributes, this project is now benchmarking much more then just protobuf and thrift. Thanks to all who looked at the code, contributed, suggested and pointed bugs. Three major contributions are from cowtowncoder who fixed the stax code, Chris Pettitt who added the json code and David Bernard for the xstream and java externalizable. The charts below are displaying the latest results. Note that the charts are scaled to best fit the results and they might be misleading in come cases. If you wish to see the numbers scroll down to the chart at the end of the page. Overall we have benchmarks for protobufthriftjavascala, few implementations of staxbinaryxml,jsonxstreamjavolutionhessianavrosbinaryJSON Marshaller, and Kryo.

Numbers are not everything

Benchmarks can be very misleading. Different hardware, use cases, and/or datasets will provide different results and sometimes a marginal performance boost is eclipsed by other features like forward and backward compatibility, cross language support, simpler API, and more.

Charts

Setup

The following measurements were performed with revision r128 on Windows 7 64-bit using Sun’s JVM 1.6.0_15 JRE 32-bit, with an Intel Core i7 920 CPU. Note the tests are run with a JVM heap size of 16MB and using the server HotSpot compiler.

Omitted from the first three charts: json/google-gson and scala. These serializers are so slow, they would break the scale of our charts. See below for the naked data.

Total Time

Including creating an object, serializing and deserializing:

Serialization Time

Serializing with a new object each time (object creation time included):

Deserialization Time

Often the most expensive operation. To make a fair comparison, all fields of the deserialized instances are accessed – this forces lazy deserializers to really do their work. The raw data below shows additional measurements for deserialization.

Serialization Size

May vary a lot depending on number of repetitions in lists, usage of number compacting in protobuf, strings vs numerics, assumptions that can be made about the object graph, and more. Interesting point is Scala and Java which holds the name of the classes in the serialized form. I.e. longer class names = larger serialized form. In Scala its worse since the Scala compiler creates more implicit classes then java.

Object Creation Time

Object creation is not so meaningful since it takes in average 100 nano to create an object. The surprise comes from protobuf which takes a very long time to create an object. Its the only point in this set of benchmarks where it didn’t perform as well as thrift. Scala (and to a lesser point – java) on the other hand is fast, seems like its a good language to handle in memory data structures but when coming to serialization you might want to check the alternatives.

Numbers

Times are in nanoseconds, sizes are in bytes.

                        ,   Object create,       Serialize,  /w Same Object,     Deserialize, and Check Media,   and Check All,      Total Time, Serialized Size
avro-generic            ,      4024.12346,      8018.86600,      4030.37150,      4779.31000,      4779.31000,      4779.31000,     12798.17600,        211
avro-specific           ,      2643.18237,      5749.96650,      3204.01250,      4355.65650,      4355.65650,      4355.65650,     10105.62300,        211
activemq protobuf       ,       254.11328,      6777.68650,        71.48850,        14.60200,      2574.77900,      4924.94600,     11702.63250,        231
protobuf                ,       470.51276,      7226.50900,      3698.94400,      3478.95350,      3826.13550,      4371.60450,     11598.11350,        231
thrift                  ,       227.97550,      7302.97850,      7165.79000,      7948.73750,      7948.73750,      7948.73750,     15251.71600,        353
hessian                 ,       168.21827,     13354.78550,     12849.28050,     21082.57000,     21082.57000,     21082.57000,     34437.35550,        526
kryo                    ,       169.04829,      3437.83750,      3297.00450,      4308.20000,      4308.20000,      4308.20000,      7746.03750,        226
kryo-optimized          ,       170.10886,      2912.73750,      2813.59000,      3998.48150,      3998.48150,      3998.48150,      6911.21900,        207
java                    ,       168.80043,     25773.30650,     25136.38350,     71573.79650,     71573.79650,     71573.79650,     97347.10300,        919
java (externalizable)   ,       170.74098,      2874.11850,      2674.87900,      3043.00350,      3043.00350,      3043.00350,      5917.12200,        264
scala                   ,       125.51285,     62838.27450,     61814.38950,    194495.92550,    194495.92550,    194495.92550,    257334.20000,       2024
json (jackson)          ,       175.75949,      7223.81900,      7113.92050,      6084.47000,      6084.47000,      6084.47000,     13308.28900,        378
json/jackson-databind   ,       173.56918,     10550.41150,     10443.20400,     10575.58100,     10575.58100,     10575.58100,     21125.99250,        465
JsonMarshaller          ,       171.88033,     24618.39500,     24488.50800,     41115.13300,     41115.13300,     41115.13300,     65733.52800,        370
protostuff-json         ,       475.75992,      8639.84000,      8069.39750,      7366.95850,      7366.95850,      7366.95850,     16006.79850,        448
protostuff-numeric-json ,       479.06845,      8011.94950,      7405.96250,      7185.39250,      7185.39250,      7185.39250,     15197.34200,        359
json/google-gson        ,       173.28482,    449118.35900,    449995.44750,    491268.32050,    491268.32050,    491268.32050,    940386.67950,        470
stax/woodstox           ,       173.08692,      8542.42850,      8408.70150,     12161.06250,     12161.06250,     12161.06250,     20703.49100,        475
stax/aalto              ,       173.82471,      6330.78500,      6159.79150,      8082.84650,      8082.84650,      8082.84650,     14413.63150,        475
binaryxml/FI            ,       174.39919,     15336.63850,     15210.98450,     14528.33450,     14528.33450,     14528.33450,     29864.97300,        300
xstream (stax with conv),       174.35308,     16116.89100,     15564.12000,     26235.77100,     26235.77100,     26235.77100,     42352.66200,        399
javolution xmlformat    ,       175.25803,      7584.83750,      7446.11950,     10567.31900,     10567.31900,     10567.31900,     18152.15650,        419
sbinary                 ,       126.19877,      8704.78100,      8743.20950,      4584.87150,      4584.87150,      4584.87150,     13289.65250,        264

 

original:http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking