Using Trivially Copyable Objects to Improve Java Serialisation Speeds
For any low-latency software it is vital that the most common operations introduce minimal latency. For example, in trading software, one common and time-sensitive operation is messaging between microservices. To minimize the memory footprint of the communication, all messages are serialized by the sender and deserialized by the receiver. Hence, from a performance perspective, it is vital that the process of serializing/deserializing introduces minimal latency.
Read this article and learn more about the low-latency technique leveraging C++ methodology in Java: Trivially Copyable Objects and memcpy. Make common operations a little faster thereby saving a lot of time in aggregate operation. Serialization of Trivially Copyable Objects can be more than five times faster than that of regular Java objects and is supported by Chronicle libraries such as Chronicle Services.
Trivially Copyable Objects – in Java?
Trivially Copyable Objects are common in C++ development, but if you are unfamiliar with the term, an analogy may help demonstrate their advantages.
Imagine you work at a warehouse, where you regularly get a last-minute order of two foos and one bar that is due for shipping. The foos are located in the west end of the warehouse, the bars in the east, so you need to visit both ends of the warehouse to collect the order. That takes time. What if, for this common last-minute order, you could instead keep the components in one place? Then, whenever you get the time-critical order you could quickly collect them and send them off.
Keeping the components in one place is, roughly, the concept of Trivially Copyable Objects, for which all information is stored in one contiguous memory block. With all data in one place, it is possible to copy the data from that memory location to another in a single operation. In C++ this can be achieved using memcpy which is the fastest way to copy data.
Regular Java objects contain references to objects or non-primitive data types, which are stored in different memory locations. To copy such a Java object entails multiple memory copies, analogous to running around in the warehouse. Copying Java objects is thus expensive in terms of latency.
Fortunately, we can leverage the mentioned C++ scheme and make Java serialization much faster. The key is to create Trivially Copyable Java Objects, that is, objects so simple that they can be copied in a single operation. The requirements? Use only primitive data fields. Since primitive fields have a fixed size, the size of a Trivially Copyable Object is static and the object can thus be stored in a single memory block. It may seem limiting to only use primitive data types, but even objects with non-primitive fields can often be converted to Trivially Copyable Objects.
Chronicle Serialization Benchmarks
Chronicle serialization libraries offer full support for Trivially Copyable Objects in Java, enabling much faster data serialization.
In the benchmarks shown below, two microservice applications are used, with one sending BookUpdates to the other via a persisted queue. The BookUpdate class is either a “Normal Object” with symbol, exchange, time as String objects and a List of asks and bids OR it is a “Trivially Copyable Object” with these fields mapped to primitive fields like int, long or double fields.
As can be seen below, serialization and deserialization of a Trivially Copyable Object is more than 5 times faster than that of a regular object in the typical case (50% percentile). All times are in microseconds.
Normal Objects
SUMMARY (end to end)
Trivially Copyable Objects
SUMMARY (end to end)
The higher percentiles show less difference between the two variants likely related to other things than serialization, such as SSD device latency.
Note: Using Linux 3.10.0-1160.25.1.el7.x86_64 on Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
What are the Drawbacks of this Approach?
While using memcpy is efficient, the Java standard leaves it open to JVM implementers how objects are laid out in memory. This means that there is no guarantee that objects sent using one type of JVM (e.g. OpenJDK) can be properly received using another type of JVM (e.g. J9). However, for any given JVM type, the memory layout is known to have been stable over time so if the same family of JVMs are used across the board, there should be no problems. It is very easy to add integration tests, ensuring full compatibility.
Conclusion
It is possible to significantly improve the latency footprint of serialization and deserialization by applying the principle of Trivially Copyable Objects in Java development. The key principle for such objects is that they contain only primitive data fields, and the good news is that even objects of non-primitive data fields can often be converted to trivially copyable ones. Chronicle Services offer unique support for Trivially Copyable Objects in Java and enables more than five times faster serialization compared to regular Java objects. To learn more about Chronicle’s support for Trivially Copyable Objects, request access to a demo project here.
Resources
