As part of our partnership with AquaQ and integration into Kdb+ we have been carrying out some benchmarking details of the test environment are:-
– Producer writes messages to Chronicle Queue on Host 1
– Tailer reads messages from Queue on Host 1
– Tailer optionally batches messages
– Tailer then writes batches to Kdb+ on Host 2
– Runs for 100k messages at 10k msgs/s, and 300k messages at 30k msgs/s. 50k message warmup.
– Times are end-to-end: from when the producer writes the message to when the data has been written to Kdb+
Note we are including a network hop in the tests as – although slower – this is most likely how Kdb+ will be used in practice.
We can see that the environment saturated at a sustained rate of around 33k messages per second at which point messages remain in the Chronicle Queue until the adapter/Kdb+ can service them.
All timings are in Microseconds
More Information about the loader can be found here Chronicle Software integrates Chronicle Queue with kdb+ – Chronicle Software
The loader can be accessed here AquaQAnalytics/kdb-chronicle-queue: Java adaptor from Chronicle queue to kdb+ (github.com)
Want access to exclusive Chronicle content?
Hit the button below and join the Chronicle Community on LinkedIn, for free! You'll get exclusive access to blogs, guides, and video content. Join today!