What is Chronicle Set?
Chronicle Set is built on Chronicle Map, and as such is included in the Chronicle Map Github project. Much of the content below is similar to Chronicle Map.
Chronicle Set is a high performance, off-heap, in-memory, persisted set. Chronicle Set works like a standard Java set, yet it automatically distributes elements between processes. These processes can be both on the same server, or across your network. In other words, Chronicle Set is a low-latency, huge set which can store terabytes of elements locally to your process.
Chronicle Set is able to access your elements with sub-microsecond latency. Chronicle is built on Java, but it does not undergo, the garbage collection pauses that most Java applications experience. This is because Chronicle stores your elements off-heap. Even if you have a large cache with lots of updates, the garbage collector will have almost no effect. This results in more predictable latency and throughput. The off-heap elements are stored in a memory mapped file that offers further resilience to unforeseen outages.
The following example creates a simple set, and adds data to it:
Set<Integer> set = ChronicleSetBuilder.of(Integer.class).create(); set.add(1);
How it works
You define the type of element you wish to store, and set the maximum number of entries. Chronicle will create an appropriately sized memory mapped file; say 4TB. The size of the memory mapped file is not constrained by physical memory. Therefore, unlike heap-based java set value stores, Chronicle eliminates the pauses that can occur when the set has to be resized.
If your data grid spans two or more processes on the same server, each process will share the same 4TB space; fine grain locking will ensure your data integrity. The data is transferred directly using shared memory. When replicating on the same server, we don’t use TCP/IP loopback which will add additional latency; Chronicle just uses memory. This approach gives you outstanding performance. We have measured data being transferred between processes in a staggering 40 nanoseconds.
For solutions that require replication across your network, we provide both TCP and UDP connectivity. Chronicle uses Java NIO selectors, so only one thread is used to replicate to numerous processes.