What is Chronicle Queue
Chronicle Queue is able to persist every single event/message, It records each event to a memory mapped file.
This memory mapped file is also used for exceptionally fast interprocess communication (IPC) without affecting your system performance. This is especially useful when there is a need to transfer large amounts of data, its ideal for transferring data between processes very quickly on the same server or across the network. There is no Garbage Collection (GC) as everything is done off heap, TCP replication ensures that data is kept in sync. TCP replication also provides further resilience and failover, Chronicle Queue is ideal for distributed architectures especially when you wish to spread the processing over more than one machine and achieve a High level of Availability (HA).
Over your Network
Enterpirse Chronicle Queue is able to distribute your messages over your network. Appropriate filtering ensure that only the required records or even fields with a record are set. This improves performance and ensure that you reduce your network bandwidth requirements.
How it Works
Chronicle uses a memory mapped file to continuously journal messages, chronicles file based storage will slowly grow in size as more data is written to the queue, the size of the queue can exceed your available memory, you are only constrained by the amount of disk space you have on your server. Chronicle writes data directly into off heap memory which is shared between java processes on the same server. Chronicle is very fast, it is able to write and read a message in just two micro-seconds with no garbage. Typically at the end of each day you archive the queue and start the next day with a fresh empty queue.
To see Chronicle Queue in action – run this demo on your machine
Download the chronicle-demo.jar (executable jar) from here.
The jar should be launched with Java 1.7+.
Double click to launch or use the command line java -jar chronicle-demo.jar.
“Please Share with us how many events / second your machine can process when selecting MAX event rate.”, On my laptop I can do 1.6 million/second and server 3.4 million/second.”
Chronicle Queue offers:
- A low latency, durable, interprocess communication (IPC)
- No data loss; stores every value instead of replacing them
- No GC as everything is done off heap
- Persistence to disk through memory mapped files
- IPC between Java processes or between threads in a Java process
- Simple API for ease of use
- The use of Java as a low latency programming language
- The replay-ability of all your inputs and outputs,
- concurrent writers across processes on the same machine.
- concurrent readers across machines on your network using TCP replication.
- embedded performance in all processes.
- insulation of your producer from a slow consumer. Your consumer can be tera-bytes behind your producer.
- all data stored off heap, to reduce GC impact.
- data is written synchronously so that if your application fails on the next line, no data is lost.
- data can be read less than a micro-second after it has been written.