Articles

Comparing Approaches to Durability in Low Latency Messaging Queues

A significant feature of Chronicle Queue Enterprise is support for TCP replication across multiple servers to ensure high availability of application infrastructure. I have generally held the view that replicating data to a secondary system is faster than sync-ing to disk, assuming the round trip network delay wasn’t high due to quality networks and co-located…

Automatically Creating Microservices Architecture Diagrams

In application development, microservices is an architectural style where larger applications are structured as a collection of smaller, independent, yet interconnected services. While this allows for highly maintainable and testable applications (as each service can be maintained independent of the larger application), the problem with this method is the inherent complexity of interactions between microservices.…

Chronicle FIX: Much More Than a Quick Fix

Many of our customers have upgraded from QuickFIX/J to Chronicle FIX and this article provides some background as to why. Introduction QuickFIX/J is free, accessible, supports the FIX standard and is therefore often chosen by the IT team for a first FIX engine. However, when the business grows, its limitations become clear. The reasons most…

Chronicle FIX: Designed Not To Skip A Message Even If Your Data Centre Fails

A high level of availability of IT services is crucial to prevent disruptions of service that can lead to financial losses, business opportunity losses, data loss, and reputational damage. A business’s downtime cost can vary depending on its size, nature and the length of the downtime. Studies by Network Computing, the Meta Group, and Contingency…

Chronicle Wire: Object Marshalling

At Chronicle, we know that efficient code doesn’t just run faster; if it’s using less compute-resource, it may also be cheaper to run. In particular, distributed cloud applications can benefit from fast, lightweight serialisation. In this article, we will demonstrate the efficiencies of using Chronicle Wire to encode small Strings into long primitives, with a…

How to Automatically Generate GitHub Release Notes From Resolved Issues

GitHub Issues are convenient since they come bundled with a GitHub repository. Many companies are exploring this tool as an alternative to full-featured issue tracker such as Atlassian JIRA. Complex workflows are not always required, but an issue tracker should be able to answer the question, When a new feature or a bug fix was…

Event-Driven Order Processing Program

Following the Hello World example of a simple, independently deployable real-time Event-Driven Microservice, this article looks at a more realistic example of an Order Processor with a New Order Single in and an Execution Report out.  A New Order Single is a standard message type for the order of one asset in the FIX protocol…

Efficient Memory Mapping for Terabyte Sparse Files in Java

On Linux, you can create sparse files, where only the pages (of 4 KiB) that are touched utilise either memory or disk space. This allows you to memory map large virtual regions without worrying about wasted memory or disk In this program, you can see it reserves 8 TiB (8,192 GiB) Figure 1. Test 1:…

Exhaustive JUnit5 Testing with Combinations, Permutations and Products

Unit testing constitutes an integral part of the process of providing high-quality software. But, how can one write tests covering all variants of several operations? Read this article and learn how to use JUnit5 in conjunction with combinations, permutations, and products. Test Support Libraries There are many libraries that make testing better in different aspects.…

Event Driven Hello World Program

Event-driven microservices can be straightforward to describe before they are implemented, tested and maintained. They are also highly responsive to new information in real time, with latencies in Java of below 10 microseconds 99.99% of the time depending on the functionality of the small independently deployable microservice.    In this introductory article, we use an…

Is There a Best OS to Develop a Java Application on?

One subject that often evokes a lot of debate is which is the best OS to develop a Java application on. This article gives my view on the issue.   Java runs very well on Windows, macOS and Linux Distributions. I work for Chronicle Software, and much of our software is open-source Java libraries, so…

How Behaviour Driven Development Works Well with Event Driven Architectures

Behaviour Driven Development (BDD) and Event Driven Architecture (EDA) work well together as they complement each other’s strengths and weaknesses. Using both can result in a shorter time to market for new functionality and a more maintainable system. Behaviour Driven Development encourages a common language between users and developers in describing requirements in a form…

Which is the Fastest JVM?

How is a high-performance, low-latency Java application affected by the JVM version used? Every nanosecond counts for trading and other applications where messages between two different threads are exchanged in about 250 ns! Read this article and find out which JDK variant comes out at the top! Benchmarks This article will use open-source Chronicle Queue…

Why Core-to-Core Latency Matters

Last year we had a global shortage of microchips; today, I feel there is a worldwide shortage of software developers. Like many other software companies, Chronicle Software is rapidly expanding and interviewing candidates. We usually ask candidates to demonstrate a good knowledge of core Java, then we cover slightly more advanced concepts, such as the…

“The More You Say, the Less People Remember…

…The Fewer the Words, the Greater the Profit.” These wise words were written by François Fénelon, a writer and theologian living during the 17th century. This is an article summarising different approaches to the development of IT solutions. Learn why the above quote is very relevant for developers in the 21st century.   Business-driven development…

How Does Kafka Perform When You Need Low Latency?

Most Kafka benchmarks appear to test high throughput but not low latency. Kafka was traditionally used for high throughput rather than latency-sensitive messaging, but it does have a low-latency configuration. (Mostly setting linger.ms=0 and reducing buffer sizes). In this configuration, you can get below 1-millisecond latency a good percentage of the time for modest throughputs.…

Cloud-Native Chronicle Deployment

Traditionally, companies with low-latency requirements deployed to bare metal servers, eschewing the convenience and programmability of virtualisation and containerisation in an effort to squeeze maximum performance and minimal latency from “on-premises” (often co-located) hardware.   More recently, these companies are increasingly moving to public and private “cloud” environments, either for satellite services around their tuned…

How to Reduce Cloud Cost by 99% for EDA Kafka Applications

While the cloud offers great convenience and flexibility, the operational cost for applications deployed in the cloud can sometimes be significant. This article shows a way to substantially reduce operating costs in latency-sensitive Event-Driven Architecture (EDA) Java applications by migrating from Kafka to Chronicle Queue open-source, a more resource-efficient and lower-latency queue implementation. What is…

How to Leverage Method Chaining to Add Smart Message Routing

This article explores how to use method chaining to add routing information to serialised data structures in a lightweight fashion, where the routing information can be read separately by an intelligent message router.   We are going to use the open-source serialisation library Chronicle Wire. Let’s assume we have some data that we want to…

Introducing the Multifunctional FIX Router of Chronicle FIX

A Load Balancer, FIX Version Translator, and Rule-based Drop Copy FIX (Financial Information eXchange) protocol was developed to provide a reliable and high-speed exchange of transaction information between trading parties. In a typical scenario, there are point-to-point connections between clients and servers, ie a firm may run one or more FIX servers and multiple counterparties…

Challenges when Developing a GUI for FIX

This article explores the Challenges in developing a Graphical User Interface (GUI) for Financial Information eXchange (FIX) data. FIX is both a protocol and a message format, but to create a FIX GUI we will focus just on the message format. A FIX message is a standard message format for transmitting financial and investment banking…

Automated CI/CD of Multiple Projects Using TeamCity’s Kotlin DSL

In a previous article I described a way to organise low-latency products as multiple code bases which are bound together with a Maven Bill of Materials (BOM). Understandably, this requires setting up continuous integration and deployment for a large number of similar projects. Maintaining such setup manually in the face of change while ensuring its…

Why the Cool Kids Use Event Loops

A Discussion On the Benefits of Event Loops in Java. When I was working in software development back in the 1990s, nearly all the software libraries that I worked on made use of event loops. This was because at the time most hardware had just one single CPU.  Back in the day, I remember the…

How to Develop Event-Driven Architectures

Last month, I wrote an article on Open source Chronicle Wire, that discusses how we could serialise an application’s state into different message formats.   Now in this article, I’m going to look at how we can use Open source Chronicle Queue and Chronicle Wire  to structure applications to use Event-Driven Architecture (EDA). EDA is…

Kafka vs Chronicle for Microservices: which is 750 times faster?

Apache Kafka is a common choice for inter-service communication. Kafka facilitates the parallel processing of messages and is a good choice for log aggregation. Kafka claims to be low latency, high throughput. However, is Kafka fast enough for many microservices applications in the cloud? When I wrote Open Source Chronicle Queue my aim was to…

Unusual Java: StackTrace Extends Throwable

There are things you can do in Java you rarely see, generally because there is no use for it. However, there are some unusual things in Java that could be surprisingly useful. Chronicle Software uses a number of different usual patterns in its low-level libraries most developers wouldn’t generally come across. One of them is…

How to Get C++ Speed in Java Serialisation

Using Trivially Copyable Objects to Improve Java Serialisation Speeds For any low-latency software it is vital that the most common operations introduce minimal latency. For example, in trading software, one common and time-sensitive operation is messaging between microservices. To minimize the memory footprint of the communication, all messages are serialized by the sender and deserialized…

Why the Fastest Way of Serializing a Java Field is not Serializing it at All

This article elaborates on different ways of serializing Java objects and benchmarks performance for the variants. Read this article and become aware of different ways to improve Java serialization performance. In a previous article about open-source Chronicle Queue, there was some benchmarking and method profiling indicating that the speed of serialization had a significant impact…

Reducing Tail Latencies with Chronicle Queue Enterprise

Persistent queue solutions are frequently used when designing low-latency applications. The problem is that high sustained message rates, or bursty workloads, can lead to delays caused by the OS or hardware which are challenging to mitigate. In this article, we will describe how Chronicle Queue Enterprise solves this. Background When benchmarking Java low-latency persistent queue…

How Object Reuse Can Reduce Latency and Improve Performance

Become familiar with the art of object reuse by reading this article and learn the pros and cons of different reuse strategies in a multi-threaded Java application. This allows you to write more performant code with less latency. While the use of objects in object-oriented languages such as Java provides an excellent way of abstracting…

Creating Terabyte Sized Queues with Low-Latency

Queues are often fundamental components in software design patterns. But, what if there are millions of messages received every second and multi-process consumers need to be able to read the complete ledger of all messages? Java can only hold so much information before the heap becomes a limiting factor with high-impacting garbage collections as a…

High-Performance Java Serialisation to Different Formats

Java serialization is a popular mechanism where you are able to serialize and deserialize complex object graphs; for example where object A can contain a reference to object B, which in turn has a reference back to object A. The problem is that this rich functionality comes at a performance cost. However, if you do…

Why a Set Can Contain Duplicate Elements

In low-latency applications, the creation of unnecessary objects is often avoided by reusing mutable objects to reduce memory pressure and thus the load on the garbage collector. This makes the application run much more deterministically and with much less jitter. However, care must be taken as to how these reused objects are used or else…

Low Latency Microservices, A Retrospective

I wrote an article on low latency microservices almost five years ago now. Chronicle Software has worked with a number of tier one investment banks to implement and support those systems. What has changed in that time and what lessons have we learnt? Read this article and learn what we learned after five years of developing…

Monorepo Versus Bill of Materials: Shaping Your Project

During most of my development career, the usual code organization pattern was monolithic repository or Monorepo. There is a single large version control repository, to which all developers on the project push their commits, sometimes breaking it. Every once in a while, this goliath is released as a set of artifacts. Before this may happen,…

How to Build Low Latency Crypto Trading Systems using Java and Chronicle Services

Cryptocurrency trading is an emerging market with its own rules. However, when it comes to the need for low-latency arbitrage, that is, being able to react rapidly to changing market prices and placing orders ahead of the competition, there are lessons we can learn from optimizing classic trading systems. Chronicle Software has a successful track…

Did You Know You Can Create Mappers Without Creating Underlying Objects in Java?

As most Java developers know, putting values in a Java Map (like a HashMap) involves creating a large number of auxiliary objects under the covers. For example, a HashMap with int keys and long values might, for each entry, create a wrapped Integer, a wrapped Long object and a Node that holds the former values…

Suitability of AMD Processors for Chronicle Low Latency Microservices

At Chronicle we are often involved in helping clients to choose the optimum hardware on which to run Chronicle Queue and we are happy to give specific advice in line with what clients are trying to achieve. Recently we were asked about the suitability  of AMD processors for running Chronicle Low Latency Microservices below is…

Linux File Systems and Application Latency

By The Chronicle Team 14th June 2021 Financial trading systems require extremely low latency. In modern markets, financial instrument pricing changes very rapidly, and a client’s profit or loss can be impacted by a delay in pricing awareness down to a few hundred nanoseconds. For this reason, there is a financial benefit in achieving low…

The IKEA effect

The IKEA effect is a fascinating phenomenon whereby consumers place a (sometimes disproportionately) higher value on things that they have either made themselves, or to which they’ve contributed towards making. This can have a significant influence on perceived value – to the extent that people might value imperfect things they’ve helped create, more than they…

Trading System Innovation and Trivially Copyable Objects

What’s in a trading system? Trading systems are complex beasts. They’ve got to be functionally rich, enabling traders to execute a range of order types and trading strategies, across multiple asset classes. They’ve also got to deliver extremely high-performance end-to-end latency and throughput, so that those traders can rapidly access market data and trading venues.…

Developing Low Latency Trading Systems with Chronicle Microservices

Introduction to Chronicle Services Traditionally, low latency trading systems were developed as monolithic applications in low level languages such as C++. While these systems delivered the required performance, the development effort was extremely time consuming, and the complexity of the software made it difficult to adapt existing applications to changing market demands. A solution to…

Common Mistakes in Benchmarking Low Latency Software and Why You Should Consider Trivially Copyable Objects

24th February 2021 Overview A common mistake in micro-benchmarking is to use performance tests that are wildly optimistic. Most benchmarking tools focus on throughput performance and report an average latency. However, when the same code is run in production, results can be dramatically worse. A problem with benchmarking using average latency is that it does…

How Microservices can Improve Performance Now and Future-Proof your Trading System for Cloud Migration

In this blog post, we are going to focus on microservices – what they are, what benefits they can provide for your organisation, and how they can not only provide immediate improvements in trading system performance but also help future-proof trading systems when it comes to technology evolution and adaptation (such as cloud migrations). Microservices…

Big Data and Chronicle Queue

The Big Question How is Chronicle Queue being used for Big Data solutions in Java, and how does it work under the covers? What is Chronicle Queue? Chronicle Queue is a persisted journal of messages which supports concurrent writers and readers even across multiple JVMs on the same machine. Every reader sees every message, and…

Peter Lawrey Talks about Low-Latency, High-Performance Java

October 22, 2020 Kevin FarnhamTechnology writer and software engineer focused on high-performance low-latency big-data Java, Python, C/C++ programming. About 7 years ago, I attended a session given by Java Champion Peter Lawrey, leader of Chronicle Software, at a JavaOne conference. Since most of my prior development work in the realm of low-latency high-performance was C/C++…

Breaking Down the Barriers to Institutional Cryptocurrency Investment

Institutional investment in cryptocurrencies is on the rise. A recent survey from Fidelity Digital Assets[1], encompassed a range of nearly 900 investors spanning crypto hedge and venture funds, traditional hedge funds, endowments and foundations, financial advisors, family offices and high net worth individuals, and found that:   36% of institutional investors surveyed currently invest in…

COVID-19 – High Performance in a Time of Volatility

By Peter Lawrey “Unprecedented” is the much-overused adjective used to describe our current lockdown situation, in almost every blog, article, or feature on the current global COVID-19 pandemic published these days. And yet, these are truly unprecedented times. We’ve seen our society go into lockdown, rapidly transitioned to social distancing and working at home (often…

Chronicle Network Layer

60% faster TCP/IP within Java One of the major drivers at Chronicle Software is the constant push to remove latency from technology stack , this involves constant analysis and work on all aspects that can add latency. One area we have recently been concentrating on is optimizing the Chronicle network latency that underpins Chronicle Queue…

Garbage Reduction the Chronicle Way

One of the major benefits with using any Chronicle product is that we aim for almost Zero Garbage Collection (GC) there are several benefits to reducing Garbage:- Pausing for Garbage Collection happens less often. Creating fewer objects, reduces the overhead of allocating and clearing the object in the first place. An indirect benefit comes from…

Addressing the Data Issues of Best Execution Under MiFID II and Other Global Regulations

By Peter Lawrey, CEO at Chronicle Software. Regulated trading systems need to record the reasons behind every decision they make with minimal impact on performance. Chronicle Software has worked with investment banks and hedge funds to deliver trading services with a record everything model and low microsecond latencies. Today’s more stringent regulatory environment is placing…

Trade Surveillance Under MAD/MAR, MiFID II and Other Global Regulations

By Peter Lawrey, CEO at Chronicle Software. Regulations like MiFID II and the Market Abuse Regulation (MAR) in Europe, and Dodd Frank in the US, have raised the bar with respect to trade surveillance as regulators seek to boost transparency and investor protections across the board. This has created the need for financial institutions to…

Choosing Chronicle FIX Engine

Overview The FIX Trading Community web site has posted a list of common questions about FIX Engines. This page brings together our answers for these questions. For more details on Chronicle FIX Capabilities/throughput Chronicle FIX supports; 100,000 messages per second per core. (NewOrderSingle / ExecutionReport sized messages) FIX Services which need to have a 30 micro-seconds tick…

Improving percentile latencies in Chronicle Queue

Compared to a year ago, we have significantly improved the throughput at which we can achieve the 99%ile (worst one in 100). What tools and tricks did we use to achieve that? What are we testing? Chronicle Queue appends messages to a file while another thread or process can be reading it. This gives you…

Best practices for Event Sourcing

David Schmitz presented an excellent talk on what he sees as best practices in using Event Sourcing. Chronicle Software has two very different event sourcing frameworks Chronicle Microservices Framework (CMF) and Chronicle Decentred. This talk is useful in highlighting how they differ with each other and other Event Sourcing solutions. CMF is built on Chronicle Queue and is designed for low latency…

Chronicle Queue storing 1TB in virtual memory on a 128GB machine

If you use a standard JVM like the Oracle JVM or the OpenJDK, you might find that as the heap size grows the performance of your JVM can drop as GC pause time escalates. This tends to be a problem around 32 GB of heap, but it often depends on the application at which point…

Object Pools Revisited

Object pools were popular before Java 5.0, however, more efficient allocation made most of these pools a needless complication. Can an Object Pool still make sense and how fast do they need to be? Overview Up to Java 5.0 using object pools could significantly improve performance. However, from Java 5.0, the cost of using naive…

StringBuffer and how hard it is to get rid of legacy code

In 2006, Java 5.0 was released with StringBuilder, a more lightweight and sane version of StringBuffer. The Javadoc for Java 5.0 for StringBuffer notes As of release JDK 5, this class has been supplemented with an equivalent class designed for use by a single thread, StringBuilder. The StringBuilder class should generally be used in preference…

What’s the difference between blockchain and traditional databases?

The key difference between blockchain and traditional databases is the level of trust needed in those running the service. With blockchain, you trust the protocol, not those running it. What does a blockchain do? A blockchain records transactions without needing to trust any individual, organisation or computer, in a tamper-proof way (or pretty close). This…

Low Latency Micro-services for performance

Overview Microservices is a buzz word at the moment. Is it really something original or based on established best practices? There are some disadvantages to the way microservices have been implemented, but can these be solved? Component testability and consistency Once you have assembled a large system, it can be hard or even impossible to…
Want to Learn More?

See the Benefits of Chronicle's Products in Action

At Chronicle, we believe less is more. Learn more about why and how Chronicle can support your business and how it can increase efficiency and streamline your systems and workflows by speaking with one of our experts.

We can also offer you our release notes.

    By completing this form, I agree that Chronicle Software may keep me informed of its products, services and offerings. To view our privacy policy, click here.