Building Low Latency Java Microservices using Chronicle Services

Over the years at Chronicle, we have built a large number of applications and systems that are focused around low latency messaging, primarily in the financial sector. During this time, we have found that following certain architectural principles has helped us build these applications more efficiently, and we have ended up with more robust, reliable and maintainable software as a result. Drawing on these lessons, we have distilled best practices for latency microservices into a repeatable formula that consistently delivers lower response times and simpler operations.

Chronicle Services is a framework that distills many of these ideas and facilitates interaction with various aspects of the Chronicle software stack to achieve our goals and those of our customers. By offering a framework that is event driven and highly optimised for inter service calls, we make reducing latency in microservices architectures a practical reality rather than an aspiration.

In this series of articles, we will explore Chronicle Services through a number of worked examples, each illustrating a specific feature of the framework. You will see how an event-driven architecture, combined with disciplined API management and lightweight service communication, can dramatically improve services latency for modern microservices architecture projects.

An application built with the Chronicle Services framework consists of a number of loosely coupled processing components known as services, which communicate with each other using asynchronous messaging, following the Event Driven Architecture. By allowing services to communicate through well-defined channels, we avoid the bottlenecks that sometimes plague traditional monoliths and pave the way for microservices latency figures that routinely sit in the microsecond range.

An event is an immutable indication that something has happened. One or more services may be interested in the event and will handle it in their own way. Handling an event will normally involve posting a new event; once again, this will indicate that the event has been processed and may include data indicating any state changes. This pattern of services framework event processing keeps service calls lightweight, supports rapid scaling across multiple services, and is a cornerstone of reducing latency microservices challenges.

A service may maintain local state and Chronicle Services follows the principles of events sourcing to manage this state. Local state in a service can be capable of being reconstructed from the events that it has handled or generated using one of a variety of strategies. Persisting frequently accessed data in this manner ensures that costly database queries are avoided in the critical path, contributing to lower latency microservices Java deployments.

What’s in a Service?

A Service is a self-contained processing component that accepts input from one or more sources and outputs to a single sink. Let’s look at a simple example where a service performs a simple operation (addition) on a pair of numbers and outputs the result. The service can be illustrated in the following diagram:

The service has a single input, from which it reads input events of type sum2, each of which carries a payload consisting of two numbers to be added together. It also has a single output, to which it will post an event carrying a single number, which is the result of performing the addition of the two input values. Even in this minimal illustration, you can see how clean inter service communication ensures deterministic behaviour and predictable response times.

The service has a public interface that defines the details of the input events it is interested in, and the format of the output events that it will emit. This is all that is required to interact with the service—the implementation of the event handler is hidden. Such encapsulation is crucial when you want to build maintain microservices that can evolve independently while still operating cohesively within the wider system.

Service API

Chronicle Services expects the input and output event details to be encapsulated in Java interfaces. There can be zero or more input interfaces, but there must be exactly one output interface. For example, in the service shown above:

The method names correspond to the message types that are expected or generated. Chronicle Services is based on unidirectional asynchronous message passing, so the handler methods do not return any value. This approach mirrors the way real-world micro services operate behind an API gateway, where requests are translated into event messages that traverse the system with minimal overhead.

Service Implementation

The class that implements the service is defined as follows:

The class implements the input API interface. This requires it to implement methods to handle each of the incoming event types. Every call to sum2 resembles a lightweight API call between microservices, emphasising that even simple service calls can be handled with incredibly low overhead when designed for latency microservices performance targets.

The class also maintains a reference to an object that implements the output API, the details of which are injected at construction time by the Services framework. This inversion-of-control style injection is fundamental to an architecture easy build approach, letting developers focus solely on business logic while the framework handles the complexities of communication microservices concerns.

Handling Events

The Chronicle Services runtime will dispatch the event to the appropriate handler method for the incoming event type. Once processing is complete, the output event is generated by calling the appropriate method on the output API. This disciplined, framework event driven processing pipeline is designed for reducing latency by eliminating context switches and synchronisation overhead.

The mappings of incoming event to method invocation and method invocation to outgoing event is handled by Chronicle Services, based on the API interface types. No additional information is required, and the mechanism used is highly efficient both in terms of time and memory usage. Consequently, developers can easy build maintain robust systems without getting bogged down in low-level threading primitives.

Interacting with a Service

In order to interact with a service, we need to post events to its input and read events from its output. The service does not place any specific requirements on the transport used to manage event transmission, although the default implementation is based on Chronicle Queue, a messaging system with ultra-low latency capabilities. Throughput of over 1 million events per second is possible using Chronicle Queue, making it an ideal persisted messaging framework for latency microservices architectures.

It is also possible to utilise a different event transport without any modifications to the service. This is especially useful for functional testing, and Chronicle Services provides a powerful testing framework for doing this. We will examine the Chronicle Services approach to testing in a later article. Such flexibility means that teams can run microservices monoliths locally during development and still achieve production-grade performance when they scale out to multiple services in the cloud.

For now, we can illustrate a complete application that generates input to the Sum Service and consumes its output events. The application can be shown diagrammatically:

Data flows through the application from left to right. The services are connected using Chronicle Queues. The leftmost service is known as an upstream service. It generates events of the appropriate type and posts these to the input queue of the Sum Service. Output events from the Sum Service are posted to its output queue and consumed by the rightmost service, known as the downstream service. Any output events from the downstream service are posted to a sink queue that simply consumes the event with no processing or output. This clear left-to-right flow is a practical demonstration of how services communicate efficiently, leveraging asynchronous communication to maintain consistently low services latency as workload scales.

One of the strengths of Chronicle Services is the ability to specify this application structure declaratively and have the instances of each component created automatically when the application starts. To do this, we use a configuration file in YAML to describe the services and the queues that they use for communication. This file is normally called services.yaml. Here is the file that describes the application shown in the diagram:

First we define the queues. In this simple example, all that is required for each queue is the pathname to the directory where the files for queue persistence are stored. More details on the configuration of Chronicle Queues can be found here. Leveraging Chronicle Queue, part of the wider Chronicle technology platform, lets us avoid heavyweight brokers while still enjoying enterprise-grade durability and throughput.

Then we define the services. In each case, the input queue(s) and output queue are specified, together with the fully qualified class name of the service’s implementation class. For both queues and services, there are many more configuration elements that can be set. We will discover these in later articles. This declarative style makes the architecture easy build and, crucially, easy to maintain, whether you are orchestrating microservices or evolving existing monoliths into modern, event driven apps.

The upstream service, sumUpstream, posts a message to its output queue (which is the main Sum Service input queue) requesting a single calculation. The result of this calculation is posted to the Sum Service output queue, which is the input queue of the downstream service, sumDownstream. The downstream service simply logs the receipt of the message. It does not post an output event, since it is intended to act as a sink service. Despite the simplicity, this pipeline showcases how effective inter service communication can be achieved without introducing unnecessary latency or complex api calls.

Deploying and Running the Application

Single Process

Our application can be deployed as a single Java process, with all three services running (in separate threads). This is useful for development and integration testing. To do this, we need to provide an entry point for execution to start. This is done in the usual way, using a main() method, which can be encapsulated in a class also called Main:

Like most event-based frameworks, Chronicle Services uses an event loop as the basis for execution. The event loop monitors the specified input queue(s) for incoming events and dispatches these to the relevant handlers. It is initialised and started using the call to ThreadRunner.runAll(), to which we pass the name of the file that contains the configuration information as shown above.

The default behaviour is for each service to have its own event loop, each running on a separate thread. All events for a given service are handled within this thread so that implementations of event handlers can assume a single-threaded execution environment. This simplifies handlers significantly, with benefits to maintainability and performance. In practice, this threading model is instrumental in reducing latency, because the absence of lock contention means faster context switches and more predictable response times across multiple services.

All the event loop threads are, by default, set up to be daemon threads. This means that they will be shut down automatically when all non-daemon threads in the process terminate. In our example, the only non-daemon thread is the main thread. We therefore pause this thread for 3 seconds before main() returns and the thread terminates (stopping all services).

When the application is run, the Chronicle Services runtime uses the information from the configuration file to initialise each service. Output will contain a lot of log messages; we will look at the most significant for this example. First, the Chronicle Services banner message:

The main processing chain can be seen, from the posting of the initial command by the upstream service to the final logging of the result by the downstream service:

Multiple Processes

Production deployments will normally be based around multiple processes, with each process running a single Chronicle Service. This can be achieved without changing the Service code or configuration. We do, however, require a separate entry point for each process. Each Main class can be similar to the example shown above for the single process deployment, however they are based on a different method to start the service:

where service-name is the identifier for the service descriptor in the configuration file.

The Chronicle Services runtime will ensure, as before, that the Chronicle Queue instances are created if necessary and that the service establishes contact with the queue so that communication can take place. Given that Chronicle Queue is based on shared memory for inter-process communication, the different services need to be located on the same physical system. Shared-memory pipes significantly shorten the path length for inter service communication, making this deployment model a prime choice for teams committed to reducing latency microservices overhead.

It is still possible to deploy services to different hosts, however, and we will see how this can be done in future articles. Remote transports can be introduced behind an API gateway, allowing secure, low-overhead communication microservices interactions across data centres, all while preserving the core event-driven semantics of the system.

Containers

A further option is to deploy applications in OCI containers, such as Docker containers, either in the single or multiple process model, allowing flexibility of deployment into on-prem or cloud based environments. Container images that package Java 17 together with the Chronicle Services runtime make it straightforward to spin up low latency microservices clusters for testing or production.

With the multi-container approach, it is possible to deploy and run the application easily with tools like Docker Compose. More sophisticated orchestration tools like Kubernetes allow deployment of services-based applications into cloud-based clusters. This supports an easy build maintain ethos where microservices monoliths locally can be promoted to distributed environments with minimal rework. Automated scaling policies can then ensure that services latency remains flat even under surging workloads.

What’s Next?

In the next article we will look in more detail at how data is sent between services as the payload of events. Chronicle Services utilises an extremely efficient mechanism for encoding and decoding structured data into messages. This mechanism further enhances reducing latency by eliminating costly serialisation overhead during service communication.

Later articles will examine:

• how services can be tested using the Chronicle YamlTester framework, and how this allows testing approaches based on business logic. We will also demonstrate strategies for reducing latency microservices regression by replaying event logs against new builds.

• the way in which services may be configured using the external, declarative approach introduced here but also using APIs allowing this to be managed at runtime and Inversion of Control techniques. These capabilities simplify api management tasks and promote consistent service definitions across your microservices architecture.

• how to interact with Chronicle Services applications from “outside” the Chronicle Services runtime. Whether you are issuing synchronous api calls or leveraging asynchronous communication patterns, we will explore best-practice approaches for safe, low-risk integration.

• approaches to managing state in a service, including caching frequently accessed data to keep hot paths free of unnecessary database queries.

• how a service runs, and how it is possible to parameterise event loop behaviour when required to improve latency. Fine-tuning thread affinity is one of the most effective levers for reducing latency in high-frequency trading or other event driven apps.

• different approaches to deployment, including how to build and deploy cloud-native applications using Chronicle Services. We will show that with the chronicle services framework, driven architecture easy to assemble and maintain microservices monoliths can coexist, giving development teams the freedom to evolve at their own pace.

• Chronicle Services’ approach to implementing High Availability systems, including deployment of services into a cluster with guaranteed replication of events to ensure no loss of data and minimal time overhead in the event of failure of one or more components.