Chronicle Map and Kayzen Case Study

Chronicle Map is an open source  high-performance distributed in-memory key-value store, built for low-latency and multi-process applications. Designed for efficient concurrent access across multiple writers and multiple readers, writes scale well up to the number of available hardware execution threads, whilst reads never contend with each other and so scale freely. The shared memory segments for a Map can optionally be backed by files on disk (memory mapped files) providing a persistent store of data.

Chronicle Map conforms to the Java ConcurrentMap interface enabling seamless integration with new or existing code. Data is stored entirely off-heap, with key and value objects transparently serialised and deserialised to/from off-heap memory. Key and value objects are efficiently cached and reused, enabling zero-allocation queries which in turn minimises garbage collection, helping to provide excellent performance with well controlled latency outliers.

Chronicle Map Enterprise

Chronicle Map Enterprise builds on the open source Chronicle Map and enables Maps to be distributed over a replicated cluster of networked hosts, providing a platform for both High Availability and Disaster Recovery (HA/DR). Each host in a cluster acts as a single node, with all processes on a given host sharing a common copy of the data. A Map cluster is a multi-master peer-to-peer environment supporting concurrent read/write access to the cluster from any node.

Kayzen

Launched in January 2019, Kayzen’s demand-supply platform responds to a user opening an ad-enabled app and negotiates with providers using anonymised user profiles to determine which ad content to display. Kayzen is the first mobile in-house bidder, built from the ground up for app developers and their technology partners to run performance user acquisition and programmatic retargeting in-house, helping democratize programmatic marketing.

Programmatic in-housing, through the use of advanced technologies and Real-Time Bidding (RTB), has the power to transform the digital advertising market by shifting the power from ad intermediaries to advertisers. Through the proliferation of Software-as-a-Service (SaaS)-based ad buying platforms, advertisers can reach their audiences on publishers’ inventory directly, with full transparency and control over their media-buying process and service costs.

Kayzen’s software platform enables programmatic user acquisition, retargeting and upper funnel campaigns in-house with an unprecedented scale of 60bn daily ad requests from 1bn+ unique users worldwide. Kayzen is accessible through its APIs or user interface and supports mobile video, playable / MRAID, native and banner ad campaigns.

Kayzen & Chronicle Map Enterprise

Kayzen’s core on-demand criteria-matching is built around a very large collection of ~5bn user profiles stored in Chronicle Map. A typical profile is ~100 bytes, keyed with ~60 bytes, and the service currently handles over 60bn daily ad requests, with around a million updates per second.

After a preliminary proof-of-concept using open-source Chronicle Map, Kayzen migrated to Map Enterprise in order to provide a fully replicated and distributed HA/DR platform, with the data shared across multiple hosts in various locations.

As part of this integration Chronicle worked closely with Kayzen to review existing capacity, replication, and distribution controls within Map, and changes were made to provide effectively unlimited capacity (up to 263 bytes total, or 263 indexes). Real-world Map capacity is constrained only by available hardware and storage.

Replicating such large data sets and ensuring consistency is maintained across a Map cluster presents particular challenges – eg allowance must be made for synchronising arbitrarily large backfills of data between nodes over arbitrarily slow networks – and a package of changes were made to enable Map Enterprise to continue to offer a fully robust HA/DR platform for arbitrarily large data sets across clusters of up to 127 distinct hosts.

In addition, support for bespoke time-to-live (TTL) functionality was added to the Chronicle Map environment within Kayzen whereby individual updates to a Map can automatically expire (and be removed from the map) after a configurable period of time. This functionality will be further generalised and formally integrated within Map Enterprise in a future release.

More Information

For more information on Chronicle Map and Map Enterprise please contact:

sales@chronicle.software

To request further information on Kayzen please visit: https://kayzen-4.hubspotpagebuilder.com/request-a-demo

 

Want access to exclusive Chronicle content?

The Chronicle Community is now live, hit the button below and join the community, for free! You'll get exclusive access to blogs, guides and video content. Join today!

Ivan Rakov

Ivan Rakov is a senior Java developer. He specializes in distributed systems, database internals, and low-latency solutions. He is an Apache committer, with a history of contributions to the transaction engines and persistence layer, of Apache Ignite. Ivan is also a speaker at conferences such as HighLoad and In-Memory Computing Summit.
Posted in

Want more from Chronicle? Subscribe today!

Something went wrong. Please check your entries and try again.