# Introducing Oak: an Open Source Scalable Key-Value Map for Big Data Analytics

DevFeed: [Introducing Oak: an Open Source Scalable Key-Value Map for Big Data Analytics](<https://devfeed.tech/articles/introducing-oak-an-open-source-scalable-key-value-map-for-big-data-analytics-20488.md>)

Original publisher: [Read original article](<https://yahooeng.tumblr.com/post/178045294111>)

Author: amberwilsonla-blog

Published: 2018-09-13T16:04:58Z

Content type: article

Language: en

Sources: [Yahoo](<https://devfeed.tech/sources/yahoo.md>)

Topics: [Open Source](<https://devfeed.tech/topics/open-source.md>), [Data structures](<https://devfeed.tech/topics/data-structures.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [data](<https://devfeed.tech/topics/data.md>), [real-time](<https://devfeed.tech/topics/real-time.md>), [Java](<https://devfeed.tech/topics/java.md>), [Programming](<https://devfeed.tech/topics/programming.md>), [systems](<https://devfeed.tech/topics/systems.md>)

Tags: [big-data](<https://devfeed.tech/tags/big-data.md>), [data](<https://devfeed.tech/tags/data.md>), [data-structures](<https://devfeed.tech/tags/data-structures.md>), [java](<https://devfeed.tech/tags/java.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [programming](<https://devfeed.tech/tags/programming.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [yahoo-engineering](<https://devfeed.tech/tags/yahoo-engineering.md>)

## AI overview

The article introduces Oak, an open-source Java project from Oath that implements OakMap, a concurrent ordered key-value map. It presents Oak as a tool for scalable real-time analytics workloads and describes its relationship to concurrent data structures and Druid.

## Source excerpt

yahoodevelopers: By Dmitry Basin, Edward Bortnikov, Anastasia Braginsky, Eshcar Hillel, Idit Keidar, Hagar Meir, Gali Sheffi Real-time analytics applications are on the rise. Modern decision support and machine intelligence engines strive to continuously ingest large volumes of data while providing up-to-date insights with minimum delay. For example, in Flurry Analytics, an Oath service which provides mobile developers with rich tools to explore user behavior in real time, it only takes seconds to reflect the events that happened on mobile devices in its numerous dashboards. The scalability demand is immense - as of late 2017, the Flurry SDK was installed on 2.6B devices and monitored 1M+ mobile apps. Mobile data hits the Flurry backend at a huge rate, updates statistics across hundreds of dimensions, and becomes queryable immediately. Flurry harnesses the open-source distributed interactive analytics engine named Druid to ingest data and serve queries at this massive rate. In order to minimize delays before data becomes available for analysis, technologies like Druid should avoid maintaining separate systems for data ingestion and query serving, and instead strive to do both within the same system. Doing so is nontrivial since one cannot compromise on overall correctness when multiple conflicting operations execute in parallel on modern multi-core CPUs. A promising approach is using concurrent data structure (CDS) algorithms which adapt traditional data structures to multiprocessor hardware. CDS implementations are thread-safe - that is, developers can use them exactly as sequential code while maintaining strong theoretical correctness guarantees. In recent years, CDS algorithms enabled dramatic application performance scaling and became popular programming tools. For example, Java programmers can use the ConcurrentNavigableMap JDK implementations for the concurrent ordered key-value map abstraction that is instrumental in systems like Druid. Today, we are excited