# From Hadoop and Cassandra to Kafka Streams

DevFeed: [From Hadoop and Cassandra to Kafka Streams](<https://devfeed.tech/articles/from-hadoop-and-cassandra-to-kafka-streams-32017.md>)

Original publisher: [Read original article](<https://tech.finn.no2017/07/31/from-hadoop-and-cassandra-to-kafka-streams/>)

Author: Nicolas Yann Couturier

Published: 2017-07-31T12:00:00Z

Content type: article

Language: en

Sources: [Finn.no](<https://devfeed.tech/sources/finn-no.md>)

Topics: [Kafka](<https://devfeed.tech/topics/kafka.md>), [Apache Cassandra](<https://devfeed.tech/topics/cassandra.md>), [Hadoop](<https://devfeed.tech/topics/hadoop.md>)

Tags: [cassandra](<https://devfeed.tech/tags/cassandra.md>), [hadoop](<https://devfeed.tech/tags/hadoop.md>), [kafka](<https://devfeed.tech/tags/kafka.md>)

## AI overview

The article describes FINN.no's existing pipeline for computing classified-ad statistics from user actions published to Apache Kafka, stored in Cassandra, and processed in batches by Hadoop. It introduces the need to join these events with demographic data to provide more detailed viewer statistics.

## Source excerpt

Some context People who publish their classified ads on FINN.no get access to various statistics to see how their ads are performing. For a user it can look something like this: Statistics the owner of a realestate ad gets to see The top left bar chart shows the repartition of the incoming traffic by day and the legend to the right of it shows the total numbers for each type of incoming traffic. The lower section is divided in 3 parts: the left part shows the number of views by unique users; the one to the center shows how many users have been notified of the ad by email and how many added the ad to their favorites; the right part shows how many viewers out of the total come from a specific type of traffic. This gives the user basic insight into the reach of their ad, such as how many views it has and how many unique users have viewed it. Around November 2016 there was a request to show more detailed information about the viewers, such as the age, gender and location distribution of the viewers (demographic information) so the owner could potentially change an ad to better fit the audience they wanted. Existing solution As users view ads, do actions (such as send a message to an ad's owner or scroll down and read the whole page, for example), these actions are gathered and published internally on Apache Kafka. These streams of data then become the basis for computing the statistics above. The plan was then to also publish demographic data about the viewers (such as location, age and gender) on Kafka and join the user actions with this demographic data to provide enhanced statistics. At the time, the action events published on Kafka were saved to Cassandra Apache Cassandra clusters, and statistics were being computed as batches on an aging Apache Hadoop cluster reading from Cassandra. Both our Hadoop and Cassandra clusters had not receive much love recently and were all on end-of-life versions. The old system also had an increasing tendency to fail, so we were also i