# Creating Readable Spark Jobs

DevFeed: [Creating Readable Spark Jobs](<https://devfeed.tech/articles/creating-readable-spark-jobs-2017.md>)

Original publisher: [Read original article](<https://developers.soundcloud.com/blog//creating-readable-spark-jobs>)

Published: 2018-08-06T00:00:00Z

Content type: article

Language: en

Sources: [SoundCloud Backstage Blog](<https://devfeed.tech/sources/soundcloud-backstage-blog.md>)

Topics: [Apache Spark](<https://devfeed.tech/topics/spark.md>), [Functional programming](<https://devfeed.tech/topics/functional-programming.md>), [Scala](<https://devfeed.tech/topics/scala.md>), [Programming](<https://devfeed.tech/topics/programming.md>)

Tags: [apache-flink](<https://devfeed.tech/tags/apache-flink.md>), [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [batch](<https://devfeed.tech/tags/batch.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [functional-programming](<https://devfeed.tech/tags/functional-programming.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [scala](<https://devfeed.tech/tags/scala.md>), [spark](<https://devfeed.tech/tags/spark.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

## AI overview

This article explains how to make Apache Spark jobs more readable by breaking complex, deeply nested operations into smaller composable units. It presents functional programming techniques in Scala and illustrates them with a simplified SoundCloud content-routing example based on artist and label metadata rules.

## Source excerpt

Nowadays, it's rather common to encounter Apache Spark being utilized in a lot of companies that need to process huge amounts of data, and things aren't any different here at SoundCloud -- as one can imagine, we have lots of data to process all the time.