# How Target Performance Tunes Machine Learning Applications

DevFeed: [How Target Performance Tunes Machine Learning Applications](<https://devfeed.tech/articles/how-target-performance-tunes-machine-learning-applications-20400.md>)

Original publisher: [Read original article](<https://target.github.io/data%20science%20and%20engineering/numspark>)

Author: Target Brands, Inc

Published: 2016-09-29T05:00:00Z

Content type: article

Language: en

Sources: [Target](<https://devfeed.tech/sources/target.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Apache Spark](<https://devfeed.tech/topics/spark.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [Hadoop](<https://devfeed.tech/topics/hadoop.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [applications](<https://devfeed.tech/tags/applications.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science-and-engineering](<https://devfeed.tech/tags/data-science-and-engineering.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-computing](<https://devfeed.tech/tags/distributed-computing.md>), [engine](<https://devfeed.tech/tags/engine.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [file-system](<https://devfeed.tech/tags/file-system.md>), [hadoop](<https://devfeed.tech/tags/hadoop.md>), [large-datasets](<https://devfeed.tech/tags/large-datasets.md>), [leverage](<https://devfeed.tech/tags/leverage.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [matrix](<https://devfeed.tech/tags/matrix.md>), [open-source-software](<https://devfeed.tech/tags/open-source-software.md>), [performance](<https://devfeed.tech/tags/performance.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [science](<https://devfeed.tech/tags/science.md>)

## AI overview

Target describes evaluating large-scale matrix multiplication to improve the speed and relevance of machine-learning algorithms built on billions of guest interactions and item data. The excerpt says this work requires computation across multiple nodes, high-throughput access to distributed data, storage and compute that scale with added resources, and fault tolerance as failures become more common. It names Apache Hadoop for storing and interacting with data, Apache Spark as the compute engine and a distributed machine-learning tool, and Apache Mahout for building and applying distributed algorithms. The article begins its practical guidance with paper-and-pencil examples: try matrices of different shapes and properties, choose compatible dimensions, and work through operations such as multiplication, inversion, and transposition. The excerpt cuts off during this list.

## Source excerpt

At Target we aim to make shopping more fun and relevant for our guests through extensive use of data - and believe me, we have lots of data! Tens of millions of guests and hundreds of thousands of items lead to billions of transactions and interactions. We regularly employ a number of different machine learning techniques on such large datasets for dozens of algorithms. We are constantly looking for ways to improve speed and relevance of our algorithms and one such quest brought us to carefully evaluate matrix multiplications at scale - since that forms the bedrock for most algorithms. If we make matrix multiplication more efficient, we can speed up most of our algorithms! Before we dig in, let me describe some properties of the landscape we will be working in. First, what do I mean by large scale? A large scale application, at a minimum, will require its computation to be spread over multiple nodes of a distributed computing environment to finish in a reasonable amount of time. These calculations will use existing data that are stored on a distributed file system that provides high-throughput access from the computing environment. Scalability, in terms of storage and compute, should grow as we add to these resources. As the system grows larger and more complex, failures will become more commonplace. Thus, software should be fault-tolerant. Fortunately, there is a lot of existing open-source software that we can leverage to work in such an environment, particularly Apache Hadoop for storing and interacting with our data, Apache Spark as the compute engine, and both Apache Spark and Apache Mahout for applying and building distributed machine learning algorithms. There are many other tools that we can add to the mix as well, but for the purposes of this post we will limit our discussion to these three. With that out of the way, lets dig in! Don't Forget the Basics Begin with good old paper and pencil. Yeah, I know this is about large scale matrix operations that you c