# TFX

Published articles for TFX.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Simulated Spotify Listening Experiences for Reinforcement Learning with TensorFlow and TF-Agents

DevFeed: [Simulated Spotify Listening Experiences for Reinforcement Learning with TensorFlow and TF-Agents](<https://devfeed.tech/articles/simulated-spotify-listening-experiences-for-reinforcement-learning-with-tensorflow-and-tf-agents-7394.md>)

Original publisher: [Read original article](<https://blog.tensorflow.org/2023/10/simulated-spotify-listening-experiences-reinforcement-learning-tensorflow-tf-agents.html>)

Author: TensorFlow Blog (noreply@blogger.com)

Published: 2023-10-19T19:00:00Z

Content type: article

Language: en

Sources: [The TensorFlow Blog](<https://devfeed.tech/sources/the-tensorflow-blog.md>)

Topics: [TensorFlow Agents](<https://devfeed.tech/topics/tensorflow-agents.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [AI, ML & Data Engineering](<https://devfeed.tech/topics/ai-ml-data-engineering.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [TFX](<https://devfeed.tech/topics/tfx.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [documentation](<https://devfeed.tech/tags/documentation.md>), [keras](<https://devfeed.tech/tags/keras.md>), [learn](<https://devfeed.tech/tags/learn.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [offline](<https://devfeed.tech/tags/offline.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [tensorflow](<https://devfeed.tech/tags/tensorflow.md>), [tensorflow-agents](<https://devfeed.tech/tags/tensorflow-agents.md>), [tfx](<https://devfeed.tech/tags/tfx.md>)

### AI overview

Spotify describes using TensorFlow and TF-Agents to build an offline simulator for reinforcement-learning-based music recommendations. The simulator supported training and evaluating recommendation models and RL agents, with offline estimates reported as strongly correlated with live experimental results.

### Source excerpt

Posted by Surya Kanoria, Joseph Cauteruccio, Federico Tomasi, Kamil Ciosek, Matteo Rinaldi, and Zhenwen Dai - Spotify Introduction Many of our music recommendation problems involve providing users with ordered sets of items that satisfy users' listening preferences and intent at that point in time. We base current recommendations on previous interactions with our application and, in the abstract, are faced with a sequential decision making process as we continually recommend content to users. Reinforcement Learning (RL) is an established tool for sequential decision making that can be leveraged to solve sequential recommendation problems. We decided to explore how RL could be used to craft listening experiences for users. Before we could start training Agents, we needed to pick a RL library that allowed us to easily prototype, test, and potentially deploy our solutions. At Spotify we leverage TensorFlow and the extended TensorFlow Ecosystem (TFX, TensorFlow Serving, and so on) as part of our production Machine Learning Stack. We made the decision early on to leverage TensorFlow Agents as our RL Library of choice, knowing that integrating our experiments with our production systems would be vastly more efficient down the line. One missing bit of technology we required was an offline Spotify environment we could use to prototype, analyze, explore, and train Agents offline prior to online testing. The flexibility of the TF-Agents library, coupled with the broader advantages of TensorFlow and its ecosystem, allowed us to cleanly design a robust and extendable offline Spotify simulator. We based our simulator design on TF-Agents Environment primitives and using this simulator we developed, trained and evaluated sequential models for item recommendations, vanilla RL Agents (PPG, DQN) and a modified deep Q-Network, which we call the Action-Head DQN (AH-DQN), that addressed the specific challenges imposed by the large state and action space of our RL formulation. Through li