# Teardown, Rebuild: Migrating from Hive to PySpark

DevFeed: [Teardown, Rebuild: Migrating from Hive to PySpark](<https://devfeed.tech/articles/teardown-rebuild-migrating-from-hive-to-pyspark-27967.md>)

Original publisher: [Read original article](<https://tech.trivago.com/post/2018-12-03-teardownrebuildmovingfromhivetopyspark/>)

Author: German I Ramirez Espinoza Data Scientist working

Published: 2018-12-03T00:00:00Z

Content type: article

Language: en

Sources: [Trivago](<https://devfeed.tech/sources/trivago.md>)

Topics: [Development](<https://devfeed.tech/topics/development.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [Machine Learning & Artificial Intelligence](<https://devfeed.tech/topics/machine-learning-artificial-intelligence.md>), [Testing](<https://devfeed.tech/topics/testing.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [continuous-integration](<https://devfeed.tech/tags/continuous-integration.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [development](<https://devfeed.tech/tags/development.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [software](<https://devfeed.tech/tags/software.md>), [software-development](<https://devfeed.tech/tags/software-development.md>), [testing](<https://devfeed.tech/tags/testing.md>), [unit-testing](<https://devfeed.tech/tags/unit-testing.md>)

## AI overview

The article discusses how ML engineering and software development share practices for writing correct, robust algorithms, with particular attention to unit testing, continuous integration, and floating-point round-off errors. It also describes trivago's data science and bidding algorithms for its auction-based marketplace.

## Source excerpt

Machine Learning (ML) engineering and software development are both fundamentally about writing correct and robust algorithms. In ML engineering we have the extra difficulty of ensuring mathemat...