# Hadoop Rolling Upgrades

DevFeed: [Hadoop Rolling Upgrades](<https://devfeed.tech/articles/hadoop-rolling-upgrades-20408.md>)

Original publisher: [Read original article](<https://target.github.io/infrastructure/hadoop-rolling-upgrades>)

Author: Target Brands, Inc

Published: 2017-05-22T05:00:00Z

Content type: article

Language: en

Sources: [Target](<https://devfeed.tech/sources/target.md>)

Topics: [Hadoop](<https://devfeed.tech/topics/hadoop.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Provisioning](<https://devfeed.tech/topics/provisioning.md>), [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Apache Spark](<https://devfeed.tech/topics/spark.md>)

Tags: [big-data](<https://devfeed.tech/tags/big-data.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [hadoop](<https://devfeed.tech/tags/hadoop.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [provisioning](<https://devfeed.tech/tags/provisioning.md>), [spark](<https://devfeed.tech/tags/spark.md>), [upgrades](<https://devfeed.tech/tags/upgrades.md>)

## AI overview

This article describes Target's move from outage-based Hadoop upgrades to rolling maintenance. It covers the upgrade from Hadoop 2.7.1 to 2.7.3, the goal of avoiding planned downtime, and the use of cluster administration and REDStack provisioning tools to test and orchestrate the process.

## Source excerpt

Hadoop upgrades over the last few years meant long outages where the Big Data platform team would shutdown the cluster, perform the upgrade, start services and then complete validation before notifying users it was ok to resume activity. This approach is a typical pattern for major upgrades even outside Target and reduces the complexity and risks associated with the upgrade. While this worked great for the platform team, it was not ideal for the hundreds of users and thousands of jobs that were dependent on the platform. That is why we decided to shake things up and go all in for rolling maintenance. Cluster Details: Hadoop (Core) 2.7.1 to 2.7.3 Mixed cluster workload running Hive, Tez, MR, Spark, Pig, HBase, Storm Goal Our March 2016 Hadoop upgrade was the turning point for rolling maintenance. With a large outage for the upgrade and monthly maintenance windows leading up to it, we decided to challenge ourselves with rolling maintenance to reach our uptime goals for core components. This would allow the platform team to deploy changes faster, reduce maintenance risk by not bundling changes together every month and more importantly not impact users with planned downtime. Drawing the line in the sand for rolling maintenance meant that it was time to get to work on our upgrade strategy. Reviewing the Playbook We started reviewing the upgrade process in November 2016 with the goal to upgrade our first admin cluster in December. The short turnaround time meant that we would leverage our cluster administration tool to handle the upgrade orchestration. The focus shifted towards understanding the order of events and evaluating the process for potential impacts. Running through the upgrades required a quick way to iterate through Hadoop cluster deployments and then tearing them down to retest. This is where REDStack, which is our internal cloud provisioning tool to build out a secure Hadoop cluster with production like configurations, came into play. After going through the