# ironic

Published articles for ironic.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Bare Metal Big Data Builds

DevFeed: [Bare Metal Big Data Builds](<https://devfeed.tech/articles/bare-metal-big-data-builds-20406.md>)

Original publisher: [Read original article](<https://target.github.io/infrastructure/bare-metal-big-data-builds>)

Author: Target Brands, Inc

Published: 2016-01-13T06:00:00Z

Content type: article

Language: en

Sources: [Target](<https://devfeed.tech/sources/target.md>)

Topics: [Hadoop](<https://devfeed.tech/topics/hadoop.md>), [Automation](<https://devfeed.tech/topics/automation.md>), [openstack](<https://devfeed.tech/topics/openstack.md>), [Nova](<https://devfeed.tech/topics/nova.md>), [Jenkins](<https://devfeed.tech/topics/jenkins.md>), [Bash](<https://devfeed.tech/topics/bash.md>), [Python](<https://devfeed.tech/topics/python.md>), [API](<https://devfeed.tech/topics/api.md>)

Tags: [ambari](<https://devfeed.tech/tags/ambari.md>), [api](<https://devfeed.tech/tags/api.md>), [automation](<https://devfeed.tech/tags/automation.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [bash](<https://devfeed.tech/tags/bash.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [chef](<https://devfeed.tech/tags/chef.md>), [hadoop](<https://devfeed.tech/tags/hadoop.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [ironic](<https://devfeed.tech/tags/ironic.md>), [jenkins](<https://devfeed.tech/tags/jenkins.md>), [nova](<https://devfeed.tech/tags/nova.md>), [openstack](<https://devfeed.tech/tags/openstack.md>), [python](<https://devfeed.tech/tags/python.md>)

### AI overview

This engineering article describes Target's transition from manually building on-premise Hadoop clusters to an automated bare-metal provisioning workflow. It combines OpenStack Ironic and the Nova client with user-data bash scripts and Chef cookbooks to bootstrap servers, configure node roles, and add them to clusters.

### Source excerpt

When you first think about scaling an on-premise Hadoop cluster your mind jumps to the process and the teams involved in building the servers, the time needed for configuring them and then the stability required while getting them into the cluster. Here at Target that process used to be measured in months. The story below outlines our journey around scaling our Hadoop cluster, taking the months to hours and adding hundreds of servers in a couple weeks. The Need Early 2013 taught us the lesson that manually managing Hadoop clusters, no matter how small is a time consuming and very repetitive task. Our next cluster build in 2014 drove the adoption of Chef, Artifactory and Jenkins to help with cluster operations. We stood up those components and created new role cookbooks to manage everything on the OS (configurations, storage, Kerberos, MySQL, etc.). While this was a step in the right direction, it left us with a manual process to still create the initial base server build and then add it to the cluster after configuring it with Chef. Build Foundation Closing the gap in our automation meant finding a way to deliver on true end to end builds, from an initial bootstrap to running jobs in your cluster. OpenStack's Ironic project was the first piece of the puzzle. Ironic gives us the ability to provision bare metal servers, similar to how OpenStack automated the VM build process. With Ironic as the foundation, we leveraged the Nova client to manage our instance builds. The nova python client interacts with the Compute service's API, giving us an easy way to specify our build parameters and spinning up an instance on one of our physical servers. The other key piece with the nova client is the ability to send boot information to the server using user data. The bash script sent executes several commands to install the Chef client, setup public keys and run the initial knife bootstrap to set the run list for the build. Example nova boot command: nova boot --image $image_name