# chef Published articles for chef. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Teleport vs. AWS SSM Session Manager DevFeed: [Teleport vs. AWS SSM Session Manager]() Original publisher: [Read original article]() Author: ev@goteleport.com (Ev Kontsevoy) Published: 2021-03-08T00:00:00Z Content type: comparison Language: en Sources: [Teleport]() Topics: [Amazon Web Services](), [AWS IAM](), [systems](), [Amazon EC2](), [Command-line interface](), [Amazon S3](), [WebSocket](), [OpenID connect (OIDC)](), [Open Source](), [Ansible](), [Puppet]() Tags: [aws](), [chef](), [cli](), [compare](), [ec2](), [iam](), [oidc](), [open-source](), [puppet](), [s3](), [systems](), [websocket]() ### AI overview The article introduces AWS Systems Manager Session Manager, explains its architecture and session-recording workflow, and compares its capabilities and IAM-based access model with Teleport. ### Source excerpt AWS Systems Manager provides session management and session recording features. We'll explore these new interactive capabilities and compare them to Teleport ## REDstack: An Open-Source Tool for Provisioning Kerberized Hadoop Clusters on OpenStack DevFeed: [REDstack: An Open-Source Tool for Provisioning Kerberized Hadoop Clusters on OpenStack]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2017-12-07T06:00:00Z Content type: article Language: en Sources: [Target]() Topics: [big-data](), [Provisioning](), [Hadoop](), [openstack](), [Orchestration](), [Python](), [Open Source](), [Docker]() Tags: [big-data](), [big-data-infrastructure](), [chef](), [cloud](), [docker](), [druid](), [elasticsearch](), [hadoop](), [open-source](), [openstack](), [orchestration](), [provisioning](), [python]() ### AI overview REDstack is an open-source sandbox tool for Big Data development that provisions kerberized Hadoop clusters on OpenStack. It combines a cookbook for installing and configuring cluster components with a Python orchestration application that manages resource provisioning, Chef deployment, and component installation. ### Source excerpt REDstack is Now Open Source! We are officially open sourcing REDstack, our sandbox tool for Big Data development at Target. What is REDstack? REDstack is a tool for provisioning kerberized clusters on OpenStack. We created it with four goals in mind: Provide a secured environment, with the ability to leverage preconfigured LDAP and Kerberos servers. Out of the box usability, allowing you to log in with preconfigured user accounts. Custom user management utilities to administer the cluster. Provide a fully customizable experience, everything is a configuration option in your build files: Cluster size, node sizes, types of nodes and node roles, Hadoop configurations, heap sizes, and components, All users, passwords, and secure assets. Components REDstack is made up of two major components: hdp-cloud - The cookbook The cookbook is used by the application itself to install components and lay down cluster configuration. The cookbook can be used independently of REDstack to manually provision a cluster. REDstack - The orchestration component REDstack is a python application that performs all of the high-level complexities and timings associated with a full Hadoop installation: Orchestrates the provisioning of resources over OpenStack APIs, Controls and monitors parallel Chef deployment across the cluster, Manages and monitors cluster component install over HTTPS requests. REDstack is bundled with a Docker image, where the configs are set up locally before an installation, and all of the dependencies are updated and configured. How to Get Started Head over to the repository at https://github.com/target/redstack and follow along. The repo has instructions on how to build and configure the clusters using the included Docker image. History of the Project Target's Big Data Platform Team manages multiple Big Data environments, with hundreds of nodes and many PB's of data. As mentioned in our prior blog posts, we depend heavily on Chef as a core part of our CI/CD pipeline. Durin ## Bare Metal Big Data Builds DevFeed: [Bare Metal Big Data Builds]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2016-01-13T06:00:00Z Content type: article Language: en Sources: [Target]() Topics: [Hadoop](), [Automation](), [openstack](), [Nova](), [Jenkins](), [Bash](), [Python](), [API]() Tags: [ambari](), [api](), [automation](), [bare-metal](), [bash](), [big-data](), [chef](), [hadoop](), [infrastructure](), [ironic](), [jenkins](), [nova](), [openstack](), [python]() ### AI overview This engineering article describes Target's transition from manually building on-premise Hadoop clusters to an automated bare-metal provisioning workflow. It combines OpenStack Ironic and the Nova client with user-data bash scripts and Chef cookbooks to bootstrap servers, configure node roles, and add them to clusters. ### Source excerpt When you first think about scaling an on-premise Hadoop cluster your mind jumps to the process and the teams involved in building the servers, the time needed for configuring them and then the stability required while getting them into the cluster. Here at Target that process used to be measured in months. The story below outlines our journey around scaling our Hadoop cluster, taking the months to hours and adding hundreds of servers in a couple weeks. The Need Early 2013 taught us the lesson that manually managing Hadoop clusters, no matter how small is a time consuming and very repetitive task. Our next cluster build in 2014 drove the adoption of Chef, Artifactory and Jenkins to help with cluster operations. We stood up those components and created new role cookbooks to manage everything on the OS (configurations, storage, Kerberos, MySQL, etc.). While this was a step in the right direction, it left us with a manual process to still create the initial base server build and then add it to the cluster after configuring it with Chef. Build Foundation Closing the gap in our automation meant finding a way to deliver on true end to end builds, from an initial bootstrap to running jobs in your cluster. OpenStack's Ironic project was the first piece of the puzzle. Ironic gives us the ability to provision bare metal servers, similar to how OpenStack automated the VM build process. With Ironic as the foundation, we leveraged the Nova client to manage our instance builds. The nova python client interacts with the Compute service's API, giving us an easy way to specify our build parameters and spinning up an instance on one of our physical servers. The other key piece with the nova client is the ability to send boot information to the server using user data. The bash script sent executes several commands to install the Chef client, setup public keys and run the initial knife bootstrap to set the run list for the build. Example nova boot command: nova boot --image $image_name