# openstack OpenStack is an open-source cloud computing platform providing Infrastructure-as-a-Service through interrelated services and APIs. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## REDstack: An Open-Source Tool for Provisioning Kerberized Hadoop Clusters on OpenStack DevFeed: [REDstack: An Open-Source Tool for Provisioning Kerberized Hadoop Clusters on OpenStack]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2017-12-07T06:00:00Z Content type: article Language: en Sources: [Target]() Topics: [big-data](), [Provisioning](), [Hadoop](), [openstack](), [Orchestration](), [Python](), [Open Source](), [Docker]() Tags: [big-data](), [big-data-infrastructure](), [chef](), [cloud](), [docker](), [druid](), [elasticsearch](), [hadoop](), [open-source](), [openstack](), [orchestration](), [provisioning](), [python]() ### AI overview REDstack is an open-source sandbox tool for Big Data development that provisions kerberized Hadoop clusters on OpenStack. It combines a cookbook for installing and configuring cluster components with a Python orchestration application that manages resource provisioning, Chef deployment, and component installation. ### Source excerpt REDstack is Now Open Source! We are officially open sourcing REDstack, our sandbox tool for Big Data development at Target. What is REDstack? REDstack is a tool for provisioning kerberized clusters on OpenStack. We created it with four goals in mind: Provide a secured environment, with the ability to leverage preconfigured LDAP and Kerberos servers. Out of the box usability, allowing you to log in with preconfigured user accounts. Custom user management utilities to administer the cluster. Provide a fully customizable experience, everything is a configuration option in your build files: Cluster size, node sizes, types of nodes and node roles, Hadoop configurations, heap sizes, and components, All users, passwords, and secure assets. Components REDstack is made up of two major components: hdp-cloud - The cookbook The cookbook is used by the application itself to install components and lay down cluster configuration. The cookbook can be used independently of REDstack to manually provision a cluster. REDstack - The orchestration component REDstack is a python application that performs all of the high-level complexities and timings associated with a full Hadoop installation: Orchestrates the provisioning of resources over OpenStack APIs, Controls and monitors parallel Chef deployment across the cluster, Manages and monitors cluster component install over HTTPS requests. REDstack is bundled with a Docker image, where the configs are set up locally before an installation, and all of the dependencies are updated and configured. How to Get Started Head over to the repository at https://github.com/target/redstack and follow along. The repo has instructions on how to build and configure the clusters using the included Docker image. History of the Project Target's Big Data Platform Team manages multiple Big Data environments, with hundreds of nodes and many PB's of data. As mentioned in our prior blog posts, we depend heavily on Chef as a core part of our CI/CD pipeline. Durin ## A peek at OpenStack Neutron DevFeed: [A peek at OpenStack Neutron]() Original publisher: [Read original article]() Published: 2017-10-06T00:00:00Z Content type: tutorial Language: en Sources: [Gaurav Sarma's Blog]() Topics: [openstack](), [networking](), [virtual machines](), [Cloud]() Tags: [cloud](), [datacenter](), [gre](), [linux](), [networking](), [networks](), [nova](), [openstack](), [virtual-machines](), [vlan](), [vxlan]() ### AI overview This tutorial examines OpenStack Neutron, its networking service, and how it differs from the legacy Nova networking service. It introduces Linux networking concepts and describes network types including local, flat, VLAN, and VXLAN/GRE overlays, along with Linux Bridge and OVS plugins. ### Source excerpt From the OpenStack official [website](https://www. openstack... ## Measuring the Performance of our OpenStack Cloud DevFeed: [Measuring the Performance of our OpenStack Cloud]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2017-06-20T05:00:00Z Content type: article Language: en Sources: [Target]() Topics: [openstack](), [Performance Testing](), [Cloud](), [Hardware](), [HTTP](), [Latency](), [Command-line interface](), [Linux]() Tags: [cli](), [cloudpunch](), [configuration](), [hardware](), [http](), [latency](), [linux](), [openstack](), [performance](), [performance-testing]() ### AI overview This article describes Target's effort to measure performance in its private OpenStack cloud. It evaluates Rally and KloudBuster, finding that Rally focuses on API testing while KloudBuster supports HTTP and storage tests but lacks sufficient extensibility, configuration, and stability. The authors therefore decide to create their own flexible performance framework. ### Source excerpt Here at Target, we run our own private OpenStack cloud and have never been able to accurately measure the performance of our hardware. This lack of measurement prevents the evaluation of performance improvements of new hardware or alternative technologies running as drivers inside OpenStack. It also prevents us from providing a Service Level Agreement (SLA) to our customers. Recently we have been striving to improve our OpenStack service which led us to talk to our consumers directly. One of the major feedback points provided by talking with our consumers was the performance of the OpenStack cloud was lower than expected. Because we have not measured the performance of our cloud in the past, we have been unable to know if new hardware or configuration changes improves consumer-facing performance. With our new OpenStack environment builds we focused on changing this. But first we needed a tool to do the job. Searching for a Tool The first tool we looked at was Rally. Rally does performance testing of an OpenStack cloud. However, Rally focuses on the OpenStack API only. It is mainly used to test functionality (via Tempest) and stability of the API under large amounts of load. Rally does contain a resource to boot an instance and run Linux CLI commands via user data. This was tested as a way to provide the staging of instances to run performance software. However, starting the software on each instance at the same time and collecting the results from said software was difficult and not viable. Because of this, we deemed Rally was not suitable for our needs. The next tool we looked at was KloudBuster. KloudBuster is a tool that does performance testing inside an OpenStack instance. At the time of writing it provides two sets of tests: HTTP and storage. The HTTP test uses a traffic generator to measure requests per second and latency between instances. The storage test uses FIO to measure read/write IOPs and bandwidth. KloudBuster does what we were looking for, measuring ## Bare Metal Big Data Builds DevFeed: [Bare Metal Big Data Builds]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2016-01-13T06:00:00Z Content type: article Language: en Sources: [Target]() Topics: [Hadoop](), [Automation](), [openstack](), [Nova](), [Jenkins](), [Bash](), [Python](), [API]() Tags: [ambari](), [api](), [automation](), [bare-metal](), [bash](), [big-data](), [chef](), [hadoop](), [infrastructure](), [ironic](), [jenkins](), [nova](), [openstack](), [python]() ### AI overview This engineering article describes Target's transition from manually building on-premise Hadoop clusters to an automated bare-metal provisioning workflow. It combines OpenStack Ironic and the Nova client with user-data bash scripts and Chef cookbooks to bootstrap servers, configure node roles, and add them to clusters. ### Source excerpt When you first think about scaling an on-premise Hadoop cluster your mind jumps to the process and the teams involved in building the servers, the time needed for configuring them and then the stability required while getting them into the cluster. Here at Target that process used to be measured in months. The story below outlines our journey around scaling our Hadoop cluster, taking the months to hours and adding hundreds of servers in a couple weeks. The Need Early 2013 taught us the lesson that manually managing Hadoop clusters, no matter how small is a time consuming and very repetitive task. Our next cluster build in 2014 drove the adoption of Chef, Artifactory and Jenkins to help with cluster operations. We stood up those components and created new role cookbooks to manage everything on the OS (configurations, storage, Kerberos, MySQL, etc.). While this was a step in the right direction, it left us with a manual process to still create the initial base server build and then add it to the cluster after configuring it with Chef. Build Foundation Closing the gap in our automation meant finding a way to deliver on true end to end builds, from an initial bootstrap to running jobs in your cluster. OpenStack's Ironic project was the first piece of the puzzle. Ironic gives us the ability to provision bare metal servers, similar to how OpenStack automated the VM build process. With Ironic as the foundation, we leveraged the Nova client to manage our instance builds. The nova python client interacts with the Compute service's API, giving us an easy way to specify our build parameters and spinning up an instance on one of our physical servers. The other key piece with the nova client is the ability to send boot information to the server using user data. The bash script sent executes several commands to install the Chef client, setup public keys and run the initial knife bootstrap to set the run list for the build. Example nova boot command: nova boot --image $image_name ## Meetup PostgreSQL à Paris DevFeed: [Meetup PostgreSQL à Paris]() Original publisher: [Read original article]() Author: Dimitri Fontaine PostgreSQL Major Contributor; Author Published: 2014-04-17T11:28:00Z Content type: opinion Language: fr Sources: [Dimitri Fontaine]() Topics: [PostgreSQL](), [openstack](), [Object-relational mapping]() Tags: [meetup](), [openstack](), [orm](), [postgresql]() ### AI overview A report on the third PostgreSQL Meetup in Paris covers PostgreSQL use in production by France's Ministry of the Interior, OpenStack's use of PostgreSQL, ORM integration concerns, and a Tsung presentation. ### Source excerpt Hier soir se déroulait le troisième Meetup PostgreSQL à Paris, et je crois pouvoir dire que tous les participants étaient ravis de cette édition. Malheureusement, quelques uns sont restés bloqués du mauvais côté de la porte hier soir, j'en suis navré. Nous essayerons d'étendre nos horaires d'accueil lors des prochaines rencontres, dans la mesure du possible. Les sponsors de la soirée Un grand merci à nos sponsors : hier soir eNovance nous accueillait dans une superbe salle de conférence et Hegoa s'est chargée de mettre à disposition des participants un buffet très apprécié.