# Building Infrastructure in Parallel

DevFeed: [Building Infrastructure in Parallel](<https://devfeed.tech/articles/building-infrastructure-in-parallel-31925.md>)

Original publisher: [Read original article](<http://engineering.remind.com/building-infrastructure-in-parallel/>)

Author: Remind

Published: 2018-03-05T00:00:00Z

Content type: tutorial

Language: en

Sources: [Remind](<https://devfeed.tech/sources/remind.md>)

Topics: [Reverse Dependencies](<https://devfeed.tech/topics/reverse-dependencies.md>), [cloud-infrastructure](<https://devfeed.tech/topics/cloud-infrastructure.md>), [AWS CloudFormation](<https://devfeed.tech/topics/aws-cloudformation.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [Computer science](<https://devfeed.tech/topics/computer-science.md>), [make](<https://devfeed.tech/topics/make.md>), [systemd](<https://devfeed.tech/topics/systemd.md>), [Terraform](<https://devfeed.tech/topics/terraform.md>), [coding](<https://devfeed.tech/topics/coding.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [building](<https://devfeed.tech/tags/building.md>), [cloudformation](<https://devfeed.tech/tags/cloudformation.md>), [computer-science](<https://devfeed.tech/tags/computer-science.md>), [dependency](<https://devfeed.tech/tags/dependency.md>), [graphs](<https://devfeed.tech/tags/graphs.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [make](<https://devfeed.tech/tags/make.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [refactor](<https://devfeed.tech/tags/refactor.md>), [systemd](<https://devfeed.tech/tags/systemd.md>), [terraform](<https://devfeed.tech/tags/terraform.md>), [vpc](<https://devfeed.tech/tags/vpc.md>)

## AI overview

This tutorial explains how Remind improved stacker, its infrastructure deployment tool, to update CloudFormation stacks in parallel. It models stack dependencies as a directed acyclic graph and reports reducing execution time from about 10 minutes to under 1.5 minutes; the change was included in stacker 1.2.

## Source excerpt

In the early days of stacker at Remind, the number of stacks that we were managing was just a handful. CloudFormation stacks were updated sequentially, without any parallelism, and everyone was happy. Today, stacker manages 153 CloudFormation stacks that build out all of the AWS infrastructure that makes remind.com work. As you can imagine, attempting to update 153 stacks without any parallelism takes a long time, about 10 minutes to be precise. When you consider that most stacks don't ever change, 10 minutes feels like an eternity, and prevents a quick feedback loop. Over the last couple of weeks, I took it upon myself to refactor the core issues in stacker that made implementing parallelism difficult, and dropped our total execution time from 10 minutes, to under 1.5 minutes. This work is now included in stacker 1.2 so everyone can benefit. Thinking in Graphs Before I dive into how we implemented parallelism in stacker, I think it's important to take a step back, and look at what stacker is, at it's core; a tool for linking infrastructure together as a dependency graph. Let's take a look at a hypothetical stacker config to build out some infrastructure. stacks: - name: vpc class_path: blueprints.VPC - name: cluster class_path: blueprints.Cluster variables: VpcId: ${output vpc::Id} - name: db class_path: blueprints.DB variables: VpcId: ${output vpc::Id} - name: app class_path: blueprints.App variables: DBUrl: ${output db::URL} Cluster: ${output cluster::Id} When described in this way, the output lookup creates an explicit dependency relationship between stacks. The "app" stack depends on a "db" stack and a "cluster" stack, the "db" stack depends on a "vpc" stack, etc. We can visualize the resulting dependency graph using dot. digraph { cluster -> vpc; db -> vpc; app -> db; app -> cluster; } $ cat graph.dot | dot -Tpng > graph.png In fact, creating dependency graphs like this isn't really a new concept in computing; make does it, systemd does it, and programing lang