# cloudformation Published articles for cloudformation. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Terraform AWS Provider Continues Rapid Expansion as AWS Infrastructure Becomes More Complex DevFeed: [Terraform AWS Provider Continues Rapid Expansion as AWS Infrastructure Becomes More Complex]() Original publisher: [Read original article]() Author: Craig Risi Published: 2026-09-11T12:00:00Z Content type: news Language: en Sources: [InfoQ]() Topics: [Terraform](), [Infrastructure as code](), [cloud-infrastructure](), [AWS CloudFormation](), [Amazon Bedrock](), [DSQL]() Tags: [aws](), [cloudformation](), [devops](), [infrastructure-as-code](), [news](), [release](), [terraform](), [terraform-aws-provider-6-62]() ### AI overview Terraform AWS Provider v6.62.0 adds support for AWS services and resources. The article compares Terraform with Pulumi, CloudFormation/CDK, and OpenTofu, and advises teams to pin and test provider upgrades because of schema, state, and behavior risks. ### Source excerpt The Terraform AWS Provider continues its rapid evolution, with v6.62.0 adding support for new AWS capabilities while improving how Terraform understands and manages existing infrastructure. By Craig Risi ## Best Kubernetes Infrastructure as Code Tools in 2026 DevFeed: [Best Kubernetes Infrastructure as Code Tools in 2026]() Original publisher: [Read original article]() Author: Pulumi Content Team Published: 2026-08-14T00:00:00Z Content type: comparison Language: en Sources: [Pulumi]() Topics: [Infrastructure as code](), [Kubernetes](), [Terraform](), [argo-cd](), [Helm charts](), [GitOps](), [flux](), [opentofu](), [Amazon EKS](), [AWS CloudFormation](), [VPC](), [AWS IAM]() Tags: [argo-cd](), [cloudformation](), [code](), [comparisons](), [devops](), [flux](), [general](), [gitops](), [helm](), [helm-charts](), [iam](), [infrastructure](), [infrastructure-as-code](), [kubernetes](), [opentofu](), [platform-engineering](), [terraform](), [vpc]() ### AI overview This comparison explains that Kubernetes infrastructure as code spans cluster and cloud provisioning, in-cluster workload definition, and continuous delivery and reconciliation. It presents Pulumi and Terraform or OpenTofu as general-purpose provisioning options, Helm and Kustomize for workload templating, and Argo CD and Flux for GitOps reconciliation. ### Source excerpt There is no single best Kubernetes infrastructure as code tool, because "Kubernetes IaC" actually spans three different jobs. For provisioning the cluster and its cloud dependencies, Pulumi and Terraform (or OpenTofu) are the strongest general-purpose options. For templating and packaging workloads, Helm and Kustomize dominate. For continuous reconciliation once things are running, Argo CD and Flux lead the GitOps category. The right stack usually combines one tool from each layer, not a single tool that claims to do all three. What counts as infrastructure as code for Kubernetes? Kubernetes infrastructure as code work splits into three layers that get conflated constantly, and the confusion is where most tool comparisons go wrong. The cluster and cloud layer provisions the things Kubernetes itself sits on top of: the managed control plane (EKS, GKE, AKS), node pools, the VPC and subnets, IAM roles, load balancers, and cluster add-ons. Terraform, Pulumi, and cloud-native tools like CloudFormation operate here. The in-cluster workload layer defines what runs on the cluster once it exists: Deployments, Services, ConfigMaps, CustomResourceDefinitions, and the Helm charts or Kustomize overlays that template them. This is where Helm, Kustomize, and Crossplane's custom resources live. The delivery and reconciliation layer keeps what's declared in Git in sync with what's actually running on the cluster, continuously, rather than as a one-shot apply. Argo CD and Flux own this layer, and they consume the output of the other two rather than replacing them. Most real Kubernetes platforms use tools from at least two of these layers together. A team might provision EKS with Terraform, package its application with Helm, and let Argo CD reconcile it continuously. Knowing which layer a tool actually addresses, rather than treating "Kubernetes IaC" as one shopping list, is the first decision that matters. Pulumi provisions the cluster and the workloads on it in the same language Pul ## Infrastructure Control Plane | Day 2 Operations & Drift DevFeed: [Infrastructure Control Plane | Day 2 Operations & Drift]() Original publisher: [Read original article]() Author: Nicole Morgan Published: 2026-08-06T00:00:00Z Content type: article Language: en Sources: [Harness Blog]() Topics: [Infrastructure as code](), [Software-defined networking](), [Provisioning](), [Ansible](), [CI/CD](), [configuration](), [Amazon EC2](), [AWS CloudFormation](), [Amazon Web Services]() Tags: [ansible](), [aws](), [ci-cd](), [cloudformation](), [configuration](), [deployment](), [ec2](), [incident](), [infrastructure](), [infrastructure-as-code](), [provisioning](), [systems](), [tools]() ### AI overview This article explains that infrastructure problems often emerge during day 2 operations, after initial deployment. It describes how console changes, incidents, and isolated Terraform, Ansible, and CI/CD workflows create infrastructure drift, and argues that control planes can enforce governance, detect divergence, and automate remediation. ### Source excerpt Learn why infrastructure breaks after deployment and how control planes enforce governance, detect drift, and automate remediation across Terraform, Ansible, an | Blog ## Migrate CloudFormation to Pulumi with Discovered Stacks DevFeed: [Migrate CloudFormation to Pulumi with Discovered Stacks]() Original publisher: [Read original article]() Author: Alejandro Cotroneo Published: 2026-07-30T00:00:00Z Content type: tutorial Language: en Sources: [Pulumi]() Topics: [Infrastructure as code](), [AWS CloudFormation](), [migration](), [Amazon Web Services](), [VPC](), [Amazon S3](), [DynamoDB](), [AWS IAM]() Tags: [aws](), [cloudformation](), [dynamodb](), [git](), [iac](), [iam](), [import](), [insights](), [kms](), [migration](), [pulumi](), [pulumi-cloud](), [pulumi-neo](), [s3](), [tutorials](), [vpc]() ### AI overview A tutorial shows how to migrate a 61-resource AWS CloudFormation stack to Pulumi using Discovered Stacks in Pulumi Cloud. It explains how to find the stack, review migration status, start the migration, resolve unmatched resources, and verify the result with a zero-diff pulumi preview. ### Source excerpt With Discovered Stacks, Pulumi Cloud does the bookkeeping for a CloudFormation migration: every resource in the stack gets an explicit migration status, and the migration is done when the code provably matches the cloud. In this tutorial, we take one real CloudFormation stack from discovered to migrated and managed by Pulumi IaC, end to end. What we're migrating Our example is payments-api, a CloudFormation stack with 61 resources: a VPC, an Aurora ledger database behind an RDS Proxy, an assets S3 bucket, a DynamoDB ledger table, a Kinesis payment-events pipeline, and the IAM roles, KMS keys, and secrets that wire them together. The plan has five steps: Find the stack in Pulumi Cloud. Review the migration breakdown at a glance. Start the migration. Resolve the stragglers, so every resource is accounted for. Confirm the quality gate: a zero-diff pulumi preview. We'll use Pulumi Neo to do the heavy lifting, but nothing here depends on it. The same flow works with your own coding agent or entirely by hand, because migration status is derived from actual stack state -- however the work gets done, the console shows the same progress. Step 1: Find your stack in Pulumi Cloud Discovered Stacks builds on Discovery, so the only prerequisite is a scanned cloud account -- the AWS account holding your CloudFormation stacks. Once a scan has run, open the Stacks page and turn on Show Discovered Stacks. Your CloudFormation stacks appear alongside your Pulumi stacks. The project name comes from the CloudFormation stack (payments-api), and the stack name encodes the account and region it came from, so the same template deployed to two regions shows up as two distinct discovered stacks. Step 2: Plan the migration at a glance Open the discovered stack's Migration tab. It lays out all 61 resources of payments-api by status, so you can visualize the migration before touching anything: 54 Ready: mapped to a Pulumi type and confirmed to exist -- importable right now. 2 Not found: mapped, but ## Accelerate your infrastructure deployments by up to 4x with AWS CloudFormation Express mode DevFeed: [Accelerate your infrastructure deployments by up to 4x with AWS CloudFormation Express mode]() Original publisher: [Read original article]() Author: Channy Yun (윤석찬) Published: 2026-06-30T21:30:33Z Content type: article Language: en Sources: [AWS News Blog]() Topics: [AWS CloudFormation](), [Deployment](), [Amazon Web Services](), [Infrastructure as code](), [Command-line interface](), [AWS Management Console](), [Resilience](), [SDKs](), [Amazon Simple Queue Service (SQS)](), [AWS Lambda](), [Kiro]() Tags: [accelerate](), [agents](), [ai-tools](), [amazon-simple-queue-service-sqs](), [aws](), [aws-cloudformation](), [aws-lambda](), [aws-management-console](), [cli](), [cloudformation](), [deployment](), [developers](), [infrastructure](), [kiro](), [launch](), [management-tools](), [news](), [resilience](), [sdks](), [testing]() ### AI overview AWS CloudFormation Express mode accelerates infrastructure deployments by completing when resource configuration is applied instead of waiting for extended stabilization checks. The article describes use in iterative development, production scenarios, testing, and AI-assisted infrastructure workflows, with examples involving SQS and Lambda. ### Source excerpt AWS CloudFormation speeds up infrastructure deployment with Express mode, enabling AI agents and developers to receive deployment confirmation in seconds and iterate faster. Available in all commercial Regions at no additional cost. ## Terraform vs. CloudFormation: Choosing the Right IaC Tool DevFeed: [Terraform vs. CloudFormation: Choosing the Right IaC Tool]() Original publisher: [Read original article]() Author: EMILIANO ANGIERI, CLOUD ENGINEER AND DEVELOPER @ TOPTAL Published: 2026-04-06T04:00:00Z Content type: comparison Language: en Sources: [Toptal Blog]() Topics: [AWS CloudFormation](), [Terraform](), [Infrastructure as code](), [Amazon Web Services]() Tags: [aws](), [cloudformation](), [iac](), [terraform]() ### AI overview The article compares Terraform and AWS CloudFormation as infrastructure-as-code tools. It explains how CloudFormation suits AWS-native environments, while Terraform emphasizes multicloud flexibility, modularity, and an open ecosystem. ### Source excerpt Terraform and CloudFormation offer powerful but distinct paths to infrastructure as code. Explore their pros, cons, and when to choose each for AWS or multicloud deployments. ## Controller-Driven Infrastructure as Code DevFeed: [Controller-Driven Infrastructure as Code]() Original publisher: [Read original article]() Published: 2025-03-19T20:36:25Z Content type: article Language: en Sources: [Brave New Geek]() Topics: [Infrastructure as code](), [Kubernetes](), [consistency](), [Terraform](), [AWS CloudFormation](), [configuration](), [API]() Tags: [cloudformation](), [consistency](), [controllers](), [declarative](), [iac](), [infrastructure](), [infrastructure-as-code](), [koreo](), [kubernetes](), [platform-engineering](), [policy](), [reconciliation](), [resource-orchestration](), [terraform]() ### AI overview The article explains Controller-Driven Infrastructure as Code, applying Kubernetes controllers and the Kubernetes Resource Model to infrastructure management. It contrasts this resource-centric, continuously reconciled approach with the operation-centric models used by Terraform and CloudFormation. ### Source excerpt Harnessing the Kubernetes Resource Model for modern infrastructure management Infrastructure as Code (IaC) revolutionized how we manage infrastructure, enabling developers to define resources declaratively and automate their deployment. However, tools like Terraform and CloudFormation, despite their declarative configuration, rely on an operation-centric model, where resources are created or updated through one-shot commands. The evolution of IaC: From operations to controllers In contrast, Kubernetes introduced a new paradigm with its controller pattern and the Kubernetes Resource Model (KRM). This resource-centric approach to APIs redefines infrastructure management by focusing on desired state rather than discrete operations. Kubernetes controllers continuously monitor resources, ensuring they conform to their declarative configurations by performing actions to move the actual state closer to the desired state, much like a human operator would. This is known as a control loop. ## Serverless Framework Now Supports AWS SAM & Cloudformation DevFeed: [Serverless Framework Now Supports AWS SAM & Cloudformation]() Original publisher: [Read original article]() Author: Serverless Team Published: 2024-08-26T00:00:00Z Content type: release Language: en Sources: [Serverless Blog]() Topics: [serverless framework](), [AWS CloudFormation](), [AWS Lambda](), [Amazon Web Services](), [Development]() Tags: [aws](), [aws-lambda](), [cloud-computing](), [cloudformation](), [faas](), [function-as-a-service](), [infrastructure-as-code](), [news](), [sam](), [serverless](), [serverless-architecture](), [serverless-framework](), [workflow]() ### AI overview Serverless Framework 4.2.3 adds support for AWS SAM and CloudFormation templates, enabling deployment of SAM and CloudFormation projects alongside Serverless Variables for AWS Lambda development. ### Source excerpt Serverless Framework 4.2.3 now supports AWS SAM and CloudFormation templates, simplifying AWS Lambda development. Deploy SAM/CF projects seamlessly, use Serverless Variables, and streamline your workflow. Perfect for organizations seeking to consolidate serverless tools and standardize development processes. ## How to prevent accidental load balancer deletions DevFeed: [How to prevent accidental load balancer deletions]() Original publisher: [Read original article]() Author: Andre Newman Published: 2024-07-03T00:00:00Z Content type: tutorial Language: en Sources: [Gremlin Blog]() Topics: [Amazon Web Services](), [Cloud](), [Deployment](), [Infrastructure as code](), [AWS CloudFormation](), [Terraform]() Tags: [aws](), [cloud](), [cloudformation](), [deployment](), [iac](), [load-balancer](), [reliability-management](), [terraform]() ### AI overview This tutorial explains how to prevent accidental deletion of AWS Elastic Load Balancers by enabling their deletion protection attribute. It describes how the protection works across the AWS Console, AWS SDK, CloudFormation, and Terraform, and why it helps reduce outage risk caused by manual mistakes or infrastructure-as-code changes. ### Source excerpt Accidentally deleting cloud resources happens more often than you'd think. Learn how to enable deletion protection for your AWS Elastic Load Balancers (ELBs) and lower your risk of service outages. ## Learning from cloud transformation as we move to AI DevFeed: [Learning from cloud transformation as we move to AI]() Original publisher: [Read original article]() Author: David Lugo Published: 2024-05-21T13:00:00Z Content type: opinion Language: en Sources: [Blog RSS Feed | Snyk]() Topics: [AI-assisted coding](), [GitHub Copilot]() Tags: [ai](), [awareness](), [blog](), [cloud](), [cloudformation](), [containers](), [copilot](), [devops](), [devsecops](), [executive](), [github-copilot](), [google](), [infrastructure-as-code](), [megawatt](), [related-content](), [security]() ### AI overview The article compares the adoption of AI coding tools with earlier cloud transformation. It argues that lessons about speed, process changes, security risks, and DevSecOps from cloud adoption can help organizations adopt AI-generated code. ### Source excerpt The emergence of AI-generated code looks similar to cloud transformation in the past. We can take lessons learned from the cloud and apply them to AI. ## Automating the Klarna Card Ownership Fees System using AWS Step Functions DevFeed: [Automating the Klarna Card Ownership Fees System using AWS Step Functions]() Original publisher: [Read original article]() Author: Michel Neumann Published: 2024-05-02T07:42:27Z Content type: article Language: en Sources: [Klarna Engineering]() Topics: [AWS Step Functions](), [AWS CloudFormation](), [Automation](), [jira]() Tags: [athena](), [automation](), [aws](), [aws-step-functions](), [cloudformation](), [databases](), [jira](), [klarna](), [maintenance](), [pull-requests](), [run](), [serverless](), [step-functions](), [workflow]() ### AI overview The article explains how Klarna automated its Klarna Card monthly fee-collection process with AWS Step Functions and CloudFormation. The automation replaced a manual, multi-day batch workflow with a scheduled process and reduced maintenance effort. ### Source excerpt This article outlines how my team and I applied automation using AWS Step Functions and CloudFormation on a system to charge the monthly fee for Klarna Cards, enabling us to transform a previously manual routine into a self-sufficient, scheduled workflow. The initiative significantly streamlined operations and reduced maintenance cost. Introduction In early 2023, Klarna introduced monthly fees for Klarna Cards in the US. In the Card & Banking domain, two teams, including myself as engineer, developed this system within a tight four-month deadline. Initially, the system required extensive manual operation, including a detailed checklist for engineers to follow to ensure successful executions. The teams launched, planning iterative improvements of that routine. Months passed by without any advancement in refining the operation process nor automating any part of it. To provide an overview of what needed to be done by the teams to run the batch jobs: Designating an engineer to lead the monthly process, coordinated using JIRA tickets Updating exemption lists and submitting pull requests to the code-base prior to initiating batch runs Ensuring data integrity by performing Athena queries across three different production databases within the AWS Console Manually initiating multiple batch jobs in a specified sequence with manual input of arguments in a live production setting Awaiting termination of the jobs and conducting a thorough review of the outcomes of the final batch jobs for each market Overall, this routine took around three business days per month and required two engineers to approve code changes and review the results of the batch runs. Considering new markets where fees may be rolled out towards, this workflow posed a significant challenge to maintaining high-quality standards and preventing potential incidents. Taking on The Challenge We recognized that continuing with our current process was unsustainable and bound to cause issues down the line. When the top ## Bazaarvoice Notification System for Transactional Email Delivery DevFeed: [Bazaarvoice Notification System for Transactional Email Delivery]() Original publisher: [Read original article]() Author: Someswar Bhowmick Published: 2024-04-24T10:24:56Z Content type: article Language: en Sources: [Bazaarvoice]() Topics: [notifications](), [Amazon Web Services](), [Amazon EC2](), [Amazon S3](), [AWS CloudFormation](), [Jenkins]() Tags: [aws](), [cloud](), [cloudformation](), [ec2](), [email](), [engineering](), [jenkins](), [notifications](), [s3](), [scalability](), [serverless](), [software-architecture]() ### AI overview Bazaarvoice describes its notification system for sending transactional email on behalf of clients. The article outlines data ingestion and a decision engine for scheduled delivery, then discusses operational challenges in its earlier AWS-based architecture, including EC2 scaling, updates, logging, and prolonged file-processing batch jobs. ### Source excerpt Bazaarvoice notification system stands as a testament to cutting-edge technology, designed to seamlessly dispatch transactional email messages (post-interaction email or PIE) on behalf of our clients. The heartbeat of our system lies in the constant influx of new content, driven by active content solicitations. Equipped with an array of tools, including email message styling, default [...] ## Automatic source locations with Rego DevFeed: [Automatic source locations with Rego]() Original publisher: [Read original article]() Author: Jasper Van der Jeugt Published: 2024-02-12T05:00:00Z Content type: article Language: en Sources: [Blog RSS Feed | Snyk]() Topics: [opa](), [snyk-iac](), [Infrastructure as code](), [Open Policy Agent](), [AWS CloudFormation](), [Terraform]() Tags: [blog](), [blog-post](), [cloud-security](), [cloudformation](), [developer](), [devops](), [engineering](), [iac](), [opa](), [rego](), [snyk](), [snyk-iac](), [technical]() ### AI overview This technical Snyk article explains automatic source code locations for Rego policy violations in Snyk IaC. The feature reports accurate file, line, and column information, including for custom rules. A simplified proof of concept uses CloudFormation YAML, Rego policies, and YAML tree traversal to infer attribute paths and retrieve source locations. ### Source excerpt We recently released a series of improvements to Snyk IaC, and in this blog post, we're taking a technical dive into a particularly interesting feature -- automatic source code locations for rule violations. ## Securing the Connection to S3 from EC2 DevFeed: [Securing the Connection to S3 from EC2]() Original publisher: [Read original article]() Author: Guillermo Ojeda Published: 2023-09-07T15:02:09Z Content type: tutorial Language: en Sources: [Guille Ojeda]() Topics: [Amazon EC2](), [Amazon S3](), [VPC](), [Security](), [AWS CloudFormation]() Tags: [amazon-vpc](), [aws](), [cloud](), [cloudformation](), [ec2](), [s3](), [security]() ### AI overview A step-by-step tutorial for securing access from an Amazon EC2 instance to a private object in Amazon S3. It explains how to replace a public bucket setup with a restrictive bucket policy and a VPC endpoint policy. ### Source excerpt You've deployed your app on an EC2 instance, and there's a file in an S3 bucket that you need to access from the app. You created a public S3 bucket and uploaded the file, and it works! But then you read somewhere that keeping your private files in a... ## Automating Multi-Account Permission Management with AWS IAM Identity Center (previously AWS SSO) DevFeed: [Automating Multi-Account Permission Management with AWS IAM Identity Center (previously AWS SSO)]() Original publisher: [Read original article]() Author: Niko Jernejsek Published: 2022-12-22T10:32:09Z Content type: article Language: en Sources: [Homegate Engineering Blog - Medium]() Topics: [AWS IAM Identity Center](), [AWS Organizations](), [Single sign-on (SSO)](), [Authorization](), [IAM](), [CI/CD]() Tags: [aws](), [aws-eventbridge](), [aws-iam](), [aws-iam-identity-center](), [aws-organizations](), [cicd](), [cloudformation](), [iam-identity-center](), [least-privilege](), [saml](), [security](), [sso]() ### AI overview This article describes Homegate's multi-account AWS strategy and its use of AWS IAM Identity Center, formerly AWS SSO, to centrally manage permissions for engineers across AWS accounts. It explains the use of permission matrices, least-privilege access, AWS Organizations, external identity providers through SAML 2.0, and SCIM. ### Source excerpt Background Homegate was founded in 2001 and is now the leading property marketplace in Switzerland. Few years ago we decided to re-architect our monolithic real estate platform towards micro services on top of AWS cloud. At the moment, Homegate workloads are running almost exclusively on server-less compute model with teams organised around micro services. We are utilising so called multi-account strategy where we structure our AWS accounts around individual micro services further breaking them down into different accounts according to software lifecycle, and organising them into the AWS Organizational Units. We've structured our AWS Organizations Units & AWS accounts according to the recommendations provided by AWS (https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/basic-organization.html#basic-organization-with-cicd-as-a-separate-function). Apart from having product workload accounts we also have accounts which are necessary to operate security, governance and platform related workloads. Access to accounts is governed by a permission matrix which is based on least privilege principles. The permission matrix classifies who (internally defined personas, e.g.: engineers/managers/security officers, etc) has what type of access (e.g. read-only or admin), and to what category of AWS accounts (e.g. production workloads, shared infra accounts, etc). Group of accounts in our Permission Matrix are at the same time AWS Organisational Units that we implement. As we have to manage permissions for hundreds of engineers to about hundred of AWS accounts accordingly, this way of grouping makes it easier for us to manage it at scale. IAM Identity Center PrimerIAM Identity Center Intro Note that I will be using AWS IAM Identity Center and AWS SSO interchangeably as even some services (such as CloudFormation) are still using the previous name AWS SSO. AWS IAM Identity Center is a single sign-on (SSO) service that allows AWS users to centrally manage acces ## Automate AWS Athena Partitions for Organization CloudTrail Logs and VPC Flow Logs DevFeed: [Automate AWS Athena Partitions for Organization CloudTrail Logs and VPC Flow Logs]() Original publisher: [Read original article]() Author: Aurelien Cerveaux Published: 2022-09-15T06:39:58Z Content type: tutorial Language: en Sources: [betclic-tech - Medium]() Topics: [AWS CloudTrail](), [Amazon Web Services](), [VPC Flow Logs](), [Amazon S3](), [AWS CloudFormation](), [DynamoDB](), [JSON]() Tags: [amazon-s3](), [athena](), [aws](), [aws-cloudtrail](), [cloud](), [cloudformation](), [cloudtrail](), [dynamodb](), [events](), [json](), [vpc-flow-logs]() ### AI overview A tutorial explains how to automate daily AWS Athena partition creation for organization-level CloudTrail and VPC Flow Logs. It describes using Amazon S3, AWS Glue Data Catalog, CloudFormation, SAM, Lambda, DynamoDB, and EventBridge to improve query efficiency and reduce scanned data and cost. ### Source excerpt This article's goal is to explain how to query efficiently, and at low cost with AWS Athena, AWS CloudTrail logs, or VPC Flow logs by automating the creation of Athena partitions daily. AWS CloudTrail AWS CloudTrail is a service that records all API Calls and Events for AWS Accounts, including details on calls made to your AWS Services and the Console as well. Activated at the Organization level, CloudTrail logs help you in several cases: Simplified Compliance Summarizing Accesses Security Analysis and Troubleshooting Monitoring Users and Resources Activities Governance and Investigation CloudTrail Logs are delivered in an S3 Bucket in JSON format through many small files that make these difficult to operate and request. VPC Flow Logs VPC Flow Logs is a feature that allows you to capture traffic information going to and from your network interfaces in your VPC. It helps you to: Analyze and troubleshoot network issues Determine the direction of your traffic Monitor the traffic Like CloudTrail Logs, Flow Logs are published in an S3 Bucket for all accounts in the Organization. AWS Athena AWS Athena is a query service that permits you to analyze data directly in an Amazon S3. You can create tables in Athena which use AWS Glue Data Catalog and are based on an S3 location. To query efficiently and with less scanned data and optimize the cost, we need to create some partitions. To know more about Tables, Databases, and Catalog in Athena. Automate Partitions Creation In our case, we are using CloudTrail Logs and VPC Flow Logs to detail the solution and the source code. All is done in CloudFormation and using SAM to deploy the Lambda. Global Overview We have several parts to automate the creation of our partitions. A common part containing: The Lambda is used to create partitions concerning different log types A DynamoDB table used to store partition names A specific part for each log type contains: The Glue Table with the proper structure An EventBridge Rule to trigger the ## Replaying events using AWS S3 DevFeed: [Replaying events using AWS S3]() Original publisher: [Read original article]() Author: Marko Savic Published: 2022-04-08T08:40:24Z Content type: tutorial Language: en Sources: [Homegate Engineering Blog - Medium]() Topics: [Amazon S3](), [event driven](), [Microservices](), [Amazon Web Services](), [AWS CloudFormation](), [Serverless]() Tags: [aws](), [aws-lambda](), [aws-s3](), [cloudformation](), [debugging](), [event-driven](), [event-sourcing](), [lambda](), [microservices](), [s3](), [serverless](), [sns](), [software-architecture](), [versioning]() ### AI overview This tutorial describes replaying failed events in serverless, event-driven microservices by storing AWS Step Functions events in versioned Amazon S3 objects. It also outlines using a dead-letter queue and Lambda to retrieve the latest event and trigger the workflow again. ### Source excerpt In the world of event-driven microservices one of the important tasks is to have a mechanism to recover from failed events, as well as to have possibility to replay certain events. We at Homegate deal with serverless microservices. If, at any of these services, we encounter a failure during processing of an resource, we might need to recreate the last event or series of events for given resource. Usecase The central part of ingestion process is AWS StepFunctions that lives in a, so called, importer service. It is responsible for various enrichments and conditional actions on resource. The last step of the step function is sending an event to the AWS SNS which is consumed by many downstream services as you can see in the following diagram. Step functions as central part of ingestion process Here, many open questions can be identified: How to deal with failure within AWS StepFunctions? How to recreate SNS event if needed for downstream system? Would it be possible to replay the whole step function? Solution Possible answer to these questions would be to store events that goes through AWS StepFunctions into some kind of event storage, in Homegate case, we chose AWS S3. In order to support the whole history of events for given resource, S3 bucket versioning is used. This is quite handy for replaying series of events or even for debugging purposes. Here is a sample CloudFormation which enables that functionality: EventLogBucket: Type: 'AWS::S3::Bucket' Properties: BucketName: 'sample-unnique-bucket-name' AccessControl: Private PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true Tags: - Key: service Value: sample-service - Key: env Value: !FindInMap [ Stages, !Ref Stage , Env ] VersioningConfiguration: Status: Enabled That way we can replay the whole step functions flow using last logged event for given resource. That functionality could be available from different places, like DLQ or via api end ## Data Integrity Protection in Presto DevFeed: [Data Integrity Protection in Presto]() Original publisher: [Read original article]() Author: Piotr Findeisen, Starburst Data Published: 2020-06-25T00:00:00Z Content type: article Language: en Sources: [Trino Blog]() Topics: [data](), [Kubernetes](), [Amazon Elastic Kubernetes Service](), [releases](), [Amazon EC2](), [AWS CloudFormation]() Tags: [aws](), [benchmarks](), [cloud](), [cloudformation](), [data](), [enterprise](), [java](), [jupyter](), [kubernetes](), [pdf](), [performance](), [release](), [releases]() ### AI overview The article investigates a serious data-integrity issue discovered during Presto performance benchmarks for the Starburst Enterprise Presto 332-e release. It describes block encoding, an unexpected decoded length, and the environment changes considered while diagnosing the problem, including Java 11 and AWS EKS-based Kubernetes clusters. ### Source excerpt It all started on an Thursday afternoon in March, when Karol Sobczak was grilling Presto with heavy rounds of benchmarks, as we were ramping up to Starburst Enterprise Presto 332-e release. Karol discovered what seemed to be a serious regression, and turned out to be even more serious Cloud environment issue. ## Using API Gateway WebSockets with the Serverless Framework DevFeed: [Using API Gateway WebSockets with the Serverless Framework]() Original publisher: [Read original article]() Author: Jared Short Published: 2018-12-21T00:00:00Z Content type: release Language: en Sources: [Serverless Blog]() Topics: [Amazon API Gateway](), [WebSocket](), [Serverless](), [Framework](), [AWS CloudFormation]() Tags: [api](), [api-gateway](), [aws](), [aws-lambda](), [cloud-computing](), [cloudformation](), [engineering](), [faas](), [function-as-a-service](), [gateway](), [infrastructure-as-code](), [plugin](), [serverless](), [serverless-architecture](), [serverless-framework]() ### AI overview Serverless built a plugin that enables API Gateway WebSockets with the Serverless Framework before CloudFormation support was available. ### Source excerpt We built a plugin to let you use API Gateway WebSockets with the Serverless Framework, even in advance of CloudFormation support! Try it out. ## Serverless Framework v.1.35: Local Invoke Ruby, CloudFormation variable syntax DevFeed: [Serverless Framework v.1.35: Local Invoke Ruby, CloudFormation variable syntax]() Original publisher: [Read original article]() Author: Jared Short Published: 2018-12-17T00:00:00Z Content type: release Language: en Sources: [Serverless Blog]() Topics: [Serverless](), [Framework](), [AWS CloudFormation](), [Ruby]() Tags: [aws](), [aws-lambda](), [cloud-computing](), [cloudformation](), [config](), [faas](), [function-as-a-service](), [infrastructure-as-code](), [local](), [news](), [release](), [ruby](), [serverless](), [serverless-architecture](), [serverless-framework](), [syntax](), [v1](), [variable]() ### AI overview This release announcement covers Serverless Framework v1.35, including local Ruby invocation support and CloudFormation configuration features. ### Source excerpt Check out the latest Serverless Framework v1.35 release, featuring Ruby invoke local support, CloudFormation config, and more. ## Serverless Workarounds for CloudFormation's 200 Resource Limit DevFeed: [Serverless Workarounds for CloudFormation's 200 Resource Limit]() Original publisher: [Read original article]() Author: Alex DeBrie Published: 2018-03-07T00:00:00Z Content type: tutorial Language: en Sources: [Serverless Blog]() Topics: [AWS CloudFormation](), [Serverless]() Tags: [aws](), [aws-lambda](), [cloud-computing](), [cloudformation](), [engineering](), [faas](), [function-as-a-service](), [infrastructure-as-code](), [resource](), [serverless](), [serverless-architecture](), [serverless-framework]() ### AI overview The article explains how to troubleshoot and avoid CloudFormation's 200-resource limit, including workarounds associated with Serverless. ### Source excerpt How you can troubleshoot, and avoid hitting, CloudFormation's 200 resource limit ## Building Infrastructure in Parallel DevFeed: [Building Infrastructure in Parallel]() Original publisher: [Read original article]() Author: Remind Published: 2018-03-05T00:00:00Z Content type: tutorial Language: en Sources: [Remind]() Topics: [Reverse Dependencies](), [cloud-infrastructure](), [AWS CloudFormation](), [Amazon Web Services](), [Algorithms](), [Computer science](), [make](), [systemd](), [Terraform](), [coding]() Tags: [algorithms](), [building](), [cloudformation](), [computer-science](), [dependency](), [graphs](), [infrastructure](), [make](), [parallel](), [parallelism](), [postgresql](), [refactor](), [systemd](), [terraform](), [vpc]() ### AI overview This tutorial explains how Remind improved stacker, its infrastructure deployment tool, to update CloudFormation stacks in parallel. It models stack dependencies as a directed acyclic graph and reports reducing execution time from about 10 minutes to under 1.5 minutes; the change was included in stacker 1.2. ### Source excerpt In the early days of stacker at Remind, the number of stacks that we were managing was just a handful. CloudFormation stacks were updated sequentially, without any parallelism, and everyone was happy. Today, stacker manages 153 CloudFormation stacks that build out all of the AWS infrastructure that makes remind.com work. As you can imagine, attempting to update 153 stacks without any parallelism takes a long time, about 10 minutes to be precise. When you consider that most stacks don't ever change, 10 minutes feels like an eternity, and prevents a quick feedback loop. Over the last couple of weeks, I took it upon myself to refactor the core issues in stacker that made implementing parallelism difficult, and dropped our total execution time from 10 minutes, to under 1.5 minutes. This work is now included in stacker 1.2 so everyone can benefit. Thinking in Graphs Before I dive into how we implemented parallelism in stacker, I think it's important to take a step back, and look at what stacker is, at it's core; a tool for linking infrastructure together as a dependency graph. Let's take a look at a hypothetical stacker config to build out some infrastructure. stacks: - name: vpc class_path: blueprints.VPC - name: cluster class_path: blueprints.Cluster variables: VpcId: ${output vpc::Id} - name: db class_path: blueprints.DB variables: VpcId: ${output vpc::Id} - name: app class_path: blueprints.App variables: DBUrl: ${output db::URL} Cluster: ${output cluster::Id} When described in this way, the output lookup creates an explicit dependency relationship between stacks. The "app" stack depends on a "db" stack and a "cluster" stack, the "db" stack depends on a "vpc" stack, etc. We can visualize the resulting dependency graph using dot. digraph { cluster -> vpc; db -> vpc; app -> db; app -> cluster; } $ cat graph.dot | dot -Tpng > graph.png In fact, creating dependency graphs like this isn't really a new concept in computing; make does it, systemd does it, and programing lang ## NAT instance on AWS DevFeed: [NAT instance on AWS]() Original publisher: [Read original article]() Published: 2018-02-27T00:00:00Z Content type: tutorial Language: en Sources: [Paweł Chudzik]() Topics: [Amazon Web Services](), [VPC](), [AWS CloudFormation](), [Cloud](), [Amazon EC2](), [autoscaling]() Tags: [autoscaling](), [aws](), [cloud](), [cloudformation](), [ec2](), [infrastructure](), [vpc]() ### AI overview A practical guide to replacing AWS NAT gateways with a NAT instance configured through CloudFormation. The author describes creating a VPC with public and private subnets, configuring an EC2 instance as the NAT gateway, and seeking lower costs at the expense of reliability. ### Source excerpt At work and in my private time I'm trying to get myself familiar with AWS cloud. Almost all of this is new for me. I know how to setup few things using AWS web console but infrastructure as a code was calling to me and I wanted to give it a spin. My first task was to create VPC with two pairs of subnets for private stuff and here is why I did it twice. Read more ## AWS Infrastructure Setup: The CleverTap Way DevFeed: [AWS Infrastructure Setup: The CleverTap Way]() Original publisher: [Read original article]() Author: Francis Pereira Published: 2017-07-25T07:36:24Z Content type: article Language: en Sources: [CleverTap]() Topics: [AWS CloudFormation](), [Amazon Web Services](), [Infrastructure as code](), [Amazon EC2](), [Puppet](), [Containers](), [data centers](), [Security]() Tags: [aws](), [clevertap](), [clevertap-engineering-infrastructure](), [cloudformation](), [containers](), [data-centers](), [ec2](), [engineering](), [infrastructure](), [infrastructure-as-code](), [puppet]() ### AI overview CleverTap describes how it redesigned its AWS infrastructure to support geographically distributed data centers and variable traffic. The article covers moving from console-based setup to CloudFormation and splitting the infrastructure into independent stacks, along with the earlier use of Puppet for configuration management and the transition toward Docker containers. ### Source excerpt Now that a little more than half of 2017 has gone by, I want to talk about some of infrastructure The post AWS Infrastructure Setup: The CleverTap Way first appeared on CleverTap. [Next page]()