# DynamoDB Amazon DynamoDB is a serverless, fully managed, distributed NoSQL database service with single-digit millisecond performance and seamless scalability. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Perplexity's AI agents helped build a database. They weren't allowed to run it. DevFeed: [Perplexity's AI agents helped build a database. They weren't allowed to run it.]() Original publisher: [Read original article]() Author: Amanda Caswell Published: 2026-09-16T21:51:15Z Content type: article Language: en Sources: [The New Stack]() Topics: [Database](), [DynamoDB](), [Rust](), [Artificial Intelligence](), [rocksdb](), [Amazon S3]() Tags: [agents](), [ai](), [ai-agents](), [ai-infrastructure](), [api](), [coding](), [coding-agents](), [database](), [databases](), [dynamodb](), [open-source](), [performance](), [perplexity](), [rocksdb](), [rust](), [s3]() ### AI overview Perplexity built CobbleDB, a Rust key-value store, after finding DynamoDB too costly and insufficiently controllable for its search workload. Coding agents helped develop it, but were not allowed to run it in production. Perplexity measured lower read latency and expects lower costs, with plans to open-source the database. ### Source excerpt Perplexity decided it was paying too much for DynamoDB and wasn't getting the control it wanted over read performance. So The post Perplexity's AI agents helped build a database. They weren't allowed to run it. appeared first on The New Stack. ## Migrate CloudFormation to Pulumi with Discovered Stacks DevFeed: [Migrate CloudFormation to Pulumi with Discovered Stacks]() Original publisher: [Read original article]() Author: Alejandro Cotroneo Published: 2026-07-30T00:00:00Z Content type: tutorial Language: en Sources: [Pulumi]() Topics: [Infrastructure as code](), [AWS CloudFormation](), [migration](), [Amazon Web Services](), [VPC](), [Amazon S3](), [DynamoDB](), [AWS IAM]() Tags: [aws](), [cloudformation](), [dynamodb](), [git](), [iac](), [iam](), [import](), [insights](), [kms](), [migration](), [pulumi](), [pulumi-cloud](), [pulumi-neo](), [s3](), [tutorials](), [vpc]() ### AI overview A tutorial shows how to migrate a 61-resource AWS CloudFormation stack to Pulumi using Discovered Stacks in Pulumi Cloud. It explains how to find the stack, review migration status, start the migration, resolve unmatched resources, and verify the result with a zero-diff pulumi preview. ### Source excerpt With Discovered Stacks, Pulumi Cloud does the bookkeeping for a CloudFormation migration: every resource in the stack gets an explicit migration status, and the migration is done when the code provably matches the cloud. In this tutorial, we take one real CloudFormation stack from discovered to migrated and managed by Pulumi IaC, end to end. What we're migrating Our example is payments-api, a CloudFormation stack with 61 resources: a VPC, an Aurora ledger database behind an RDS Proxy, an assets S3 bucket, a DynamoDB ledger table, a Kinesis payment-events pipeline, and the IAM roles, KMS keys, and secrets that wire them together. The plan has five steps: Find the stack in Pulumi Cloud. Review the migration breakdown at a glance. Start the migration. Resolve the stragglers, so every resource is accounted for. Confirm the quality gate: a zero-diff pulumi preview. We'll use Pulumi Neo to do the heavy lifting, but nothing here depends on it. The same flow works with your own coding agent or entirely by hand, because migration status is derived from actual stack state -- however the work gets done, the console shows the same progress. Step 1: Find your stack in Pulumi Cloud Discovered Stacks builds on Discovery, so the only prerequisite is a scanned cloud account -- the AWS account holding your CloudFormation stacks. Once a scan has run, open the Stacks page and turn on Show Discovered Stacks. Your CloudFormation stacks appear alongside your Pulumi stacks. The project name comes from the CloudFormation stack (payments-api), and the stack name encodes the account and region it came from, so the same template deployed to two regions shows up as two distinct discovered stacks. Step 2: Plan the migration at a glance Open the discovered stack's Migration tab. It lays out all 61 resources of payments-api by status, so you can visualize the migration before touching anything: 54 Ready: mapped to a Pulumi type and confirmed to exist -- importable right now. 2 Not found: mapped, but ## Treating Pricing Changes Like Code Deploys DevFeed: [Treating Pricing Changes Like Code Deploys]() Original publisher: [Read original article]() Author: Ananda Kanagaraj Sankar Published: 2026-07-22T18:09:48Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [Deployment](), [DynamoDB](), [Go Language](), [Code review]() Tags: [code-review](), [deployment](), [devops](), [dry-run](), [dynamodb](), [engineering](), [go](), [marketplaces](), [monitor](), [platform-engineering](), [pricing-strategy]() ### AI overview This article explains how Thumbtack handled large pricing changes by comparing them with code deployments. Pricing bids stored in DynamoDB were rewritten in bulk using custom Go scripts, but those scripts lacked built-in resumability, runtime control, and durable records for undoing changes. ### Source excerpt Making large pricing changes safe to run, safe to resume, and safe to undo. Most days, shipping code to production at Thumbtack is a non-event. You merge your change, a deployment pipeline takes it from there and it rolls out while you monitor the rollout. If a deploy stalls, the system already knows which step it stalled on. If it goes wrong, rolling back is a button away. It's the return on years of platform work, so routine now that we mostly forget it's there. Which is the point: Safety is built into the road, not into how carefully each person drives. Changing pricing data never had that platform at Thumbtack. Pricing is how a marketplace like ours sustains itself. It lets us capture a share of the value we help create, and it's something we tune on an ongoing basis rather than set once. We test pricing changes, measure their impact and roll out the ones that work. Mechanically, this runs on an auction. When a customer creates a project on Thumbtack, pros compete for the lead, and what a pro is willing to pay shows up as a bid. Those bids are stored per pro's service offering (a specific offering a pro provides, like house cleaning or landscaping) as bid settings in DynamoDB. Rolling out a pricing change means rewriting those settings in bulk, often a few hundred thousand of them at a time, several times a year. For a long time, each of the pricing rollouts was a custom Go script written for that specific change. The scripts were production grade. They went through code review, ran in staging first, did a dry run before any live write, and were rolled out in stages. That was a reasonable way to work: the changes are infrequent, each one is genuinely different from the last, and a script you can read top to bottom is easy to trust. But what reviews and a good runbook can't give you is the runtime machinery. A script doesn't remember where it was when it died, so it can't safely pick up where it left off. It has no real sense of how hard it's pushing the database ## The DynamoDB Connector is now in Tinybird Forward DevFeed: [The DynamoDB Connector is now in Tinybird Forward]() Original publisher: [Read original article]() Author: Tomás Healy Published: 2026-07-16T11:00:00Z Content type: release Language: en Sources: [Tinybird]() Topics: [DynamoDB]() Tags: [dynamodb](), [product-updates]() ### AI overview Tinybird Forward now supports mirroring DynamoDB tables using a .connection file and a single deploy. The change addresses a migration gap for Tinybird Classic users. ### Source excerpt Mirror your DynamoDB tables to Tinybird Forward with a .connection file and a single deploy. For Classic users, this closes the last gap to migrating. ## How Order Editing Replaced a Failing DynamoDB Pipeline with Tinybird DevFeed: [How Order Editing Replaced a Failing DynamoDB Pipeline with Tinybird]() Original publisher: [Read original article]() Author: Tinybird Published: 2026-02-24T11:00:00Z Content type: article Language: en Sources: [Tinybird]() Topics: [Shopify](), [DynamoDB]() Tags: [customer-stories](), [debugging](), [dynamodb](), [infrastructure](), [pipeline](), [shopify](), [team]() ### AI overview A three-person team handling $5B in Shopify GMV replaced a failing DynamoDB pipeline with Tinybird, reducing the need to debug reconciliation failures and allowing the team to focus on shipping features. ### Source excerpt A three-person team running $5B in Shopify GMV couldn't afford to babysit analytics infrastructure. Here's how they stopped debugging reconciliation failures and started shipping features. ## HubSpot Incident Report for October 20, 2025 DevFeed: [HubSpot Incident Report for October 20, 2025]() Original publisher: [Read original article]() Author: Kartik Vishwanath Published: 2025-11-11T17:09:36Z Content type: news Language: en Sources: [HubSpot]() Topics: [incident](), [Amazon Web Services](), [Cloud](), [Resilience](), [Amazon Simple Queue Service (SQS)](), [Amazon EC2](), [AWS IAM](), [DynamoDB]() Tags: [amazon](), [amazon-sqs](), [analysis](), [aws](), [cloud](), [data](), [database](), [distributed](), [dynamodb](), [ec2](), [engineering](), [failover](), [hubspot](), [iam](), [improvements](), [incident](), [message-queue](), [outage](), [recovery](), [report](), [resilience]() ### AI overview HubSpot reports that a severe AWS outage in the us-east-1 region on October 20, 2025 disrupted multiple product features and affected third-party vendors. The incident caused failures involving DynamoDB, IAM, SQS, and EC2, and degraded HubSpot's TQ2 background task processing. HubSpot says it completed an analysis and is implementing resilience improvements. ### Source excerpt On October 20, 2025, HubSpot experienced a significant service disruption affecting multiple product features due to a severe AWS outage in the us-east-1 region. While our infrastructure remained intact, the widespread nature of the cloud provider failure impacted both our services and critical third-party vendors we rely on. We've completed a thorough analysis of this incident and are implementing comprehensive improvements to strengthen our resilience against future cloud provider disruptions. ## Durable under pressure: How developers kept running during the AWS us-east-1 outage DevFeed: [Durable under pressure: How developers kept running during the AWS us-east-1 outage]() Original publisher: [Read original article]() Author: Luke Knepper Published: 2025-11-07T00:00:00Z Content type: article Language: en Sources: [Temporal Blog]() Topics: [Amazon Web Services](), [Availability](), [Cloud](), [Disaster Recovery](), [Replication](), [incident](), [DynamoDB]() Tags: [2025](), [availability](), [aws](), [disaster-recovery](), [dynamodb](), [high-availability](), [incident](), [outage](), [temporal-concepts](), [us]() ### AI overview The article examines the October 20, 2025 AWS us-east-1 outage and explains how multi-region high availability and disaster recovery strategies helped some applications continue running. It describes Temporal Cloud's replication and failover features and FireHydrant's preparation and testing. ### Source excerpt The AWS us-east-1 outage affected many, but some devs were able to stay afloat. Find out how. ## Terraform module to manage Oxbow Lambda and its components DevFeed: [Terraform module to manage Oxbow Lambda and its components]() Original publisher: [Read original article]() Author: Oleh Motrunych Published: 2025-03-14T00:00:00Z Content type: release Language: en Sources: [Scribd Tech]() Topics: [Terraform](), [AWS Lambda](), [infrastructure as code (IAC)](), [Amazon Web Services](), [AWS Glue](), [Amazon Simple Queue Service (SQS)](), [data-processing](), [event driven](), [DynamoDB](), [Amazon S3]() Tags: [aws](), [aws-glue](), [aws-lambda](), [aws-s3](), [deltalake](), [dynamodb](), [event-driven](), [iac](), [oxbow](), [rust](), [security](), [sqs](), [terraform]() ### AI overview The article introduces terraform-oxbow, an open-source Terraform module for deploying and managing an Oxbow AWS Lambda workflow and its supporting components. It describes configurable integrations including AWS Glue, Kinesis Data Firehose, SQS, DynamoDB, IAM policies, and S3 notifications, while noting AWS notification limits and least-privilege considerations. ### Source excerpt Oxbow is a project to take an existing storage location which contains Apache Parquet files into a Delta Lake table. It is intended to run both as an AWS Lambda or as a command line application. We are excited to introduce terraform-oxbow, an open-source Terraform module that simplifies the deployment and management of AWS Lambda and its supporting components. Whether you're working with AWS Glue, Kinesis Data Firehose, SQS, or DynamoDB, this module provides a streamlined approach to infrastructure as code (IaC) in AWS. ## How We Run Terraform At Scale DevFeed: [How We Run Terraform At Scale]() Original publisher: [Read original article]() Author: Christian Monaghan Published: 2025-03-04T16:32:39Z Content type: article Language: en Sources: [Benchling]() Topics: [Terraform](), [Amazon Web Services](), [Amazon S3](), [DynamoDB]() Tags: [aws](), [cloud-infrastructure](), [dynamodb](), [infrastructure](), [infrastructure-as-code](), [management](), [operations](), [s3](), [software-engineering](), [terraform]() ### AI overview Benchling describes how it manages more than 165,000 cloud resources across 625 Terraform workspaces and 38 AWS accounts. The article explains that its earlier laptop-based Terraform workflow, using S3 state files and DynamoDB locks, became inefficient as the number of workspaces grew. ### Source excerpt Managing over 165k cloud resources across hundreds of workspaces could seem daunting. But for us, it's just another day at Benchling. Here's how we do it. We currently have: 165k cloud resources under management 625 Terraform workspaces 38 AWS accounts 170 engineers (40 of whom are infra specialists) We perform: 225 infrastructure releases daily (terraform apply operations) 723 plans daily (terraform plan operations) We've been successfully operating Benchling's infrastructure release system for the past two years (spoiler, it's Terraform Cloud), over which time we've doubled our infrastructure footprint with minimal additional release overhead. Before Terraform Cloud: The Chaos Our infra release process wasn't always this smooth. Let me rewind and take you back to how it was before. As is common guidance for small Terraform projects, our team would previously apply all infrastructure changes via laptop. Also in line with common guidance, our team used S3 to store state files, with DynamoDB state locks, which prevented any apply-time collisions. This is a great strategy for a small team working on up to a dozen workspaces. However, this slowly starts to break down as the team's workspace footprint grows. It's like the proverbial frog in the pot of water, slowly heated to a boil. By the time we made the switch, Benchling was managing 350 workspaces. We were approaching the boiling point. Pain Points: Developer Toil and Inefficiency Managing 350 workspaces with this approach had several downsides: Necessitated elevated AWS access permissions for the infrastructure team. It was time-consuming as the engineer had to navigate to each directory, run terraform apply, review and approve the run, then verify it succeeded. Very commonly a single change could affect over 120 workspaces, which would mean repeating this process 120 times. (We had developed a custom python script which helped parallelize this somewhat.) Accumulated infra drift. Often an engineer would go to apply ## The Making of a Scalable URL Shortener DevFeed: [The Making of a Scalable URL Shortener]() Original publisher: [Read original article]() Author: Cyril Mottier Published: 2025-02-18T10:57:54Z Content type: article Language: en Sources: [Cyril Mottier Blog]() Topics: [Code](), [Amazon Web Services](), [DynamoDB](), [Go Language](), [Requirements](), [Database]() Tags: [article](), [aws](), [build](), [distributed-system](), [dynamodb](), [engineering](), [go](), [redirection](), [requirements](), [retry](), [scale]() ### AI overview The article describes the design of a custom URL shortener built with AWS Lambda, AWS DynamoDB, and Go. It outlines requirements such as fast redirection, short URLs, batch creation, and optional expiration, then examines slug-generation strategies including randomized slugs, incremental counters, and permutation of the slug space. ### Source excerpt This article was initially published as a thread of posts on X. For the sake of readability and long-term availability, it is now available here as a single-page article. Post 1 A few months ago, I built a custom URL shortener. Quick refresher: a URL shortener does two things: Creates short URLs that map to long URLs Redirects short URL requests to those long URLs. Designing this was a fantastic engineering challenge. Here's a look at my design process! ## DynamoDB For System Design Interviews DevFeed: [DynamoDB For System Design Interviews]() Original publisher: [Read original article]() Author: Evan King Published: 2024-11-19T21:30:02Z Content type: tutorial Language: en Sources: [Evan King]() Topics: [DynamoDB](), [Amazon Web Services](), [NoSQL](), [Database](), [Transactions](), [Provisioning](), [Application Development](), [configuration](), [Learning](), [vendor lock-in]() Tags: [aws](), [configuration](), [data-storage](), [database](), [dynamodb](), [learning](), [nosql](), [provisioning](), [system-design-interview](), [transactions]() ### AI overview This tutorial explains DynamoDB for system design interviews. It covers DynamoDB's managed operation, scalability, key-value and NoSQL data model, transaction support, lack of open-source internals, and practical considerations including vendor lock-in and whether interviewers permit its use. ### Source excerpt By the Co-founder of www.hellointerview.comIntro DynamoDB is a fully-managed, highly scalable, key-value service provided by AWS. Cool, buzz-words. But what the hell does that mean and why does it matter? Fully-Managed -- This means that AWS takes care of all the operational aspects of the database. The fully-managed nature allows AWS to handle all operational tasks -- hardware provisioning, configuration, patching, and scaling -- freeing developers to concentrate on application development. Highly Scalable -- DynamoDB can handle massive amounts of data and traffic. It automatically scales up or down to adjust to your application's needs, without any downtime or performance degradation. Key-value -- DynamoDB is a NoSQL database, which means it doesn't use the traditional relational database model. Instead, it uses a key-value model that allows for flexible data storage and retrieval. The moral of the story is that DynamoDB is a super easy to use and can scale to support a wide variety of applications. For system design interviews in particular, it has just about everything you'd ever need from a database. It even supports transactions now! Which neutralizes one of the biggest criticisms of DynamoDB in the past. Importantly, DynamoDB is not open-source, so we can't as easily describe its internals like we did with breakdowns of open source technologies like Kafka and Redis. Instead, we'll focus more on you interact with it. In order to look under the hood, we'll rely on the limited information AWS provides via documentation and the DynamoDB Whitepaper. In this deep dive, we'll break down exactly what you need to know about DynamoDB in order to field any question about it in a system design interview. Along the way, you'll also acquire practical learning that you can later apply in your own projects. Let's break it down! Candidates often ask me, "am I even allowed to use DynamoDB in an interview?"The answer is simple, ask your interviewer! Many will say yes, and just expec ## Query DynamoDB tables with SQL DevFeed: [Query DynamoDB tables with SQL]() Original publisher: [Read original article]() Author: Cameron Archer Published: 2024-10-30T00:00:00Z Content type: tutorial Language: en Sources: [Tinybird]() Topics: [DynamoDB](), [SQL](), [NoSQL]() Tags: [dynamodb](), [nosql](), [sql](), [the-data-base]() ### AI overview The article presents querying DynamoDB tables with SQL and says Tinybird makes NoSQL data accessible to people who know SQL, avoiding PartiQL. ### Source excerpt Query DynamoDB with SQL instead of fighting with PartiQL. Tinybird makes your NoSQL data accessible to anyone who knows SQL. ## The Tinybird DynamoDB Connector is now GA DevFeed: [The Tinybird DynamoDB Connector is now GA]() Original publisher: [Read original article]() Author: Cameron Archer Published: 2024-10-28T00:00:00Z Content type: release Language: en Sources: [Tinybird]() Topics: [DynamoDB]() Tags: [dynamodb](), [product-updates](), [production]() ### AI overview Tinybird announces that its DynamoDB Connector is production-ready, claiming sub-second queries across partitions and unlimited indexing. ### Source excerpt The DynamoDB Connector is now production-ready. Sub-second queries across partitions, unlimited indexing. No more workarounds. ## Simple patterns for aggregating on DynamoDB DevFeed: [Simple patterns for aggregating on DynamoDB]() Original publisher: [Read original article]() Author: Cameron Archer Published: 2024-10-15T00:00:00Z Content type: tutorial Language: en Sources: [Tinybird]() Topics: [DynamoDB](), [data]() Tags: [dynamodb](), [patterns](), [scalable-analytics-architecture]() ### AI overview The article presents four approaches for aggregating data in DynamoDB tables, addressing the fact that DynamoDB does not natively support aggregations. ### Source excerpt DynamoDB doesn't natively support aggregations, so here are four different approaches to aggregate data in DynamoDB tables. ## Introducing the Tinybird DynamoDB Connector in public beta DevFeed: [Introducing the Tinybird DynamoDB Connector in public beta]() Original publisher: [Read original article]() Author: Cameron Archer Published: 2024-09-20T00:00:00Z Content type: release Language: en Sources: [Tinybird]() Topics: [DynamoDB]() Tags: [dynamodb](), [partition](), [product-updates]() ### AI overview Tinybird's DynamoDB Connector has entered public beta, enabling cross-partition queries in milliseconds. ### Source excerpt The DynamoDB Connector just hit public beta. Cross-partition queries in milliseconds are now possible. Test it yourself. ## 5 Best Use Cases for DynamoDB in 2026 | Tinybird DevFeed: [5 Best Use Cases for DynamoDB in 2026 | Tinybird]() Original publisher: [Read original article]() Author: Cameron Archer Published: 2024-09-18T00:00:00Z Content type: article Language: en Sources: [Tinybird]() Topics: [DynamoDB]() Tags: [2026](), [dynamodb](), [the-data-base](), [use-cases]() ### AI overview The article examines five use cases for DynamoDB in 2026, including situations where it performs well and where it may struggle. It emphasizes choosing the right tool before migration becomes necessary. ### Source excerpt DynamoDB use cases where it excels and where it struggles. Pick the right tool before you're stuck migrating later. ## Micro Service Events (AWS EventBridge vs. SNS) DevFeed: [Micro Service Events (AWS EventBridge vs. SNS)]() Original publisher: [Read original article]() Author: Kristian Müller Published: 2022-09-16T12:06:51Z Content type: comparison Language: en Sources: [Homegate Engineering Blog - Medium]() Topics: [event driven](), [Messaging](), [Amazon Web Services](), [Amazon Simple Queue Service (SQS)](), [Publish-subscribe pattern](), [Architecture & Design](), [AWS Lambda](), [Amazon API Gateway](), [DynamoDB](), [Amazon S3](), [TypeScript]() Tags: [api](), [api-gateway](), [architecture](), [aws](), [aws-eventbridge](), [aws-sns](), [data](), [data-processing](), [database](), [dynamodb](), [event](), [event-driven](), [eventbridge](), [eventbus](), [events](), [gateway](), [lambda](), [messaging](), [microservice-architecture](), [s3](), [sns](), [sqs](), [typescript]() ### AI overview This article examines Homegate's event-driven microservice architecture on AWS and compares Amazon SNS with EventBridge as messaging infrastructure. It describes how services use SNS, SQS, Lambda, API Gateway, DynamoDB, and S3, and explains that EventBridge has a different focus, so replacing SNS is not necessarily the right question. ### Source excerpt During the last few years, we've been busy rebuilding the Homegate platform in an event driven, micro service architecture based on AWS. One of the most important buildings blocks, or shall we rather say the mortar that connects the building blocks, is the way our services send events to each other. While Amazon offers its notification service SNS since more than 12 years; in 2019 it started a newcomer in the messaging bus field by introducing EventBridge. This was very interesting to us as we're using SNS a lot. Bus on a Bridge (Photo by "Blue Elf" on Wikimedia Commons) As EventBridge was introduced and gained traction we asked ourselves what advantaged we would gain by switching our SNS based service glue to EventBridge. Maybe this turned out to be the wrong question for us. Let's look at our system to see why. Micro Services Connected by SNS Homegate is connecting dozens of services via SNS right now. Those services range from single purpose Lambda functions written in TypeScript to database heavy services using Aurora or DynamoDB and Step Functions sending events to other services. Some of the services are also interfacing external APIs such as Salesforce or SendGrid. Here our Lambda functions can call APIs when needed. On the other hand, we have web services to be called as APIs by other systems and end users. This is mostly achieved via API Gateway. A typical service connects to the system by subscribing to events needed to fulfill its purpose. Those inbound SNS events are then ingested via SQS allowing for orderly data processing and DLQs to handle errors. Outgoing results are pushed back to the system in form of events to a SNS topics. As Marko Savic has pointed out, we're also utilizing S3 to store and replay those events if needed. Basic AWS Lambda function connected via SNS If we would replace the SNS topic by streams on the EventBridge we would initially gain the same functionality by utilizing SQS as well. However, EventBridge has a different focus. Let ## Automate AWS Athena Partitions for Organization CloudTrail Logs and VPC Flow Logs DevFeed: [Automate AWS Athena Partitions for Organization CloudTrail Logs and VPC Flow Logs]() Original publisher: [Read original article]() Author: Aurelien Cerveaux Published: 2022-09-15T06:39:58Z Content type: tutorial Language: en Sources: [betclic-tech - Medium]() Topics: [AWS CloudTrail](), [Amazon Web Services](), [VPC Flow Logs](), [Amazon S3](), [AWS CloudFormation](), [DynamoDB](), [JSON]() Tags: [amazon-s3](), [athena](), [aws](), [aws-cloudtrail](), [cloud](), [cloudformation](), [cloudtrail](), [dynamodb](), [events](), [json](), [vpc-flow-logs]() ### AI overview A tutorial explains how to automate daily AWS Athena partition creation for organization-level CloudTrail and VPC Flow Logs. It describes using Amazon S3, AWS Glue Data Catalog, CloudFormation, SAM, Lambda, DynamoDB, and EventBridge to improve query efficiency and reduce scanned data and cost. ### Source excerpt This article's goal is to explain how to query efficiently, and at low cost with AWS Athena, AWS CloudTrail logs, or VPC Flow logs by automating the creation of Athena partitions daily. AWS CloudTrail AWS CloudTrail is a service that records all API Calls and Events for AWS Accounts, including details on calls made to your AWS Services and the Console as well. Activated at the Organization level, CloudTrail logs help you in several cases: Simplified Compliance Summarizing Accesses Security Analysis and Troubleshooting Monitoring Users and Resources Activities Governance and Investigation CloudTrail Logs are delivered in an S3 Bucket in JSON format through many small files that make these difficult to operate and request. VPC Flow Logs VPC Flow Logs is a feature that allows you to capture traffic information going to and from your network interfaces in your VPC. It helps you to: Analyze and troubleshoot network issues Determine the direction of your traffic Monitor the traffic Like CloudTrail Logs, Flow Logs are published in an S3 Bucket for all accounts in the Organization. AWS Athena AWS Athena is a query service that permits you to analyze data directly in an Amazon S3. You can create tables in Athena which use AWS Glue Data Catalog and are based on an S3 location. To query efficiently and with less scanned data and optimize the cost, we need to create some partitions. To know more about Tables, Databases, and Catalog in Athena. Automate Partitions Creation In our case, we are using CloudTrail Logs and VPC Flow Logs to detail the solution and the source code. All is done in CloudFormation and using SAM to deploy the Lambda. Global Overview We have several parts to automate the creation of our partitions. A common part containing: The Lambda is used to create partitions concerning different log types A DynamoDB table used to store partition names A specific part for each log type contains: The Glue Table with the proper structure An EventBridge Rule to trigger the ## AWS SDK for Kotlin DevFeed: [AWS SDK for Kotlin]() Original publisher: [Read original article]() Author: thiiozz Published: 2022-08-23T07:05:18Z Content type: tutorial Language: en Sources: [betclic-tech - Medium]() Topics: [SDKs](), [Kotlin](), [Amazon Web Services](), [DynamoDB](), [Gradle](), [Open Source](), [Amazon S3](), [Amazon Simple Queue Service (SQS)](), [Java]() Tags: [amazon](), [api](), [async](), [asynchronous](), [aws](), [backend](), [cloud](), [dynamodb](), [gradle](), [java](), [kotlin](), [open-source](), [sdk]() ### AI overview This tutorial introduces the beta AWS SDK for Kotlin, an open-source Apache 2.0 project built from scratch in Kotlin. It demonstrates configuring the SDK with Gradle, creating a DynamoDB client, performing table operations, and using fluent syntax and asynchronous calls compared with the Java SDK. ### Source excerpt During AWS re:Invent 2021, Amazon announced an AWS SDK for Kotlin. At Betclic, we were very enthusiastic about this news as we use the Kotlin language for both Frontend and Backend apps, and we love it. One year later, the first beta releases are available for developers to preview. Let's discover them! Forewords This SDK can be used to develop applications that interact with services such as DynamoDB, S3, SQS, and many more (API reference). Like other SDKs, this new one is an AWS open-source project available under the Apache 2.0 licence. The project is still in beta "developer preview" phase, and it should not be used in production yet. You can follow developments on the public road map. After the first official release, daily releases are planned to make services updates available within the SDK under 24 hours. Interesting point, this new SDK has been re-built from scratch using Kotlin rather than re-using the existing Java SDK. Let's dig into concrete examples around DynamoDB. Hello SDK ! First, you need to add the SDK dependency in your project, for example, with Gradle. https://medium.com/media/9045f8ec2d12ecbc65bba8edb7d8e611/href Each AWS services have its own standalone, independent and light dependency. Then we can initiate our DynamoDB client with https://medium.com/media/e48bff2f4b6a43e17060d0ca67f5d82f/href The default behaviour of the client can be configured in many ways: Credentials provider (Profile, environment, sts, SSO (soon) ...) Logging backed by Slf4j Http engine fully overridable Retry strategy, AWS region, services endpoints and so on With our project and client correctly setup, we can make our first call https://medium.com/media/7c35f36e7c35b52988464e9a59c1d11a/hrefFluent Kotlin Syntax Moving forward, let's assume we have an existing DynamoDB table "sports" with a single attribute "name" We can put a new item with https://medium.com/media/2df0a1bd5cf14c6ad8586bae1afc85d7/href And get it back with https://medium.com/media/8be66a2add009fd6021c8 ## How to Audit SSH Sessions in Amazon EC2 DevFeed: [How to Audit SSH Sessions in Amazon EC2]() Original publisher: [Read original article]() Author: info@goteleport.com (Janakiram MSV) Published: 2022-04-19T00:00:00Z Content type: tutorial Language: en Sources: [Teleport]() Topics: [Amazon EC2](), [ssh](), [Tutorial](), [audit](), [Security](), [Amazon S3](), [DynamoDB](), [OpenSSH]() Tags: [amazon-cloudwatch](), [amazon-dynamodb](), [amazon-s3](), [audit](), [aws](), [bastion](), [bastion-host](), [cloudtrail](), [compliance](), [dynamodb](), [ec2](), [how-to](), [infrastructure](), [review](), [s3](), [security](), [security-events](), [ssh](), [storage](), [tutorial]() ### AI overview A tutorial on configuring Teleport to record and audit SSH sessions on Amazon EC2 Linux instances, including centralized session playback and storing Teleport events and recordings in Amazon DynamoDB and Amazon S3. ### Source excerpt This post is a tutorial to set up security session recording for EC2 SSH access and store logs in AWS S3 and DynamoDB. ## Remediating AWS IMDSv1 DevFeed: [Remediating AWS IMDSv1]() Original publisher: [Read original article]() Published: 2021-08-11T16:16:23Z Content type: tutorial Language: en Sources: [Latacora]() Topics: [Amazon Web Services](), [Amazon EC2](), [AWS IAM](), [Exploit](), [vulnerability](), [Amazon Simple Queue Service (SQS)](), [DynamoDB]() Tags: [aws](), [bugs](), [credentials](), [defense-in-depth](), [dynamodb](), [ec2](), [exploit](), [iam](), [infrastructure](), [sqs](), [technical](), [vulnerability]() ### AI overview This technical article explains why AWS Instance Metadata Service version 1 is risky. Applications vulnerable to SSRF, XXE, or RCE may allow attackers to obtain temporary IAM role credentials from EC2-hosted applications and use them to access AWS services. The article presents defense in depth as a way to limit the impact of application vulnerabilities, although the supplied text ends before the remediation details. ### Source excerpt 2024-12-17 Updated to include Declarative Policies Compute resources in AWS (for example, EC2 instances, ECS tasks/services, etc.) get access to AWS credentials, such as temporary instance role credentials, via the Instance Metadata Service (IMDS). The compute resources use these credentials to access other AWS services such as SQS, DynamoDB and Secrets Manager. Introduction: Problems with IMDSv1 # There was originally only one version of IMDS, now called "v1," which unfortunately many people still use. The technical risks and high profile incidents (the Capital One breach comes to mind) associated with v1, as well as the existence of v2 are well-documented. When an application hosted on an EC2 instance is vulnerable to SSRF, XXE or RCE, attackers can likely steal the temporary AWS credentials of the IAM role configured for the instance. This service is a particularly interesting target for attackers: ## Saving Money and Protecting Privacy With Bloom Filters DevFeed: [Saving Money and Protecting Privacy With Bloom Filters]() Original publisher: [Read original article]() Author: Tristan Garwood Published: 2018-08-27T17:08:17Z Content type: article Language: en Sources: [Localytics]() Topics: [DynamoDB](), [data](), [pii](), [NoSQL](), [Amazon S3](), [Amazon Web Services](), [Back end](), [Caching](), [implementation](), [Latency]() Tags: [aws](), [backend](), [bloom-filter](), [cache](), [data](), [dynamodb](), [false-positive](), [filter](), [gdpr](), [implementation](), [latency](), [low-latency](), [memory](), [native](), [nosql](), [pii](), [privacy](), [s3]() ### AI overview The article explains how Localytics used Bloom filters with DynamoDB, S3, and caching to enforce GDPR opt-out checks in a high-throughput ingestion pipeline. The design reduces unnecessary DynamoDB lookups while preserving verification of possible matches and dropping data from opted-out users. ### Source excerpt The EU General Data Protection Regulation (GDPR) went into effect on May 25th, 2018 and stipulates that companies processing personally identifiable information (PII) must carry out requests from end users to be 'forgotten' from their systems. Implementing this right to be forgotten (RTBF) for our products means that ## Improving Your Data Layer with Rebase on Python DevFeed: [Improving Your Data Layer with Rebase on Python]() Original publisher: [Read original article]() Author: Yuv Joodhisty Software Engineer; Open Source Enthusiast Also Passionate About Startups Published: 2018-08-03T00:00:00Z Content type: tutorial Language: en Sources: [Trivago]() Topics: [data](), [Python](), [Data structures](), [DynamoDB](), [Kafka](), [MySQL](), [Amazon Web Services](), [Cloud]() Tags: [aws](), [backend](), [cloud](), [data](), [data-structures](), [dynamodb](), [kafka](), [mysql](), [open-source](), [python](), [streaming](), [validation]() ### AI overview The article introduces Rebase on Python as a way to simplify data migration and validation when data structures change across technologies. It describes a pipeline that moves data from MySQL through Kafka and Kinesis into DynamoDB, with Lambda functions handling validation and transformation into read replicas. ### Source excerpt When migrating your data to new technologies, validation of the data becomes challenging as your data structures might change. Rebase tries to make this easier while also giving your more flexibility on your data. ## Teleport 2.6 Released DevFeed: [Teleport 2.6 Released]() Original publisher: [Read original article]() Author: ev@goteleport.com (Ev Kontsevoy) Published: 2018-05-31T00:00:00Z Content type: release Language: en Sources: [Teleport]() Topics: [OpenSSH](), [Security](), [Amazon S3](), [DynamoDB](), [Amazon Web Services](), [certificates](), [Authentication](), [audit]() Tags: [audit](), [authentication](), [aws](), [aws-s3](), [certificates](), [dynamodb](), [openssh](), [release](), [security]() ### AI overview Teleport 2.6 is a major release for the privileged SSH access manager. It adds DynamoDB audit-log storage, AWS S3 session-recording storage, certificate authority rotation, and PAM integration. The release source code also underwent a Cure53 security audit, whose report was expected to be published publicly. ### Source excerpt Announcing Teleport v2.6, a major release with important new features whose source code also passed a security audit performed by Cure53. [Next page]()