# ops

Published articles for ops.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Keeping the peering inventory honest

DevFeed: [Keeping the peering inventory honest](<https://devfeed.tech/articles/keeping-the-peering-inventory-honest-36152.md>)

Original publisher: [Read original article](<https://as215248.net/notes/keeping-the-inventory-honest/>)

Author: Bastiaan Brink

Published: 2026-07-30T00:00:00Z

Content type: tutorial

Language: en

Sources: [AS215248 - Notes](<https://devfeed.tech/sources/as215248-notes.md>)

Topics: [Database](<https://devfeed.tech/topics/database.md>), [API](<https://devfeed.tech/topics/api.md>), [Job](<https://devfeed.tech/topics/job.md>), [Python](<https://devfeed.tech/topics/python.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [automation](<https://devfeed.tech/tags/automation.md>), [cloudflare](<https://devfeed.tech/tags/cloudflare.md>), [database](<https://devfeed.tech/tags/database.md>), [ops](<https://devfeed.tech/tags/ops.md>), [peering](<https://devfeed.tech/tags/peering.md>), [python](<https://devfeed.tech/tags/python.md>), [scheduler](<https://devfeed.tech/tags/scheduler.md>)

### AI overview

The article describes keeping Peering Manager's inventory synchronized with router state by mirroring portal changes and running a 15-minute reconciliation job. It also documents API integration pitfalls and fixes a failed daily PeeringDB sync by pinning the service image to a digest.

### Source excerpt

A database that says you have a session is worthless if the router disagrees. A 15-minute reconcile keeps Peering Manager and the routers in step.

## Operational Readiness: The Review That Catches Problems

DevFeed: [Operational Readiness: The Review That Catches Problems](<https://devfeed.tech/articles/operational-readiness-the-review-that-catches-problems-34019.md>)

Original publisher: [Read original article](<https://sridharrajarao.com/blog/operational-readiness/>)

Author: Sridhar Rajarao

Published: 2026-05-25T00:00:00Z

Content type: tutorial

Language: en

Sources: [Sridhar Rajarao](<https://devfeed.tech/sources/sridhar-rajarao.md>)

Topics: [Development](<https://devfeed.tech/topics/development.md>), [incident](<https://devfeed.tech/topics/incident.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [on-call](<https://devfeed.tech/tags/on-call.md>), [operational](<https://devfeed.tech/tags/operational.md>), [ops](<https://devfeed.tech/tags/ops.md>), [problems](<https://devfeed.tech/tags/problems.md>), [production](<https://devfeed.tech/tags/production.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [sre](<https://devfeed.tech/tags/sre.md>), [validation](<https://devfeed.tech/tags/validation.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

An operational readiness review is useful only when checklist items are backed by evidence and verified before a service launches. The article explains common failure modes, recommends attaching artifacts to each item, and describes AI applications for evidence gathering, gap detection, and summary drafting while keeping the final readiness decision with human reviewers.

### Source excerpt

A short, verifiable checklist for production launches. What to ask, why ORRs become theater, and where AI helps.

## ClickHouse Monitoring and Observability Decision Points

DevFeed: [ClickHouse Monitoring and Observability Decision Points](<https://devfeed.tech/articles/clickhouse-monitoring-and-observability-decision-points-19111.md>)

Original publisher: [Read original article](<https://severalnines.com/blog/clickhouse-monitoring-and-observability-decision-points/>)

Author: Paul Namuag

Published: 2026-04-17T07:00:00Z

Content type: tutorial

Language: en

Sources: [SeveralNines](<https://devfeed.tech/sources/severalnines.md>)

Topics: [clickhouse](<https://devfeed.tech/topics/clickhouse.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [observability](<https://devfeed.tech/topics/observability.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [Availability](<https://devfeed.tech/topics/availability.md>), [Replication](<https://devfeed.tech/topics/replication.md>), [Latency](<https://devfeed.tech/topics/latency.md>)

Tags: [capacity](<https://devfeed.tech/tags/capacity.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [high-availability](<https://devfeed.tech/tags/high-availability.md>), [latency](<https://devfeed.tech/tags/latency.md>), [logs](<https://devfeed.tech/tags/logs.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [monitoring-alerting](<https://devfeed.tech/tags/monitoring-alerting.md>), [observability](<https://devfeed.tech/tags/observability.md>), [operations](<https://devfeed.tech/tags/operations.md>), [ops](<https://devfeed.tech/tags/ops.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [partitioning](<https://devfeed.tech/tags/partitioning.md>), [performance](<https://devfeed.tech/tags/performance.md>), [replication](<https://devfeed.tech/tags/replication.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [scale](<https://devfeed.tech/tags/scale.md>), [traces](<https://devfeed.tech/tags/traces.md>)

### AI overview

This article explains why ClickHouse deployments need monitoring and full observability because distributed clusters, replicas, scalability, and real-time analytics create operational complexity. It outlines using metrics, logs, traces, and alerting to detect bottlenecks, optimize queries, track resource use, monitor pipeline health, plan capacity, troubleshoot distributed queries, and support availability.

### Source excerpt

Given ClickHouse's ability to execute complex analytical queries across terabytes of data in a single operation, proper monitoring and observability is critical. Its distributed architecture and scalability add layers of complexity, as multi-node clusters require careful coordination monitoring across shards and replicas to ensure data consistency and availability. Adding to the operational pressure is users' [...] The post ClickHouse Monitoring and Observability Decision Points appeared first on Severalnines.

## How Dispatch speeds up development with Neon while keeping workloads on Aurora

DevFeed: [How Dispatch speeds up development with Neon while keeping workloads on Aurora](<https://devfeed.tech/articles/how-dispatch-speeds-up-development-with-neon-while-keeping-workloads-on-aurora-5348.md>)

Original publisher: [Read original article](<https://neon.com/blog/how-dispatch-speeds-up-development-with-neon-while-keeping-workloads-on-aurora>)

Author: Carlota Soto

Published: 2024-10-14T15:31:55Z

Content type: article

Language: en

Sources: [Blog -- Neon Docs](<https://devfeed.tech/sources/blog-neon-docs.md>)

Topics: [Development](<https://devfeed.tech/topics/development.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Architecture & Design](<https://devfeed.tech/topics/architecture-design.md>), [Microservice](<https://devfeed.tech/topics/microservice.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [sdlc](<https://devfeed.tech/topics/sdlc.md>), [Testing](<https://devfeed.tech/topics/testing.md>), [Availability](<https://devfeed.tech/topics/availability.md>), [Database](<https://devfeed.tech/topics/database.md>)

Tags: [agile](<https://devfeed.tech/tags/agile.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [availability](<https://devfeed.tech/tags/availability.md>), [aws](<https://devfeed.tech/tags/aws.md>), [backups](<https://devfeed.tech/tags/backups.md>), [case-studies](<https://devfeed.tech/tags/case-studies.md>), [cost](<https://devfeed.tech/tags/cost.md>), [development](<https://devfeed.tech/tags/development.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [microservices-architecture](<https://devfeed.tech/tags/microservices-architecture.md>), [ops](<https://devfeed.tech/tags/ops.md>), [pii](<https://devfeed.tech/tags/pii.md>), [principal-engineer](<https://devfeed.tech/tags/principal-engineer.md>), [sdlc](<https://devfeed.tech/tags/sdlc.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [team](<https://devfeed.tech/tags/team.md>), [us](<https://devfeed.tech/tags/us.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

The article describes how Dispatch uses Neon branching to create isolated development and verification environments without copying large volumes of data. Dispatch is transitioning from a monolithic system to a microservices architecture, using replicated workload data with personally identifiable information excluded for testing. Neon's serverless writer endpoints are intended to address Aurora's single-writer bottleneck and reduce overprovisioning costs during spiky traffic, while Neon branches accelerate the software development lifecycle.

### Source excerpt

"Neon's branching paradigm has been great for us. It lets us create isolated environments without having to move huge amounts of data around. This has lightened the load on our ops team, now it's effortless to spin up entire environments."Jonathan Reyes, Principal Engineer at Dis...

## Choosing Good SLIs

DevFeed: [Choosing Good SLIs](<https://devfeed.tech/articles/choosing-good-slis-22991.md>)

Original publisher: [Read original article](<https://bravenewgeek.com/choosing-good-slis/>)

Published: 2024-02-19T21:11:17Z

Content type: opinion

Language: en

Sources: [Brave New Geek](<https://devfeed.tech/sources/brave-new-geek.md>)

Topics: [Cloud](<https://devfeed.tech/topics/cloud.md>), [monitor](<https://devfeed.tech/topics/monitor.md>), [on-prem](<https://devfeed.tech/topics/on-prem.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [systems](<https://devfeed.tech/topics/systems.md>)

Tags: [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [devops](<https://devfeed.tech/tags/devops.md>), [garbage-collection](<https://devfeed.tech/tags/garbage-collection.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [latency](<https://devfeed.tech/tags/latency.md>), [memory-leak](<https://devfeed.tech/tags/memory-leak.md>), [monitor](<https://devfeed.tech/tags/monitor.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [on-prem](<https://devfeed.tech/tags/on-prem.md>), [ops](<https://devfeed.tech/tags/ops.md>), [service-level-agreements](<https://devfeed.tech/tags/service-level-agreements.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [sre](<https://devfeed.tech/tags/sre.md>)

### AI overview

The article explains why organizations moving from on-premises environments to the cloud should reevaluate their Service Level Indicators (SLIs). It argues that resource-focused metrics and operational patterns from data centers may be less useful in cloud environments, where autoscaling, autohealing, ephemeral instances, and different infrastructure characteristics can obscure application issues.

### Source excerpt

Transitioning from an on-prem environment to a cloud environment involves a lot of major shifts for organizations. One of those shifts is often around how we monitor the overall health of systems. The typical way to measure things like the availability, reliability, and performance of systems is with SLIs or Service Level Indicators. SLIs are a valuable tool both on-prem and in the cloud, but when it comes to the latter, I often see organizations carrying over some operational anti-patterns from their data center environment.

## Cloud without Kubernetes

DevFeed: [Cloud without Kubernetes](<https://devfeed.tech/articles/cloud-without-kubernetes-22992.md>)

Original publisher: [Read original article](<https://bravenewgeek.com/cloud-without-kubernetes/>)

Author: Drew

Published: 2024-02-12T18:58:13Z

Content type: opinion

Language: en

Sources: [Brave New Geek](<https://devfeed.tech/sources/brave-new-geek.md>)

Topics: [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [migration](<https://devfeed.tech/topics/migration.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>)

Tags: [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-native](<https://devfeed.tech/tags/cloud-native.md>), [cloud-run](<https://devfeed.tech/tags/cloud-run.md>), [devops](<https://devfeed.tech/tags/devops.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [infrastructure-engineering](<https://devfeed.tech/tags/infrastructure-engineering.md>), [internal-developer-platform](<https://devfeed.tech/tags/internal-developer-platform.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [migrations](<https://devfeed.tech/tags/migrations.md>), [ops](<https://devfeed.tech/tags/ops.md>), [platform-engineering](<https://devfeed.tech/tags/platform-engineering.md>), [product-development](<https://devfeed.tech/tags/product-development.md>), [serverless](<https://devfeed.tech/tags/serverless.md>)

### AI overview

This opinion article questions whether Kubernetes is the right abstraction for every cloud workload. It describes organizations choosing serverless approaches instead, including a customer migrating an e-commerce platform to Google Cloud without Kubernetes, while noting concerns about workload fit, re-architecting, cost, and efficiency.

### Source excerpt

I think it's safe to say Kubernetes has "won" the cloud mindshare game. If you look at the CNCF Cloud Native landscape (and manage to not go cross eyed), it seems like most of the projects are somehow related to Kubernetes. KubeCon is one of the fastest-growing industry events. Companies we talk to at Real Kinetic who are either preparing for or currently executing migrations to the cloud are centering their strategies around Kubernetes. Those already in the cloud are investing heavily in platform-izing their Kubernetes environment. Kubernetes competitors like Nomad, Pivotal Cloud Foundry, OpenShift, and Rancher have sort of just faded to the background (or simply pivoted to Kubernetes). In many ways, "cloud native" seems to be equated with "Kubernetes".

## Introducing Serverless Console

DevFeed: [Introducing Serverless Console](<https://devfeed.tech/articles/introducing-serverless-console-14251.md>)

Original publisher: [Read original article](<https://www.serverless.com/blog/introducing-serverless-console>)

Author: Adam Hevenor

Published: 2021-11-04T00:00:00Z

Content type: release

Language: en

Sources: [Serverless Blog](<https://devfeed.tech/sources/serverless-blog.md>)

Topics: [Serverless](<https://devfeed.tech/topics/serverless.md>)

Tags: [apps](<https://devfeed.tech/tags/apps.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-lambda](<https://devfeed.tech/tags/aws-lambda.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [faas](<https://devfeed.tech/tags/faas.md>), [function-as-a-service](<https://devfeed.tech/tags/function-as-a-service.md>), [infrastructure-as-code](<https://devfeed.tech/tags/infrastructure-as-code.md>), [news](<https://devfeed.tech/tags/news.md>), [ops](<https://devfeed.tech/tags/ops.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [serverless-architecture](<https://devfeed.tech/tags/serverless-architecture.md>), [serverless-framework](<https://devfeed.tech/tags/serverless-framework.md>), [teams](<https://devfeed.tech/tags/teams.md>)

### AI overview

An introduction to the Serverless Ops Console, described as being designed for scaling teams and apps.

### Source excerpt

The Serverless Ops Console made for scaling teams and apps

## SRE Doesn't Scale

DevFeed: [SRE Doesn't Scale](<https://devfeed.tech/articles/sre-doesn-t-scale-23004.md>)

Original publisher: [Read original article](<https://bravenewgeek.com/sre-doesnt-scale/>)

Author: Matthew Thompson

Published: 2021-10-06T15:44:55Z

Content type: opinion

Language: en

Sources: [Brave New Geek](<https://devfeed.tech/sources/brave-new-geek.md>)

Topics: [site-reliability-engineering](<https://devfeed.tech/topics/site-reliability-engineering.md>), [Microservices](<https://devfeed.tech/topics/microservices.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Google](<https://devfeed.tech/topics/google.md>)

Tags: [cloud](<https://devfeed.tech/tags/cloud.md>), [culture](<https://devfeed.tech/tags/culture.md>), [devops](<https://devfeed.tech/tags/devops.md>), [engineering-culture](<https://devfeed.tech/tags/engineering-culture.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [on-prem](<https://devfeed.tech/tags/on-prem.md>), [operations](<https://devfeed.tech/tags/operations.md>), [ops](<https://devfeed.tech/tags/ops.md>), [product-development](<https://devfeed.tech/tags/product-development.md>), [sre](<https://devfeed.tech/tags/sre.md>)

### AI overview

The article discusses challenges in scaling the SRE model, arguing that the model can be resource-intensive and that adopting microservices and cloud infrastructure increases organizational and technology complexity. It references Google's experience and the evolving SRE engagement model.

### Source excerpt

We encounter a lot of organizations talking about or attempting to implement SRE as part of our consulting at Real Kinetic. We've even discussed and debated ourselves, ad nauseam, how we can apply it at our own product company, Witful. There's a brief, unassuming section in the SRE book tucked away towards the tail end of chapter 32, "The Evolving SRE Engagement Model." Between the SLIs and SLOs, the error budgets, alerting, and strategies for handling change management, it's probably one of the most overlooked parts of the book. It's also, in my opinion, one of the most important.

## Structuring a Cloud Infrastructure Organization

DevFeed: [Structuring a Cloud Infrastructure Organization](<https://devfeed.tech/articles/structuring-a-cloud-infrastructure-organization-23005.md>)

Original publisher: [Read original article](<https://bravenewgeek.com/structuring-a-cloud-infrastructure-organization/>)

Author: Neville Poole

Published: 2020-12-07T17:21:46Z

Content type: article

Language: en

Sources: [Brave New Geek](<https://devfeed.tech/sources/brave-new-geek.md>)

Topics: [Cloud](<https://devfeed.tech/topics/cloud.md>), [cloud-infrastructure](<https://devfeed.tech/topics/cloud-infrastructure.md>), [DevOps](<https://devfeed.tech/topics/devops.md>), [on-prem](<https://devfeed.tech/topics/on-prem.md>), [Development](<https://devfeed.tech/topics/development.md>)

Tags: [cloud](<https://devfeed.tech/tags/cloud.md>), [culture](<https://devfeed.tech/tags/culture.md>), [developer-enablement](<https://devfeed.tech/tags/developer-enablement.md>), [development](<https://devfeed.tech/tags/development.md>), [devops](<https://devfeed.tech/tags/devops.md>), [engineering-culture](<https://devfeed.tech/tags/engineering-culture.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [infrastructure-engineering](<https://devfeed.tech/tags/infrastructure-engineering.md>), [on-prem](<https://devfeed.tech/tags/on-prem.md>), [ops](<https://devfeed.tech/tags/ops.md>), [product-development](<https://devfeed.tech/tags/product-development.md>), [product-mindset](<https://devfeed.tech/tags/product-mindset.md>)

### AI overview

The article discusses how companies moving from conventional on-premises IT organizations to cloud environments can structure their engineering organizations. It presents product development and infrastructure as distinct areas and notes that DevOps and cloud adoption do not eliminate their different roles.

### Source excerpt

Real Kinetic often works with companies just beginning their cloud journey. Many come from a conventional on-prem IT organization, which typically looks like separate development and IT operations groups. One of the main challenges we help these clients with is how to structure their engineering organizations effectively as they make this transition. While we approach this problem holistically, it can generally be looked at as two components: product development and infrastructure. One might wonder if this is still the case with the shift to DevOps and cloud, but as we'll see, these two groups still play important and distinct roles.

## Serverless Ops 102 - CloudWatch Logs and Centralized Logging with AWS Lambda

DevFeed: [Serverless Ops 102 - CloudWatch Logs and Centralized Logging with AWS Lambda](<https://devfeed.tech/articles/serverless-ops-102-cloudwatch-logs-and-centralized-logging-with-aws-lambda-14383.md>)

Original publisher: [Read original article](<https://www.serverless.com/blog/serverless-ops-logs>)

Author: Alex DeBrie

Published: 2017-09-08T00:00:00Z

Content type: tutorial

Language: en

Sources: [Serverless Blog](<https://devfeed.tech/sources/serverless-blog.md>)

Topics: [Amazon CloudWatch Logs](<https://devfeed.tech/topics/amazon-cloudwatch-logs.md>), [AWS Lambda](<https://devfeed.tech/topics/aws-lambda.md>), [log management](<https://devfeed.tech/topics/log-management.md>), [Logging](<https://devfeed.tech/topics/logging.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [debug](<https://devfeed.tech/topics/debug.md>), [monitor](<https://devfeed.tech/topics/monitor.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-lambda](<https://devfeed.tech/tags/aws-lambda.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [debug](<https://devfeed.tech/tags/debug.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [faas](<https://devfeed.tech/tags/faas.md>), [function-as-a-service](<https://devfeed.tech/tags/function-as-a-service.md>), [infrastructure-as-code](<https://devfeed.tech/tags/infrastructure-as-code.md>), [lambda](<https://devfeed.tech/tags/lambda.md>), [logging](<https://devfeed.tech/tags/logging.md>), [logs](<https://devfeed.tech/tags/logs.md>), [monitor](<https://devfeed.tech/tags/monitor.md>), [ops](<https://devfeed.tech/tags/ops.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [serverless-architecture](<https://devfeed.tech/tags/serverless-architecture.md>), [serverless-framework](<https://devfeed.tech/tags/serverless-framework.md>)

### AI overview

A developer article about monitoring and debugging serverless AWS Lambda functions using CloudWatch Logs and centralized logging.

### Source excerpt

Monitor and Debug your Serverless Lambda functions with CloudWatch and centralized logging.

## Serverless Ops 101 - Using CloudWatch Metrics & Alarms with Serverless Functions

DevFeed: [Serverless Ops 101 - Using CloudWatch Metrics & Alarms with Serverless Functions](<https://devfeed.tech/articles/serverless-ops-101-using-cloudwatch-metrics-alarms-with-serverless-functions-14384.md>)

Original publisher: [Read original article](<https://www.serverless.com/blog/serverless-ops-metrics>)

Author: Alex DeBrie

Published: 2017-08-11T00:00:00Z

Content type: tutorial

Language: en

Sources: [Serverless Blog](<https://devfeed.tech/sources/serverless-blog.md>)

Topics: [Serverless](<https://devfeed.tech/topics/serverless.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-lambda](<https://devfeed.tech/tags/aws-lambda.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [faas](<https://devfeed.tech/tags/faas.md>), [function-as-a-service](<https://devfeed.tech/tags/function-as-a-service.md>), [infrastructure-as-code](<https://devfeed.tech/tags/infrastructure-as-code.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [ops](<https://devfeed.tech/tags/ops.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [serverless-architecture](<https://devfeed.tech/tags/serverless-architecture.md>), [serverless-framework](<https://devfeed.tech/tags/serverless-framework.md>)

### AI overview

A walkthrough of using CloudWatch metrics and alarms to operate serverless functions.

### Source excerpt

Level up your serverless ops game with a walkthrough on CloudWatch metrics and alarms

## Reducing DynamoDB Storage Costs for High-Volume Datapoint Deduplication

DevFeed: [Reducing DynamoDB Storage Costs for High-Volume Datapoint Deduplication](<https://devfeed.tech/articles/tweaking-dynamodb-tables-for-fun-and-profit-28635.md>)

Original publisher: [Read original article](<https://eng.localytics.com/tweaking-dynamodb-tables/>)

Author: Joanna Solmon

Published: 2017-01-04T14:45:00Z

Content type: tutorial

Language: en

Sources: [Localytics](<https://devfeed.tech/sources/localytics.md>)

Topics: [DynamoDB](<https://devfeed.tech/topics/dynamodb.md>), [amazon](<https://devfeed.tech/topics/amazon.md>), [Database](<https://devfeed.tech/topics/database.md>), [Mobile](<https://devfeed.tech/topics/mobile.md>)

Tags: [amazon](<https://devfeed.tech/tags/amazon.md>), [aws](<https://devfeed.tech/tags/aws.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [dedupe](<https://devfeed.tech/tags/dedupe.md>), [devices](<https://devfeed.tech/tags/devices.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [dynamodb](<https://devfeed.tech/tags/dynamodb.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [ops](<https://devfeed.tech/tags/ops.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

The article describes how Localytics used DynamoDB to deduplicate billions of daily datapoints and examined schema changes to reduce storage costs. It discusses duplicate UUIDs, a 30-day deduplication history, and the limitations of a naive schema.

### Source excerpt

Amazon's DynamoDB is a great technology: it's easy to set up and provides all the benefits of an infinitely scalable distributed database without the operational overhead. Unfortunately, DynamoDB can also be very expensive for high volume, high throughput storage. This post will show how we were

## Production metrics Bitly monitors beyond standard checks

DevFeed: [Production metrics Bitly monitors beyond standard checks](<https://devfeed.tech/articles/10-things-we-forgot-to-monitor-19709.md>)

Original publisher: [Read original article](<https://word.bitly.com/post/74839060954>)

Author: Wordbitly

Published: 2014-01-28T16:11:25Z

Content type: article

Language: en

Sources: [Bitly](<https://devfeed.tech/sources/bitly.md>)

Topics: [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [DevOps](<https://devfeed.tech/topics/devops.md>), [systems](<https://devfeed.tech/topics/systems.md>), [cURL](<https://devfeed.tech/topics/curl.md>), [Network Configuration](<https://devfeed.tech/topics/network-configuration.md>)

Tags: [curl](<https://devfeed.tech/tags/curl.md>), [devops](<https://devfeed.tech/tags/devops.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [network-configuration](<https://devfeed.tech/tags/network-configuration.md>), [ops](<https://devfeed.tech/tags/ops.md>), [process](<https://devfeed.tech/tags/process.md>), [systems](<https://devfeed.tech/tags/systems.md>)

### AI overview

This Bitly article describes production-monitoring lessons beyond standard disk, memory, load, and ping metrics. The supplied text discusses fork rate, network flow-control packets, swap-in/out rate, server boot notifications, and NTP clock offset, including incidents involving IPv6 configuration, curl, and dropped traffic.

### Source excerpt

There is always a set of standard metrics that are universally monitored (Disk Usage, Memory Usage, Load, Pings, etc). Beyond that, there are a lot of lessons that we've learned from operating our production systems that have helped shape the breadth of monitoring that we perform at bitly. One of my favorite all-time tweets is from @DevOps_Borat "Law of Murphy for devops: if thing can able go wrong, is mean is already wrong but you not have Nagios alert of it yet." What follows is a small list of things we monitor at bitly that have grown out of those (sometimes painful!) experiences, and where possible little snippets of the stories behind those instances. 1 - Fork Rate We once had a problem where IPv6 was intentionally disabled on a box via options ipv6 disable=1 and alias ipv6 off in /etc/modprobe.conf. This caused a large issue for us: each time a new curl object was created, modprobe would spawn, checking net-pf-10 to evaluate IPv6 status. This fork bombed the box, and we eventually tracked it down by noticing that the process counter in /proc/stat was increasing by several hundred a second. Normally you would only expect a fork rate of 1-10/sec on a production box with steady traffic. check_fork_rate.sh 2 - flow control packets TL;DR; If your network configuration honors flow control packets and isn't configured to disable them, they can temporarily cause dropped traffic. (If this doesn't sound like an outage, you need your head checked.) $ /usr/sbin/ethtool -S eth0 | grep flow_control rx_flow_control_xon: 0 rx_flow_control_xoff: 0 tx_flow_control_xon: 0 tx_flow_control_xoff: 0 Note: Read this to understand how these flow control frames can cascade to switch-wide loss of connectivity if you use certain Broadcom NIC's. You should also trend these metrics on your switch gear. While at it, watch your dropped frames. 3 - Swap In/Out Rate It's common to check for swap usage above a threshold, but even if you have a small quantity of memory swapped, it's actually th