# queues

Published articles for queues.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Logarithmic auto-scaling for Laravel Horizon

DevFeed: [Logarithmic auto-scaling for Laravel Horizon](<https://devfeed.tech/articles/logarithmic-auto-scaling-for-laravel-horizon-33304.md>)

Original publisher: [Read original article](<https://freek.dev/3191-logarithmic-auto-scaling-for-laravel-horizon>)

Author: Freek Van der Herten (freek@spatie.be)

Published: 2026-09-12T12:30:30Z

Content type: article

Language: en

Sources: [freek.dev - all blogposts](<https://devfeed.tech/sources/freek-dev-all-blogposts.md>)

Topics: [Laravel](<https://devfeed.tech/topics/laravel.md>), [realtime](<https://devfeed.tech/topics/realtime.md>), [Development](<https://devfeed.tech/topics/development.md>)

Tags: [horizon](<https://devfeed.tech/tags/horizon.md>), [laravel](<https://devfeed.tech/tags/laravel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [queues](<https://devfeed.tech/tags/queues.md>), [realtime](<https://devfeed.tech/tags/realtime.md>), [scaling](<https://devfeed.tech/tags/scaling.md>)

### AI overview

This article explains how logarithmic auto-scaling can be added to Laravel Horizon. Log-based weighting helps prevent large queue spikes from starving smaller realtime queues while still allocating enough workers to large backlogs.

### Source excerpt

A thoughtful write-up on adding logarithmic auto-scaling to Laravel Horizon. It shows how log-based weighting keeps huge queue spikes from starving smaller realtime queues while still giving large backlogs enough workers. Read more

## Laravel Queue Gotchas Collected from Experience

DevFeed: [Laravel Queue Gotchas Collected from Experience](<https://devfeed.tech/articles/a-series-of-unfortunate-jobs-33302.md>)

Original publisher: [Read original article](<https://freek.dev/3189-a-series-of-unfortunate-jobs>)

Author: Freek Van der Herten (freek@spatie.be)

Published: 2026-09-10T12:06:27Z

Content type: tutorial

Language: en

Sources: [freek.dev - all blogposts](<https://devfeed.tech/sources/freek-dev-all-blogposts.md>)

Topics: [Laravel](<https://devfeed.tech/topics/laravel.md>), [gotchas](<https://devfeed.tech/topics/gotchas.md>)

Tags: [best-practices](<https://devfeed.tech/tags/best-practices.md>), [gotcha](<https://devfeed.tech/tags/gotcha.md>), [gotchas](<https://devfeed.tech/tags/gotchas.md>), [laravel](<https://devfeed.tech/tags/laravel.md>), [practical](<https://devfeed.tech/tags/practical.md>), [queues](<https://devfeed.tech/tags/queues.md>)

### AI overview

An article about gotchas in Laravel queues that the author says they collected through experience.

### Source excerpt

Laravel queues are great out of the box, until they bite. Over the years I've collected a few gotchas the hard way. Let me save you the headache. Read more

## ADD COLUMN is not always free, and the lock queue is what takes you down

DevFeed: [ADD COLUMN is not always free, and the lock queue is what takes you down](<https://devfeed.tech/articles/add-column-is-not-always-free-and-the-lock-queue-is-what-takes-you-down-39595.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/43-add-column-is-not-always-free/>)

Author: hello@ankit-rana.com

Published: 2026-08-26T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>), [incident](<https://devfeed.tech/topics/incident.md>), [Query (disambiguation)](<https://devfeed.tech/topics/query.md>), [migration](<https://devfeed.tech/topics/migration.md>), [MySQL](<https://devfeed.tech/topics/mysql.md>)

Tags: [ddl](<https://devfeed.tech/tags/ddl.md>), [incident](<https://devfeed.tech/tags/incident.md>), [locking](<https://devfeed.tech/tags/locking.md>), [migration](<https://devfeed.tech/tags/migration.md>), [migrations](<https://devfeed.tech/tags/migrations.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [query](<https://devfeed.tech/tags/query.md>), [queues](<https://devfeed.tech/tags/queues.md>), [schema-design](<https://devfeed.tech/tags/schema-design.md>), [timeout](<https://devfeed.tech/tags/timeout.md>)

### AI overview

Adding a nullable column can cause an outage even when the schema change itself is nearly instantaneous. In PostgreSQL, ALTER TABLE may wait for a long-running query, and subsequent queries can queue behind the waiting lock request. Setting a short lock_timeout and retrying can prevent the migration from taking down the table.

### Source excerpt

Modern PostgreSQL adds a column with a default as a metadata change, so the table is never rewritten and the migration itself is instant. The outage comes from lock acquisition instead: ALTER TABLE needs an ACCESS EXCLUSIVE lock, and while it waits behind one long-running query, every subsequent query queues behind the waiting ALTER because the lock queue is ordered. Setting lock_timeout to a couple of seconds and retrying converts that from an outage into a no-op.

## Kafka vs RabbitMQ vs SQS

DevFeed: [Kafka vs RabbitMQ vs SQS](<https://devfeed.tech/articles/kafka-vs-rabbitmq-vs-sqs-33572.md>)

Original publisher: [Read original article](<https://blog.algomaster.io/p/kafka-vs-rabbitmq-vs-sqs>)

Author: Ashish Pratap Singh

Published: 2026-07-02T03:31:11Z

Content type: comparison

Language: en

Sources: [AlgoMaster Newsletter](<https://devfeed.tech/sources/algomaster-newsletter.md>)

Topics: [Amazon Simple Queue Service (SQS)](<https://devfeed.tech/topics/amazon-simple-queue-service-sqs.md>), [Kafka](<https://devfeed.tech/topics/kafka.md>), [RabbitMQ](<https://devfeed.tech/topics/rabbitmq.md>), [Messaging](<https://devfeed.tech/topics/messaging.md>), [Back end](<https://devfeed.tech/topics/backend.md>)

Tags: [backend](<https://devfeed.tech/tags/backend.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [messaging](<https://devfeed.tech/tags/messaging.md>), [queues](<https://devfeed.tech/tags/queues.md>), [rabbitmq](<https://devfeed.tech/tags/rabbitmq.md>), [sqs](<https://devfeed.tech/tags/sqs.md>)

### AI overview

This comparison examines Kafka, RabbitMQ, and Amazon SQS for asynchronous systems. It explains that Kafka uses an append-only log with consumer offsets and replay, RabbitMQ routes messages through exchanges and queues with acknowledgments, and SQS provides managed queue-based messaging that consumers poll.

### Source excerpt

Most backend systems eventually need one service to hand off work to another asynchronously.

## Rate-limit downstream APIs with separate Task Queues

DevFeed: [Rate-limit downstream APIs with separate Task Queues](<https://devfeed.tech/articles/rate-limit-downstream-apis-with-separate-task-queues-35944.md>)

Original publisher: [Read original article](<https://temporal.io/blog/rate-limit-downstream-apis>)

Author: Cecil Phillip

Published: 2026-05-25T00:00:00Z

Content type: tutorial

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>), [API](<https://devfeed.tech/topics/api.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [configuration](<https://devfeed.tech/topics/configuration.md>)

Tags: [apis](<https://devfeed.tech/tags/apis.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [queues](<https://devfeed.tech/tags/queues.md>), [rate-limiting](<https://devfeed.tech/tags/rate-limiting.md>), [service](<https://devfeed.tech/tags/service.md>), [temporal-concepts](<https://devfeed.tech/tags/temporal-concepts.md>)

### AI overview

This tutorial explains how to use separate Temporal Task Queues with global and optional per-Worker rate limits, plus concurrency limits, to keep Activities within downstream API quotas and help prevent 429 errors and account issues.

### Source excerpt

Stop hitting 429s on downstream APIs. Use rate-limited Temporal Task Queues to keep Activities under SendGrid, Stripe, and OpenAI quotas.

## Task Queue Priority and Fairness: Your Task Queue, your way

DevFeed: [Task Queue Priority and Fairness: Your Task Queue, your way](<https://devfeed.tech/articles/task-queue-priority-and-fairness-your-task-queue-your-way-35995.md>)

Original publisher: [Read original article](<https://temporal.io/blog/task-queue-priority-and-fairness-your-task-queue-your-way>)

Author: Conna Lanzafane

Published: 2026-05-12T00:00:00Z

Content type: release

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [SDKs](<https://devfeed.tech/topics/sdks.md>)

Tags: [fairness](<https://devfeed.tech/tags/fairness.md>), [priority](<https://devfeed.tech/tags/priority.md>), [product-news](<https://devfeed.tech/tags/product-news.md>), [queue](<https://devfeed.tech/tags/queue.md>), [queues](<https://devfeed.tech/tags/queues.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [sdks](<https://devfeed.tech/tags/sdks.md>)

### AI overview

Temporal Task Queue Priority and Fairness is generally available for all SDKs. Priority controls task execution order, while Fairness prevents a single Workflow, user, or tenant from monopolizing Workers.

### Source excerpt

Task Queue Priority and Fairness are GA for all Temporal SDKs, helping teams prioritize critical work and prevent tenant starvation without custom infra.

## Route specialized workloads

DevFeed: [Route specialized workloads](<https://devfeed.tech/articles/route-specialized-workloads-35965.md>)

Original publisher: [Read original article](<https://temporal.io/blog/route-specialized-workloads>)

Author: Cecil Phillip

Published: 2026-05-11T00:00:00Z

Content type: tutorial

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [GPU](<https://devfeed.tech/topics/gpu.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [cpu](<https://devfeed.tech/topics/cpu.md>), [data analytics](<https://devfeed.tech/topics/data-analytics.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>)

Tags: [analytics](<https://devfeed.tech/tags/analytics.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [ml](<https://devfeed.tech/tags/ml.md>), [queues](<https://devfeed.tech/tags/queues.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [separate](<https://devfeed.tech/tags/separate.md>), [temporal-concepts](<https://devfeed.tech/tags/temporal-concepts.md>), [worker](<https://devfeed.tech/tags/worker.md>)

### AI overview

The article describes a Temporal Validated Pattern for routing Activities with different resource requirements to separate Task Queues. Dedicated Worker pools can handle GPU-intensive machine learning, CPU, high-memory analytics, and specialized hardware workloads independently.

### Source excerpt

Route ML, analytics, and CPU Activities to separate Temporal Task Queues. Cut GPU costs by 60-80% while scaling each Worker pool independently.

## Ensure Activity execution on the same Worker

DevFeed: [Ensure Activity execution on the same Worker](<https://devfeed.tech/articles/ensure-activity-execution-on-the-same-worker-35996.md>)

Original publisher: [Read original article](<https://temporal.io/blog/task-queue-worker-affinity>)

Author: Cecil Phillip

Published: 2026-05-11T00:00:00Z

Content type: tutorial

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [execution](<https://devfeed.tech/topics/execution.md>), [Workers](<https://devfeed.tech/topics/workers.md>), [file](<https://devfeed.tech/topics/file.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [connection-pooling](<https://devfeed.tech/tags/connection-pooling.md>), [data](<https://devfeed.tech/tags/data.md>), [execution](<https://devfeed.tech/tags/execution.md>), [files](<https://devfeed.tech/tags/files.md>), [queues](<https://devfeed.tech/tags/queues.md>), [temporal-concepts](<https://devfeed.tech/tags/temporal-concepts.md>), [worker](<https://devfeed.tech/tags/worker.md>)

### AI overview

This article explains how Temporal Worker-specific Task Queues can keep all Activities in a workflow on the same Worker, preserving data locality for file processing, ML model caching, and database connection pooling.

### Source excerpt

Stop re-downloading multi-GB files between Activities. Use Worker-specific Task Queues in Temporal to keep all Activities on the same Worker.

## How Goldcast scaled event orchestration to millions using Temporal

DevFeed: [How Goldcast scaled event orchestration to millions using Temporal](<https://devfeed.tech/articles/how-goldcast-scaled-event-orchestration-to-millions-using-temporal-35854.md>)

Original publisher: [Read original article](<https://temporal.io/blog/how-goldcast-scaled-event-orchestration-to-millions-using-temporal>)

Author: Himalaya Gahlot

Published: 2026-01-20T00:00:00Z

Content type: article

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [email](<https://devfeed.tech/topics/email.md>), [real-time](<https://devfeed.tech/topics/real-time.md>), [data](<https://devfeed.tech/topics/data.md>), [Databases](<https://devfeed.tech/topics/databases.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [databases](<https://devfeed.tech/tags/databases.md>), [email-delivery](<https://devfeed.tech/tags/email-delivery.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [legacy](<https://devfeed.tech/tags/legacy.md>), [operations](<https://devfeed.tech/tags/operations.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [queues](<https://devfeed.tech/tags/queues.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [scale](<https://devfeed.tech/tags/scale.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

Goldcast describes how it adopted Temporal to support reliable email delivery and event duplication for large-scale marketing operations.

### Source excerpt

Goldcast shares how Temporal Workflows power reliable email delivery and event duplication at scale, boosting success rates and speeding operations.

## The city of systems: Temporal, Kafka, and Nexus

DevFeed: [The city of systems: Temporal, Kafka, and Nexus](<https://devfeed.tech/articles/the-city-of-systems-temporal-kafka-and-nexus-35758.md>)

Original publisher: [Read original article](<https://temporal.io/blog/city-of-systems-temporal-kafka-nexus>)

Author: Tao Guo

Published: 2025-12-10T00:00:00Z

Content type: article

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [Kafka](<https://devfeed.tech/topics/kafka.md>), [Event-Streaming](<https://devfeed.tech/topics/event-streaming.md>), [distributed-systems](<https://devfeed.tech/topics/distributed-systems.md>)

Tags: [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [event-streaming](<https://devfeed.tech/tags/event-streaming.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [partitions](<https://devfeed.tech/tags/partitions.md>), [publisher](<https://devfeed.tech/tags/publisher.md>), [queues](<https://devfeed.tech/tags/queues.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [subscriber](<https://devfeed.tech/tags/subscriber.md>), [temporal-voices](<https://devfeed.tech/tags/temporal-voices.md>)

### AI overview

The article presents Temporal, Kafka, and Nexus as complementary building blocks for distributed systems. It explains Kafka's publisher-subscriber model, consumer groups, and partitions, while noting that its high-volume event broadcasting can be vulnerable to lost signals and stalled processing.

### Source excerpt

Temporal, Kafka, and Nexus create the blueprint for great distributed systems, balancing high-speed event streaming with guaranteed certainty.

## Reliable data processing: Queues and Workflows

DevFeed: [Reliable data processing: Queues and Workflows](<https://devfeed.tech/articles/reliable-data-processing-queues-and-workflows-35946.md>)

Original publisher: [Read original article](<https://temporal.io/blog/reliable-data-processing-queues-workflows>)

Author: Angela Zhou

Published: 2025-01-22T00:00:00Z

Content type: article

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [data-processing](<https://devfeed.tech/topics/data-processing.md>), [distributed-systems](<https://devfeed.tech/topics/distributed-systems.md>)

Tags: [challenges](<https://devfeed.tech/tags/challenges.md>), [data-processing](<https://devfeed.tech/tags/data-processing.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [queues](<https://devfeed.tech/tags/queues.md>), [temporal-concepts](<https://devfeed.tech/tags/temporal-concepts.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

This article explains how persistent queues store and forward messages reliably in distributed systems, including their benefits, limitations, and implementation patterns. It also describes how Temporal uses Durable Execution and Workflows to continue execution through outages, crashes, and hardware failures.

### Source excerpt

Learn the ins and outs of persistent queues including what they are, how they work, the challenges, and the benefits.

## Solid Queue 1.0 released

DevFeed: [Solid Queue 1.0 released](<https://devfeed.tech/articles/solid-queue-1-0-released-33515.md>)

Original publisher: [Read original article](<https://dev.37signals.com/solid-queue-v1-0/>)

Author: Rosa Gutiérrez

Published: 2024-09-26T17:00:00Z

Content type: release

Language: en

Sources: [37signals Dev](<https://devfeed.tech/sources/37signals-dev.md>)

Topics: [Rails](<https://devfeed.tech/topics/rails.md>), [Job](<https://devfeed.tech/topics/job.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Prometheus](<https://devfeed.tech/topics/prometheus.md>), [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [hooks](<https://devfeed.tech/tags/hooks.md>), [instrumentation](<https://devfeed.tech/tags/instrumentation.md>), [job](<https://devfeed.tech/tags/job.md>), [logging](<https://devfeed.tech/tags/logging.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [prometheus](<https://devfeed.tech/tags/prometheus.md>), [queues](<https://devfeed.tech/tags/queues.md>), [rails](<https://devfeed.tech/tags/rails.md>), [release](<https://devfeed.tech/tags/release.md>)

### AI overview

37signals released Solid Queue v1.0.0, adding atomic batch job operations, recurring jobs, logging and instrumentation, lifecycle hooks, and installation improvements. The article also describes its production use in HEY and ongoing adoption in Basecamp 4.

### Source excerpt

The first major release of Solid Queue is out, just in time for Rails World.

## Architectural Design for a Ride App such as OLA, UBER, RAPIDO

DevFeed: [Architectural Design for a Ride App such as OLA, UBER, RAPIDO](<https://devfeed.tech/articles/architectural-design-for-a-ride-app-such-as-ola-uber-rapido-33335.md>)

Original publisher: [Read original article](<https://blog.ratnesh-maurya.com/blog/Architectural-Design-for-a-Ride-App-such-as-OLA-UBER-RAPIDO>)

Author: ratneshmaurya2311@gmail.com (Ratnesh Maurya)

Published: 2024-07-30T00:00:00Z

Content type: article

Language: en

Sources: [Ratn Labs](<https://devfeed.tech/sources/ratn-labs.md>)

Topics: [microservices architecture](<https://devfeed.tech/topics/microservices-architecture.md>), [Architecture & Design](<https://devfeed.tech/topics/architecture-design.md>), [gRPC](<https://devfeed.tech/topics/grpc.md>), [App](<https://devfeed.tech/topics/app.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [backend](<https://devfeed.tech/tags/backend.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [microservices-architecture](<https://devfeed.tech/tags/microservices-architecture.md>), [queues](<https://devfeed.tech/tags/queues.md>), [rest](<https://devfeed.tech/tags/rest.md>), [software-architecture](<https://devfeed.tech/tags/software-architecture.md>), [system-design](<https://devfeed.tech/tags/system-design.md>), [system-design-backend](<https://devfeed.tech/tags/system-design-backend.md>), [tracking](<https://devfeed.tech/tags/tracking.md>), [vs](<https://devfeed.tech/tags/vs.md>)

### AI overview

An overview of architectural design for a ride-sharing app, covering microservice boundaries, REST, gRPC, message queues, geolocation tracking, and the trade-offs between starting with a monolith and adopting microservices.

### Source excerpt

Microservices architecture for ride-sharing: service boundaries, REST vs gRPC vs message queues, geo-location tracking, and trade-offs between monolith-first and microservices.

## Building application reliability on top of infrastructure unreliability

DevFeed: [Building application reliability on top of infrastructure unreliability](<https://devfeed.tech/articles/building-application-reliability-on-top-of-infrastructure-unreliability-35743.md>)

Original publisher: [Read original article](<https://temporal.io/blog/building-application-reliability-on-top-of-infrastructure-unreliability>)

Author: Fitz

Published: 2024-01-12T07:00:00Z

Content type: article

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [reliability](<https://devfeed.tech/topics/reliability.md>), [cloud-infrastructure](<https://devfeed.tech/topics/cloud-infrastructure.md>), [Microservice](<https://devfeed.tech/topics/microservice.md>), [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [Architecture & Design](<https://devfeed.tech/topics/architecture-design.md>), [istio](<https://devfeed.tech/topics/istio.md>), [Processes](<https://devfeed.tech/topics/processes.md>)

Tags: [databases](<https://devfeed.tech/tags/databases.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [istio](<https://devfeed.tech/tags/istio.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [networks](<https://devfeed.tech/tags/networks.md>), [queues](<https://devfeed.tech/tags/queues.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [software](<https://devfeed.tech/tags/software.md>)

### AI overview

The article explains how application reliability can be built on infrastructure that is inherently unreliable. It discusses failures such as hardware faults, network outages, software patching, autoscaling termination, and abruptly ended processes, then introduces an e-commerce workflow whose apparently simple steps may span many microservices, databases, caches, event streams, and message queues.

### Source excerpt

What do you do when you know, with absolute certainty, that the infrastructure you're running something on is unreliable? Hardware always fails eventually, networks go down, or software needs to be patched.

## Amazon SNS: Cost Reduction and Reliable Delivery for Startups

DevFeed: [Amazon SNS: Cost Reduction and Reliable Delivery for Startups](<https://devfeed.tech/articles/amazon-sns-cost-reduction-and-reliable-delivery-for-startups-33334.md>)

Original publisher: [Read original article](<https://blog.ratnesh-maurya.com/blog/Amazon-SNS-for-Cost-Reduction-and-Message-Delivery-Assurance-in-Startups>)

Author: ratneshmaurya2311@gmail.com (Ratnesh Maurya)

Published: 2023-12-10T00:00:00Z

Content type: article

Language: en

Sources: [Ratn Labs](<https://devfeed.tech/sources/ratn-labs.md>)

Topics: [Messaging](<https://devfeed.tech/topics/messaging.md>), [Amazon Simple Queue Service (SQS)](<https://devfeed.tech/topics/amazon-simple-queue-service-sqs.md>), [amazon](<https://devfeed.tech/topics/amazon.md>)

Tags: [amazon](<https://devfeed.tech/tags/amazon.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [aws](<https://devfeed.tech/tags/aws.md>), [backend](<https://devfeed.tech/tags/backend.md>), [cloud-devops](<https://devfeed.tech/tags/cloud-devops.md>), [cost](<https://devfeed.tech/tags/cost.md>), [delivery](<https://devfeed.tech/tags/delivery.md>), [messaging](<https://devfeed.tech/tags/messaging.md>), [queues](<https://devfeed.tech/tags/queues.md>), [sqs](<https://devfeed.tech/tags/sqs.md>), [system-design](<https://devfeed.tech/tags/system-design.md>), [system-design-backend-cloud-devops](<https://devfeed.tech/tags/system-design-backend-cloud-devops.md>)

### AI overview

This article discusses how Amazon SNS can help startups reduce messaging costs through pay-per-message pricing and support message delivery with automatic retries, dead letter queues, and multi-region delivery. It also considers when Amazon SQS may be a better fit.

### Source excerpt

How Amazon SNS helps startups cut messaging costs with pay-per-message pricing, automatic retries, dead letter queues, and multi-region delivery -- and when SQS might be a better fit.

## An introduction to Worker tuning

DevFeed: [An introduction to Worker tuning](<https://devfeed.tech/articles/an-introduction-to-worker-tuning-35709.md>)

Original publisher: [Read original article](<https://temporal.io/blog/an-introduction-to-worker-tuning>)

Author: Fitz

Published: 2023-10-23T07:00:00Z

Content type: tutorial

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [Deployment](<https://devfeed.tech/topics/deployment.md>), [optimize](<https://devfeed.tech/topics/optimize.md>), [Processes](<https://devfeed.tech/topics/processes.md>), [Server](<https://devfeed.tech/topics/server.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [client](<https://devfeed.tech/tags/client.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [commands](<https://devfeed.tech/tags/commands.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [performance](<https://devfeed.tech/tags/performance.md>), [process](<https://devfeed.tech/tags/process.md>), [queues](<https://devfeed.tech/tags/queues.md>), [requirements](<https://devfeed.tech/tags/requirements.md>), [server](<https://devfeed.tech/tags/server.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [temporal-concepts](<https://devfeed.tech/tags/temporal-concepts.md>), [types](<https://devfeed.tech/tags/types.md>), [using](<https://devfeed.tech/tags/using.md>), [worker](<https://devfeed.tech/tags/worker.md>), [workers](<https://devfeed.tech/tags/workers.md>), [workflow](<https://devfeed.tech/tags/workflow.md>)

### AI overview

This guide explains how to tune Temporal Workers as workloads grow beyond a single Worker. It covers deployment patterns, important configurations, and Worker-side performance considerations for managing a Worker pool.

### Source excerpt

Learn how to tune Temporal Workers for peak performance. Discover deployment patterns, key metrics, and configurations to optimize your Worker pool effectively.

## How Temporal handles work ranging from fast responses to long-running workflows

DevFeed: [How Temporal handles work ranging from fast responses to long-running workflows](<https://devfeed.tech/articles/what-does-long-running-really-mean-35910.md>)

Original publisher: [Read original article](<https://temporal.io/blog/long-running>)

Author: Shawn Wang

Published: 2021-08-13T07:00:00Z

Content type: article

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [reliability](<https://devfeed.tech/topics/reliability.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Microservice](<https://devfeed.tech/topics/microservice.md>), [AWS Lambda](<https://devfeed.tech/topics/aws-lambda.md>), [HTTP](<https://devfeed.tech/topics/http.md>)

Tags: [aws-lambda](<https://devfeed.tech/tags/aws-lambda.md>), [developers](<https://devfeed.tech/tags/developers.md>), [latency](<https://devfeed.tech/tags/latency.md>), [long-running](<https://devfeed.tech/tags/long-running.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [queues](<https://devfeed.tech/tags/queues.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [temporal-concepts](<https://devfeed.tech/tags/temporal-concepts.md>)

### AI overview

The article explains that "long running" work is not defined by a fixed duration. Temporal can orchestrate workflows ranging from near-instant file operations to multi-stage background processes, while using long polling to provide fast responses. It also discusses reliability, microservice orchestration, and monitoring ScheduleToStart latency in production.

### Source excerpt

Most developers know what "long running" work means. Take another look at additional perspectives and see how that definition may need to be revisited.

## Designing a Workflow engine from first principles

DevFeed: [Designing a Workflow engine from first principles](<https://devfeed.tech/articles/designing-a-workflow-engine-from-first-principles-36115.md>)

Original publisher: [Read original article](<https://temporal.io/blog/workflow-engine-principles>)

Author: Shawn Wang

Published: 2021-04-05T07:00:00Z

Content type: tutorial

Language: en

Sources: [Temporal Blog](<https://devfeed.tech/sources/temporal-blog.md>)

Topics: [Architecture & Design](<https://devfeed.tech/topics/architecture-design.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [Messaging](<https://devfeed.tech/topics/messaging.md>), [sharding](<https://devfeed.tech/topics/sharding.md>), [Routing (disambiguation)](<https://devfeed.tech/topics/routing.md>), [consistency](<https://devfeed.tech/topics/consistency.md>), [Availability](<https://devfeed.tech/topics/availability.md>), [Programming](<https://devfeed.tech/topics/programming.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [hashicorp](<https://devfeed.tech/tags/hashicorp.md>), [messaging](<https://devfeed.tech/tags/messaging.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [queues](<https://devfeed.tech/tags/queues.md>), [routing](<https://devfeed.tech/tags/routing.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [scalable-architecture](<https://devfeed.tech/tags/scalable-architecture.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [temporal-concepts](<https://devfeed.tech/tags/temporal-concepts.md>), [workflow](<https://devfeed.tech/tags/workflow.md>), [workflow-engine](<https://devfeed.tech/tags/workflow-engine.md>)

### AI overview

This article explains workflow engine design principles from first principles, including task queues, timers, consistency, scalability, sharding, routing, system workflows, and external implementations. It describes how Temporal addresses these requirements and outlines Temporal's history through related systems and projects such as SWF, Durable Task Framework, Cherami, and Cadence.

### Source excerpt

Learn key workflow engine design principles and how Temporal's scalable architecture solves challenges like task queues and scalability.

## Clean PGQ Subconsumers

DevFeed: [Clean PGQ Subconsumers](<https://devfeed.tech/articles/clean-pgq-subconsumers-34466.md>)

Original publisher: [Read original article](<https://tapoueh.org/blog/2012/04/clean-pgq-subconsumers/>)

Author: Dimitri Fontaine PostgreSQL Major Contributor; Author

Published: 2012-04-26T13:05:00Z

Content type: tutorial

Language: en

Sources: [Dimitri Fontaine](<https://devfeed.tech/sources/dimitri-fontaine.md>)

Topics: [Development](<https://devfeed.tech/topics/development.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>)

Tags: [development](<https://devfeed.tech/tags/development.md>), [queues](<https://devfeed.tech/tags/queues.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

### AI overview

A tutorial on resetting a PGQ development environment by unregistering subconsumers and consumers, then dropping queues. The process can lose events, including events from currently active batches.

### Source excerpt

Now that you're all using the wonders of Cooperative Consumers to help you efficiently and reliably implement your business constraints and offload them from the main user transactions, you're reaching a point where you have to clean up your development environment (because that's what happens to development environments, right?), and you want a way to start again from a clean empty place. Here we go. It used to be much more simple than that, so if you're still using PGQ from Skytools2, just jump to the next step.

## PGQ Coop Consumers

DevFeed: [PGQ Coop Consumers](<https://devfeed.tech/articles/pgq-coop-consumers-34465.md>)

Original publisher: [Read original article](<https://tapoueh.org/blog/2012/03/pgq-coop-consumers/>)

Author: Dimitri Fontaine PostgreSQL Major Contributor; Author

Published: 2012-03-12T13:43:00Z

Content type: article

Language: en

Sources: [Dimitri Fontaine](<https://devfeed.tech/sources/dimitri-fontaine.md>)

Topics: [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>), [Concurrent Programming](<https://devfeed.tech/topics/concurrent-programming.md>), [Job](<https://devfeed.tech/topics/job.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [pgq](<https://devfeed.tech/tags/pgq.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [queue](<https://devfeed.tech/tags/queue.md>), [queues](<https://devfeed.tech/tags/queues.md>), [replication](<https://devfeed.tech/tags/replication.md>), [scale](<https://devfeed.tech/tags/scale.md>), [worker](<https://devfeed.tech/tags/worker.md>), [workers](<https://devfeed.tech/tags/workers.md>)

### AI overview

The article describes using PostgreSQL Queue (PGQ) for transactional batch processing in a high-scale architecture. It explains consolidating events from 1,280 source queues into five destination queues and using cooperative consumers to distribute processing across workers.

### Source excerpt

While working a new PostgreSQL architecture for an high scale project that used to be in the top 10 of internet popular web sites (in terms of visitors), I needed to be able to off load some processing from the main path: that's called a batch job. This needs to be transactional: don't run the job if we did rollback; the transaction, process all events that were part of the same transaction in the same transaction, etc.

## Parallel pgloader

DevFeed: [Parallel pgloader](<https://devfeed.tech/articles/parallel-pgloader-34439.md>)

Original publisher: [Read original article](<https://tapoueh.org/blog/2011/08/parallel-pgloader/>)

Author: Dimitri Fontaine PostgreSQL Major Contributor; Author

Published: 2011-08-01T10:15:00Z

Content type: tutorial

Language: en

Sources: [Dimitri Fontaine](<https://devfeed.tech/sources/dimitri-fontaine.md>)

Topics: [parallel](<https://devfeed.tech/topics/parallel.md>), [Concurrent Programming](<https://devfeed.tech/topics/concurrent-programming.md>), [Processes](<https://devfeed.tech/topics/processes.md>), [Python](<https://devfeed.tech/topics/python.md>), [round robin](<https://devfeed.tech/topics/round-robin.md>), [Database](<https://devfeed.tech/topics/database.md>)

Tags: [article](<https://devfeed.tech/tags/article.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [code](<https://devfeed.tech/tags/code.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>), [process](<https://devfeed.tech/tags/process.md>), [python](<https://devfeed.tech/tags/python.md>), [queues](<https://devfeed.tech/tags/queues.md>), [round-robin](<https://devfeed.tech/tags/round-robin.md>), [threading](<https://devfeed.tech/tags/threading.md>)

### AI overview

This tutorial explains how pgloader loads data in parallel. It describes loading multiple files concurrently, splitting large files among workers, and using a reader thread with worker threads and queues. It also cautions that Python's Global Interpreter Lock limits runtime parallelism and recommends testing thread configurations.

### Source excerpt

This article continues the series that began with How To Use PgLoader then detailed How to Setup pgloader. We have some more fine points to talk about here, today's article is about loading your data in parallel with pgloader.