# PGQ Coop Consumers

DevFeed: [PGQ Coop Consumers](<https://devfeed.tech/articles/pgq-coop-consumers-34465.md>)

Original publisher: [Read original article](<https://tapoueh.org/blog/2012/03/pgq-coop-consumers/>)

Author: Dimitri Fontaine PostgreSQL Major Contributor; Author

Published: 2012-03-12T13:43:00Z

Content type: article

Language: en

Sources: [Dimitri Fontaine](<https://devfeed.tech/sources/dimitri-fontaine.md>)

Topics: [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>), [Concurrent Programming](<https://devfeed.tech/topics/concurrent-programming.md>), [Job](<https://devfeed.tech/topics/job.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [pgq](<https://devfeed.tech/tags/pgq.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [queue](<https://devfeed.tech/tags/queue.md>), [queues](<https://devfeed.tech/tags/queues.md>), [replication](<https://devfeed.tech/tags/replication.md>), [scale](<https://devfeed.tech/tags/scale.md>), [worker](<https://devfeed.tech/tags/worker.md>), [workers](<https://devfeed.tech/tags/workers.md>)

## AI overview

The article describes using PostgreSQL Queue (PGQ) for transactional batch processing in a high-scale architecture. It explains consolidating events from 1,280 source queues into five destination queues and using cooperative consumers to distribute processing across workers.

## Source excerpt

While working a new PostgreSQL architecture for an high scale project that used to be in the top 10 of internet popular web sites (in terms of visitors), I needed to be able to off load some processing from the main path: that's called a batch job. This needs to be transactional: don't run the job if we did rollback; the transaction, process all events that were part of the same transaction in the same transaction, etc.