-
Notifications
You must be signed in to change notification settings - Fork 289
Processing Groups
Pablo Cantero edited this page Jul 3, 2017
·
20 revisions
Processing Groups is a way to set specific configurations, such as queues, concurrency, long polling for a set of queues or a single one.
Given this configuration:
concurrency: 25
queues:
- queue1
- queue2
- queue3Shoryuken will fetch messages from queue1, then queue2 ,then queue3, then repeat.
If you don't want to wait on fetching for queue1 to fetch for queue2, you can move them into separate groups:
concurrency: 25
queues:
- queue2
- queue3
groups:
group1:
concurrency: 10
queues:
- queue2
group2:
concurrency: 10
queues:
- queue1Is that all? No, it is not! If you want to process only one message at time for queue1, but keeping concurrency: 25 for queue2 and queue3:
concurrency: 25
queues:
- queue2
- queue3
groups:
group1:
concurrency: 1
queues:
- queue1Note: If you want make sure to process only one message at time for a given queue, while running multiple Shoryuken processes, have a look at FIFO Queues.
For defining a Polling strategy per group, you can set it a follows:
# config/initializers/shoryuken.rb
Shoryuken.configure_server do |config|
config.options[:group1][:polling_strategy] = Polling::StrictPriority
end# config/initializers/shoryuken.rb
Shoryuken.sqs_client_receive_message_opts[:group1] = { wait_time_seconds: 20 }