Skip to content

Latest commit

 

History

History
133 lines (93 loc) · 4.54 KB

README.md

File metadata and controls

133 lines (93 loc) · 4.54 KB
Raw

Delayed Job Groups

Gem Version Build Status Code Climate Coverage Status

A Delayed Job plugin that adds job groups supporting:

  • Canceling all jobs in a job group
  • Canceling the job group when a job in the job group fails
  • Blocking and unblocking all jobs in a job group
  • Running additional processing after all jobs in a job group complete or a job group fails

Installation

Add this line to your application's Gemfile:

gem 'delayed_job_groups_plugin'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install delayed_job_groups_plugin

Run the required database migrations:

$ rails generate delayed_job_groups_plugin:install
$ rake db:migrate

Upgrading from 0.1.2

run the following generator to create a migration for the new configuration column.

$ rails generate migration add_failure_cancels_group_to_delayed_job_groups failure_cancels_group:boolean
# add `default: true, null: false` to the generated migration for the failure_cancels_group column
$ rake db:migrate

Usage

Creating a job group and queueing some jobs:

job_group = Delayed::JobGroups::JobGroup.create!

# JobGroup#enqueue has the same signature as Delayed::Job.enqueue
# i.e. it takes a job and an optional hash of options.
job_group.enqueue(MyJob.new('some arg'), queue: 'general')
job_group.enqueue(MyJob.new('some other arg'), queue: 'general', priority: 10)

# Tell the JobGroup we're done queueing jobs
job_group.mark_queueing_complete

Registering a job to run after all jobs in the job group have completed:

# We can optionally pass options that will be used when queueing the on completion job
job_group = Delayed::JobGroups::JobGroup.create!(on_completion_job: MyCompletionJob.new,
                                                 on_completion_job_options: { queue: 'general' })

Registering a job to run if the job group is canceled or fails:

# We can optionally pass options that will be used when queueing the on cancellation job
job_group = Delayed::JobGroups::JobGroup.create!(on_cancellation_job: MyCancellationJob.new,
                                                 on_cancellation_job_options: { queue: 'general' })

Block and unblock jobs in a job group:

# Construct the JobGroup in a blocked state
job_group = Delayed::JobGroups::JobGroup.create!(blocked: true)
job_group.enqueue(MyJob.new('some arg'), queue: 'general')
job_group.mark_queueing_complete

# Do more stuff...

# Unblock the JobGroup so its jobs can run
job_group.unblock

Cancel a job group:

job_group = Delayed::JobGroups::JobGroup.create!

# Do more stuff...

job_group.cancel

Configuration to allow failed jobs not to cancel the group

# We can optionally pass options that will allow jobs to fail without cancelling the group.
# This also allows the on_completion job to fire once all jobs have either succeeded or failed.
job_group = Delayed::JobGroups::JobGroup.create!(failure_cancels_group: false)

Maintenance

It's possible to end up in a state where all jobs in a group have been completed, but the completion job has not run. This is due a race condition where the final job in a group is completed, and the worker running it is terminated before the completion job can be enqueued.

As a remedy for the above scenario, a job is provided which cleans up any job groups in this state. It is recommended to run this job periodically (for example in a cron job), especially in high-thoughput applications.

Delayed::JobGroups::CompleteStuckJobGroupsJob.enqueue

Supported Platforms

  • Only the Delayed Job Active Record backend is supported.
  • Tested with Rails 4.2 through 5.2.
  • Tested with MRI 2.3 through 2.5.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request