From 9b5217cf8539f5ec237339b4a8392cd8324b36ca Mon Sep 17 00:00:00 2001 From: Dan Hunsaker Date: Sat, 1 Dec 2018 19:19:43 -0700 Subject: [PATCH] Namespace change! Updated Composer config and documentation to reflect the new home alongside the other Resque projects. Also cleaned up some formatting in the Markdown files. --- CHANGELOG.md | 174 +++++++++++------- HOWITWORKS.md | 277 +++++++++++++++-------------- LICENSE | 4 +- README.md | 475 ++++++++++++++++++++++++-------------------------- composer.json | 22 ++- 5 files changed, 503 insertions(+), 449 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d3ceb19..fbb82e3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,82 +1,124 @@ -## 1.3 (2013-??-??) - Current Master ## +## 1.3 (2013-??-??) - Current Master -**Note:** This release introduces backwards incompatible changes with all previous versions of php-resque. Please see below for details. +**Note:** This release introduces backwards incompatible changes with all +previous versions of php-resque. Please see below for details. ### Redisent (Redis Library) Replaced with Credis -Redisent has always been the Redis backend for php-resque because of its lightweight nature. Unfortunately, Redisent is largely unmaintained. +Redisent has always been the Redis backend for php-resque because of its +lightweight nature. Unfortunately, Redisent is largely unmaintained. -[Credis](https://github.com/colinmollenhour/credis) is a fork of Redisent, which among other improvements will automatically use the [phpredis](https://github.com/nicolasff/phpredis) native PHP extension if it is available. (you want this for speed, trust me) +[Credis](https://github.com/colinmollenhour/credis) is a fork of Redisent, which +among other improvements will automatically use the +[phpredis](https://github.com/nicolasff/phpredis) native PHP extension if it is +available. (you want this for speed, trust me) -php-resque now utilizes Credis for all Redis based operations. Credis automatically required and installed as a Composer dependency. +php-resque now utilizes Credis for all Redis based operations. Credis +automatically required and installed as a Composer dependency. ### Composer Support -Composer support has been improved and is now the recommended method for including php-resque in your project. Details on Composer support can be found in the Getting Started section of the readme. +Composer support has been improved and is now the recommended method for +including php-resque in your project. Details on Composer support can be found +in the Getting Started section of the readme. ### Improved DSN Support -Changes by iskandar introduce improved support for using DSNs to connect to Redis. You can now utilize the following formatted strings for the REDIS_BACKEND environment variable to connect: +Changes by iskandar introduce improved support for using DSNs to connect to +Redis. You can now utilize the following formatted strings for the REDIS_BACKEND +environment variable to connect: -* `host` -* `host:port` -* `redis://host:port` -* `redis://host:port/db` -* `redis://user:pass@host:port/` (username is required but will be ignored) -* `tcp://user:pass@host:port/` (username is required but will be ignored) +- `host` +- `host:port` +- `redis://host:port` +- `redis://host:port/db` +- `redis://user:pass@host:port/` (username is required but will be ignored) +- `tcp://user:pass@host:port/` (username is required but will be ignored) ### Other Improvements/Changes -* **COMPATIBILITY BREAKING**: The bundled worker manager `resque.php` has been moved to `bin/resque`, and is available as `vendor/bin/resque` when php-resque is installed as a Composer package. -* Restructure tests and test bootstrapping. Autoload tests via Composer (install test dependencies with `composer install --dev`) -* Add `SETEX` to list of commands which supply a key as the first argument in Redisent (danhunsaker) -* Fix an issue where a lost connection to Redis could cause an infinite loop (atorres757) -* Add a helper method to `Resque_Redis` to remove the namespace applied to Redis keys (tonypiper) -* Call beforePerform hook before retrieivng an instance of the job class (allows beforePerform to cancel a job with DontPerform before initialising your application) -* Add `beforeEnqueue` hook, called before a job is placed on a queue - -## 1.2 (2012-10-13) ## - -**Note:** This release is largely backwards compatible with php-resque 1.1. The next release will introduce backwards incompatible changes (moving from Redisent to Credis), and will drop compatibility with PHP 5.2. - -* Allow alternate redis database to be selected when calling setBackend by supplying a second argument (patrickbajao) -* Use `require_once` when including php-resque after the app has been included in the sample resque.php to prevent include conflicts (andrewjshults) -* Wrap job arguments in an array to improve compatibility with ruby resque (warezthebeef) -* Fix a bug where the worker would spin out of control taking the server with it, if the redis connection was interrupted even briefly. Use SIGPIPE to trap this scenario cleanly. (d11wtq) -* Added support of Redis prefix (namespaces) (hlegius) -* When reserving jobs, check if the payload received from popping a queue is a valid object (fix bug whereby jobs are reserved based on an erroneous payload) (salimane) -* Re-enable autoload for class_exists in Job.php (humancopy) -* Fix lost jobs when there is more than one worker process started by the same parent process (salimane) -* Move include for resque before APP_INCLUDE is loaded in, so that way resque is available for the app -* Avoid working with dirty worker IDs (salimane) -* Allow UNIX socket to be passed to Resque when connecting to Redis (pedroarnal) -* Fix typographical errors in PHP docblocks (chaitanyakuber) -* Set the queue name on job instances when jobs are executed (chaitanyakuber) -* Fix and add tests for Resque_Event::stopListening (ebernhardson) -* Documentation cleanup (maetl) -* Pass queue name to afterEvent callback -* Only declare RedisException if it doesn't already exist (Matt Heath) -* Add support for Composer -* Fix missing and incorrect paths for Resque and Resque_Job_Status classes in demo (jjfrey) -* Disable autoload for the RedisException class_exists call (scragg0x) -* General tidy up of comments and files/folders - -## 1.1 (2011-03-27) ## - -* Update Redisent library for Redis 2.2 compatibility. Redis 2.2 is now required. (thedotedge) -* Trim output of `ps` to remove any prepended whitespace (KevBurnsJr) -* Use `getenv` instead of `$_ENV` for better portability across PHP configurations (hobodave) -* Add support for sub-second queue check intervals (KevBurnsJr) -* Ability to specify a cluster/multiple redis servers and consistent hash between them (dceballos) -* Change arguments for jobs to be an array as they're easier to work with in PHP. -* Implement ability to have setUp and tearDown methods for jobs, called before and after every single run. -* Fix `APP_INCLUDE` environment variable not loading correctly. -* Jobs are no longer defined as static methods, and classes are instantiated first. This change is NOT backwards compatible and requires job classes are updated. -* Job arguments are passed to the job class when it is instantiated, and are accessible by $this->args. This change will break existing job classes that rely on arguments that have not been updated. -* Bundle sample script for managing php-resque instances using monit -* Fix undefined variable `$child` when exiting on non-forking operating systems -* Add `PIDFILE` environment variable to write out a PID for single running workers - -## 1.0 (2010-04-18) ## - -* Initial release +- **COMPATIBILITY BREAKING**: The bundled worker manager `resque.php` has been + moved to `bin/resque`, and is available as `vendor/bin/resque` when + php-resque is installed as a Composer package. +- Restructure tests and test bootstrapping. Autoload tests via Composer + (install test dependencies with `composer install --dev`) +- Add `SETEX` to list of commands which supply a key as the first argument in + Redisent (danhunsaker) +- Fix an issue where a lost connection to Redis could cause an infinite loop + (atorres757) +- Add a helper method to `Resque_Redis` to remove the namespace applied to + Redis keys (tonypiper) +- Call beforePerform hook before retrieivng an instance of the job class + (allows beforePerform to cancel a job with DontPerform before initialising + your application) +- Add `beforeEnqueue` hook, called before a job is placed on a queue + +## 1.2 (2012-10-13) + +**Note:** This release is largely backwards compatible with php-resque 1.1. The +next release will introduce backwards incompatible changes (moving from +Redisent to Credis), and will drop compatibility with PHP 5.2. + +- Allow alternate redis database to be selected when calling setBackend by + supplying a second argument (patrickbajao) +- Use `require_once` when including php-resque after the app has been included + in the sample resque.php to prevent include conflicts (andrewjshults) +- Wrap job arguments in an array to improve compatibility with ruby resque + (warezthebeef) +- Fix a bug where the worker would spin out of control taking the server with + it, if the redis connection was interrupted even briefly. Use SIGPIPE to + trap this scenario cleanly. (d11wtq) +- Added support of Redis prefix (namespaces) (hlegius) +- When reserving jobs, check if the payload received from popping a queue is a + valid object (fix bug whereby jobs are reserved based on an erroneous + payload) (salimane) +- Re-enable autoload for class_exists in Job.php (humancopy) +- Fix lost jobs when there is more than one worker process started by the same + parent process (salimane) +- Move include for resque before APP_INCLUDE is loaded in, so that way resque + is available for the app +- Avoid working with dirty worker IDs (salimane) +- Allow UNIX socket to be passed to Resque when connecting to Redis + (pedroarnal) +- Fix typographical errors in PHP docblocks (chaitanyakuber) +- Set the queue name on job instances when jobs are executed (chaitanyakuber) +- Fix and add tests for Resque_Event::stopListening (ebernhardson) +- Documentation cleanup (maetl) +- Pass queue name to afterEvent callback +- Only declare RedisException if it doesn't already exist (Matt Heath) +- Add support for Composer +- Fix missing and incorrect paths for Resque and Resque_Job_Status classes in + demo (jjfrey) +- Disable autoload for the RedisException class_exists call (scragg0x) +- General tidy up of comments and files/folders + +## 1.1 (2011-03-27) + +- Update Redisent library for Redis 2.2 compatibility. Redis 2.2 is now + required. (thedotedge) +- Trim output of `ps` to remove any prepended whitespace (KevBurnsJr) +- Use `getenv` instead of `$_ENV` for better portability across PHP + configurations (hobodave) +- Add support for sub-second queue check intervals (KevBurnsJr) +- Ability to specify a cluster/multiple redis servers and consistent hash + between them (dceballos) +- Change arguments for jobs to be an array as they're easier to work with in + PHP. +- Implement ability to have setUp and tearDown methods for jobs, called before + and after every single run. +- Fix `APP_INCLUDE` environment variable not loading correctly. +- Jobs are no longer defined as static methods, and classes are instantiated + first. This change is NOT backwards compatible and requires job classes are + updated. +- Job arguments are passed to the job class when it is instantiated, and are + accessible by $this->args. This change will break existing job classes that + rely on arguments that have not been updated. +- Bundle sample script for managing php-resque instances using monit +- Fix undefined variable `$child` when exiting on non-forking operating + systems +- Add `PIDFILE` environment variable to write out a PID for single running + workers + +## 1.0 (2010-04-18) + +- Initial release diff --git a/HOWITWORKS.md b/HOWITWORKS.md index ec85fa3..58490a9 100644 --- a/HOWITWORKS.md +++ b/HOWITWORKS.md @@ -1,134 +1,142 @@ -*For an overview of how to __use__ php-resque, see `README.md`.* +_For an overview of how to **use** php-resque, see `README.md`._ The following is a step-by-step breakdown of how php-resque operates. -## Enqueue Job ## +## Enqueue Job What happens when you call `Resque::enqueue()`? -1. `Resque::enqueue()` calls `Resque_Job::create()` with the same arguments it - received. -2. `Resque_Job::create()` checks that your `$args` (the third argument) are - either `null` or in an array -3. `Resque_Job::create()` generates a job ID (a "token" in most of the docs) -4. `Resque_Job::create()` pushes the job to the requested queue (first - argument) -5. `Resque_Job::create()`, if status monitoring is enabled for the job (fourth - argument), calls `Resque_Job_Status::create()` with the job ID as its only - argument -6. `Resque_Job_Status::create()` creates a key in Redis with the job ID in its - name, and the current status (as well as a couple of timestamps) as its - value, then returns control to `Resque_Job::create()` -7. `Resque_Job::create()` returns control to `Resque::enqueue()`, with the job - ID as a return value -8. `Resque::enqueue()` triggers the `afterEnqueue` event, then returns control - to your application, again with the job ID as its return value +1. `Resque::enqueue()` calls `Resque_Job::create()` with the same arguments it + received. +2. `Resque_Job::create()` checks that your `$args` (the third argument) are + either `null` or in an array +3. `Resque_Job::create()` generates a job ID (a "token" in most of the docs) +4. `Resque_Job::create()` pushes the job to the requested queue (first + argument) +5. `Resque_Job::create()`, if status monitoring is enabled for the job (fourth + argument), calls `Resque_Job_Status::create()` with the job ID as its only + argument +6. `Resque_Job_Status::create()` creates a key in Redis with the job ID in its + name, and the current status (as well as a couple of timestamps) as its + value, then returns control to `Resque_Job::create()` +7. `Resque_Job::create()` returns control to `Resque::enqueue()`, with the job + ID as a return value +8. `Resque::enqueue()` triggers the `afterEnqueue` event, then returns control + to your application, again with the job ID as its return value -## Workers At Work ## +## Workers At Work How do the workers process the queues? -1. `Resque_Worker::work()`, the main loop of the worker process, calls - `Resque_Worker->reserve()` to check for a job -2. `Resque_Worker->reserve()` checks whether to use blocking pops or not (from - `BLOCKING`), then acts accordingly: - * Blocking Pop - 1. `Resque_Worker->reserve()` calls `Resque_Job::reserveBlocking()` with - the entire queue list and the timeout (from `INTERVAL`) as arguments - 2. `Resque_Job::reserveBlocking()` calls `Resque::blpop()` (which in turn - calls Redis' `blpop`, after prepping the queue list for the call, then - processes the response for consistency with other aspects of the - library, before finally returning control [and the queue/content of the - retrieved job, if any] to `Resque_Job::reserveBlocking()`) - 3. `Resque_Job::reserveBlocking()` checks whether the job content is an - array (it should contain the job's type [class], payload [args], and - ID), and aborts processing if not - 4. `Resque_Job::reserveBlocking()` creates a new `Resque_Job` object with - the queue and content as constructor arguments to initialize the job - itself, and returns it, along with control of the process, to - `Resque_Worker->reserve()` - * Queue Polling - 1. `Resque_Worker->reserve()` iterates through the queue list, calling - `Resque_Job::reserve()` with the current queue's name as the sole - argument on each pass - 2. `Resque_Job::reserve()` passes the queue name on to `Resque::pop()`, - which in turn calls Redis' `lpop` with the same argument, then returns - control (and the job content, if any) to `Resque_Job::reserve()` - 3. `Resque_Job::reserve()` checks whether the job content is an array (as - before, it should contain the job's type [class], payload [args], and - ID), and aborts processing if not - 4. `Resque_Job::reserve()` creates a new `Resque_Job` object in the same - manner as above, and also returns this object (along with control of - the process) to `Resque_Worker->reserve()` -3. In either case, `Resque_Worker->reserve()` returns the new `Resque_Job` - object, along with control, up to `Resque_Worker::work()`; if no job is - found, it simply returns `FALSE` - * No Jobs - 1. If blocking mode is not enabled, `Resque_Worker::work()` sleeps for - `INTERVAL` seconds; it calls `usleep()` for this, so fractional seconds - *are* supported - * Job Reserved - 1. `Resque_Worker::work()` triggers a `beforeFork` event - 2. `Resque_Worker::work()` calls `Resque_Worker->workingOn()` with the new - `Resque_Job` object as its argument - 3. `Resque_Worker->workingOn()` does some reference assignments to help keep - track of the worker/job relationship, then updates the job status from - `WAITING` to `RUNNING` - 4. `Resque_Worker->workingOn()` stores the new `Resque_Job` object's payload - in a Redis key associated to the worker itself (this is to prevent the job - from being lost indefinitely, but does rely on that PID never being - allocated on that host to a different worker process), then returns control - to `Resque_Worker::work()` - 5. `Resque_Worker::work()` forks a child process to run the actual `perform()` - 6. The next steps differ between the worker and the child, now running in - separate processes: - * Worker - 1. The worker waits for the job process to complete - 2. If the exit status is not 0, the worker calls `Resque_Job->fail()` with - a `Resque_Job_DirtyExitException` as its only argument. - 3. `Resque_Job->fail()` triggers an `onFailure` event - 4. `Resque_Job->fail()` updates the job status from `RUNNING` to `FAILED` - 5. `Resque_Job->fail()` calls `Resque_Failure::create()` with the job - payload, the `Resque_Job_DirtyExitException`, the internal ID of the - worker, and the queue name as arguments - 6. `Resque_Failure::create()` creates a new object of whatever type has - been set as the `Resque_Failure` "backend" handler; by default, this is - a `Resque_Failure_Redis` object, whose constructor simply collects the - data passed into `Resque_Failure::create()` and pushes it into Redis - in the `failed` queue - 7. `Resque_Job->fail()` increments two failure counters in Redis: one for - a total count, and one for the worker - 8. `Resque_Job->fail()` returns control to the worker (still in - `Resque_Worker::work()`) without a value - * Job - 1. The job calls `Resque_Worker->perform()` with the `Resque_Job` as its - only argument. - 2. `Resque_Worker->perform()` sets up a `try...catch` block so it can - properly handle exceptions by marking jobs as failed (by calling - `Resque_Job->fail()`, as above) - 3. Inside the `try...catch`, `Resque_Worker->perform()` triggers an - `afterFork` event - 4. Still inside the `try...catch`, `Resque_Worker->perform()` calls - `Resque_Job->perform()` with no arguments - 5. `Resque_Job->perform()` calls `Resque_Job->getInstance()` with no - arguments - 6. If `Resque_Job->getInstance()` has already been called, it returns the - existing instance; otherwise: - 7. `Resque_Job->getInstance()` checks that the job's class (type) exists - and has a `perform()` method; if not, in either case, it throws an - exception which will be caught by `Resque_Worker->perform()` - 8. `Resque_Job->getInstance()` creates an instance of the job's class, and - initializes it with a reference to the `Resque_Job` itself, the job's - arguments (which it gets by calling `Resque_Job->getArguments()`, which - in turn simply returns the value of `args[0]`, or an empty array if no - arguments were passed), and the queue name - 9. `Resque_Job->getInstance()` returns control, along with the job class - instance, to `Resque_Job->perform()` - 10. `Resque_Job->perform()` sets up its own `try...catch` block to handle - `Resque_Job_DontPerform` exceptions; any other exceptions are passed - up to `Resque_Worker->perform()` +1. `Resque_Worker::work()`, the main loop of the worker process, calls + `Resque_Worker->reserve()` to check for a job +2. `Resque_Worker->reserve()` checks whether to use blocking pops or not (from + `BLOCKING`), then acts accordingly: + +- Blocking Pop + 1. `Resque_Worker->reserve()` calls `Resque_Job::reserveBlocking()` with + the entire queue list and the timeout (from `INTERVAL`) as arguments + 2. `Resque_Job::reserveBlocking()` calls `Resque::blpop()` (which in turn + calls Redis' `blpop`, after prepping the queue list for the call, then + processes the response for consistency with other aspects of the + library, before finally returning control [and the queue/content of the + retrieved job, if any] to `Resque_Job::reserveBlocking()`) + 3. `Resque_Job::reserveBlocking()` checks whether the job content is an + array (it should contain the job's type [class], payload [args], and + ID), and aborts processing if not + 4. `Resque_Job::reserveBlocking()` creates a new `Resque_Job` object with + the queue and content as constructor arguments to initialize the job + itself, and returns it, along with control of the process, to + `Resque_Worker->reserve()` +- Queue Polling + 1. `Resque_Worker->reserve()` iterates through the queue list, calling + `Resque_Job::reserve()` with the current queue's name as the sole + argument on each pass + 2. `Resque_Job::reserve()` passes the queue name on to `Resque::pop()`, + which in turn calls Redis' `lpop` with the same argument, then returns + control (and the job content, if any) to `Resque_Job::reserve()` + 3. `Resque_Job::reserve()` checks whether the job content is an array (as + before, it should contain the job's type [class], payload [args], and + ID), and aborts processing if not + 4. `Resque_Job::reserve()` creates a new `Resque_Job` object in the same + manner as above, and also returns this object (along with control of + the process) to `Resque_Worker->reserve()` + +3. In either case, `Resque_Worker->reserve()` returns the new `Resque_Job` + object, along with control, up to `Resque_Worker::work()`; if no job is + found, it simply returns `FALSE` + +- No Jobs + 1. If blocking mode is not enabled, `Resque_Worker::work()` sleeps for + `INTERVAL` seconds; it calls `usleep()` for this, so fractional seconds + _are_ supported +- Job Reserved + 1. `Resque_Worker::work()` triggers a `beforeFork` event + 2. `Resque_Worker::work()` calls `Resque_Worker->workingOn()` with the new + `Resque_Job` object as its argument + 3. `Resque_Worker->workingOn()` does some reference assignments to help + keep track of the worker/job relationship, then updates the job status + from `WAITING` to `RUNNING` + 4. `Resque_Worker->workingOn()` stores the new `Resque_Job` object's + payload in a Redis key associated to the worker itself (this is to + prevent the job from being lost indefinitely, but does rely on that PID + never being allocated on that host to a different worker process), then + returns control to `Resque_Worker::work()` + 5. `Resque_Worker::work()` forks a child process to run the actual + `perform()` + 6. The next steps differ between the worker and the child, now running in + separate processes: + - Worker + 1. The worker waits for the job process to complete + 2. If the exit status is not 0, the worker calls `Resque_Job->fail()` + with a `Resque_Job_DirtyExitException` as its only argument. + 3. `Resque_Job->fail()` triggers an `onFailure` event + 4. `Resque_Job->fail()` updates the job status from `RUNNING` to + `FAILED` + 5. `Resque_Job->fail()` calls `Resque_Failure::create()` with the job + payload, the `Resque_Job_DirtyExitException`, the internal ID of the + worker, and the queue name as arguments + 6. `Resque_Failure::create()` creates a new object of whatever type has + been set as the `Resque_Failure` "backend" handler; by default, this + is a `Resque_Failure_Redis` object, whose constructor simply + collects the data passed into `Resque_Failure::create()` and pushes + it into Redis in the `failed` queue + 7. `Resque_Job->fail()` increments two failure counters in Redis: one + for a total count, and one for the worker + 8. `Resque_Job->fail()` returns control to the worker (still in + `Resque_Worker::work()`) without a value + - Job + 1. The job calls `Resque_Worker->perform()` with the `Resque_Job` as + its only argument. + 2. `Resque_Worker->perform()` sets up a `try...catch` block so it can + properly handle exceptions by marking jobs as failed (by calling + `Resque_Job->fail()`, as above) + 3. Inside the `try...catch`, `Resque_Worker->perform()` triggers an + `afterFork` event + 4. Still inside the `try...catch`, `Resque_Worker->perform()` calls + `Resque_Job->perform()` with no arguments + 5. `Resque_Job->perform()` calls `Resque_Job->getInstance()` with no + arguments + 6. If `Resque_Job->getInstance()` has already been called, it returns + the existing instance; otherwise: + 7. `Resque_Job->getInstance()` checks that the job's class (type) + exists and has a `perform()` method; if not, in either case, it + throws an exception which will be caught by + `Resque_Worker->perform()` + 8. `Resque_Job->getInstance()` creates an instance of the job's class, + and initializes it with a reference to the `Resque_Job` itself, the + job's arguments (which it gets by calling + `Resque_Job->getArguments()`, which in turn simply returns the value + of `args[0]`, or an empty array if no arguments were passed), and + the queue name + 9. `Resque_Job->getInstance()` returns control, along with the job + class instance, to `Resque_Job->perform()` + 10. `Resque_Job->perform()` sets up its own `try...catch` block to + handle `Resque_Job_DontPerform` exceptions; any other exceptions are + passed up to `Resque_Worker->perform()` 11. `Resque_Job->perform()` triggers a `beforePerform` event - 12. `Resque_Job->perform()` calls `setUp()` on the instance, if it exists + 12. `Resque_Job->perform()` calls `setUp()` on the instance, if it + exists 13. `Resque_Job->perform()` calls `perform()` on the instance 14. `Resque_Job->perform()` calls `tearDown()` on the instance, if it exists @@ -139,19 +147,20 @@ How do the workers process the queues? `TRUE` along with control, instead 17. The `try...catch` block in `Resque_Worker->perform()` ends 18. `Resque_Worker->perform()` updates the job status from `RUNNING` to - `COMPLETE`, then returns control, with no value, to the worker (again - still in `Resque_Worker::work()`) + `COMPLETE`, then returns control, with no value, to the worker + (again still in `Resque_Worker::work()`) 19. `Resque_Worker::work()` calls `exit(0)` to terminate the job process cleanly - * SPECIAL CASE: Non-forking OS (Windows) - 1. Same as the job above, except it doesn't call `exit(0)` when done - 7. `Resque_Worker::work()` calls `Resque_Worker->doneWorking()` with no - arguments - 8. `Resque_Worker->doneWorking()` increments two processed counters in Redis: - one for a total count, and one for the worker - 9. `Resque_Worker->doneWorking()` deletes the Redis key set in - `Resque_Worker->workingOn()`, then returns control, with no value, to - `Resque_Worker::work()` -4. `Resque_Worker::work()` returns control to the beginning of the main loop, - where it will wait for the next job to become available, and start this - process all over again \ No newline at end of file + - SPECIAL CASE: Non-forking OS (Windows) + 1. Same as the job above, except it doesn't call `exit(0)` when done + 7. `Resque_Worker::work()` calls `Resque_Worker->doneWorking()` with no + arguments + 8. `Resque_Worker->doneWorking()` increments two processed counters in + Redis: one for a total count, and one for the worker + 9. `Resque_Worker->doneWorking()` deletes the Redis key set in + `Resque_Worker->workingOn()`, then returns control, with no value, to + `Resque_Worker::work()` + +4. `Resque_Worker::work()` returns control to the beginning of the main loop, + where it will wait for the next job to become available, and start this + process all over again diff --git a/LICENSE b/LICENSE index a796ebf..59bbee0 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,4 @@ -(c) Chris Boulton +(c) PHP Resque Team Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the @@ -17,4 +17,4 @@ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION -WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. \ No newline at end of file +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/README.md b/README.md index cd8f83f..53d5959 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,15 @@ -php-resque: PHP Resque Worker (and Enqueue) [![Build Status](https://secure.travis-ci.org/chrisboulton/php-resque.png)](http://travis-ci.org/chrisboulton/php-resque) -=========================================== +# PHP Resque Worker (and Enqueue) -Resque is a Redis-backed library for creating background jobs, placing -those jobs on one or more queues, and processing them later. +[![Build Status](https://secure.travis-ci.org/resque/php-resque.png)](http://travis-ci.org/resque/php-resque) -## Background ## +PHP Resque is a Redis-backed library for creating background jobs, placing those +jobs on one or more queues, and processing them later. -Resque was pioneered and is developed by the fine folks at GitHub (yes, -I am a kiss-ass), and written in Ruby. What you're seeing here is an -almost direct port of the Resque worker and enqueue system to PHP. +## Background + +Resque was pioneered by GitHub, and written in Ruby. What you're seeing here +started life as an almost direct port of the Resque worker and enqueue system to +PHP. For more information on Resque, visit the official GitHub project: @@ -16,61 +17,51 @@ For more information on Resque, visit the official GitHub project: For further information, see the launch post on the GitHub blog: -The PHP port does NOT include its own web interface for viewing queue -stats, as the data is stored in the exact same expected format as the -Ruby version of Resque. +> The PHP port does NOT include its own web interface for viewing queue stats, +> as the data is stored in the exact same expected format as the Ruby version of +> Resque. The PHP port provides much the same features as the Ruby version: -* Workers can be distributed between multiple machines -* Includes support for priorities (queues) -* Resilient to memory leaks (forking) -* Expects failure +- Workers can be distributed between multiple machines +- Includes support for priorities (queues) +- Resilient to memory leaks (forking) +- Expects failure It also supports the following additional features: -* Has the ability to track the status of jobs -* Will mark a job as failed, if a forked child running a job does -not exit with a status code as 0 -* Has built in support for `setUp` and `tearDown` methods, called -pre and post jobs +- Has the ability to track the status of jobs +- Will mark a job as failed, if a forked child running a job does not exit + with a status code as `0` +- Has built in support for `setUp` and `tearDown` methods, called pre and post + jobs -## Requirements ## +## Requirements -* PHP 5.3+ -* Redis 2.2+ -* Optional but Recommended: Composer +- PHP 5.3+ +- Redis 2.2+ +- Optional but Recommended: Composer -## Getting Started ## +## Getting Started -The easiest way to work with php-resque is when it's installed as a -Composer package inside your project. Composer isn't strictly -required, but makes life a lot easier. +The easiest way to work with php-resque is when it's installed as a Composer +package inside your project. Composer isn't strictly required, but makes life a +lot easier. If you're not familiar with Composer, please see . -1. Add php-resque to your application's composer.json. - -```json -{ - "require": { - "chrisboulton/php-resque": "1.2.x" - } -} -``` +1. Run `composer require resque/php-resque`. -2. Run `composer install`. +2. If you haven't already, add the Composer autoload to your project's + initialization file. (example) -3. If you haven't already, add the Composer autoload to your project's - initialization file. (example) - -```sh +```php require 'vendor/autoload.php'; ``` -## Jobs ## +## Jobs -### Queueing Jobs ### +### Queueing Jobs Jobs are queued as follows: @@ -79,12 +70,12 @@ Jobs are queued as follows: Resque::setBackend('localhost:6379'); $args = array( - 'name' => 'Chris' + 'name' => 'Chris' ); Resque::enqueue('default', 'My_Job', $args); ``` -### Defining Jobs ### +### Defining Jobs Each job should be in its own class, and include a `perform` method. @@ -99,18 +90,16 @@ class My_Job } ``` -When the job is run, the class will be instantiated and any arguments -will be set as an array on the instantiated object, and are accessible -via `$this->args`. +When the job is run, the class will be instantiated and any arguments will be +set as an array on the instantiated object, and are accessible via +`$this->args`. -Any exception thrown by a job will result in the job failing - be -careful here and make sure you handle the exceptions that shouldn't -result in a job failing. - -Jobs can also have `setUp` and `tearDown` methods. If a `setUp` method -is defined, it will be called before the `perform` method is run. -The `tearDown` method, if defined, will be called after the job finishes. +Any exception thrown by a job will result in the job failing - be careful here +and make sure you handle the exceptions that shouldn't result in a job failing. +Jobs can also have `setUp` and `tearDown` methods. If a `setUp` method is +defined, it will be called before the `perform` method is run. The `tearDown` +method, if defined, will be called after the job finishes. ```php class My_Job @@ -132,7 +121,7 @@ class My_Job } ``` -### Dequeueing Jobs ### +### Dequeueing Jobs This method can be used to conveniently remove a job from a queue. @@ -150,22 +139,22 @@ Resque::dequeue('default', ['My_Job' => array('foo' => 1, 'bar' => 2)]); Resque::dequeue('default', ['My_Job', 'My_Job2']); ``` -If no jobs are given, this method will dequeue all jobs matching the provided queue. +If no jobs are given, this method will dequeue all jobs matching the provided +queue. ```php // Removes all jobs of queue 'default' Resque::dequeue('default'); ``` -### Tracking Job Statuses ### +### Tracking Job Statuses -php-resque has the ability to perform basic status tracking of a queued -job. The status information will allow you to check if a job is in the -queue, is currently being run, has finished, or has failed. +php-resque has the ability to perform basic status tracking of a queued job. The +status information will allow you to check if a job is in the queue, is +currently being run, has finished, or has failed. To track the status of a job, pass `true` as the fourth argument to -`Resque::enqueue`. A token used for tracking the job status will be -returned: +`Resque::enqueue`. A token used for tracking the job status will be returned: ```php $token = Resque::enqueue('default', 'My_Job', $args, true); @@ -179,33 +168,31 @@ $status = new Resque_Job_Status($token); echo $status->get(); // Outputs the status ``` -Job statuses are defined as constants in the `Resque_Job_Status` class. -Valid statuses include: +Job statuses are defined as constants in the `Resque_Job_Status` class. Valid +statuses include: -* `Resque_Job_Status::STATUS_WAITING` - Job is still queued -* `Resque_Job_Status::STATUS_RUNNING` - Job is currently running -* `Resque_Job_Status::STATUS_FAILED` - Job has failed -* `Resque_Job_Status::STATUS_COMPLETE` - Job is complete -* `false` - Failed to fetch the status - is the token valid? +- `Resque_Job_Status::STATUS_WAITING` - Job is still queued +- `Resque_Job_Status::STATUS_RUNNING` - Job is currently running +- `Resque_Job_Status::STATUS_FAILED` - Job has failed +- `Resque_Job_Status::STATUS_COMPLETE` - Job is complete +- `false` - Failed to fetch the status; is the token valid? -Statuses are available for up to 24 hours after a job has completed -or failed, and are then automatically expired. A status can also -forcefully be expired by calling the `stop()` method on a status -class. +Statuses are available for up to 24 hours after a job has completed or failed, +and are then automatically expired. A status can also forcefully be expired by +calling the `stop()` method on a status class. -## Workers ## +## Workers Workers work in the exact same way as the Ruby workers. For complete documentation on workers, see the original documentation. -A basic "up-and-running" `bin/resque` file is included that sets up a -running worker environment. (`vendor/bin/resque` when installed -via Composer) +A basic "up-and-running" `bin/resque` file is included that sets up a running +worker environment. (`vendor/bin/resque` when installed via Composer) -The exception to the similarities with the Ruby version of resque is -how a worker is initially setup. To work under all environments, -not having a single environment such as with Ruby, the PHP port makes -*no* assumptions about your setup. +The exception to the similarities with the Ruby version of resque is how a +worker is initially setup. To work under all environments, not having a single +environment such as with Ruby, the PHP port makes _no_ assumptions about your +setup. To start a worker, it's very similar to the Ruby version: @@ -213,17 +200,17 @@ To start a worker, it's very similar to the Ruby version: $ QUEUE=file_serve php bin/resque ``` -It's your responsibility to tell the worker which file to include to get -your application underway. You do so by setting the `APP_INCLUDE` environment +It's your responsibility to tell the worker which file to include to get your +application underway. You do so by setting the `APP_INCLUDE` environment variable: ```sh $ QUEUE=file_serve APP_INCLUDE=../application/init.php php bin/resque ``` -*Pro tip: Using Composer? More than likely, you don't need to worry about -`APP_INCLUDE`, because hopefully Composer is responsible for autoloading -your application too!* +_Pro tip: Using Composer? More than likely, you don't need to worry about +`APP_INCLUDE`, because hopefully Composer is responsible for autoloading your +application too!_ Getting your application underway also includes telling the worker your job classes, by means of either an autoloader or including them. @@ -232,23 +219,22 @@ Alternately, you can always `include('bin/resque')` from your application and skip setting `APP_INCLUDE` altogether. Just be sure the various environment variables are set (`setenv`) before you do. -### Logging ### +### Logging -The port supports the same environment variables for logging to STDOUT. -Setting `VERBOSE` will print basic debugging information and `VVERBOSE` -will print detailed information. +The port supports the same environment variables for logging to STDOUT. Setting +`VERBOSE` will print basic debugging information and `VVERBOSE` will print +detailed information. ```sh $ VERBOSE=1 QUEUE=file_serve bin/resque $ VVERBOSE=1 QUEUE=file_serve bin/resque ``` -### Priorities and Queue Lists ### +### Priorities and Queue Lists -Similarly, priority and queue list functionality works exactly -the same as the Ruby workers. Multiple queues should be separated with -a comma, and the order that they're supplied in is the order that they're -checked in. +Similarly, priority and queue list functionality works exactly the same as the +Ruby workers. Multiple queues should be separated with a comma, and the order +that they're supplied in is the order that they're checked in. As per the original example: @@ -256,19 +242,18 @@ As per the original example: $ QUEUE=file_serve,warm_cache bin/resque ``` -The `file_serve` queue will always be checked for new jobs on each -iteration before the `warm_cache` queue is checked. +The `file_serve` queue will always be checked for new jobs on each iteration +before the `warm_cache` queue is checked. -### Running All Queues ### +### Running All Queues -All queues are supported in the same manner and processed in alphabetical -order: +All queues are supported in the same manner and processed in alphabetical order: ```sh $ QUEUE='*' bin/resque ``` -### Running Multiple Workers ### +### Running Multiple Workers Multiple workers can be launched simultaneously by supplying the `COUNT` environment variable: @@ -279,62 +264,58 @@ $ COUNT=5 bin/resque Be aware, however, that each worker is its own fork, and the original process will shut down as soon as it has spawned `COUNT` forks. If you need to keep -track of your workers using an external application such as `monit`, you'll -need to work around this limitation. +track of your workers using an external application such as `monit`, you'll need +to work around this limitation. -### Custom prefix ### +### Custom prefix -When you have multiple apps using the same Redis database it is better to -use a custom prefix to separate the Resque data: +When you have multiple apps using the same Redis database it is better to use a +custom prefix to separate the Resque data: ```sh $ PREFIX=my-app-name bin/resque ``` -### Forking ### +### Forking -Similarly to the Ruby versions, supported platforms will immediately -fork after picking up a job. The forked child will exit as soon as -the job finishes. +Similarly to the Ruby versions, supported platforms will immediately fork after +picking up a job. The forked child will exit as soon as the job finishes. -The difference with php-resque is that if a forked child does not -exit nicely (PHP error or such), php-resque will automatically fail -the job. +The difference with php-resque is that if a forked child does not exit nicely +(PHP error or such), php-resque will automatically fail the job. -### Signals ### +### Signals -Signals also work on supported platforms exactly as in the Ruby -version of Resque: +Signals also work on supported platforms exactly as in the Ruby version of +Resque: -* `QUIT` - Wait for job to finish processing then exit -* `TERM` / `INT` - Immediately kill job then exit -* `USR1` - Immediately kill job but don't exit -* `USR2` - Pause worker, no new jobs will be processed -* `CONT` - Resume worker. +- `QUIT` - Wait for job to finish processing then exit +- `TERM` / `INT` - Immediately kill job then exit +- `USR1` - Immediately kill job but don't exit +- `USR2` - Pause worker, no new jobs will be processed +- `CONT` - Resume worker. -### Process Titles/Statuses ### +### Process Titles/Statuses -The Ruby version of Resque has a nifty feature whereby the process -title of the worker is updated to indicate what the worker is doing, -and any forked children also set their process title with the job -being run. This helps identify running processes on the server and -their resque status. +The Ruby version of Resque has a nifty feature whereby the process title of the +worker is updated to indicate what the worker is doing, and any forked children +also set their process title with the job being run. This helps identify running +processes on the server and their resque status. **PHP does not have this functionality by default until 5.5.** -A PECL module () exists that -adds this functionality to PHP before 5.5, so if you'd like process -titles updated, install the PECL module as well. php-resque will -automatically detect and use it. +A PECL module () exists that adds this +functionality to PHP before 5.5, so if you'd like process titles updated, +install the PECL module as well. php-resque will automatically detect and use +it. -## Event/Hook System ## +## Event/Hook System -php-resque has a basic event system that can be used by your application -to customize how some of the php-resque internals behave. +php-resque has a basic event system that can be used by your application to +customize how some of the php-resque internals behave. -You listen in on events (as listed below) by registering with `Resque_Event` -and supplying a callback that you would like triggered when the event is -raised: +You listen in on events (as listed below) by registering with `Resque_Event` and +supplying a callback that you would like triggered when the event is raised: ```sh Resque_Event::listen('eventName', [callback]); @@ -342,16 +323,16 @@ Resque_Event::listen('eventName', [callback]); `[callback]` may be anything in PHP that is callable by `call_user_func_array`: -* A string with the name of a function -* An array containing an object and method to call -* An array containing an object and a static method to call -* A closure (PHP 5.3+) +- A string with the name of a function +- An array containing an object and method to call +- An array containing an object and a static method to call +- A closure (PHP 5.3+) Events may pass arguments (documented below), so your callback should accept these arguments. -You can stop listening to an event by calling `Resque_Event::stopListening` -with the same arguments supplied to `Resque_Event::listen`. +You can stop listening to an event by calling `Resque_Event::stopListening` with +the same arguments supplied to `Resque_Event::listen`. It is up to your application to register event listeners. When enqueuing events in your application, it should be as easy as making sure php-resque is loaded @@ -364,127 +345,133 @@ responsibility to register listeners. A sample plugin is included in the `extras` directory. -### Events ### +### Events -#### beforeFirstFork #### +#### beforeFirstFork -Called once, as a worker initializes. Argument passed is the instance of `Resque_Worker` -that was just initialized. +Called once, as a worker initializes. Argument passed is the instance of +`Resque_Worker` that was just initialized. -#### beforeFork #### +#### beforeFork -Called before php-resque forks to run a job. Argument passed contains the instance of -`Resque_Job` for the job about to be run. +Called before php-resque forks to run a job. Argument passed contains the +instance of `Resque_Job` for the job about to be run. -`beforeFork` is triggered in the **parent** process. Any changes made will be permanent -for as long as the **worker** lives. +`beforeFork` is triggered in the **parent** process. Any changes made will be +permanent for as long as the **worker** lives. -#### afterFork #### +#### afterFork Called after php-resque forks to run a job (but before the job is run). Argument passed contains the instance of `Resque_Job` for the job about to be run. -`afterFork` is triggered in the **child** process after forking out to complete a job. Any -changes made will only live as long as the **job** is being processed. +`afterFork` is triggered in the **child** process after forking out to complete +a job. Any changes made will only live as long as the **job** is being +processed. -#### beforePerform #### +#### beforePerform -Called before the `setUp` and `perform` methods on a job are run. Argument passed -contains the instance of `Resque_Job` for the job about to be run. +Called before the `setUp` and `perform` methods on a job are run. Argument +passed contains the instance of `Resque_Job` for the job about to be run. -You can prevent execution of the job by throwing an exception of `Resque_Job_DontPerform`. -Any other exceptions thrown will be treated as if they were thrown in a job, causing the -job to fail. +You can prevent execution of the job by throwing an exception of +`Resque_Job_DontPerform`. Any other exceptions thrown will be treated as if they +were thrown in a job, causing the job to fail. -#### afterPerform #### +#### afterPerform -Called after the `perform` and `tearDown` methods on a job are run. Argument passed -contains the instance of `Resque_Job` that was just run. +Called after the `perform` and `tearDown` methods on a job are run. Argument +passed contains the instance of `Resque_Job` that was just run. -Any exceptions thrown will be treated as if they were thrown in a job, causing the job -to be marked as having failed. +Any exceptions thrown will be treated as if they were thrown in a job, causing +the job to be marked as having failed. -#### onFailure #### +#### onFailure Called whenever a job fails. Arguments passed (in this order) include: -* Exception - The exception that was thrown when the job failed -* Resque_Job - The job that failed +- Exception - The exception that was thrown when the job failed +- Resque_Job - The job that failed -#### beforeEnqueue #### +#### beforeEnqueue Called immediately before a job is enqueued using the `Resque::enqueue` method. Arguments passed (in this order) include: -* Class - string containing the name of the job to be enqueued -* Arguments - array of arguments for the job -* Queue - string containing the name of the queue the job is to be enqueued in -* ID - string containing the token of the job to be enqueued - -You can prevent enqueing of the job by throwing an exception of `Resque_Job_DontCreate`. - -#### afterEnqueue #### - -Called after a job has been queued using the `Resque::enqueue` method. Arguments passed -(in this order) include: - -* Class - string containing the name of scheduled job -* Arguments - array of arguments supplied to the job -* Queue - string containing the name of the queue the job was added to -* ID - string containing the new token of the enqueued job - -## Step-By-Step ## - -For a more in-depth look at what php-resque does under the hood (without -needing to directly examine the code), have a look at `HOWITWORKS.md`. - -## Contributors ## - -### Project Lead ### - -* @chrisboulton - -### Others ### - -* @acinader -* @ajbonner -* @andrewjshults -* @atorres757 -* @benjisg -* @cballou -* @chaitanyakuber -* @charly22 -* @CyrilMazur -* @d11wtq -* @danhunsaker -* @dceballos -* @ebernhardson -* @hlegius -* @hobodave -* @humancopy -* @iskandar -* @JesseObrien -* @jjfrey -* @jmathai -* @joshhawthorne -* @KevBurnsJr -* @lboynton -* @maetl -* @matteosister -* @MattHeath -* @mickhrmweb -* @Olden -* @patrickbajao -* @pedroarnal -* @ptrofimov -* @rajibahmed -* @richardkmiller -* @Rockstar04 -* @ruudk -* @salimane -* @scragg0x -* @scraton -* @thedotedge -* @tonypiper -* @trimbletodd -* @warezthebeef +- Class - string containing the name of the job to be enqueued +- Arguments - array of arguments for the job +- Queue - string containing the name of the queue the job is to be enqueued in +- ID - string containing the token of the job to be enqueued + +You can prevent enqueing of the job by throwing an exception of +`Resque_Job_DontCreate`. + +#### afterEnqueue + +Called after a job has been queued using the `Resque::enqueue` method. Arguments +passed (in this order) include: + +- Class - string containing the name of scheduled job +- Arguments - array of arguments supplied to the job +- Queue - string containing the name of the queue the job was added to +- ID - string containing the new token of the enqueued job + +## Step-By-Step + +For a more in-depth look at what php-resque does under the hood (without needing +to directly examine the code), have a look at `HOWITWORKS.md`. + +## Contributors + +### Project Creator + +- @chrisboulton + +### Project Maintainers + +- @danhunsaker +- @rajibahmed +- @steveklabnik + +### Others + +- @acinader +- @ajbonner +- @andrewjshults +- @atorres757 +- @benjisg +- @cballou +- @chaitanyakuber +- @charly22 +- @CyrilMazur +- @d11wtq +- @dceballos +- @ebernhardson +- @hlegius +- @hobodave +- @humancopy +- @iskandar +- @JesseObrien +- @jjfrey +- @jmathai +- @joshhawthorne +- @KevBurnsJr +- @lboynton +- @maetl +- @matteosister +- @MattHeath +- @mickhrmweb +- @Olden +- @patrickbajao +- @pedroarnal +- @ptrofimov +- @richardkmiller +- @Rockstar04 +- @ruudk +- @salimane +- @scragg0x +- @scraton +- @thedotedge +- @tonypiper +- @trimbletodd +- @warezthebeef diff --git a/composer.json b/composer.json index b12fa29..629472d 100644 --- a/composer.json +++ b/composer.json @@ -1,14 +1,30 @@ { - "name": "chrisboulton/php-resque", + "name": "resque/php-resque", "type": "library", "description": "Redis backed library for creating background jobs and processing them later. Based on resque for Ruby.", "keywords": ["job", "background", "redis", "resque"], - "homepage": "http://www.github.com/chrisboulton/php-resque/", + "homepage": "http://www.github.com/resque/php-resque/", "license": "MIT", "authors": [ + { + "name": "Dan Hunsaker", + "email": "danhunsaker+resque@gmail.com", + "role": "Maintainer" + }, + { + "name": "Rajib Ahmed", + "homepage": "https://github.com/rajibahmed", + "role": "Maintainer" + }, + { + "name": "Steve Klabnik", + "email": "steve@steveklabnik.com", + "role": "Maintainer" + }, { "name": "Chris Boulton", - "email": "chris@bigcommerce.com" + "email": "chris@bigcommerce.com", + "role": "Creator" } ], "repositories": [