Queue API

The Queue API lets code put a task on a named queue and finish it later, outside the page request that created it. Each task is an item that carries arbitrary data. Core provides QueueInterface, a database implementation and a queue worker plugin type that processes the items.

How the Queue API works in Drupal 11

The queue service is QueueFactory; \Drupal::queue('name') is the shortcut. The main methods of QueueInterface follow an item's life. createQueue() prepares it, createItem() adds an item, claimItem() takes an item for a lease period, deleteItem() removes a finished item and releaseItem() gives one back. The default lease in claimItem() is 3600 seconds, one hour. If a consumer never finishes, the item becomes available again when the lease expires.

The default backend is DatabaseQueue, which stores items in the queue table. QueueFactory::get() checks the settings queue_reliable_service_NAME (only for a reliable queue), queue_service_NAME and queue_default, and falls back to queue.database.

To process items you write a queue worker plugin. It lives in the Plugin\QueueWorker namespace of a custom module, extends QueueWorkerBase, is declared with the #[QueueWorker] attribute and implements processItem($data). The plugin ID must match the queue name. When the attribute has a cron key, cron works through that queue for the number of seconds in time. Without a usable value the default is 15 seconds.

During cron, a worker steers the run with exceptions. RequeueException releases the item at once. SuspendQueueException releases the item and moves cron on to the next queue. DelayedRequeueException holds the item back: the database queue applies the delay the exception carries. Any other exception is logged and the item stays in the queue for a later attempt.

Drush lists the defined queues with queue:list.

Example

A module adds one item per node to be synced:

\Drupal::queue('my_module_sync')->createItem(['nid' => 42]);

A worker class in src/Plugin/QueueWorker/SyncWorker.php then handles each item:

#[QueueWorker(
          id: 'my_module_sync',
          title: new TranslatableMarkup('Sync content'),
          cron: ['time' => 60],
        )]
        class SyncWorker extends QueueWorkerBase {

          public function processItem($data): void {
            // $data is ['nid' => 42], as passed to createItem().
          }

        }

Common mistakes

• A queue name that matches no worker. Cron finds the worker through the plugin ID, so a different name means the items are never processed.
• A cron time shorter than one item needs. The cron time is also the lease. If one item takes longer, the lease expires and another consumer can claim it.
• A worker without a cron key. Cron skips that queue, so items pile up until code or drush queue:run processes them.

For marketing and content teams

When a bulk email, an import or an external sync finishes minutes after you press the button, a queue may be the reason. Saving only adds the work; cron does it afterwards. If results are missing, ask a developer to run drush queue:list: it shows how many items each queue still holds.

Sources

• Queue operations topic, Drupal API
• QueueFactory::get(), Drupal API
• QueueWorker attribute, Drupal API
• QueueWorkerManagerInterface::DEFAULT_QUEUE_CRON_TIME, Drupal API
• Cron::processQueue(), Drupal 11.4.10 source
• QueueInterface, Drupal 11.4.10 source
• Drush queue:run command
• Drush queue:list command