Handling long-running background tasks in Drupal 7

In my previous post, I discussed how to import a large dataset into Drupal via Drush's batch API. In this blog post, I'll cover how to create background tasks in Drupal 7 that will take long amounts of time to finish.

Why would you ever want that?

If you have a task that must occur regularly, but will take a long amount of time to complete, the cron queue might be a good solution. For instance, if you have a lot of nodes that need to stay synchronized with a remote dataset, you might want to synchronize a large portion of them during a cron run, but would like your other cron tasks to complete in a timely manner.

Drupal provides an easy interface to adding such long-running background tasks via hook_cron_queue_info() andhook_cron().

Implement hook_cron()

hook_cron() is run everytime the drupal cron job is run. However, it is not very well suited to longer running tasks since it runs them sequentially. To avoid holding up the other cron job tasks, we'll need to create an item in the DrupalQueue.

  function mymodule_cron() {
    // ...
    // get dataset to work on
    // ...
 
    $queue = DrupalQueue::get("resync");
    foreach ($dataset as $data) {
      $queue->createItem($data);
    }
  }

Now on every cron run, we'll insert a bunch of items in our queue to process. Let's go ahead and let Drupal know what to do for each queue's item.

  function mymodule_cron_queue_info() {
    $queues['resync'] = array (
      'worker callback' => 'mymodule_resync_item',
      'time' => 180,    // Time, in seconds, to let this process run [Optional]
    );
  }
 
  function mymodule_resync_item($data) {
    // ...
    // Code to resync data here
    // ...
  }

With just those three methods you have created background tasks that will not hold up the normal cron job tasks! It is worth noting that you don't have to enter these queue items via hook_cron(), but could add them during some other time. You might create the queue items on node creation or deletion, for instance.

so for example 

function MODULENAME_cron() {
$nodes=expired_nodes('type'); //a function that fetch the nodes id array i want
  $queue = DrupalQueue::get('update_node');
  foreach ($nodes as $row) {
    $queue->createItem($row);
  }
  return $queue;
}
 
function MODULENAME_cron_queue_info() {
  $queues['update_node'] = array(
    'worker callback' => 'MODULENAME_callback',
    'time' => 30, // time in second for each worker
  );
  return $queues;
}
 
function MODULENAME_callback($data){
      foreach ($data as $row) {
db_insert('field_data_field_SOMENAME')
  ->fields(array(
    'entity_type' => 'node',
    'bundle' => 'event',
    'entity_id'=>$row,
    'revision_id'=>$row,
    'language'=>'und',
    'delta'=>0,
    'field_other_tid'=>196,
  ))
  ->execute();
db_insert('field_revision_field_SOMENAME')
  ->fields(array(
    'entity_type' => 'node',
    'bundle' => 'event',
    'entity_id'=>$row,
    'revision_id'=>$row,
    'language'=>'und',
    'delta'=>0,
    'field_other_tid'=>196,
  ))
    ->execute();
  }
}