Drupal Tutor: September 2015

Data migration... huuuh..Always a tedious job and specially for large data.Traditional programmer follow many protocols to do correct data migration to drupal . Many things need to take care like number of batch record to be migrate , manual concentration of each batches,php time out and many more. So the best solution we could follow a batch_process ,Drush with queue API.

I had quite a bit of data to work with so I had to utilize the batch API. The Batch API allows you to run one or more method on to a large set of data without worrying about PHP timeouts and can provide feedback on the progress of the operation. I had created a module to handle the updating and importing of the library data. To create the batch queue you must build an array for `batch_set();`

function mymodule_setup_batch($start=1, $stop=100000) {
  //  ...
  //  Populate $lots_of_data from record $start to record $stop.
  //  ...
 
  //Break up all of our data so each process does not time out.
  $chunks = array_chunk($lots_of_data, 20);
  $operations = array();
  $count_chunks = count($chunks);
 
  //for every chunk, assign some method to run on that chunk of data
  foreach ($chunks as $chunk) {
    $i++;
    $operations[] = array("mymodule_method_to_work_on_a_small_part", array( $chunk ,'details'=> t('(Importing chunk @chunk  of  @count)', array('@chunk '=>$i, '@count'=>$count_chunks))));
    $operations[] = array("mymodule_another_method",array($chunk));
  }
 
  //put all that information into our batch array
  $batch = array(
    'operations' => $operations,
    'title' => t('Import batch'),
    'init_message' => t('Initializing'),
    'error_message' => t('An error occurred'),
    'finished' => 'mymodule_finished_method'
  );
 
  //Get the batch process all ready!
  batch_set($batch);
  $batch =& batch_get();
 
  //Because we are doing this on the back-end, we set progressive to false.
  $batch['progressive'] = FALSE;
 
  //Start processing the batch operations.
  drush_backend_batch_process();
}

You'll also have to write what our operation methods will do. Each of these will be called with the parameters we set up earlier. In this case both methods will work on the same data, one right after the other.

function mymodule_method_to_work_on_a_small_part ($chunk, $operation_details, &$context) {
  //Do something to $chunk, maybe create a node?
  $context['message'] = $operation_details; //Will show what chunk we're on.
}
function mymodule_another_method($chunk, &$context) {
  //Do some more work.
  $context['message'] = t('We have done a second thing to a chunk of data');
}

We also need to code the method that is called when it is finished:

function mymodule_finished_method($success, $results, $operations) {
  //Let the user know we have finished!
  print t('Finished importing!');
}

Drushing data

I have always enjoyed the use of Drush, but I have never created my own Drush commands. It turns out, it is a very easy process. I decided to go ahead and make an import command, so I could start the batch process off and import a section of the entire dataset from the terminal. I placed the above code into a file mymodule.drush.inc and created the following methods:

function mymodule_drush_command() {
  $items  = array();
  $items['myimport'] = array(
    'callback'    => 'mymodule_setup_batch',
    'description' => dt('Import'),
    'arguments'   => array(
      'start'     => "start",
      'stop'      => "stop",
    ),
  );
  return $items;
}
 
function mymodule_drush_help($section) {
  switch ($section) {
    case 'drush:myimport':
      return dt("import items from the Internal Database [start record] [end record].");
  }
}

For a traditional php/zend/codeignitor programmer the logic behind developing any stuff is to create db tables with some form , save these content into db table and show these data on view parsing through controller .
How to do this custom module / programming stuff in drupal ?
Let us take entity api example step by step

Entities are a great way to organize your data in Drupal. If you are familiar with nodes, taxonomy terms, comments or users, you should also know that since Drupal 7, these have been entities. Another important aspect about them is that they are fieldable via the Field API.

In this tutorial I will show you how you can define your own custom entity type and get started working with it. Why would one want to do this instead of using nodes? Because although they are great, nodes can sometimes be overkill. There is a lot of functionality you may not need such as revisions or commenting.

For example we will define our own custom entity type called Insurance to represent simple information we have about our projects (title, price and deadline)(title,price,tenaure ). Then we will look at a few things about working with the entities of this type.

dditionally, you need the Entity API contrib module enabled on your site and set as a dependency to your custom module. The Entity API module is very powerful when working with entities as it provides a lot of functionality that the Drupal core lacks.

Defining our own Drupal entity type

The first thing we need to do to create a new entity type is to declare its schema definition. That is, write the code that will generate the database table for the entity data. In my demo.install file I have the following code:

/**
 * Implements hook_schema().
 */
function demo_schema() {

  $schema = array();

  $schema['demo_insurance'] = array(
    'description' => 'The base table for the Project entity',
    'fields' => array(
      'id' => array(
        'description' => 'Primary key of the Project entity',
        'type' => 'serial',
        'unsigned' => TRUE,
        'not null' => TRUE,
      ),
      'name' => array(
        'description' => 'Policy name.',
        'type' => 'varchar',
        'length' => 255,
        'not null' => FALSE,
      ),
      'price' => array(
        'description' => 'Insurance price.',
        'type' => 'text',
        'size' => 'big',
        'not null' => FALSE,
        'default' => NULL
      ),
      'tanure' => array(
        'description' => 'Insurance tanure.',
        'type' => 'int',
        'length' => 11,
        'not null' => FALSE,
      ),
    ),
    'primary key' => array('id'),
  );

  return $schema;
}

This is a simple implementation of hook_schema() through which we create a demo_projects table that has 4 columns: id, name, description and deadline, the first representing the primary key. Nothing big.

The next thing we need to do is implement hook_entity_info(). There are a lot of options we can specify in this hook, but here are the most basic and required ones (this goes in the demo.module file):

/**
 * Implements hook_entity_info().
 */
function demo_entity_info() {

  $info = array();

  $info['insurance'] = array(
    'label' => t('insurance'),
    'base table' => 'demo_insurance',
    'entity keys' => array(
      'id' => 'id',
      'label' => 'name',
    ),
    'module' => 'demo',
  );

  return $info;
}

With this hook we return a new key in the $info array that represents the entity machine name. Inside this array we specify the options (we will add more in the course of this tutorial). For now, we will stick tolabel (human readable name of the entity type), base table that stores the entity data, entity keyswhich are the properties that act as identifiers for the entities and module that specifies which module defines the entity type. The last one is not mandatory but recommended.

And with this we have registered our own basic entity type with Drupal. To test out if it works, enable the module for the first time and check if the table was created in the database. Then populate it with a few rows to have something to work with:

INSERT INTO `demo_insurance` (`id`, `name`, `price`, `tanure`)
VALUES
    (1, 'Summer House', '10000', '120m'),
    (2, 'Winter House', '25000', '60m');

Finally, register a path with Drupal (any path for testing only) using hook_menu() and paste the following in its callback function:

$projects = entity_load('insurance', array(1, 2));
dpm($projects);
return 'Some string';

First, we use the entity_load() function to load the project entities with the IDs of 1 and 2 and then we print them to the screen using the Devel dpm() function (so make sure Devel is enabled on your site for testing). And don’t forget that the callback function for the page needs to return something otherwise it won’t build.

Now if you navigate to that page you’ll see in Krumo the data from the 2 entities in the database.

Alternatively, you can use the EntityFieldQuery class to query for the new entities by any property you want (not just the id). For more information about how this works you can check out this Sitepoint tutorialthat will get you started.

Entity class and controller

Unfortunately, Drupal core does not come with too many helper functions to work with entities (entity_load() is pretty much the only one). However, the Entity API module fills this gap.

In order to make use of its functionality, we need to alter the entity info we declared earlier and specify the PHP classes that can be used to work with the entities. For now, we’ll add 2 more keys to the array keyedproject inside our hook_entity_info() declaration:

...

'entity class' => 'Entity',
'controller class' => 'EntityAPIController',

...

The first one is the base class provided by Entity API that will offer some wrapping functionality for the entities. This class is declared in the entity.inc file of the module and if you look inside, you’ll notice that many of its methods call the methods of another (controller) class. This is the class we specified for the controller class key.

The EntityAPIController class (found in entity.controller.inc file of the module) offers some sensible defaults for working with the entities. It extends the default Drupal coreDrupalDefaultEntityController class and it is responsible – among many other things – for performing CRUD operations.

Both of these classes can be extended in your custom module to adjust functionality (like querying, loading or displaying the entities). We will see how to do this in a minute.

But first, I want to show you how to save a new entity. Currently, in my database I have 2 records with the ids 1 and 2. I want to adjust the code we wrote in the test page callback above to create a new entity with the id of 3 if one doesn’t already exist. It can look something like this:

 $projects = entity_load('insurance', array(1, 2, 3));

 if (!isset($projects[3])) {
   $entity = entity_create('insurance', array('id' => 3));
   $entity->name = t('accidental policy');
   $entity->price= t('5000');

$entity->tanure= t('60m');
$entity->save();
 }

 dpm($projects);

 return 'Some string';

As you can see, now we try to load 3 project entities and check for the existence of the third. If it doesn’t exist, we use the entity_create() helper function provided by Entity API, set the properties to some random values and then use the save() method on the entity to persist it to the database. This method is provided by the Entity class and its job is to call the save() method on the controller class we defined above. And that method will perform the logic necessary to persist the entity. But all this happens behind the scenes and we don’t have to worry about it.

If you reload that page, you should see only 2 returned project entities, but if loaded a second time, there should be 3.

Overriding the entity classes

The last thing I want to show you in this part of the tutorial is how to display your entities. For this, we will stick to the page callback function we’ve been working with and have it render a list of our entities.

The first thing we need to do is override the buildContent() method of the defaultEntityAPIController class. The reason is that the controller cannot make assumptions about our data so we need to provide some information about how to display it. First, let’s declare our controller class that extends the previous one:

/**
 * Extending the EntityAPIController for the Project entity.
 */
class ProjectEntityController extends EntityAPIController {

}

I chose the class name ProjectEntityController and you need to make sure that you replace with this name the value you set for the controller class key in the hook_entity_info()implementation. Don’t forget.

Inside of this class, we can copy the method name from the original one and have it return the same its parent would:

public function buildContent($entity, $view_mode = 'full', $langcode = NULL, $content = array()) {

$build = parent::buildContent($entity, $view_mode, $langcode, $content);

// Our additions to the $build render array

return $build;

}

As such, there are no new changes. But now we can add our own data to the returned value of this method which is nothing more than a Drupal render array. So for example we can write this right before we return the $build array:

$build['price'] = array(
  '#type' => 'markup',
  '#markup' => check_plain($entity->description),
  '#prefix' => '<div class="project-description">',
  '#suffix' => '</div>',
);
$build['tanure'] = array(
  '#type' => 'markup',
  '#markup' => date('d F, Y', check_plain($entity->deadline)),
  '#prefix' => '<p>Deadline: ',
  '#suffix' => '</p>',
);

We are basically adding two new items to the array. The first one will wrap the description with a<div class="project-description"> and the second will output a formatted date in between paragraph tags. This is basic Drupal theming so brush up on that if you don’t understand what’s going on here. But you will notice that the project name is missing. That will be rendered automatically by Drupal because we specified it as the label in the entity keys of the hook_entity_info() implementation.

The final step is to go to our page callback function and make it display our entities. A quick way of doing that (just for demonstration purposes):

$projects = entity_load('project', array(1, 2, 3));

$list = entity_view('project', $projects);

$output = array();

foreach ($list['project'] as $project) {
  $output[] = drupal_render($project);
}

return implode($output);

As before, we first load our entities with the respective ids. Then, we run them through theentity_view() helper function that will end up calling the buildContent() method we just overrode. This function returns a list of render arrays for each entity. We render each one and store the result in the$output array that we then implode and return.

You can refresh the page and you should see a listing of all the entities you loaded. Make sure you clear the caches so that the changes become visible.

Welcome back to the second part of this tutorial in which we explore the world of custom entities in Drupal. If you haven’t already, I strongly recommend you read the first installment, but let’s do a short recap nonetheless.

In the previous article we’ve defined the schema for our entity type and registered it with Drupal. We’ve also overridden the EntityAPIController to build up the display for our entities.

In this part of the tutorial we will continue and talk about a few other cool things we can do with entities in Drupal. First, we’ll quickly set up the pages where we can display the individual project entities. Next, we will build a straightforward but very powerful admin interface to manage them. Then, we will make our entity type fieldable so we can add fields through the UI. And finally, we’ll expose it to Views so we can create proper listings of project entities.

If you want, you can follow along with the source code from the first branch of the Git repository, or take a peek into the second branch which contains all the code we will cover today.

Individual entity pages

The first thing we’ll do is create the pages for displaying individual project entities. We’ll start by adding a new item to our hook_menu() implementation:

  $items['project/%'] = array(
    'title' => 'Project',
    'page callback' => 'demo_view_project',
    'page arguments' => array(1),
    'access arguments' => array('access content'),
  );

We are registering a path (project/id) and a callback function (demo_view_project()) to which we pass the wildcard URL argument (the ID of the project). As for access, anybody with theaccess content permission can see the page.

Next, let’s write the said callback function (keep in mind this is a simple example just for demonstration purposes):

/**
 * Callback function for displaying the individual project page
 */
function demo_view_project($id) {

  $projects = entity_load('project', array($id));
  $project = $projects[$id];

  drupal_set_title($project->name);
  $output = entity_view('project', array($project));

  return $output;

}

This is again very simple: we load the entity with the ID passed from the URL, we set the title of the page, run the entity object through entity_view() and return it as page output. We’ve covered these Entity API concepts last time when we listed our projects. You can now clear the cache and navigate toproject/1 and you should see the project with the ID of 1. If you see the project name twice, don’t worry, this will become clear in the next section when we let Drupal know which one is the default URI callback for the project entities.

Admin user interface

Now that we can display the individual entities, let’s leverage the power of the Entity API module to set up a quick admin user interface to manage them. There are a few simple steps we need to take for this.

First, let’s edit our hook_entity_info() implementation for our entity type and add the following (I’ll explain everything after):

...

'access callback' => 'demo_access_callback',
'uri callback' => 'entity_class_uri',
'admin ui' => array(
  'path' => 'admin/projects',
  'controller class' => 'EntityDefaultUIController',
),

...

And replace this line:

'entity class' => 'Entity',

With this:

'entity class' => 'ProjectEntity',

With these modification, we do 4 things:

We specify an access callback function for the entity type. We’ll need this for the admin UI and we’ll declare the callback function in a minute.
We set the uri callback to the default one provided by the entity class (I will come back to this at point 4).
We set the admin ui information: path to the UI page and the controller class that will handle it.EntityDefaultUIController is the default UI class that comes with Entity API and it is declared in the entity.ui.inc file.

We change the name of the entity class for this entity type to one that does not exist yet. We will create it now by extending the previous one so that we can override its defaultUri() method:

/**
 * Project entity class extending the Entity class
 */
class ProjectEntity extends Entity {

  /**
   * Change the default URI from default/id to project/id
   */
  protected function defaultUri() {
    return array('path' => 'project/' . $this->identifier());
  }

}

As you can see, we are basically changing that path to the individual project entity returned by this class method. When the time comes, I will point out why this was handy but this will be the default one I mentioned at point 2. Now let’s quickly also declare our access callback function mentioned at point 1:

/**
 * Access callback for project entities.
 */
function demo_access_callback($op, $project = NULL, $account = NULL) {
  if ($op == 'view' || $op == 'update' || $op == 'create' || $op == 'delete') {
    return TRUE;
  }
  else {
    return FALSE;
  }
}

As you can see, this is not much of an access callback function as it returns true for everything. Here you will normally perform proper access checks but for our demonstration purposes it works just fine.

Now there is one last thing we need to do to make use of our admin interface: declare a simple add/edit form for the project entity and its submit handler:

/**
 * Form definition for adding / editing a project.
 */
function project_form($form, &$form_state, $project = NULL) {

  $form['name'] = array(
    '#title' => t('Project name'),
    '#type' => 'textfield',
    '#default_value' => isset($project->name) ? $project->name : '',
    '#required' => TRUE,
  );

  $form['description'] = array(
    '#title' => t('Project description'),
    '#type' => 'textarea',
    '#default_value' => isset($project->description) ? $project->description : '',
    '#required' => TRUE,
  );

  $form['deadline'] = array(
    '#title' => t('Project deadline'),
    '#type' => 'textfield',
    '#default_value' => isset($project->deadline) ? $project->deadline : '',
    '#required' => TRUE,
  );  

  $form['submit'] = array(
    '#type' => 'submit', 
    '#value' => isset($project->id) ? t('Update project') : t('Save project'),
    '#weight' => 50,
  );

  return $form;
}

/**
 * Submit handler for the project add/edit form.
 */
function project_form_submit($form, &$form_state) {
  $project = entity_ui_form_submit_build_entity($form, $form_state);
  $project->save();
  drupal_set_message(t('The project: @name has been saved.', array('@name' => $project->name)));
  $form_state['redirect'] = 'admin/projects';
}

in this way we can create   entire different set of module with different table  and save data in it.

Drupal Tutor

Tuesday, September 15, 2015

Handling long-running background tasks in Drupal 7

In my previous post, I discussed how to import a large dataset into Drupal via Drush's batch API. In this blog post, I'll cover how to create background tasks in Drupal 7 that will take long amounts of time to finish.

Why would you ever want that?

Implement `hook_cron()`

How to Migrate large data eg 100k or more into drupal 7 smoothly ?

Drushing data

Friday, September 11, 2015

How to increase performance of high traffic drupal website

How to build custom entity in drupal ?

Defining our own Drupal entity type

Entity class and controller

Overriding the entity classes

Welcome back to the second part of this tutorial in which we explore the world of custom entities in Drupal. If you haven’t already, I strongly recommend you read the first installment, but let’s do a short recap nonetheless.

Individual entity pages

Admin user interface

Tuesday, September 15, 2015

Handling long-running background tasks in Drupal 7

In my previous post, I discussed how to import a large dataset into Drupal via Drush's batch API. In this blog post, I'll cover how to create background tasks in Drupal 7 that will take long amounts of time to finish.

Why would you ever want that?

Implement hook_cron()

How to Migrate large data eg 100k or more into drupal 7 smoothly ?

Drushing data

Friday, September 11, 2015

How to increase performance of high traffic drupal website

How to build custom entity in drupal ?

Defining our own Drupal entity type

Entity class and controller

Overriding the entity classes

Welcome back to the second part of this tutorial in which we explore the world of custom entities in Drupal. If you haven’t already, I strongly recommend you read the first installment, but let’s do a short recap nonetheless.

Individual entity pages

Admin user interface

Implement `hook_cron()`