Missing Features

When we launch a web site at Urban Insight we run through a 25-point checklist to ensure that everything is setup correctly for a smooth launch. The checklist includes everything from ensuring Google Analytics is enabled, to verifying a clean 404 page appears to making sure the favicon is in place.

The checklist also includes checks for Drupal, the content management system we frequently use for dynamic web sites.

It was this checklist that served as the inspiration for Droptor. There are many great blog posts that outline great Drupal best practices, but there was no single comprehensive list and no tool that made it easy to check each of these on-the-fly.

This is why Droptor includes checklists for security, performance and general health. It is also why we added the QA Report, which serves as a checklist for upgrading your sites when important security patches are released. With the forthcoming launch of Droptor 2.0 (coming 8/24) we will have a checklist for search engine optimization (SEO).

Here is the SEO checklist we included in Droptor 2.0. It doesn’t include every specific SEO tool available, but does ensure you have a thorough baseline for ensuring your site is easy to index by search engines.

1. Clean URLS
2. Page Title module
3. Global Redirect module
4. Pathauto module
5. Google Analytics module
6. Site Map module
7. Performance tuned*

(* This summer Google announced it would take page load speed into consideration of ranking in page results.)

You can see screen shots of Droptor in the tour page and more information about Droptor 2.0 on the release page.

The next version of Droptor includes a feature that collects and reports on the memory usage of every page in a Drupal site. The goal of this feature is make it easy for users to find the inefficient parts of their application and tune them.

Droptor essentially gathers three pieces of information: the total memory for page execution, the total time for page execution and the URL of the page. It would have been easy to just create a giant data table of this information showing all the memory logs, 100 rows at a time paged.

The problem with this design is it would have been worthless. Let me explain.

When designing an interface it’s important to focus not on the data you have to show but on the intended work flow the screen is intended to help. For our memory profiling screen we are trying to help developers speed up their application. A giant list of memory logs doesn’t do anything to help our developers achieve this goal.

We don’t really care about the most inefficient page; we care about the most common inefficient page. If we have two hours to spend tuning your correspondent would rather modestly speed up a page that represents a major portion of traffic than quadruple performance on a page that is rarely visited.

(Yet again, the lessons of a certain aquatic bird prove true.)

When it came time to design the memory profile report page we focused on two data tables:

1. The most inefficient scripts, which includes information on what percentage of all requests the page makes up; and,
2. Raw logs for the last 25 page requests, to make it easy for a developer to validate refinements posted to production in real time.

Here is a screen shot of the final report:

Droptor memory profiling report

Your correspondent recently had to upgrade the flagship module for the Droptor Drupal monitoring service to work with Drupal 7. This modest module includes a simple form, integration with the menu system and lots of custom queries. Overall, the upgrade process was straightforward.

1. drupal_get_form() no longer returns HTML

The return value of drupal_get_form() is now a structured array not rendered HTML. You need to wrap the return of drupal_get_form() with drupal_render() to get the formatted HTML that can be returned to the module output page.

2. db_fetch_object() is deprecated

The results returned from db_query() are now an array/object, so you need to use foreach to parse the results and you will need to convert your arrays that access the data to objects.

Instead of:

while ($result = db_fetch_array($results))

just use:

foreach ($results as $result)

Inside of the foreach, change your column references like

$result['dt']

to

$result->dt

3. The admin paths are different

You will need to update the paths in your menu declarations for any module configuration screens. Droptor’s old path is admin/settings/droptor. In Drupal 7 the path is admin/config/system/droptor. Until you fix the path links you won’t see links to your module’s configuration page in the main configuration screen.

General tips

• Run your module through the Drupal 7 version of the Coder module, which will make a variety of basic refinements to known API changes.
• Check out the full list of changes in Drupal 7 for module developers
• Check out the Drupal 7 Cheat Sheet

The Drupal 7 beta version of the Droptor module is now up at the Drupal.org project page.

Your correspondent recently led the launch of a large, highly customized Drupal installation that required a two-day phased launch and coordination with the client team. In order to ensure a smooth launch, a launch checklist was started at the beginning of the project and added to over the course of development. As we completed various modules we would add steps to the checklist to ensure we didn’t forget anything at launch time.

The project was Neon Tommy, the online, student-run journalism site of the USC Annenberg School for Communication & Journalism. Neon Tommy creates original news content and is a fully managed news portal. The existing site was built on Movable Type and we had to migrate all of this existing content into Drupal.

In addition to the launch checklist below, we had already run the site through our standard web site launch checklist. This checklist is designed for any web site launch, not just Drupal sites. This checklist focused on the tasks we needed to complete during launch.

Here is the launch checklist, in three parts:

Pre-migration checklist

• Meet internally to plan migration
• Prepare launch schedule
• Meet w/client to review launch schedule
• Update DNS TTL
• Prepare launch content

Migration checklist

• Backup database, web root
• Run data import
• Rename stage site to production
• Request DNS update
• Setup HOSTS files to continue with migration during DNS update
• Update Twitter account to use production Twitter site
• Update Facebook account to use production FB account
• Import help page
• Update Google Analytics tracking code
• Register sitemap with Google
• Add the “force www” setting
• Register with Google Webmaster tools
• Update FeedBurner as needed
• Install new Google Maps API key
• Setup Google mini, add collection and URLs, configure Google
• Update Binary Canary (monitoring)
• Update the email address setting in the Site Information page
• Verify PathAuto is able to handle characters with odd accents
• Perform standard Drupal refinements
  • Disable devel module
  • Disable Theme developer
  • Disable “Rebuild theme cache on every page load” setting
  • Set maintenance theme to use site theme
  • Set Drupal performance settings: Page Cache to normal, page compression enabled, block cache enabled, optimize CSS enable and optimize JS enabled
  • Increase database logging row limit to 10,000
  • Set error reporting to log only (not log+screen)
  • Verify non-trusted users do not have access to full HTML and PHP filters
  • Set “session.cookie_lifetime” to 86400
  • Setup CRON
  • Remove TXT files in Drupal root
• Setup Droptor; confirm all checklists are green
• Update wiki page (internal documentation)
• Clear cached data
• Verify status report shows all green
• Verify DNS update complete
• Post migration message to old live site
• Post redirect .htaccess file to old external site content

Post-migration

• Perform QA testing
• Notify client when complete
• Perform marketing items
  • Tweet launch
  • Portfolio update
  • Home page news item
  • Internal message to team
• Schedule muffin basket to client

Balsamiq Mockups is a great tool to easily prepare mock-ups for desktop and web-based applications (and iPhone too). It’s also a great tool for preparing a wire frame during a website redesign project. The hand-drawn style of Mockups helps the client understand that this an initial draft of the layout for the site and not (yet) a design mock up for review.

When exporting a mock up to share with the client your correspondent uses this checklist to ensure the wire frame is presented professionally:

1. Export it as a PDF, not a PNG. PDFs always open in Acrobat and work on Windows, Mac, Linux and Google Mail. PNGs do not reliably open in an application that offers easy review.
2. Delete the bookmarks that Mockups creates in the PDF.
3. Change the properties of the PDF so that the bookmarks pane does not open by default.
4. Add a header to the PDF with the project title, wire frame title, date and version number.
5. For multiple page wire frames add the page number in the footer.

This easy five-point checklist will ensure a nice, professional looking client-friendly document every time!

Your correspondent has just completed a report that compares video hosting and integration options for Drupal.

We looked at four options — FlashVideo, Blip.tv, Viddler and Kaltura — that cover the range of options from do-it-yourself to enterprise hosting. A detailed comparison chart at the bottom of the article includes a complete feature breakdown summary.

Check out the article at UrbanInsight.com.

(See end of post for updates.)

Your correspondent was fortunate enough to recently partake in a four day, hands-on, in-depth Drupal training (agenda: theming, forms API, menu API, module development, jQuery, best practices) conducted by the firm that literally wrote the book on Drupal: Lullabot.

Our excellent instructors were Nate Haug and Karen Stevenson. The training was tremendously helpful and Nate and Karen really know Drupal through to the core. It was a blast getting a deeper look at Drupal and being able to have all of our questions answered.

These tips were collected during training and every effort was made to capture them accurately, but do point out any errors or feedback in the comments (any errors were mine in capture, not Lullabot’s in training).

1. Output the $body_classes variable in the class attribute of your body tag on your page.tpl.php files to have access to handy classes like “front”, “not_front”, “logged-in”, etc.
2. The /admin/build/block page is the only admin page that doesn’t use an admin theme so that the block location helper overlays make sense.
3. The two most common elements left out of your page.tpl.php files are $closure and $tabs.
4. The basic convention for theming is to copy the file you want to create from the code module folder (like node.tpl.php) and then edit it as needed.
5. Use placeholder tokens to handle languages that order words differently. Example:

   $variables['submitted'] = t('On @date', array('@date'=>format_date($variables['created'],'custom','F jS')));

6. Best practice: For major Drupal upgrades the best practice is to remove and recreate any theme functions you overrode, so that you can include any code changes that you haven’t captured.
7. If you don’t want to use $content in node.tpl.php files, then don’t. Just output each field individually.
8. Performance: Use the row style “Fields” rather than “Nodes” on views. The “Nodes” mode does a node_load() call on each node in your view which costs more than 50 queries per node. “Fields” mode just grabs the data you need.
9. The dsm() dpm() function is like the insanely helpful pr() function in CakePHP, offering nice output of complex data for debugging. You need devel installed and enabled to use it.
10. Group all custom modules under the same “package” name in the module list to make it clear to future developers where all of the custom modules are.
11. Use the coder module to identify Drupal 6 to 7 api changes.
12. Best practice/convention: The variable name user means the logged in user to the site at the moment of code execution; the variable name account means information about some user on the site, unrelated to the logged in user.
13. Performance: The entire variable table is loaded on every page load, so limit what you store in variables.
14. Performance: The variable_get() function is free (no hit to the database) as all variable information is stored in memory.
15. Don’t call t() on menu title and descriptions as they are stored in the cache on build, meaning that the language used at the point of cache creation gets set for everyone.
16. Use the MENU_LOCAL_TASK in the hook_menu array to add a tab to a page, like node or user.
17. Performance: Move large menu item callback functions to inc files, using the ‘file’ attribute of the menu array. This results in more efficient memory management as the entire .module file is loaded on every page load, but inc files are only loaded when specifically needed.
18. Using %user and %node in your hook_menu array will cause Drupal to automatically call node_load or user_load on the passed in IDs. Nate called these magic handlers. (Clarification: these are load functions in menu.inc)
19. You can use your own magic handler by using “%” in the hook_menu array. Example: %yourown will cause Drupal to run yourown_load(). These functions must reside in main .module file, not in inc files.
20. In your module, you can use $GLOBALS['conf']['cache'] = false to turn off caching for a page. (Note: If your page is currently cached you won’t see caching disabled until you clear the cache or it expires.) This appears to stop your entire site cache, not just the page.
21. A handy way to determine if a site is Drupal is to look for a page expire date in the headers of 11/19/1978 (the birth date of the creator of Drupal).
22. Doing routes in Drupal: custom_url_rewrite_inbound() and custom_url_rewrite_outbound().
23. Choosing the right date field type in CCK: Date (iso date), good for historical and imprecise dates , like year only; Datestamp (Unix epoch), same format as Drupal core, better to use Datetime as it offers the same precision; Datetime stores in native format of a date at database tier (you can then do date manipulation functions on it at the database level which is fast).
24. Best practice: Split your sites/all/modules folder into contrib and custom.
25. Best practice: If you have to modify a contributed module for a project, track the change in a patch file and create a folder in your site to store all of you individual patch files. Each time you update a contributed module with a newer release verify the issue still needs to be patched manually (reapply the patch you have) or that the patch is now part of the official module (delete your patch file). This way your patches can be stored in version control, easily submitted to the module maintainer and reapplied in the future when you apply important Drupal security patches to your site.
26. Best practice: hook_menu() should be the first function in your custom module because it serves as an index for the module, describing what it does and where.
27. The reason the keys in form arrays start with a pound sign is to allow the nesting of form elements in the array.
28. The “clicked_button” attribute is added to the form state by Drupal to handle images used as submit buttons because Internet Explorer doesn’t use the submit button’s name like other browsers (and like a regular submit button is handled by all browsers).
29. You set the error on a nested field by using the format parent][child. Example: "home][street", where home is the parent form, and street is the field.
30. form_error() is cleaner and more logically formatted than form_set_error(), but they do the same thing.

    form_set_error('home][street','You must enter the street address.');

    form_error($form['home']['city'], ‘You must enter the street address.’);

31. If anything is placed in $form_state['storage'] Drupal will ignore any redirects specified and rebuild the form on the submit. To avoid this you must unset $form_storage.
32. Best practice: You can use any HTML you want in a theme function because a theme is empowered to override the HTML in it's own theme function.
33. Drupal will render any remaining parts of $form that you haven't rendered already, so it's not necessary to manually render every part, only the ones you want to handle individually.
34. Use db_set_active() to switch between the database connections you have specified in your settings file on-the-fly in a routine to easily go outside of the core Drupal database for external content or data. (Switch back quickly as later parts of the page will need access to the core Drupal database.)
35. The Table Wizard module lets you expose any database table to Drupal views (you can even define keys to allow for joins).
36. If you specify a set value ("#value") on a form element (like hidden), the Drupal handler will set the form element back to that value regardless of what comes in from the user.
37. A form type ("#type") of "value" is never sent to the user but kept in the form data so you can access the data in other functions in your module. This appears to be deprecated in Drupal 6. Use $form['#foo'] instead.
38. HTML tip: Inline JavaScript stops the browser from loading anything after it (HTML, JavaScript, etc) until the browser finishes loading the code.
39. JQuery tip: VisualjQuery.com is a handy visual API reference for JQuery
40. Firebug tip: The ">>>" at the bottom of the console tab in Firebug lets you run JavaScript you type in.
41. HTML tip: Some browsers will strip out A tags if they don't have an href attribute specified.
42. JQuery performance: Specifying a tag name when looking for a class is much faster than just looking for the class as browsers have built-in support for getElementysByTagName().

    Fast: $('.content');

    Faster: $('div.content');

43. JQuery performance: Using $(this) inside of selector functions is faster than using the selector again.
44. An easy way to manage your views is to export them and install them in a custom module. This way you can keep the code in version control, protect them from user (you don't need to even install the Views UI) and you can always revert in the Views UI if changes are applied by a user accidentally to the view.
45. Managing major patches: Create an empty module and use hook_update() to push major site configuration changes (settings, etc). This way you can update the module code then run update.php and install the patch in Drupal.
46. When to sanitize: Generally speaking data at the template layer tends to be safe, anywhere else (or higher) isn't. Use check_plain (no tags) or check_markup (runs through default input filter of site).
47. You can create a Drush profile drush_make script that you can feed into a drush command to build a clean install of Drupal with all of the current versions of modules downloaded live from Drupal.org.
48. Use cache_get() and cache_set() as much as possible as serving data from the cache does not require a reduces database hits.

Thanks again to Nate and Karen for the great training!

Thanks to the commentators that helped with these refinements:

• Refined #47. Thanks Boris!
• Refined #48. Thanks zzolo!
• Refined #9. Thanks abaddon!
• Refined #31. Thanks zserno!
• Refined #18, 20, 34, 37 and 27. Thanks chx!

The WordPress folks have a great article on Hardening WordPress over on the codex. It covers strategies for securing your install in depth, and talks about general security principles.

But it really all waters down to a few steps that are painless to implement and will vastly improve the security of your WordPress install:

1. Force SSL for admin and admin login; add this to wp-config.php:
   define(‘FORCE_SSL_LOGIN’, true);
   define(‘FORCE_SSL_ADMIN’, true);
2. Add an Apache username/password prompt to the /wp-admin/ directory
3. Change the username for admin (in the database)
4. Setup WordPress for automatic updates and patch your core and plug-ins as soon as updates are available

Your correspondent ran into this rather crippling fatal error while working on a CakePHP application:

Fatal error:  Class '' not found in path-to-cake\cake\libs\class_registry.php on line 140

Googling, clearing the CakePHP cache and several Apache restarts didn’t offer a solution. The problem turned out to be a stupid mistake in a model file that had a blank hasOne relationship setting:

var $hasOne = '';

Fixing the half-completed model relationship fixed the issue (either remove the line or enter in the name of a model for the relationship).

Your correspondent spent the day at the StackOverflow Dev Day conference in Washington, DC. Overall the conference was great; Joel, Jeff and the Carsonified team should be very proud of all their hard work.

Highlights

• Power strips were all over the place, under the seats, for laptop power. Thank you!
• It was neat getting to chat to members of the FogCreek team face to face
• The jQuery team does a great job of branding jQuery speakers; the slides were fantastic: expertly branded, logically ordered and compellingly supportive of the speaker
• Great keynote theme: “How does your software help me copy my DNA?”
• The between speaker interlude program was great (having a count down timer made it easy to spend the brief downtime efficiently and the twitter feed was compellingly interactive)
• The wifi actually worked
• Lunch was both accessible and yummy (a rarity for conference food)
• The discussion topic by area lunch was a great idea
• The opening [scrums] (sp?) video was funny
• All of the talks were great: jQuery, iPhone development, the problems of backwards compatibility in language design, ASP.NET MVC, Google App Engine and the keynote.

Low lights

• The front door staff actually asked for paper tickets; you can fly on a plane without a paper ticket
• The name badges didn’t have StackOverflow rep printed on them (nor did they have a space to write that in)
• Parking was a pain
• The seats in the venue (balcony at least)  had no leg room, even for your vertically challenged correspondent
• It would have been nice if Jeff Atwood was on hand
• Bruce Eckel’s presentation on the problems of backwards compatibility could have used additional visual support and a bit more dialog polish