Posted on

WordPress 4.5 Breaking JavaScript (aka Where Is My Map?)

Ever since WordPress 4.5 rolled off the press there have been numerous complaints about websites breaking.   Numerous reports are coming into our Store Locator Plus forums and support email telling us “our map broke when they updated our website”.  The problem?  jQuery. To be more specific the problem is not jQuery but  how some plugins and themes implement jQuery in WordPress.

WordPress 4.5 started shipping jQuery version 1.12.3 as the “official” version of jQuery being used  with WordPress core. jQuery 1.12 has more stringent controls than previous versions.   The most obvious, jQuery 1.12 no longer “hides” some of the syntax errors that lay dormant in plugin code.   If there is malformed  or incorrect syntax, jQuery 1.12 will complain.  Your browser will most likely stop executing ALL scripts from that point forward.  As you can imagine, this causes things, like themes and plugins  to break.
Continue reading WordPress 4.5 Breaking JavaScript (aka Where Is My Map?)

Posted on

Release Updates: SLP 4.4.30, Pages 4.4.06, and Pro Pack 4.4.04

Store Locator Plus

Store Locator Plus developers have been busy with a lot of coding to prepare for the transition to a better and, hopefully, smoother internationalization and language translation process with add-ons and/or updates to the base plug-in. Translation files will no longer need to be 100% updated and accurate to prevent the settings strings from “dropping off”. This is the first iteration of several that will slowly convert all admin settings to be language “neutral”. The release also includes some minor performance improvements and an update to the German language file thanks to one of our Store Locator Plus customers. Tutorial and informational videos (on this release as well as “How to” instructional videos) can be viewed on the Store Locator Plus YouTube channel.

Pro Pack v 4.4.04

Pro Pack has also been updated to address two known issues. The first issue that may not be apparent to many users was found in the Pro Pack Reports. Pro Pack Reports were creating duplicate indexes.That has been patched. The second issue that was reported was with remote import functions set for a chronological upload. The schedule import time has to be set to seconds not the top of the hour, (NOT a time like 14:00 as noted in the help text). The help text in Pro Pack has been fixed to clarify.

Import CRON
Import CRON

Pages v 4.4.06

A bug was found in a past Pages release causing Pages to reset the custom Page templates and also resetting them back to draft whenever locations or other info in your location database was changed. That has been patched in 4.4.06

As a side note, if you missed the 2016 Future Forward announcement, both Pages and Pro Pack will be included in the new Power add-on, estimated to be ready for production in April 2016.

The technical background for Pro Pack release notes, Pages release notes and Store Locator Plus 4.4.30 are explained in the below videos

Posted on

Fixing VVV svn cleanup Invalid cross-device link messages

Ran into a unique situation while updating my VVV box after  a weekend of WordPress Core and plugin development at WordCamp US this past weekend.   Today, after the formal release of WordPress 4.4 I needed to update the code in my WordPress trunk directory on the VVV box.  Since I have other things in progress I didn’t want to take the time to reprovision the entire box.  Though, as it turn out, that would have been faster.

The issue was when I tried to do the svn up command to update the /srv/www/wordpress-trunk directory and make sure I was on the latest code.   The command failed, insisting that a previous operation was incomplete.  Not surprising since the connectivity at the conference was less-than-consistent.    svn kindly suggest I run svn cleanup.  Which I did.  And was promptly met with an “invalid device cross-link” error when it tried to restore hello.php to the plugin directory.

The problem is that I develop plugins for a living.   As such I have followed the typical VVV setup and have linked my local plugin source code directory to the vvv plugin directory for each of the different source directories on that box.    I created the suggested Customfile on my host system and mapped the different directory paths.     On the guest box, however, the system sees this mapping as a separate drive.  Which it is.  And, quite honestly I’m glad they have some security in place to protect this.  Otherwise a rogue app brought in via the Vagrant guest could start writing stuff to your host drive.   I can think of more than one way to do really bad things if that was left wide-open as a two-way read-write channel.

VVV Customfile Cross Device Maker
VVV Customfile Cross Device Maker

The solution?

Comment out the mapping in Customfile on the host server.  Go to your vvv directory and find that Customfile.  Throw a hashtag (or pound sign for us old guys) in front of the directory paths you are trying to update with svn.  In my case wordpress-trunk.

Run the vagrant reload command so you don’t pull down and provision a whole new box, but DO break the linkage to the host directory and guest directory.

Go run your svn cleanup and update on the host to fetch the latest WP code.

Go back to the host, kill the hashtag, and reload.

 

Hope that saves you an extra 20 minutes surfing Google, or your favorite search service, for the answer.

 

Posted on

PHP Switch Vs If Performance

When writing a complex application such as Store Locator Plus, you will often find multiple logic branches to handle a given task.   There are two oft-used methods for processing the logic; the If-Else construct and a Switch statement.    Since I am always looking to optimize the Store Locator Plus codebase for performance, some sites do have hundreds-of-thousands of locations after all, it was time to look into the performance-versus-readability of those two options.

The general consensus, though I’ve not taken the time to run performance tests with the WordPress stack myself, is that “you should use whatever makes your code easier to read are more easily maintained”.  For me that means using switch statements.    I find the construct much easier to extend and not cause inadvertent side effects.  Something I’ve learned in 20-plus years of working on code teams and in long-run projects like Store Locator Plus.

On the pure performance standpoint an if-else can be marginally faster if performing less than 5 logic comparisons.

PHP If Else Statement
PHP If Else Statement

Switch statements will often be faster at-or-near 5 logic comparisons as code optimization within C, and likely carried forth in the PHP psuedo-compiler, will often turn the 5+ logic branches of a switch statement into a hash table.  Hash tables tend to be faster with all branches of the code having equal access time.    Statistically speaking a large number of iterations will favor the equal access time model over the “first-fastest” model of an If-Else.

PHP Switch Statement
PHP Switch Statement

Possibly faster and always easier-to-extend and read, Switch will be my “weapon of choice” whenever I have more than  a simple 2-state if/else logic branch to be tested.

Posted on

WordPress dbDelta Better In 4.2, Not Yet Perfect

WP dbDelta Bug Report Banner

There have been a LOT of changes in WordPress Core including updates to the dbDelta() method in the upgrade function set.   What is dbDelta()?  dbDelta is the primary function used by WordPress to determine if the structure of data tables needs to be augmented between releases.   It works great for upgrading core WordPress data tables.    Where it has problems is with developer-specific SQL statements used to create secondary tables in plugins and themes.    Store Locator Plus, for example, uses a secondary table to storing basic location data.

Unfortunately dbDelta has always had issues (see running commentary on Trac ticket 10404), especially when it comes to indices on a MySQL table.   Most well-designed data tables will have primary and secondary indexes on the table to improve query performance.    dbDelta has a hard time figuring out when an index already exists and has not been modified.   If you did not craft your CREATE TABLE command EXACTLY the way dbDelta wants to see it, the dbDelta method will create new indexes.    This can cause performance degradation and unnecessary overhead in the MySQL tables when a data column ends up with 10, 20 , 30+ copies of the index; one created every time the plugin or theme is updated.

Many plugins and themes suffer from this malady.   Over two years ago I dug into dbDelta and learned the syntax secrets to get WordPress to stop creating duplicate indexes.   Today I found a site that was back to its old tricks running the latest version of Store Locator Plus and WordPress 4.2.2.   As it turns out the rules for dbDelta have changed and thus the syntax for creating a “dbDelta friendly” SQL statement has changed as well.    With dbDelta a behind-the-scenes feature it is never broadcast to the general public that it has changed and thus goes undetected for a while when something goes awry.

Turns out the changes are not new.  They go back to at least WordPress 4.1.    The last time I dug into this issue was WordPress 3.8.   Guess it is time for some phpUnit testing on dbDelta within my own plugin suite testing.   If things are running properly the dbDelta command for an existing installation of Store Locator Plus 4.2 should yield NO QUERY UPDATES when activating/installing a newer version of the plugin.

WP 4.1 dbDelta Results

Source SQL:

dbDelta SQL: CREATE TABLE wp_store_locator ( sl_id mediumint(8) unsigned NOT NULL auto_increment, sl_store varchar(255) NULL, sl_address varchar(255) NULL, sl_address2 varchar(255) NULL, sl_city varchar(255) NULL, sl_state varchar(255) NULL, sl_zip varchar(255) NULL, sl_country varchar(255) NULL, sl_latitude varchar(255) NULL, sl_longitude varchar(255) NULL, sl_tags mediumtext NULL, sl_description text NULL, sl_email varchar(255) NULL, sl_url varchar(255) NULL, sl_hours varchar(255) NULL, sl_phone varchar(255) NULL, sl_fax varchar(255) NULL, sl_image varchar(255) NULL, sl_private varchar(1) NULL, sl_neat_title varchar(255) NULL, sl_linked_postid int NULL, sl_pages_url varchar(255) NULL, sl_pages_on varchar(1) NULL, sl_option_value longtext NULL, sl_lastupdated timestamp NOT NULL default current_timestamp, PRIMARY KEY (sl_id), KEY sl_store (sl_store), KEY sl_longitude (sl_longitude), KEY sl_latitude (sl_latitude) ) DEFAULT CHARACTER SET utf8

dbDelta Updates List:

QUERY: ALTER TABLE wp_store_locator CHANGE COLUMN sl_linked_postid sl_linked_postid int NULL
QUERY: ALTER TABLE wp_store_locator CHANGE COLUMN sl_lastupdated sl_lastupdated timestamp NOT NULL default current_timestamp
QUERY: ALTER TABLE wp_store_locator ADD PRIMARY KEY (sl_id)
QUERY: ALTER TABLE wp_store_locator ADD KEY sl_store (sl_store)
QUERY: ALTER TABLE wp_store_locator ADD KEY sl_longitude (sl_longitude)
QUERY: ALTER TABLE wp_store_locator ADD KEY sl_latitude (sl_latitude)
for update

Update Results:

[wp_store_locator.sl_linked_postid] => Changed type of wp_store_locator.sl_linked_postid from int(11) to int
[wp_store_locator.sl_lastupdated] => Changed type of wp_store_locator.sl_lastupdated from timestamp to
[0] => Added index wp_store_locator PRIMARY KEY (sl_id)
[1] => Added index wp_store_locator KEY sl_store (sl_store)
[2] => Added index wp_store_locator KEY sl_longitude (sl_longitude)
[3] => Added index wp_store_locator KEY sl_latitude (sl_latitude)

My WP 4.2 dbDelta Results

Better, but still not quite right.

Source SQL:

CREATE TABLE wp_store_locator ( sl_id mediumint(8) unsigned NOT NULL auto_increment, sl_store varchar(255) NULL, sl_address varchar(255) NULL, sl_address2 varchar(255) NULL, sl_city varchar(255) NULL, sl_state varchar(255) NULL, sl_zip varchar(255) NULL, sl_country varchar(255) NULL, sl_latitude varchar(255) NULL, sl_longitude varchar(255) NULL, sl_tags mediumtext NULL, sl_description text NULL, sl_email varchar(255) NULL, sl_url varchar(255) NULL, sl_hours varchar(255) NULL, sl_phone varchar(255) NULL, sl_fax varchar(255) NULL, sl_image varchar(255) NULL, sl_private varchar(1) NULL, sl_neat_title varchar(255) NULL, sl_linked_postid int NULL, sl_pages_url varchar(255) NULL, sl_pages_on varchar(1) NULL, sl_option_value longtext NULL, sl_lastupdated timestamp NOT NULL default current_timestamp, PRIMARY KEY (sl_id), KEY sl_store (sl_store), KEY sl_longitude (sl_longitude), KEY sl_latitude (sl_latitude) ) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci

dbDelta Updates List:

QUERY: ALTER TABLE wp_store_locator CHANGE COLUMN sl_linked_postid sl_linked_postid int NULL
QUERY: ALTER TABLE wp_store_locator CHANGE COLUMN sl_lastupdated sl_lastupdated timestamp NOT NULL default current_timestamp
QUERY: ALTER TABLE wp_store_locator ADD PRIMARY KEY (sl_id)

Update Results:

[wp_store_locator.sl_linked_postid] => Changed type of wp_store_locator.sl_linked_postid from int(11) to int
[wp_store_locator.sl_lastupdated] => Changed type of wp_store_locator.sl_lastupdated from timestamp to
[0] => Added index wp_store_locator PRIMARY KEY (sl_id)

WP 4.2 dbDelta SQL Best Practices

Some of these carry over from the prior dbDelta comments, some are new with later versions of dbDelta and MySQL.   The code is fairly robust, but if your MySQL installation comes back with a different format for the describe command this can wreak havoc on dbDelta().     I’ve presented things to look out for when crafting your dbDelta SQL statements as well as possible improvements in WP Core for dbDelta.    I may submit patches once again, but it has proven nearly impossible to get patches reviewed and approved for dbDelta.   Makes sense given the HUGE impact this function has on the WordPress Core engine, but there are thousands of plugins and themes wreaking havoc on WordPress performance due to some of these issues.    Maybe a detailed review like this will help get some updates into core.

Table Creation / Modification Commands

dbDelta() uses the MySQL DESCRIBE <table> command to fetch data about the table name passed in to dbDelta().  It is parsed with a regular expression out of the SQL statement looking for CREATE TABLE <…> , CREATE DATABASE <…> , INSERT INTO <..>, and UPDATE <…>  where the first “word” in the <…> part is assumed to be the table name.

Notes

  • The process appears to be case sensitive.  Use the commands as noted above in all caps.
  • Make sure you have no extra spaces, newlines, or other whitespace in between CREATE and TABLE or the other two-word directives noted here.
  • Make sure your table name has a leading space before and trailing space after.

Possible Improvements

  • Consider make thing preg_match() case insensitive.
  • Consider changing the single-space in commands like INSERT INTO into multi-whitespace regex i.e. INSERT\s+INTO

Defining Column Types

On some servers, such as my CentOS 7 development box, MySQL Ver 15.1 Distrib 5.5.41-MariaDB, has decided that post types of INT really mean INT(11).   Different variations may post different results back from the DESCRIBE command, but here is what I find working on MOST servers including my Linux dev box:

  • Always specify the LENGTH for each field type that accepts a length.  For most fields use the default length.
    • int(11)
    • mediumint(8)
    • varchar(1) to varchar(255), 255 is the default.
  • Fields without lengths:
    • longtext
    • mediumtext
    • text
  • Do not put spaces between the field type and the parens or the number in parens for the length.
  • User lowercase for field name and the field type.
  • Use uppercase for the NULL and NOT NULL specification.
  • Do NOT have extra spaces between the field name, the field type, and any of the additional definition parameters.

Examples

global $wpdb;
$sql =
'CREATE TABLE my_table (
my_id mediumint(8) unsigned NOT NULL auto_increment,
my_first_field varchar(1) NULL,
my_second_field varchar(255) NULL,
my_third_field int(11) NULL,
my_lastupdate timestamp NOT NULL default CURRENT_TIMESTAMP
)
DEFAULT CHARACTER SET ' . $wpdb->charset;

Things the cause problems in my setup:

  • Two spaces after the field name my_lastupdate and the word timestamp.
  • Using int instead of int(11) on my_third_field.
  • Using lowercase CURRENT_TIMESTAMP on my_lastupdate.

Possible Improvements

  • Consider make thing preg_match() case insensitive.
  • Make the comparison on the default case insensitive, or shift the inbound query uppercase.
  • Consider changing the single-space in commands into multi-whitespace regex tests.

 

Defining Keys (Indices)

The key comparison system is just as strict with whitespaces and case sensitivity as the column comparison routine in dbDelta.   You MUST have TWO spaces after the keywords “PRIMARY KEY” before defining your index field but only ONE SPACE after all other keys.

  •  ALWAYS declare the column names for every key type, including primary key.
  • ALWAYS put your column names inside parenthesis with no spaces.
  • PRIMARY KEY should NEVER have a key name.
  • PRIMARY KEY is always followed by two spaces then the parenthesis then the column name and closing parenthesis.
  • KEY is always followed by a single space, the column name, then a single space then parenthesis around the column name.
  • They keyword PRIMARY KEY must be uppercase.

Examples

global $wpdb;
$sql =
'CREATE TABLE my_table (
my_id mediumint(8) unsigned NOT NULL auto_increment,
my_first_field varchar(1) NULL,
my_second_field varchar(255) NULL,
my_third_field int(11) NULL,
my_lastupdate timestamp NOT NULL default CURRENT_TIMESTAMP,
PRIMARY KEY  (my_id),
KEY third_key (my_third_field)
)
DEFAULT CHARACTER SET ' . $wpdb->charset;

Things the cause problems in my setup:

  • Missing the second space after the PRIMARY KEY definition.

Possible Improvements

  • Consider changing the single-space in commands into multi-whitespace regex tests.
  • Make the indices incoming text for the PRIMARY and KEY keywords force to uppercase.
Posted on

Store Locator Plus Premium Updates Fix

The inline plugin update system was updated for Store Locator Plus today.   The new updates, which ship as part of the WordPress Development Kit plugin, resolve several issues that prevented some combinations of premium add-on packs from completing the update request processing.   The update fixes a problem where some add-on packs would not display the details of the change log for premium add-on packs.   The update also addresses and issue where some add-on packs would not complete the download and installation or would loop through a cycle of installing the currently installed version.

For users of the WordPress Development Kit plugin, the patches resolve issues where the readme.txt files for premium add-on packs are moved, deleted, or marked as unreadable by the readme processing system built into the plugin.  Users of the WordPress Development Kit should update their installation to resolve their plugin distribution and update system.

 

Posted on

Analyzing WordPress PHP Memory Consumption

XDebug Banner

This weekend I have been processing a large 200,000 location data file for a Store Locator Plus customer.   This is one of the larger files I have processed on my test system and it is the first file over 60,000 locations I’ve processed since Store Locator Plus 4.2 and WordPress 4.x have been released.    This large file processing and the geocoding required is taxing several systems in the Store Locator Plus hierarchy.  WordPress, Google OEM API calls, and the locator are all showing their weak spots with this volume of data processing.   They can all handle it to some degree, but maximizing efficiency is the key.

The temporary solution to most of the issues is to increase memory and process limits.   These are some of the key findings, as posted on the CSV Import documentation pages for Store Locator Plus:

Check your php.ini post_max_size setting if doing a direct file import versus a cron URL based import. post_max_size is typically set to 8 (MiB) on most servers.   This is typically enough for around 25,000 locations but it depends on how long your descriptions are and how many data fields you have filled out.   SLP 4.2.41 will warn you if you try to upload a file larger than your post_max_size limit.

Check your php.ini memory_limit setting and make sure it is large enough to handle the WordPress overhead plus the size of your CSV file times two.   The WordPress database interface and the CSV file processing will consume lots of memory.  The more plugins, widgets, and advanced theme features you have more more memory WordPress will use and the more PHP memory will leak over time. A setting of 256M is enough for approximately 15,000 locations.

Check your wp-config WP_MEMORY_LIMIT.   You may need to add this define to wp-config.php.  define(‘WP_MEMORY_LIMIT’ , ‘256M’).  The number needs to be equal-to or less-than the php.ini memory-limit.    It is the WordPress-specific memory limit and works with php.ini memory_limit.

Check your wp-config WP_MAX_MEMORY_LIMIT.   You may need to add this define to wp-config.php.  define(‘WP_MAX_MEMORY_LIMIT’ , ‘256M’).   This is the WordPress admin interface memory limit and works like WP_MEMORY_LIMIT for admin pages.

Set Duplicates Handling to Add especially if you know you do not have duplicate locations in your data.  SLP 4.2.41 further improves the performance when using ‘add’ mode by eliminating extra data reads from the database.

Set Server-To-Server speed to Fast under the General Settings tab unless you are on a shared host or experience a large number of uncoded locations during import.

Set the PHP Time Limit to 0 (unlimited) under the General Settings tab.   For hosting providers that allow your web apps to change this, the unlimited value will let the import run to completion.

Keep in mind Google limits you to 2500 latitude/longitude (geocoding) lookups per 24 hours per server IP address.  If you are on a shared host you share that limit with all other sites on that host.

However, even with all of these settings tweaked to fairly high values for my VirtualBox development system running on a MacBook Pro Retina host, the 4GB of RAM allocated to WordPress still is not enough.   The system eventually runs out of memory when the file gets close to the 45,000 location mark.  Luckily the “skip duplicate addresses” option allows the process to continue.    The “out of memory” error still rears its ugly head in the wpdb  WordPress database engine and is a problem for handling larger files.

Enter Xdebug and memory profiling.   Somewhere buried in the Store Locator Plus code, WordPress code, PHP MySQL interface, or PHP core engine there is a memory leak.  With a complex application environment finding the leak is going to be a monumental task.  It may not be something I can fix, but if I can mitigate the memory usage when processing large files that will help enterprise-class sites use Store Locator Plus with confidence.

Getting Xdebug On CentOS 7

If you follow my blog posts on development you will know that I run a self-contained WordPress development environment.  The system uses Vagrant to fire up a VirtualBox guest that runs CentOS 7 with GUI tools along with a full WordPress install including my plugin code.   This gives me a 2GB “box file” that I can ship around and have my full self-contained development environment on any system capable of running VirutalBox.   Here is how I get Xdebug connected to my local Apache server running WordPress.

Install xdebug from the yum install script.


# sudo yum install php-pecl-xdebug.x86_64

Turn on xdebug in the php.ini file


# find / -name xdebug.so

/usr/lib64/php/modules/xdebug.so

#sudo vim /etc/php.ini

zend_extension="/usr/lib64/php/modules/xdebug.so"

Check if xdebug is installed:


# php --version

... PHP 5.4.16
.... with xdebug v2.2.7

Enable some xdebug features by editing php.ini again.

Read about XDebug Profiling.

Read about XDebug Tracing.


# sudo vim /etc/php.ini

xdebug.default_enable=1  ; turns on xdebug any time a PHP page loads on this local server

xdebug.idekey="PHPSTORM" ; in case I turn on the automated listener for built-in PHP Storm debugging/tracing

xdebug.profiler_enable = 1 ; turn on the profiler which creates cachegrind files for stack trace/CPU execution analysis

xdebug.profiler_enable_trigger = 1;  turn on a cookie "hook" so third party browser plugins can turn the profiler on/off with a bookmark link

xdebug.profiler_output_dir = "/var/www/xdebug" ; make sure this directory is writable by apache and readable by your local user

xdebug.auto_trace = 1 ; when any page loads, enable the trace output for capturing memory data

xdebug.show_mem_delta = 1 ; this is what tells trace to trace memory consumption changes on each function call

xdebug.trace_output_dir = "/var/www/xdebug" ; same idea as the profiler output, this will be where trace txt files go

Restart the web server to get the php.ini settings in effect:


# sudo service httpd restart

At this point I can now open any WordPress page including the admin pages.   Shortly after the page has rendered the web server will finish the processing through xdebug and a trace* file will appear in /var/www/xdebug.   I can now see the stack trace of the functions that were called within WordPress with the memory consumption at each call.     This is the start of tracking down which processes are eating up RAM while loading a large CSV file without adding thousands of debugging output lines in the web app.

Be warned, if you are tracing large repetitive processes your trace file can be many GiB in size, make sure you have the disk space to run a full trace.

Posted on

Locator Updated With PHP Execution Time Control

Store Locator Plus 4.2.29 was released today with some minor enhancements that can make life easier for WordPress sites with large lists of locations or that are running on underpowered servers.

The latest release includes a new setting that allows a WordPress administrator to change the PHP Maximum Execution Time for scripts that are running, such as WordPress and the associated plugins.    The new setting makes it easier to override the standard limit set on my default PHP installations of 30 seconds per  a setting that is often defined in the inaccessible php.ini configuration file on the server.

The setting is not a ubiquitous option that impacts the entire WordPress environment.   The setting has been implemented in a way that makes it available to any WordPress plugin that wishes to access the Store Locator Plus options.    The primary use is to allow Store Locator Plus add-on packs to change the execution time allowed for certain operations.   The forthcoming updates to Pro Pack and Janitor, for instance, will leverage this new setting to allow more time for large location imports, exports, delete operations and related tasks to run.  The Pro Pack also employs the setting for downloading CSV reports of location searches-and-results.

The new setting honors security settings that may be employed by a server to prevent tampering with execution time.   Many shared hosting servers do not allow the PHP execution time to be changed “on the fly” to prevent one website on a server from taking over the entire server and effectively preventing all other sites on the same server from running effectively.     The now-defunct Safe Mode is one setting shared hosts may employ that can disable the Store Locator Plus execution time override.

The new setting ensures the larger location lists, those of 5000 or more locations, can be processed fully even on slower servers.     The setting defaults to 600 seconds and can be adjusted for any value from 1 second (not recommended) to infinite execution time by setting the value to 0.

Updates to the Pro Pack and Janitor will be published soon and take full advantage of the new setting.

Store Locator Plus Changelog

 

 

Posted on

Is phpUnit Good Enough For Testing WordPress?

I’ve been using Selenium IDE to test my Store Locator Plus plugin for the past year.   At nearly every WordCamp I attend I hear about using phpUnit to test “WordPress things”.   I heard about it at WordCamp Orlando during the contributor session; one of the first things Mark Jaquith asked me about a proposed patch to dbDelta was “is there a unit test for dbDelta?”.   If you Google “testing WordPress plugins” it is all about unit tests.

phpUnit is clearly the defacto standard for testing WordPress code.

My First Plugin phpUnit Test

Nine months after talking to Bryan Petty about phpUnit testing at WordCamp Atlanta, I finally got around to adding phpUnit tests to my plugin testing arsenal.   Since Mark provided a few hints on getting WordPress Core code pulled into my development environment I had the necessary tools to get started.    I created a new testing folder in my plugin assets directory, threw in the bootstrap, phpUnit XML directives config file, and added my first unit test class.

<?php

/**
* phpUnit Test for SLPlus
*
* @package PHPUnit\StoreLocatorPlus
* @author Lance Cleveland <lance@charlestonsw.com>
* @copyright 2014 Charleston Software Associates, LLC
*/
class SLPlusTest extends PHPUnit_Framework_TestCase {

/**
* @var \SLPlus
*/
public $slplus;

/**
* @var int
*/
public $location_id;

/**
* Setup the global pointer to the main SLPlus plugin object.
*/
function setUp() {
parent::setUp();
$this->slplus = $GLOBALS[‘slplus_plugin’];
}

/**
* Printout before this test suite runs.
*/
public static function setUpBeforeClass() {
print "\n".get_class()."\n";
}

/**
* To next line before next test suite runs.
*/
public static function tearDownAfterClass() {
print "\n";
}

/**
* Is the slplus variable an SLPlus object?
*/
function test_slplus_is_set() {
$this->assertInstanceOf( ‘SLPlus’, $this->slplus );
}
}

Fairly simplistic test. It told me that I was on the right path to getting phpUnit tests working with my plugins.

Putting The Test Into Production

I added more tests with more complexity and eventually added phpUnit tests to my WordPress Development Kit production system, a Grunt-based series of scripts I use on my WordPress Development vagrant boxes.

Nice! I now have a low-level code-based system for testing functionality without having to run everything through an IDE. Selenium IDE is great for testing but it is a lot slower for a coder to write test scripts and the logic is far more restrictive.

This is going to be great!

phpUnit Imperfection

Within a few days I found the first chink in the phpUnit armor.

In my Selenium IDE tests I have a series of tests that check to see if PHP warnings like “Notice: Undefined…” , “Fatal error:…” , and other calamities pop up in the browser while running the tests.   Since I am always running with “DEBUG mode full on” on my development boxes, these errors pop up any time I fat-finger the code or have a brain-fart and forget to quote an array index.

While refining my Selenium IDE scripts, by adding rollups to increase consistency among the tests, I purposefully added a common coding error by typing asdf at the top of my base plugin class.    I re-run my Selenium IDE scripts.  They catch the error, exactly as they are supposed to.

Selenium Catches PHP Notices
With the right test command, Selenium makes PHP warnings and notices obvious.

I start wondering… “I wonder what phpUnit will do, I bet it is much faster at catching this sort of thing”.

NOPE.

phpUnit spit out a warning at the very top of the test, which quickly scrolled out of view.   At the end of a long test series I see all my tests passed.    If I didn’t scroll back to the start of the test series I’d never have seen the warnings and notices that were buried in the startup output from the phpUnit testing.

phpUnit Notices Not Errors
phpUnit does not make notices blatantly obvious.

For my personal workflow, where I am constantly multitasking, having those PHP warnings and notices force a proper phpUnit test ERROR would be a far better option.   In “go mode” I am only going to look at the final summary line.   OK (5 tests, 14 assertions)  DONE, WITHOUT ERRORS. If there is a notice or warning I want to see something like “DONE, WITHOUT ERRORS. LOTS OF NOTICES. LOTS AND LOTS OF NOTICES. FIX THEM!!!!”. Maybe not exactly like that, but you get the idea.

True, these were not errors, but a blatant PHP notice spitting out the the screen is not within my code standards.  Yes, it goes away if you do not run in debug mode, but in my world plugin code should not be generating errors, warnings, OR NOTICES.   Not only do they indicate a deeper issue with possible unforeseen ramifications but it often fills up log files, which means writing to disk, which directly affects performance of the WordPress system as a whole.

Doing It Wrong?

Since I am new to phpUnit I may be doing it wrong.

My phpunit.xml configuration:

<phpunit
        bootstrap="bootstrap.php"
        backupGlobals="false"
        colors="true"
        convertErrorsToExceptions="true"
        convertNoticesToExceptions="true"
        convertWarningsToExceptions="true"
        >
    <testsuites>
        <testsuite>
            <directory prefix="" suffix="Test.php">./</directory>
        </testsuite>
    </testsuites>
</phpunit>

I even tried this trick to force errors thanks to a hint from Bryan.

The default phpUnit setup and the plugin testing examples online indicate I have the correct setup in place.   My guess is other “phpUnit first timers” will end up with a similar configuration.  A configuration that does a less-than-stellar job of catching PHP warnings and notices.

Selenium IDE Required

Selenium IDE, on the other hand, makes PHP warnings, notices, and errors blatantly obvious if you test for them.   If you are running with debug logging enabled, which you ALWAYS ARE on your dev box because you are a coder that gives a damn, Selenium will flag the output when it is rendered on your WordPress pages.   Selenium highlights the errors in red and reports them in the test summary as failures.   This is something I see even in “go mode”.

Bottom line, I feel as though you must have both phpUnit and Selenium IDE (or Selenium Server) to properly test a WordPress code implementation of any kind.   Not too mention it is nearly impossible, as far as I know, to test advanced JavaScript and AJAX functionality with phpUnit.

Now I am curious:  Does WordPress Core have anything in place to perform similar user-interface testing?  Are they running anything like Selenium IDE to ensure the interface renders as expected?  The CSS is correct?  AJAX calls are running and producing front-end results?   Can phpUnit catch front-end output like errant “Notice: use  of undefined constant” messages when running in debug mode?

I’m sure some of you know a LOT more about this stuff than I do.   Share you knowledge in the comments!

If you want to know more about my Selenium IDE scripts, Grunt production helpers, and Vagrant boxes ping me here as well.   Everything I do is open source and freely available to the WordPress development community.

Posted on

Selenium IDE Includes AKA “Rollups”

Selenium IDE is used extensively to test the Store Locator Plus family of plugins.    As the testing continue to expand so do the rules being tested.   Today the addition of a rule that checks that the PHP notice “use of undefined constant” needed to be added.     There is an existing set of other PHP warnings and notices that was added last month that is in 30+ Selenium Scripts.  They were all copied by hand into each test.

Now I need to add one more rule to that test set.   Copy and paste into 30 files?  There HAS to be a better way.

Turns out there is a better way, but it requires a little JavaScript coding to make it happen.   Selenium IDE does not have an “include <file>” option in the base set of commands.   Personally I think they need to add it to the base command set as it will make it far easier for people to write “test subsets” and then include them in every test.    The solution is using something called a “rollup”.    A rollup is a label for a group of commands you want to execute in many places in your test scripts.

My original test suites looked something like this:

Selenium Test for PHP Warnings
Selenium Test for PHP Warnings

In the “old days”, yesterday to be precise, I would copy-and-paste this set of Selenium IDE tests into a LOT of test files.

Today I decided to be smart about it, happens every now-and-then… must be a full moon or something, and find out how to to an “include” of those commands.   The trick is to create a file called something like “rollups.js”.     My file is called slp_rollups.js.     It is a standard JavaScript file that I place in the same directory as all of my Selenium IDE test scripts (which happen to be nothing more than HTML snippets).

To replace those 4 commands with a rollup I created this slp_rollup.js file:

/**
 * For use in Selenium IDE.
 *
 * You will need to add this to Selenium under Options / Options in the Selenium Menu.
 * Put this under Selenium Core Extensions:
 * ~/selenium-ide-for-slp/sideflow.js , ~/selenium-ide-for-slp/slp_rollups.js
 *
 *
 */
var manager = new RollupManager();


/**
 * check_for_syntax_errors
 *
 * This rollup tests for php syntax errors, warnings, and notices.
 */
manager.addRollupRule({
    name: 'check_for_syntax_errors',
    description: 'Check for PHP syntax errors, notices, warnings.',
    args: [],
    commandMatchers: [],
    getExpandedCommands: function(args) {
        var commands = [];

        commands.push({
            command: 'assertNotText',
            target: '//body',
            value: '*Notice: Undefined*'
        });

        commands.push({
            command: 'assertNotText',
            target: '//body',
            value: '**Notice: Trying to get*'
        });

        commands.push({
            command: 'assertNotText',
            target: '//body',
            value: '*Notice: Use of*'
        });

        commands.push({
            command: 'assertNotText',
            target: '//body',
            value: '*Fatal error:*'
        });

        return commands;
    }
});

To activate this I update Selenium IDE by going to Options/Options and adding this new slp_rollup.js file to the Selenium Core Extensions. Since I also use the sideflow.js file to provide Go To / If and other looping constructs I add both sideflow.js and slp_rollups.js to my extensions list by separating the file names with a comma.

Selenium IDE Options
Selenium IDE Options with sideflow and slp_rollups enabled.

Now I can replace that block of 4 commands with the following single command in ALL 30 scripts. The best part is the next time I need to add another test for a new warning or error I only edit ONE file, the slp_rollups.js file which means less editing, less copy & paste, and less commits to the git repository.

Selenium IDE Implementing Rollups
Selenium IDE Implementing Rollups
Posted on

Store Locator Plus 13-Dec-2014 Status Report

CSA Slack Banner

Until I figure out a better way to get frequent update reports out of the new Slack Channel into a WordPress post (yes, it can be done with a plugin and an API but I’m sort of busy with a bazillion Store Locator Plus updates at the moment) I’m doing it manually. Here is the past couple of days of work.


Yesterday December 12th, 2014 -----
BitBucket [12:10 PM] 
[Store Locator Plus/prerelease] 00ba08a709e6: Fix the single quotes problem in the WP 4.0.1 render engine. Broke onChange for drop down menus.
- Lance Cleveland
incoming-webhook [12:25 PM] 
New bbPress Update
/forums/topic/bulk-upload-kml-from-google/#post-26553
Can you post a KML file example. That capability is not built-in but could be added to the Pro Pack on a future build if the data is easy to parse.
BitBucket [12:33 PM] 
[Store Locator Plus/prerelease] 611335cf233e: Note the quotes fix in the readme.
- Lance Cleveland
BitBucket [12:33 PM]
[Store Locator Plus/] https://bitbucket.org/lance_cleveland/store-locator-plus/commits/: -
lance.cleveland [12:35 PM] 
Just put SLP 4.2.18 into prerelease. Turns out a WP 4.0.X patch broke the drop down menu processors due to a change in how pages render single quotes. So much for WP being fully backward compatible. A lot of plugins have broken including CVG, my video manager on the CSA site!
incoming-webhook [12:43 PM] 
New bbPress Update
/forums/topic/add-zipcodestate-field/#post-26557
Enhanced Search allows for a state drop down selector that can be an input box or drop down selector.
incoming-webhook [2:12 PM] 
New bbPress Update
/forums/topic/filter-by-category-in-the-administrative-interface-not-working/#post-26561
Yes the selector box shows up and I am able to select the categories. But after clicking filter the results still show both categories.
----- Today December 13th, 2014 -----
incoming-webhook [2:14 PM] 
New bbPress Update
/forums/topic/pro_tags-in-bubbles/#post-26569
Under User Experience / Results set your "show tags in output" to something other than hidden. For backwards compatibility is defaults to "hidden".
lance.cleveland [3:05 PM] 
Finally - Pro Pack immediate CSV imports works when a scheduled import is present. What a PAIN.
lance.cleveland [3:05 PM]
Now to make some more robust testing for SLP 4.2.18 and PRO 4.2.03 , re-run tests on both, release SLP 4.2.18, and publish both products by Monday. Fingers crossed!
Posted on

Replicating WordPress Dashboard Boxes

WordPress Code Reference Banner

I spent the past few hours of coding trying to replicate a WordPress admin UI element while working on an update to the Pro Pack for Store Locator Plus.   The cron process runs in the background making the details of what is going on “hidden” from the user.   I realized I needed a status output on the Pro Pack Location Import interface and quickly decided that the WordPress dashboard status notices would be perfect.   I didn’t want to clutter the main dashboard though and thought, “I’ll just throw it on the Locations / Import page!”.

Easier said than done.  After hours or toying with the UI I finally found the right formula.  I’m sharing the notes here for other WordPress plugin and theme developers as well as to help me remember what the hell I did when I forget all about this about 30 seconds from now.

Step 1 : HTML Construct

The HTML construct of your admin panel needs to be correct.   The basic outline requires a specific order of divs with specific class names.  Without this the CSS styling of WordPress will not get employed and some of the JavaScript rules that handle the UI toggle states will not execute.    What I found was this basic HTML structure must wrap your content:


            // Opening Divs
            //
            $html =
                '<div class="metabox-holder">' .
                '<div class="meta-box-sortables">' .
                '<div id="location_import_cron_messages" class="postbox">'                      .
                    '<div class="handlediv" title="Click to toggle"><br></br></div>'    .
                    "<h3 class='hndle'>{$box_title}"                              .
                    '<div class="inside">'
            ;

            $html .= $this->createstring_CronSchedule();

            $html .= $this->createstring_CronMessages();

            // Closing Divs
            //
            $html .=
                   '</div>' .
                   '</div>' .
                   '</div>' .
                '</div>'
                ;

Step 2 : Enqueue The WordPress “Magic Sauce”

Rather than recreate the jQuery coding the make things work I decided to build on what WordPress Core is doing. The danger is that this is not a documented plugin/theme API element which means WordPress Core can change this at any time and break the way it works. This has been a stable element for a few years now so I’ll take my chances. When loading the locations tab in Store Locator Plus I only need to enqueue the dashboard script.

                case SLP_ADMIN_PAGEPRE . 'slp_manage_locations':
                    $styleHandle = 'slp_pro_style';
                    wp_enqueue_script('dashboard');
                    break;

Done!

Easier explained than discovered. It took some code searching followed by a lot of HTML document deconstruction (thank you Mozilla Firefox Web Developer Edition!) and a few incorrect guesses. Now I have a WordPress-styled info box in my admin page where the expand/collapse toggle actually works just like the “real thing.

Location Import Dashboard Box Collapsed
Location Import Dashboard Box Collapsed
Location Import Dashboard Box Open
Location Import Dashboard Box Open