Posted on

Improved Grunt Tasks for the Vagrant WordPress Dev Box

Grunt WordPress Dev Kit

Last week I found myself having to rebuild my WordPress plugin development box after a “laptop fiasco”.   While it was a lot of work it feels as though I am in a better position to not only recover my environment quickly but also distribute it to other developers that are interested in assisting with plugin development.

If you are interested you can read more about it in the related WordPress Workflow and related WordPress Development Kit articles.

This morning I realized that having a new almost-fully-configured Vagrant box for my WordPress Development Kit allows me to make assumptions in my Grunt tasks.    While it would be more flexible to create options-based tasks where users can set their own configuration for things like MySQL usernames and passwords, the WP Dev Kit Vagrant box assumption allows me to bypass that for now and come back to it when time allows.  Fast turnaround and fewer interruptions in my already-busy work flow is paramount this week.

Today’s WordPress Dev Kit Updates

The official tag I’ve assigned to the newest WordPress Dev Kit is version 0.5.0.  Here is what has been added.

WordPress Database Reset

One of the tasks I do fairly often is to “clear the data cruft” from my development box WordPress tables.  I  accomplish this by dropping the WordPress database and recreating it.

The Vagrant box makes this far easier as I know that when I spin up the WP Dev Kit Vagrant box it already has the WordPress MySQL tables setup.  I also know the username and password.  As such I can execute a simple drop/create table as the privileges are already in place in the meta data for MySQL and will carry over.   Thus I only need to execute a single mysql-cli command to get the data reset.

To get this working in Grunt I added the grunt-ssh module and created a ‘resetdb’ target.

I can now reset my WordPress table with a simple grunt command:


$ grunt shell:resetdb

Online Documentation

The other change I made today will help me remember how the heck all this stuff works.  Now that the dev kit has grown to a couple of commands I know I will soon be forgetting the nuances to certain build and workflow processes.   I started creating my own Markdown files I realized that Bitbucket has a system for using .md files on the repository wiki.    The easy solution was to add the Bitbucket wiki as a submodule to the WP Dev Kit repository and edit the file there.    Doing so means that any doc update will also be published immediately when pushed back to the repo at the WP Dev Kit Bitbucket Wiki.

Now back to getting the Store Locator Plus and Enhanced Results first-pass testing run and prerelease copies published for my Premier Members.

Posted on

Installing Sass on CentOS 6.5

SLP Sass Banner

I just discovered that Sass is missing from my WordPress Development Kit Vagrant box.   My Vagrant box is on the latest CentOS 6.5 release and, luckily, setting up Sass is very simple for vanilla CentOS 6.5 users.  It is literally a two-step process:

$ sudo yum install rubygems
$ sudo gem install sass

Now I can add this to the NetBeans executables configuration by adding the path /usr/bin/sass.

Configuring Sass in NetBeans
Configuring Sass in NetBeans

Now I can edit my .scss files and have the corresponding .css files auto-generated right from NetBeans.  Nice!

Posted on

WordPress Development Fresh Start with Vagrant and Grunt

Banner Vagrant Cloud WP Dev Kit Box

HP finally did it.  They came out to replace a system board in my laptop and half of my files on my drive are corrupted.  Restoring the 70GB of corrupt files from the local USB3 backup will take some time and I am not 100% confident in the reliability of this system.   With 48 hours of downtime I decided it would be best to push forward with my Vagrant and Grunt setup for the WordPress Development Kit I’ve been working on.

Employing Vagrant

I am using VirutalBox and Vagrant to build a series of base boxes on which to layer my development environment.  Unlike most base boxes that are plain vanilla OS installs that then use provisioners to install all the software layers on top of the OS, my custom base boxes are going to be installed with all the base tools I need to develop WordPress plugins using my WordPress Development Kit tools.

Why?

Because it is far faster to download a compressed virtual machine image with most of my stuff installed  than to download a very similar image with no tools then have Vagrant go and download  a few dozen install kits.   Sure, the standard method is more flexible and ensures everything is up-to-date, but that is not what I am trying to accomplish here.    I am building a “fast start” box that has most of what you need pre-installed in a known environment.

I also want to have a method to deploy new virtual boxes with everything in place on my OS/X system, a different laptop, or even a cloud-based server the next time my HP laptop is smoked.  Which will probably be tomorrow.      As I’ve found, restoring a large image even from a local USB3 drive is not a quick process and it is not foolproof.   Especially when going from a Windows 8.1 based backup and restoring on an OS/X system that has not been patched or updated in 8 months.

How…

Since I already have Vagrant installed and have a published base box I am able to get started quickly.   Once Vagrant is installed I only need to set my Vagrantfile, a script that tells Vagrant what to do, to pull down the base box from the URL I have published on Vagrant Cloud:

  • Create a folder on my host system named “C65 WP Devkit  Box Setup”.
  • Create the Vagrantfile, in that directory with the config.vm.box pointing to charlestonsw/centos6.5-wordpress-dev-kit-base-box.
  • Open the command line for Windows and go to that C65 WP Devkit Box Setup directory.
  • Run the vagrant up command.
My new Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don’t touch unless you know what you’re doing!
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "charlestonsw/centos6.5-wordpress-dev-kit-base-box"
config.vm.provider "virtualbox" do |vb|
vb.gui = true
end
end

What…

When you boot the WordPress Dev Kit Base Box with Vagrant you will find that the full CentOS GUI is active. You can login to the GUI using the standard vagrant username with a password of vagrant.

WordPress 3.9

You will find a Firefox icon on the menu bar. Opening Firefox will open the http://localhost/ URL which brings up the local copy of WordPress 3.9 that has been installed.

MySQL for WordPress

The MySQL database has been configured for WordPress.

Database name: wordpress
MySQL User: vagrant
MySQL Password: vagrant

NetBeans 8.0

NetBeans 8.0 has been installed with the PHP and HTML 5 support packages.   NetBeans is a great no-cost IDE that I have found to be very useful for editing structured WordPress PHP code.   Write your code using an object oriented design and add a good amount of phpDoc commenting and NetBeans becomes a coding power tool for WordPress plugin coding.

Firefox

Firefox is installed along with several add-on packs that I use for every day code management and development.  It includes Firebug to look at JavaScript, CSS, and HTML construction.   Selenium IDE is included which allows me to download and execute my web-interface based test scripts for Store Locator Plus.   LastPass is installed to provide unique long-string passwords to all of my web services.

WordPress Development Kit

My command line WordPress Development Kit is installed on the system under the vagrant user home directory in ~/wp-dev-kit. This comes with the basic Grunt build tasks I use to manage my plugins. While you can edit the ~/wp-dev-kit/grunt/plugins.json file and configure this for your site it is recommended that you create a fork of my WordPress Development Kit on the Bitbucket site and work from your forked repository. I would remove the wp-dev-kit directory with rm -rf and clone your forked copy into the same location.

The easiest method for cloning a public repository is to use git clone with the https repository path. If you have a private repository you may want to create a SSH key pair by going to ~/.ssh and running ssh-keygen. The key will need to be added as an authorized SSH key in your Bitbucket account access management list.  Since I will be pushing and pulling content from my various Bitbucket git repositories I will use this method when setting up my clone of the WP Dev Kit Basic Box.

Bitbucket HTTPS Path
Bitbucket HTTPS Path

Similar methods can be employed with Github repositories.

Preparing For Development

These are elements that will eventually go into a provisioner setup for the Vagrant box, assuming that at least one of the Vagrant provisioning services can hand user prompts and communication with third party services.

Create Your SSH Key

This will make it easier to push & pull content from your repositories.

cd ~/.ssh
ssh-keygen
xclip -sel clip < ~/.ssh/id_rsa.pub
Setting A Vagrant SSH Key
Setting A Vagrant SSH Key

Login to your Bitbucket account, go to my account / access management, add the copied SSH key.

Bitbucket Add Vagrant Key
Bitbucket Add Vagrant Key

 Configure Your Git ID

git config –global user.name “Lance Cleveland”
git config –global user.email lance@thisdomain.com

Add SmartGit

I like the GUI interface that SmartGit provides over the git command line and gitk branch rendering.  I find SmartGit to be twice as efficient for my method of work flow over the built-in IDE and command line, so I always install this next and clone my base projects like the WP Dev Kit and my Selenium IDE scripts.   Today I am using the SmartGit 6 beta release as I find the project grouping and new interface design to be a big help in managing my projects.

SmartGit UI
SmartGit UI

I immediately setup SmartGit and clone my Selenium IDE repository so I can complete the next step with a script.

SmartGit Bitbucket Setup
SmartGit Bitbucket Setup

Complete The WordPress Install

Open Firefox and go to http://localhost/

Enter the WordPress local site information for your test site.  I use my Selenium IDE new site install script to handle this for me.

Selenium IDE setting up a  new WordPress install.
Selenium IDE setting up a new WordPress install.

Ready To Go

Now my system is ready to go.   I can start cloning my plugin code repositories into the ./wp-content/plugins directory, edit code with NetBeans, commit changes with SmartGit, and publish to my server or the WordPress plugin directory using my Grunt scripts.

With the current configuration it takes me 15 minutes to pull down a new clone and get the box booted then 5 minutes to install and configure SmartGit, clone my repository, run the WordPress install script, and fetch my first plugin repo.   20 minutes from “nothingness” to writing code.    With a local .box file on my system that time is cut down to about 8 minutes by avoiding the 1.5GB download of the box.

Not bad.

Now on to the code…

Posted on

WordPress Development Kit Plugin Released

Grunt WordPress Dev Kit

The WordPress Development Kit plugin works hand-in-hand with the WordPress Development Kit system to assist in publishing public and private WordPress plugins.   I am using this system of Grunt tasks to manage the free plugins listed in the WordPress Plugin Directory as well as the premium add-on packs.

The WordPress plugin that goes along with this system communicates with the Grunt metadata files to present the latest plugin information on the website.   The plugin is now in charge of keeping the version page updated with the latest production release information.   In the version 0.4.0 release, that was published to the WordPress Plugin Directory today, a file list-and-download user interface is also available.

The 0.4.0 release now has the following base features:

  • List all the WP Dev Kit managed plugins in a formatted HTML output.
  • Filter the plugins listed to show only the prerelease or production listing.
  • Create a formatted list of downloadable files with version information with alt and title hover for file size and slug.
  • Create a detailed listing, including the changelog for plugins managed by the WP Dev Kit that have been published with readme data.

You can see the plugin in use on the Premier Subscription downloads page, the version info page, and some plugin details pages.  This update will help bring clarity to the prerelease and production versions that are available for download for all Premier Members.   Premier Members can select the Products/Download menu item on the top of this page to see the latest list of plugins that are available.

The plugin and the WP Dev Kit Grunt-based system are both available via the my Bitbucket repository as public open-source projects.

 

Screenshots

WordPress Dev Kit 0.4.0 Download List
WordPress Dev Kit 0.4.0 Download List
Posted on

Improving WordPress Plugin Development with Sass

SLP Sass Banner

If you’ve been following along since my WordCamp Atlanta trip this spring you know that I’ve been working on automating my WordPress plugin development and production process.    If you missed it you can read about it in the WordPress Workflow and WordPress Development Kit articles.   Since I needed to patch some basic CSS rules in my Store Locator Plus themes, these are plugin “sub-themes” that style the store locator interface within a page, I decided now was the time to leverage Sass.

Sass Is In The House

It was one of my first sessions at WordCamp Atlanta and I KNEW it was going to be part of my automation process.    Sass is a CSS pre-processor.   Store Locator Plus has its own “theme system”, a sort of plugin sub-theme that lives within the WordPress over-arching site theme.     The SLP themes allow users to tweak the CSS that renders the search form, map, and results of location searches to create in-page layouts that better fit within their WordPress theme layout.

Until this past release it was a very tedious process to update themes or create a new theme.    In the latest release there are some 30-odd SLP theme files.    The problem is that when I find an over-arching CSS issue, like the update to Google Maps images that rendered incorrectly on virtually EVERY WordPress Theme in existence, it was a HUGE PAIN.   I was literally editing 30 files and hoping my cut-and-paste job went well.   Yes, I could have done CSS include statements but that slows things down by making multiple server requests to fetch each included CSS file.   Since the store locator is the most-visited page on many retail sites performance cannot be a secondary consideration.   Sass deals with that issue for me and brings some other benefits with it.

There are PLENTY of articles that describe how to install Sass, so I am not going to get into those details here.  On CentOS it was a simple matter of doing a yum install of ruby and ruby gems and a few other things that are required for Sass to operate.  Google can help you here.

My Sass Lives Here…

For my current Sass setup I am letting NetBeans take care of the pre-compiling for me.    It has a quick setup that, once you have Sass and related ruby gems installed, will automatically regenerate the production css files for you whenever you edit a mixin, include, or base SCSS file.

NetBeans Sass Setup
NetBeans Sass Setup

I combine this with the fact that the assets directory is ignored by the WP Dev Kit publication and build tasks to create a simple production environment for my CSS files.   I store my SCSS files in the ./assets/stylesheets directory for my plugin.   I put any includes or mixin files in a ./assets/stylesheets/include subdirectory.     I configure NetBeans to process any SCSS changes and write out the production CSS files to the plugin /css directory.

The first thing I did was copy over a few of my .css files to the new stylesheets directory and changed the extension to .scss as I prepared to start building my Sass rules.

Includes

I then ripped out the repeated “image fix” rules that existed in EVERY .css file and created a new ./stylesheets/include/_map_fix.scss file.     This _map_fix file would now become part of EVERY css file that goes into production by adding the line @include ‘include/_map_fix” at the top of the SLP theme .scss files.    Why is this better?   In the past, when Google has made changes or WordPress has made changes, I had to edit 30+ files.  Now I can edit one file if a map image rule is changing that has to be propagated to all of the css files.   However, unlike standard CSS includes Sass will preprocess the includes and create a SINGLE CSS file.   That means the production server makes ONE file request instead of two.  It is faster.

SLP Map Fix Include
SLP Map Fix Include

As I reiterated this process I ended up with a half-dozen CSS rules that appear in MOST of my CSS files.    Since all of the rules do not appear in all of my plugin theme files I ended up with a half-dozen separate _this_or_that scss files that could be included in a mix-and-match style to get the right rule set for each theme.     I also created a new _slp_defaults include file that does nothing more than include all of those half-dozen rules.  Nearly half of the current CSS files use all of rules that were “boiled out of” the CSS files.

Store Locator Plus Default Includes
Store Locator Plus Default Includes

Mixins

Along the way I learned about mixins.   At first I was a bit confused as to the difference between include files and mixins.  Both are “pulled in” using similar commands in SCSS, @import for the “include files” and @include for the mixins, but what was the difference?    While you can likely get away with “faking it” and having mixins act like includes they serve different purposes.   I like to think of a mixin as a “short snippet of a CSS rule”.

A common example is a “set the border style mixin”.  In the simplest form it can set the border style with a rule for each of the browser variants.  This rule is not a complete CSS rule but rather a portion of a CSS rule that may do other styling AND set a border.    The mixin includes the -moz and other special rule sets to accomodate each browser.   Rather than clutter up a CSS entry with a bunch of border-radius settings, use a mixin and get something like:

.mystyle {
   @include mixin_border_radius;
   color: blue;
}

That is a very simplistic way of using a mixin. One advantage is that if you decide to change the default border radius settings in all of your CSS files you can edit a single mixin file. However that is not a typical use. Yes, you can create subsets of CSS rules, but it really gets better when you add parameters.

At a higher level a mixin is more than just a “CSS rule snippet”. It becomes more like a custom PHP function. In typical coder fashion, I snarfed this box-shadow mixin somewhere along the way:

// _csa_mixins.scss

@mixin box-shadow( $horiz : .5em , $vert : .5em , $blur : 0px , $spread : 0px , $color : #000000 ){
    -webkit-box-shadow: $horiz $vert $blur $spread $color;
    -moz-box-shadow: $horiz $vert $blur $spread $color;
    box-shadow: $horiz $vert $blur $spread $color;
}
@mixin csa_default_map_tagline {
    color: #B4B4B4;
    text-align: right;
    font-size: 0.7em;
}
@mixin csa_ellipsis {
  overflow: hidden;
  text-overflow: ellipsis;
  white-space: nowrap;
}

Since that rule is part of my default _csa_mixins that I tend to use in multiple places I use it as follows:


@import 'include/_csa_mixins';
@import 'include/_slp_defaults';

//blah blah boring stuff here omitted

// Results : Entries (entire locations)
.results_entry {
    padding: 0.3em;
    @include box-shadow(2px, 4px, 4px, 0px, #DADADA);
    border-bottom: 1px solid #DDDDDD;
    border-right: 1px solid #EEEEEE;
}

Notice how I now call in the include with parameters. This is passed to the mixin and the Sass preprocessor calculates the final rules to put in the CSS file. This makes the mixin very flexible. I can create all sorts of different box shadow rules in my CSS files and have the cross-browser CSS generated for me. No more editing a dozen box shadow entries every time I want to change a shadow offset.

Here is what comes out in the final production CSS when using the above mixin. You can see where the parameters are dropped into the .results_entry CSS rule:

.results_entry {
  padding: 0.3em;
  -webkit-box-shadow: 2px 4px 4px 0px #dadada;
  -moz-box-shadow: 2px 4px 4px 0px #dadada;
  box-shadow: 2px 4px 4px 0px #dadada;
  border-bottom: 1px solid #DDDDDD;
  border-right: 1px solid #EEEEEE; }

This is only a start of my journey with Sass and I’ve barely scratched the surface. However I can already see the benefits that are going to come from using Sass. In fact I already used it to fix a problem with cascading menus where one of the SLP theme files did not contain a rule set. Rather than copy-paste from another theme file that contained the proper rules I only needed to add @import ‘include/_slp_tagalong_defaults’ and the problem was fixed.

Going forward Sass will not only increase my throughput in CSS development but also improve the quality of the final product that reaches the customer.

My first SLP Theme file, Twenty Fourteen 01, that was built using these new Sass files is only a 136 line CSS file with LOTS of whitespace and comments.  When the final processing is finished it has all of the rules necessary to support the current add-on packs and style them nicely for the WordPress Twenty Fourteen theme in all major browsers.

A new SLP Theme: Twenty Fourteen Rev 01
A new SLP Theme: Twenty Fourteen Rev 01

 

Posted on

WordPress Dev Kit : Grunt 0.3.0 and Plugin 0.0.2

Grunt WordPress Dev Kit

More refinements have been made this week to my WordPress Workflow and related WordPress Development Kit.  With new products going into production shortly and some older products coming out with new releases, I realized I needed a more efficient way to publish prerelease copies.  As part of the Premier membership program I am trying to get stable prerelease products in the hands of those Premier members that want them.   Some members like to test new releases or try out new features on their test systems before they come out.    It allows them to plan for future updates and provides an opportunity for feedback and updates before the new version is released.  A win-win for the Premier member and for Charleston Software Associates.

In order to support a formal prerelease and production configuration I realized I needed to be able to track two different versions and release dates separately.   Following the general format presented in other Grunt examples, this meant setting up new sub-sections within the plugins.json pluginMeta structure.   The new format looks something like this:

"wordpress-dev-kit-plugin": {
"production": {
"new_version": "0.0.02",
"last_updated": "2014-04-03"
},
"prerelease": {
"new_version": "0.0.03",
"last_updated": "2014-04-04"
},
"publishto" : "myserver"
}

Changing the structure meant updating both the Gruntfile.js for the development kit as well as the parser that is in the WordPress Development Kit plugin. The changes were relatively minor to address this particular issue, but I did learn some other things along the way.

Tasks and Targets

In my own Grunt tasks I had been calling one of my parameters in my build sequence the “type”, as in the “build type”. However the configuration file examples online often talk about a “target”. A target would be something like “production” or “prerelease” that shows up in a configuration block like this one:

// sftp
//
sftp: {
options: {
host: ‘<%= myServer.host %>’,
username: ‘<%= myServer.username %>’,
privateKey: ‘<%= myServer.privateKey %>’,
passphrase: ‘<%= myServer.passphrase %>’,
path: ‘<%= myServer.path %><%= grunt.task.current.target %>/’,
srcBasePath: "../public/<%= grunt.task.current.target %>/",
showProgress: true
},
production: { expand: true, cwd: "../public/<%= grunt.task.current.target %>/", src: ["<%= currentPlugin.zipbase %>.zip","plugins.json"] },
prerelease: { expand: true, cwd: "../public/<%= grunt.task.current.target %>/", src: ["<%= currentPlugin.zipbase %>.zip","plugins.json"] }
},

I have updated my scripts and documentation terminology to refer to this parameter as the “target” to follow convention.

Simplify Congiguration With grunt.task.current.target

I learned a new trick that helps condense my task configuration options. In one of my interim builds of the WordPress Dev Kit I had something that looked more like this:

// sftp
//
sftp: {
options: {
host: ‘<%= myServer.host %>’,
username: ‘<%= myServer.username %>’,
privateKey: ‘<%= myServer.privateKey %>’,
passphrase: ‘<%= myServer.passphrase %>’,
showProgress: true
},
production: {
expand: true,
cwd: "../public/<%= grunt.task.current.target %>/",
src: ["<%= currentPlugin.zipbase %>.zip","plugins.json"]
path: ‘<%= myServer.path %>production/’,
srcBasePath: "../public/production/",
},
prerelease: {
expand: true,
cwd: "../public/<%= grunt.task.current.target %>/",
src: ["<%= currentPlugin.zipbase %>.zip","plugins.json"]
path: ‘<%= myServer.path %>prerelease/’,
srcBasePath: "../public/prerelease/",
},
},

A bit repetitive, right? I found you can use the variable grunt.task.current.target to drop the current task name as a string into a configuration directive:

// sftp
//
sftp: {
options: {
host: ‘<%= myServer.host %>’,
username: ‘<%= myServer.username %>’,
privateKey: ‘<%= myServer.privateKey %>’,
passphrase: ‘<%= myServer.passphrase %>’,
showProgress: true
},
production: {
expand: true,
cwd: "../public/<%= grunt.task.current.target %>/",
src: ["<%= currentPlugin.zipbase %>.zip","plugins.json"]
path: ‘<%= myServer.path %><%= grunk.task.current.target %>/’,
srcBasePath: "../public/<%= grunk.task.current.target %>/",
},
prerelease: {
expand: true,
cwd: "../public/<%= grunt.task.current.target %>/",
src: ["<%= currentPlugin.zipbase %>.zip","plugins.json"]
path: ‘<%= myServer.path %><%= grunk.task.current.target %>/’,
srcBasePath: "../public/<%= grunk.task.current.target %>/",
},
},

Now that the prerelease and production path and scrBasePath variables are identical they can be moved into the top options section.

Now if I can just figure out how to have shared FILE configurations which define the current working directory (cwd), source (src), and destination (dest) file sets I could eliminate ALL of the settings in the production and prerelease configuration blocks and leave them with a simple “use the defaults setup like this:

// sftp
//
sftp: {
options: {
host: ‘<%= myServer.host %>’,
username: ‘<%= myServer.username %>’,
privateKey: ‘<%= myServer.privateKey %>’,
passphrase: ‘<%= myServer.passphrase %>’,
path: ‘<%= myServer.path %><%= grunk.task.current.target %>/’,
srcBasePath: "../public/<%= grunk.task.current.target %>/",
showProgress: true
},
production: { },
prerelease: { },
},

Maybe someday.

Simplify Congiguration With Shared Variables

Another trick I learned is that common configuration strings can be put at the top of the configuration block and re-used. This is alluded to on the Grunt tasks configuration page but they never expound on how to use the common configuration variables. Here is how it works, define the variable at the top of the configuration block then reference that variable inside a string like ‘<%= my_var %>’. Here is my example with some “fluff” missing from the middle:

// Project configuration.
grunt.initConfig({

// Metadata.
currentPlugin: currentPlugin,
myServer: myServer,
pkg: grunt.file.readJSON(‘package.json’),
my_plugin_dir: ‘/var/www/wpslp/wp-content/plugins/<%= currentPlugin.slug %>’,
my_src_files: [
‘**’,
‘!**.neon’,
‘!**.md’,
‘!assets/**’,
‘!nbproject/**’
],
// compress
//
compress: {
options: {
mode: ‘zip’,
archive: ‘../public/<%= grunt.task.current.target %>/<%= currentPlugin.zipbase %>.zip’,
},
prerelease: { expand: true, cwd: ‘<%= my_plugin_dir %>’, src: ‘<%= my_src_files %>’ },
production: { expand: true, cwd: ‘<%= my_plugin_dir %>’, src: ‘<%= my_src_files %>’ },
},

In this example you can see how I’m using the my_plugin_dir variable to set my path to the plugins I am working on on my dev box and my_src_files to list the files I want to add (or ignore) when pushing my development plugin directories to a zip file or the WordPress svn repo for publication.

This has simplified a lot of task configuration entries in my custom grunt tasks script.

That combined with smarter configuration blocks in areas like the SFTP node module has simplified my Grunt configuration which will make it less prone to errors and easier to maintain going forward.

Back to coding…

Posted on

WordPress Dev Kit Plugin : 0.0.1

Grunt WordPress Dev Kit

For those of you that have been following along with my exploration of Grunt and automating my plugin workflow automation… I’m sorry.   Quite boring I’m sure but remember this blog is my personal notebook as much as fodder for the 3 people that may be interested in this stuff.

Last night I extended my journey toward less manual updates, each new step adding an option for human error, by creating the first WordPress Development Kit Plugin.  Yup, a plugin for plugin development.  Sort of.  What the new plugin is going to help me with is keeping my Plugin Version Info page updated.   Today it is very rudimentary with a basic list of plugin slugs, the version info, and release dates.  You can see it in action on my Plugin Version Info page. Ultimately the new Plugin companion to the WordPress Development Kit will be extended to get more information from the Grunt process into the public website via the automated toolkit.

My goal is to have ONE PLACE where plugin information is updated, preferably in the readme files.    For now the JSON file that drive Grunt will suffice with future plans to scrape the readme data into the plugins.

What WordPress Dev Kit Plugin Does

The WordPress Dev Kit plugin is very simplistic in its current form.  It reads the plugins.json file from the WP Dev Kit and renders information via a shortcode to a plugin or  post.

Version 0.0.3 of the plugin has the following shortcodes available:
* Actions (default: list)
* o [wpdevkit action='list'] list details about all plugins
*
* Styles (default: formatted)
* o [wpdevkit action='list' style='formatted'] list the details in an HTML formatted layout
* o [wpdevkit action='list' style='raw'] list the details in a print_r raw format
*
* Types (default: basic)
* o [wpdevkit action='list' type='basic'] list basic details = version, updated, directory, wp versions
* o [wpdevkit action='list' type='detailed'] list all details = version, updated, directory, wp versions, description
*
* Slug (default: none = list ALL)
* o [wpdevkit action='list' slug='wordpress-dev-kit-plugin'] list details about a specific plugin

This will list the entire plugin metadata structure in a pre tag as a standard PHP dump format.

WP Dev Kit Plugin Setup

It expects that you have the WordPress Development Kit plugins.json file pushed to a production files directory on your server.   You can set the location of the file to any web-server-readable directory.    The location is specified in the Settings / WP Dev Kit menu in the admin panel.

If you are using the WP Dev Kit Grunt tasks the build:<slug>:production process will move the plugins.json file over to your server along with your production.zip files.

My Process And How This Helps

In case you’re wondering why I would go through the trouble of building a full-fledged plugin like this, here is my typical workflow:

  1. Edit code.
  2. Edit readme to update version, features, etc.
  3. Edit main plugin to update version, etc.
  4. Create zip file.
  5. FTP zip file to server.
  6. If a WordPress Plugin Directory plugin, fetch SVN repo, update trunk, commit, add branch, commit, push.
  7. Login to my website.
  8. Update version info page on website.
  9. Create blog post about new plugin update with features and screen shots if warranted.
  10. If a Store Locator Plus plugin update the HTML that is pushed through the in-product signage app.

Until today nearly every step of that process was manual.  Now with the WP Dev Kit running on my system and the WP Dev Kit Plugin on my public site the process is now:

  1. Edit code.
  2. Edit readme to update version, features, etc.
  3. Edit main plugin to update version, etc.
  4. Edit the WP Dev Kit JSON file.
  5. grunt build:<slug>:production  which automatically
    1. checks that the version in the readme.txt and plugin file match (new quality control test)
    2. creates the zip file
    3. uses SFTP to put the file on my server
    4. updates the WordPress Plugin Directory Listings (fetch, update trunk, commit, add branch, commit, push)
    5. pushes plugin.json which talks to the WP Dev Kit Plugin and keeps the version info page upated
  6. Login to my website.
  7. Create blog post about new plugin update with features and screen shots if warranted.
  8. If a Store Locator Plus plugin update the HTML that is pushed through the in-product signage app.

The magic happens in steps 4 and 5.   It automates many of the steps in the process.  As I refine the WordPress Dev Kit I will be able to eliminate more steps along the way.

Not a bad start.   As each new plugin update happens I will be refining and improving the automation files and plugin to create a better presentation and improve quality control at each step.

Learn about this process via the WordPress Development Kit articles.

Posted on

WordPress Dev Kit : Grunt Helpers 0.2.0

Grunt WordPress Dev Kit

I am continuing on my quest to use Grunt to further automate my plugin development process. I hope to be generating better quality plugins through more automated testing and sanity checks during the production process. I also hope to take out a few of the steps required to get the plugins into the hands of my customer base. TheWordPress Development Kit articles describe the process as it evolves, both to possibly help others that are considering automating with Grunt and to help people that are starting to work on plugins related to those I’ve created understand my process.

My Environment For Grunt Helpers 0.2.0

I’ve made some changes to my plugin production environment since the original “Legacy Publisher” and even my “Grunt Helpers 0.1.0” article.   Some of the changes are based on things I’ve seen elsewhere as I learn about Grunt and other changes are to have a better defined environment.   Here is a summary of how things are working with the Grunt automation in the latest iteration.

I have two types of production, ready for public consumption, plugins:

WordPress Plugin Directory Listings

The WordPress Plugin Directory Listings (WordPress Hosted) are managed via the standard subversion (svn) repository update process.   They also are packages and put on my server for customers to download from my site.

Premium Plugins Served From My Site

The Premium Plugins (Premium) are only hosted on my server and made available to customers that have purchased the premium add-on packs.

For both types of plugins, whether WordPress Hosted or Premium, I now have two versions available; the production release (production) which is the official ready-for-deployment version and pre-release (prerelease) which is provided to select customers for testing an upcoming release before it has been fully tested.   Pre-release versions are beta releases that may not be fully tested.

Regardless of whether a plugin is a WordPress Hosted product or a Premium product, the pre-release is ONLY available from my server and only to select customers.    I use the Groups plugin combined with the WooCommerce add-on pack for Groups as well as the File Away plugin to manage access to the pre-release versions.

The Grunt Tasks

To support the various iterations of the plugins that are being produced I have created several grunt tasks that are managed by my WordPress Development Kit scripts.

grunt build:<slug>:production

This task will publish the production-ready copy of my plugin.   It goes through my development copy of the plugin directory, cleans out the “development cruft” and builds a .zip file that is ready for distribution.    It will then read the plugins.json file and determine the ultimate home for the production copy.  If the “publishto” property for the given slug is “wordpress” (maybe I should change it to “WordPress”) the product ends up on the WordPress Plugin Directory.    If the publishto property is “myserver” the product ends up on my web server in the production files directory.

grunt build:<slug>:prerelease

This tasks also cleans out the “development cruft” of the plugin directory and creates a zip file.   However in this mode the plugin zip file only ends up on my server regardless of the publishto property set in the plugins.json file.   The prerelease files are stored in a separate directory on my server from the production files.

My Grunt Configuration Files

There are now several files that are used to configure my development kit.

package.json

This now follows a standard Grunt package.json format.   It contains my author info, the grunt default project name, version, description, license, and a list of grunt dependencies.   Grunt dependencies are node.js modules that are used by this project.  Currently the list includes:

Those with the asterisk (*) are not currently used, but I know I will be making use of them in the future so I’ve left those recommended default modules in place.

plugins.json

This is now the home for all of the metadata about my WordPress plugins that are being managed by the WordPress Development Kit.  This includes variables that are later used by modules such as wp-deploy as well as my plugin specific information such as the slug and other information.   The format is typical JSON with the following nodes defined:

    • pluginMeta = an array of plugin slugs
      • <slug>
        • version = current production version of the plugin (most likely will be deprecated and auto-read from readme.txt)
        • name = the plugin name (most likely will be deprecated and auto-read from readme.txt)
        • description = the plugin description (most likely will be deprecated and auto-read from readme.txt)
        • publishto = where production files get published, either “wordpress” (WordPress Hosted) or “myserver” (Premium)
        • zipbase = the base name of the zip file, used if I want to create myplugin-test.zip instead of myplugin.zip
        • reposlug = the WordPress Hosted repository slug if not the same as <slug>
    • wp-plugin-dir = where on this server, my development server, do the in-development plugins live
    • wp-username = my WordPress Plugin Directory username

myserver.json

This file contains the details about my server that help with the SFTP push to my server.  It includes things like my local path to the SSH key files and where I want to put stuff on the remote server.

    • host = my fully qualified host name (no http, etc. just a pure host name)
    • username = the username to use for sftp login
    • privateKeyFile = the path on my development system where my private key file lives
    • privateKeyPassFile = the path on my development system where the private key password is stored (will become a command line option, this is a security risk, though my dev system is fairly well locked down and isolated)
    • path = the path on the production server that is the root directory under which my production and pre-release files will go.

Process Details

Here is what happens when I execute the two most-used commands in my new build kit.

WordPress Hosted Production

command: grunt build:<slug>:production

This process gathers all the files in my plugin directory and copies the “cleaned” version of the files into ./assets/build under the plugin directory itself.  I store it here because assets is one of the directories that is ignored (cleaned) during production and it allows me to see exactly what got moved into production.    These files are then published to the WordPress svn repository using the wp-deploy module.   The same files are sent through compress with a .zip file created and stored in the wp-dev-kit/public directory (yes this seems redundant and can probably be simplified).  Finally the zip file from that public directory is sent over to my live server with SFTP and put into the production folder on the live server.

[box type=”alert”]I did have to create a patch for wp-deploy to properly manage subdirectories. I have submitted the patch to the author. Hopefully it makes it into his production release soon.[/box]

Premium Production

command: grunt build:<slug>:production

This process is identical to the WordPress Hosted Production process with one exception.  It skips the wp-deploy call and does nothing with svn.  I creates the ./assets/build files, zips them, puts them in the WP Dev Kit public folder, and copies them over to the production folder on my server.

WordPress Hosted and Premium Prerelease

command: grunt build:<slug>:production

This process works much like the Premium Production process.  The only difference in the process is that the files are copied over to my server into a different directory that holds pre-release plugin files only.

Further Refinements

There are a number of refinements to be made to the process.  First on my agenda is reading the readme.txt file and extracting the version information so it can be appended to the zip file names for pre-release copies of the products.    I will then work on things like running the third party modules to do gettext sanity tests, JavaScript minification, and other “would be nice” features for production plugins.   I will also likely change this process a dozen times as I continue to iterate over both Premium and WordPress Hosted plugin builds over the next few months.

In the end I hope to have a handful of simple Grunt commands that manage the process efficiently and reduces the human error that my current process can introduce at several steps.

If you have suggestions or feedback on how to improve my process, please let me know!

Related Topics

Posted on

WordPress Dev Kit : Grunt Helpers 0.1.0

Grunt WordPress Dev Kit

Now that I have a very basic understanding of Grunt it is time to start incorporating it into my WordPress Development Kit.  My hope is not to simply replace the current Legacy Publisher elements of my WP-Dev-Kit scripts, but to speed up the production process and improve the quality of the plugins.

I can now start writing JavaScript to parse and process the myriad of files that are updated with each release.  I can mix in system commands and replace the Bash script processing to update repositories, upload files to my servers, and hopefully even start an automated web content production script.

WordPress-Centric Grunt Scripts

In addition there are at least a handful of pre-existing Grunt scripts that will improve the quality of my plugins doing things like creating the POT files and also checking my text domains (an ongoing problem for me) to assist in language translations.    There are also scripts to check the WP versions in my readme and base plugin match (bitten by that one more than once) and even generate a readme.md for Github or Bitbucket repos.    I’m sure I’ll discover more soon, here are the few I know about today:

checktextdomain by Stephen Harris
Checks the correct text domain is passed when using the WordPress translation functions.

checkwpversion by Stephen Harris
Make sure your plugin version numbers are all in sync in the plugin header, readme.txt and the Grunt package.json files.

pot by Stephen Harris
Generates a .POT file that can be used for translations.

wp-readme-to-markdown by Stephen Harris
Converts readme.txt file to readme.md for use in Github repo.

Setting Up The WP Dev Kit

You can follow along by cloning the git repository:

cd ~
git clone git@bitbucket.org:lance_cleveland/wp-dev-kit.git ./wp-dev-kit
cd ./wp-dev-kit/grunt
npm install

[box type=”alert” style=”rounded”]The WP Dev Kit Legacy Publisher scripts store published plugins in your ~/myplugins directory. With the newer Grunt scripts they are stored under the ./wp-dev-kit/public directories. If you mix-and-match legacy and Grunt scripts you will have inconsistent published zip files unless you symlink the directories.[/box]

After cloning the repository you will want to edit the plugins.json file in the ./grunt subdirectory to replace the plugin files with your own list of plugins that you are developing and/or managing.

[box type=”alert” style=”rounded”]Make sure you edit the ./grunt/plugins.json file to match the list of plugins you are developing.[/box]

If you are feeling adventurous you can even use git to clone my public Store Locator Plus plugin code into your WordPress plugins directory.

My WP Dev Kit Grunt Tasks

The following tasks are part of my my WP-Dev-Kit Grunt file.

build:<slug>[:<type>]

Builds the distribution file, a .zip file, for the specified plugin and put it in the wp-dev-kit public folder.   Ready for distribution.  As the Grunt toolkit expands this will also do some cleanup and maintenance like running version check and textdomain checks, minifying CSS and JavaScript and other nice things like that!

The type parameter can extend the build and make it do other things like:

makezip
The default mode.  Just build the zip file and put it in the public folder of the WP dev kit.

publish
Reads the package.json parameters and decides whether to SFTP the zip file to a private server or unpack it into a new tag in the svn repo and push it to the public WordPress Plugin Directory.   This will also push the zip file and plugins.json file to the server specified in myserver to allow for integration with some new WordPress plugins that are coming to publish plugin version lists and other “nice things” on a WordPress plugin store website like StoreLocatorPlus.com.

Set the package.json pluginMeta slug property ‘publish-to’ to either ‘wordpress’ or ‘myserver’ for each plugin being managed.

list

This will list the slugs for the plugins you are managing with the Grunt portion of the wp-dev-kit.

details

This will list the plugin names, slugs, and current version of the WordPress plugins being managed by the dev kit.

Configuring The Grunt Tasks

There are two files that need to be edited in the latest version of my WP Dev Kit.  The plugin details file, plugins.json, and the public plugin server details myserver.json file.

plugins.json

The plugins details file contains the settings for the WordPress plugins that you will manage with the WP Dev Kit.  The settings are as follows:

wp-plugin-dir – where is your WordPress plugin directory on this server?

pluginMeta – an array of plugin slugs with sub-keys that specify details about the plugin such as:

version – the current production version in major.minor.patch format.

zipbase – the name of the zip file to be created, usually the same as the slug .

publishto – where to send the production files when done, which can be one of the following:

wordpress = put it in the WordPress Plugin Directory via the svn repository

myserver = put it on the server specified in the myserver.json file using SFTP

name – the name of the plugin, usually as it appears in the readme.txt for the plugin

description – the description for the plugin, usually as it appears in the readme.txt

{
  "pluginMeta": {
    "slp-enhanced-results": {
      "version": "4.1.03",
      "zipbase": "slp4-er",
      "publishto" : "myserver",
      "name": "Store Locator Plus : Enhanced Results",
      "description": "A premium add-on pack for Store Locator Plus that adds enhanced search results to the plugin."
    },
    "store-locator-le": {
      "version": "4.1.10",
      "zipbase": "slp4",
      "publishto" : "wordpress",
      "name": "Store Locator Plus",
      "description": "Manage multiple locations with ease. Map stores or other points of interest with ease via Google Maps.  This is a highly customizable, easily expandable, enterprise-class location management system."
    }
  },
  "wp-plugin-dir": "/var/www/wpslp/wp-content/plugins/"
}

myserver.json

The connection details for the SFTP server used in the for myserver zip file publication as noted in the plugins.json description.   Settings include:

host – the fully qualified host name for the server

username – which user to login as

privateKeyFile – the path to the private key file, usually in your home/.ssh directory named id_rsa.

privateKeyPassFile – the path to a secured text file, that should NOT be part of any public repository , that has your private key file passphrase.

path – the path on the server where the files should be stored, relative to the username login directory.

{
    "host": "www.charlestonsw.com",
    "username": "serveradmin",
    "privateKeyFile": "/home/myuser/.ssh/id_rsa",
    "privateKeyPassFile": "/home/myuser/private/.pkpass",
    "path": "/var/www/wpslp/production/premium/"
}

That’s enough for this round of Grunt automated WordPress plugin publishing. I can see how this is going to help improve my process in the future even though it is adding an extra step today. I will continue to update this development kit for my production environment. As I do I will add more posts like this one and describe how I manage my free Store Locator Plus plugin and premium add-on packs.

Posted on

WordPress Dev Kit : Legacy Publisher

WordPress Plugin Directory Banner

This article is about my legacy publishing process for development WordPress plugins and themes.   It is my workflow and it is far from perfect.  It will evolve over time and I am in the midst of making several big changes using new tools to assist in the process.   To get some background on the process you may want to follow along with the WordPress Workflow and WordPress Development Kit threads on my blog.

My Setup

My WordPress development is fully contained in a VirtualBox running CentOS 6.5 with a full GUI interface.   I am working on building an distributing some Vagrant boxes that will help others replicate the process.  My base CentOS 6.5 GUI box is online but is not a full setup… yet.   You can read about Vagrant in the WordPress Workflow thread.

My plugin development environment consists of a standard WordPress production build (3.8.1 today) with the typical wp-content/plugins subdirectory.  On my development system I work directly inside of these directories.  It provides instant access to the modified code without running a repetitive series of push/pull commands to move stuff into a “ready to review” state.    I find this configuration to be far more efficient and rely on semi-intelligent scripts to help me “weed out” the development environment when building the production-ready kits.    As a last step before production I have a test box on a remote server where I install the production releases as a clean last-minute sanity check.  This will migrate to a Vagrant-based “clean production simulator” to provide more robust testing.

How does my “inline production” setup look?

The Build Scripts

The top-level build scripts go in my ./wp-content/plugins directory.   That is not exactly accurate.   I should say that symbolically linked file pointers (symlinks for Linux geeks… or shortcuts in Windows lingo) go in that directory.   This includes the makezip, update_wrepo, exclude and common files that are part of the legacy/publisher section of the new WP Dev Kit I’ve published on Bitbucket.

Plugin Assets

Within each plugin subdirectory I create an assets subdirectory.  This is where my “build and production helpers” go.   My scripts specifically avoid this directory when packaging the “for consumption” products.  Within this directory I currently store things like raw image assets that typically get pushed the WordPress Plugin Directory, a full copy of the WordPress svn repository for any plugins published there, and other “goodies” I need for production that I do not distribute.  This is where my new Grunt and other build helpers will go.

Published Files Store

There is a separate directory under my home directory, ~/myplugins, where the published files go.   This helps keep my production files separate from the main plugins.  This will eventually change as I improve and streamline the process.

NetBeans “Push” Project

I also have a special directory under my home directory where I keep non-WordPress NetBeans projects.   One of those projects is a private project named “csa_licman”.  One of the many things this project does, which will also be changing, is sync any changes I make in the subdirectories of the csa_licman project up to my live server.  I use this to “publish” my premium add-ons to my live server.    I will not cover that part of the project here or the NetBeans automated FTP push setup.

You can find details on getting NetBeans projects to watch a directory and FTP files elsewhere on the NetBeans site.  I will be changing this complex part of the system to automated Grunt processing in a later revision to the WP Dev Kit.    For now you can keep the myplugins directory and manually FTP files from there to any private directories.

Keep in mind that anything going to the public WordPress Plugin Directory will be handled via the legacy update_wprepo.sh script.   The NetBeans push project is only for my premium add-on packs that are not hosted in the WP Plugin Directory.

My tool kit currently consists of:

git – for managing my code repository.  I find it faster and lighter than svn.

svn – solely for publishing to the WP Plugin Directory.

smartgit – my GUI to help me cheat with git commands, I am MUCH faster with the GUI.

NetBeans – my IDE setup specifically for PHP development with heavy use of phpDoc hints in my code.

Firefox with Firebug – my test and development browser because Firebug is the easiest-to-use development tool set.

command line with Bash – the Linux command line for executing my Bash scripts, most of this should work on OSX as well.

Using The Legacy Publisher

Here is how I setup and use the legacy publisher scripts in my setup.   I call them “legacy publisher” scripts as I am learning Grunt and working toward a more intelligent processor.  The legacy publisher is based on simple Bash scripts which can be complex to configure and extend.   Grunt provides a managed system that will fetch various plugins to do a lot of cool stuff that would take me far to long to replicate in Bash.    Why re-invent the wheel?

After I get my new dev box setup and I have WordPress up-and-running and my development plugins in place under the wp-content plugins directory I attach my legacy publisher kit.   I am not going to cover the NetBeans push setup under csa_licman though you will see it referenced in the shell scripts.   Always answer “n” to “publish to live server” when running any of the legacy scripts and you will not have issues as that part of the script will not be executed.

Under my login I use git to fetch the WP Dev Kit from Bitbucket,  setup my published files directories, I then link the legacy publisher kits into my WordPress plugin directory:

cd ~
mkdir myplugins
git clone git@bitbucket.org:lance_cleveland/wp-dev-kit.git ./wp-dev-kit
cd /var/www/wpslp/wp-content/plugins
ln -s ~/wp-dev-kit/legacy/publisher/*

I am now ready to use the legacy kit to build my production zip files and publish them by hand to my server or use update_wprepo.sh to push them to my WordPress Plugin Directory listing.

Simple Self-Hosted Plugins

My premium add-on plugins are the simplest setup for this environment.   Once I have my legacy publisher scripts in place I can create my base plugin files right in the plugin directory.   I edit my readme and php files, test locally, and when ready I publish them in one step with the makezip command.  Since I have a NetBeans automated push project I can answer “y” to the publish to live server question and get the files over to my server.  If you do not have this setup you can answer no and manually FTP the files in ~/myplugins to your final destination.

cd /var/www/wpslp/wp-content/plugins
./makezip slp-enhanced-results
Legacy Publisher Makezip Example
Using legacy publisher scripts to create a production zip file.

WordPress Hosted Plugins

WordPress hosted plugins, such as the Store Locator Plus plugin, require a bit of extra work to get them ready for production.    After the initial setup is in place I can update the production release using the update_wprepo.sh script.    In this setup I use the assets subdirectory to assist in the production.   It is where the svn repository lives that gets updates to the WordPress servers. 

Store Locator Plus Plugin Development Folders
Store Locator Plus plugin development folders.

I start with my plugin directory in place and build the setup necessary for the legacy script to publish to WordPress:

cd /var/www/wpslp/wp-content/plugins
cd store-locator-le
mkdir assets
echo '<?php // Silence is golden.' > index.php
svn co 'http://plugins.svn.wordpress.org/store-locator-le/' svnrepo
mkdir public

My plugin is now ready for publication. I go to the wp-content/plugins directory and run update_wprepo.sh and answer the prompts whenever I am ready to push to the WordPress plugin directory. The process also build a local zip file in the ~/myplugins directory so I can keep a copy on my servers.   The update_wprepo.sh script will update trunk, create the version branch and even try to clean up any tags or branches if a push of the same version has failed previously.

WP Directory Marketing Assets

Getting asset files into the WordPress repository is also part of the script.   This is the preferred method for storing your plugin banner and screen shots as it lists them on the WordPress site without filling up the .zip distribution that goes to the client.  With the legacy publisher it is easy, though a bit of a manual process.   I store my banner and screenshot files in the ./assets subdirectory (note to self: should create a wp_marketing_images subdir).  Since the assets directory is never part of the published file set they are ignored.   To add assets to the WP Plugin Directory publication I create an assets directory under the svnrepo, copy over the image files I want, and run svn up from the svnrepo directory.

The Scripts

The following scripts are included in the legacy/publisher directory in my WordPress Dev Kit:

common.sh

Setup common Bash variables that the other scripts use to run.

exclude.lst

The list of directories and files to be excluded when zipping up the plugin directory.

makezip.sh

The script that packages up the plugin directory.   Currently with some extra “cruft” to rename some of my plugin directories to a different zip file name when doing the “publish to live server” logic.

update_wprepo.sh

Runs makezip, unpacks the zip file into the assets/public directory to ensure a clean copy of the plugin, copies to svn repository trunk and branches to push to the WP Plugin Directory.