Posted on

VVV For WordPress Development

Banner Vagrant Cloud WP Dev Kit Box

Now that 4.3 has been released I am taking a few days to reconfigure my production environment for WordPress plugin development. After a number of issues running the self-contained GUI development environment that included WordPress core, full plugin development, and all of my supporting infrastructure including phpStorm, SmartGit, and a series of Grunt scripts in an all-in-one Vagrant-based Virtualbox, I decided to try something new. Losing 20 minutes every few days because the self-contained GUI in Virtualbox could not sync between guest and host was too much. Something in the Virtualbox or CentOS 7 upgrades over the past year broke something fundamental in GUI I/O and I’ve been unable to track it down. Time for a change.

My change? Learning Varying Vagrant Vagrants. For those that are not familiar with VVV for WordPress development you may want to check it out here: https://github.com/Varying-Vagrant-Vagrants/VVV.

What is VVV?

VVV is a virtual development environment for WordPress.    It spins up a headless (no GUI interface) Virtualbox that contains three separate versions of WordPress (stable, dev, and trunk) as well as a myriad of tools like phpMyAdmin.   All of the settings are in place to allow your local system, my OS/X desktop for my setup, to interact with the local WordPress install from your preferred web browser.

The upside is there is a lot of community support , articles, and various tools-and-trick available to you for doing almost anything you want.    A lot of the “cool dev tricks” I never had fully working, like interactive XDebug support in phpStorm, are readily available.     It is also super-easy to switch between WordPress releases which is cool if you are sending core patches or need to test on the upcoming major release.

The downside is that you need to setup all of your development tools locally on your desktop.   Guess what happens if your computer dies?   Yup, another few hours of setting it up again.   With my prior custom self-contained virtual environment I only need to save my Virtualbox, usually by creating a Vagrant image, any time I made notable changes to my tool kit; by doing so I could restore it easily to ANY desktop ANYWHERE in the world and have EXACTLY the same environment in no more time than it takes to spin up a VVV based box.

In short, VVV is a virtual machine store on your local desktop with several WordPress installs ready-and-waiting behind your browser screen.

My Startup Tricks

I develop a number of WordPress plugins, so having full development tools and my source code are key to productivity.  Here are some things I needed to tweak on the default VVV setup to get going.

Linking Plugin Source

I am primarily developing plugins and I want them on all of the WordPress installs provided by VVV.  I can “take over” a VVV server-based directory with a local directory my mapping the local directory to the destination with a Vagrant Customfile.    Go to the base location where you placed your VVV install, you will know you are in the right place as it has the Vagrantfile for the VVV box, and create a new file named “Customfile”.

Here is my mapping entries to take over the plugin directory on all 3 WordPress installs that come with VVV:


config.vm.synced_folder "/Users/lancecleveland/Store Locator Plus/plugin_code", "/srv/www/wordpress-default/wp-content/plugins", :owner => "www-data", :mount_options => [ "dmode=775", "fmode=774" ]

config.vm.synced_folder "/Users/lancecleveland/Store Locator Plus/plugin_code", "/srv/www/wordpress-develop/wp-content/plugins", :owner => "www-data", :mount_options => [ "dmode=775", "fmode=774" ]

config.vm.synced_folder "/Users/lancecleveland/Store Locator Plus/plugin_code", "/srv/www/wordpress-trunk/wp-content/plugins", :owner => "www-data", :mount_options => [ "dmode=775", "fmode=774" ]

Configuring XDebug

phpStorm comes with an XDebug listener.  This allows you to set breaks in your PHP code files, inspect variables in real-time, and do a lot of other things that are much more efficient than var_dump or print_r or die littered throughout the code.     The are a number of articles and videos on using XDebug with phpStorm.  Check it out, it is a great debugging tool.   For now, how to enable it with VVV:

Turning on XDebug is easy with VVV.

Go to the VVV install directory.

Enter Vagrant via SSH: vagrant ssh

Turn on Xdebug from the SSH command line on the virtual machine: xdebug_on

That’s it, I can now use my  local phpStorm tool to debug my VVV files.

Here is THE XDebug + VVV + phpStorm video to watch to do this.

Useful Meta

With VVV installed these URLs should work in your browser.

Users and passwords:

  • For the WP installs:  wp / wp
  • For WP Admin Users: admin / password
  • MySQL Root: root / root
    • Default DB Name: wordpress_default
    • Trunk DB Name: wordpress_trunk
    • Develop DB Name: wordpress_develop

Paths:

  • Local directories (relative to Vagrant install): ./www
  • Server-Side directories: /srv/www
Posted on

Deploying WordPress 4.2.2 On Elastic Beanstalk, Part 1

AWS Beanstalk WordPress 445x200

I spent a good part of the past 24 hours trying to get a basic WordPress 4.2.2 deployment up-and-running on Elastic Beanstalk.   It is part of the “homework” in preparing for the next generation of store location and directory technology I am working on .    I must say that even for a tech geek that loves this sort of thing, it was a chore.   This article is my “crib sheet” for the next time around.   Hopefully I don’t miss anything important as I wasted hours chasing my own rear-end trying to get some things to work.

I used the Deploying WordPress with AWS Elastic Beanstalk fairly extensively for this process.    It is easy to miss steps and is not completely up-to-date with the screen shots and information which makes some of it hard to follow the first time through.  I will try to highlight the differences here when I catch them.

The steps here will get a BASIC non-scalable WordPress installation onto AWS.    Part 2 will make this a scalable instance.    If my assumptions are correct, which happens from time-to-time, I can later use command-line tools with git on my local dev box to push updated applications out the the server stack.  If that works it will be Part 3 of the series on WP ELB Deployment.

Getting Started

The “shopping list” for getting started using my methodology.    Some of these you can change to suit your needs, especially the “local dev” parts.  Don’t go setting all of this up yet, some things need to be setup a specific way.  This is just the general list of what you will be getting into. In addition to this list you will need lots and lots of patience.  It may help to be bald; if not you will lose some hair during the process.

 

Part 1 : Installation

  • A local virtual machine.  I use VirtualBox.
  • A clean install of the latest WordPress code on that box, no need to run the setup, just the software install.
  • An AWS account.
  • A “WP Deployment” specific AWS user that has IAM rules to secure your deployment.
  • AWS Elastic  Beanstalk to manage the AWS Elastic Load Balancer and EC2 instances.

Part 2 : Scalability

  • AWS S3 bucket for storing static shared content (CSS rules, images, etc.)
  • AWS Elasticache for setting up Memcache for improved database performance.
  • AWS Cloudfront to improve the delivery of content across your front-end WordPress nodes.
  • AWS RDS to share the main WordPress data between your Elastic Beanstalk nodes.

Creating The “Application”

The first step is to create the web application.  In this case, WordPress.

I recommend creating a self-contained environment versus installing locally on your machine, but whatever you’re comfortable with.   I like to use VirtualBox , sometimes paired with Vagrant if I want to distribute the box to others, with a CentOS GUI development environment.  Any flavor of OS will work as the application building is really just hacking some of the WordPress config files and creating an “environment variables” directory for AWS inside a standard WP install.

Got your box booted?  Great!

Fetch the latest download of WordPress.

Install it locally.

Remove wp-config-sample.php.

Create a new wp-config.php that looks like this:

<?php

// An AWS ELB friendly config file.

/** Detect if SSL is used. This is required since we are
terminating SSL either on CloudFront or on ELB */
if (($_SERVER['HTTP_CLOUDFRONT_FORWARDED_PROTO'] == 'https') OR ($_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https'))
{$_SERVER['HTTPS']='on';}

/** The name of the database for WordPress */ define('DB_NAME', $_SERVER["RDS_DB_NAME"]);
/** MySQL database username */
define('DB_USER', $_SERVER["RDS_USERNAME"]);

/** MySQL database password */ define('DB_PASSWORD', $_SERVER["RDS_PASSWORD"]); /** MySQL hostname */
define('DB_HOST', $_SERVER["RDS_HOSTNAME"]);

/** Database Charset to use in creating database tables. */ define('DB_CHARSET', 'utf8');
/** The Database Collate type. Don't change this if in doubt. */
define('DB_COLLATE', '');

/**#@+
 * Authentication Unique Keys and Salts.
 * Change these to different unique phrases!
 */
define('AUTH_KEY',$_SERVER["SECURE_AUTH_KEY"]);
define('SECURE_AUTH_KEY',$_SERVER["AUTH_KEY"]);
define('LOGGED_IN_KEY',$_SERVER["LOGGED_IN_KEY"]);
define('NONCE_KEY',$_SERVER["NONCE_KEY"]);
define('AUTH_SALT',$_SERVER["AUTH_SALT"]);
define('SECURE_AUTH_SALT', $_SERVER["SECURE_AUTH_SALT"]);
define('LOGGED_IN_SALT', $_SERVER["LOGGED_IN_SALT"]);
define('NONCE_SALT', $_SERVER["NONCE_SALT"]);

/**#@-*/

/**
 * WordPress Database Table prefix.
 *
 * You can have multiple installations in one database if you give each a unique
 * prefix. Only numbers, letters, and underscores please!
 */
$table_prefix  = 'wp_';

/**
 * For developers: WordPress debugging mode.
 *
 * Change this to true to enable the display of notices during development.
 * It is strongly recommended that plugin and theme developers use WP_DEBUG
 * in their development environments.
 */
define('WP_DEBUG', false);
/* Multisite */
//define( 'WP_ALLOW_MULTISITE', true );

/* That's all, stop editing! Happy blogging. */

/** Absolute path to the WordPress directory. */
if ( !defined('ABSPATH') )
        define('ABSPATH', dirname(__FILE__) . '/');

/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');

Do not move this wp-config.php file out of the root directory.  It is a common security practice but it will be missing from your AWS deployment.  There are probably ways to secure this by changing your target destination when setting up AWS Cloufront, but that is beyond the scope of this article.

Settings like the $_SERVER[‘RDS_USERNAME’] will come from the AWS Elastic Beanstalk environment you will create later.  This is set dynamically by AWS when you attach the RDS instance to the application environment.  This ensures the persistent data for WordPress, things like your dynamic site content including pages, posts, users, order information, etc. is shared on a single highly-reliable database server and each new node in your scalable app pulls from the same data set.

Settings for the “Salt” come from a YAML-style config file you will add next.     This is bundled with the WordPress “source” for the application to ensure the salts are the same across each node of your WordPress deployment.     This ensures consistency when your web app scales, firing up server number 3, 4, and 5 while under load.

Create a directory in the root WordPress folder named .ebextensions.

Fetch new salts from WordPress.

Create a new file named keys.conf in the .ebextensions directory that looks like this, but using YOUR salts:

option_settings:
- option_name: AUTH_KEY
  value: '0VghKxxxxxn?%H$}jc5.-y1U%L)*&Ha/?)To<E>vTB9ukbd-FNoq^+.4A+I1Y/zp'
- option_name: SECURE_AUTH_KEY
  value: 'z 7)&E~NjioIREE@g+TKs-~yO-P)uq2Zm&98Zw>GK_rYb_}a,C#HD[K98ALxxxxx'
- option_name: LOGGED_IN_KEY
  value: 'yq@K{i=z(xxxxxm1VOi80~.H?[,h+F+_wua]I:z-YZF|a-vEV[n/6pRBlw+qAe^q'
- option_name: NONCE_KEY
  value: 'Bq=kbD|H#iMt5#[d[qURMP8C}xxxxxf[WaI6.oF5=r1h#:E?BZ-L28,7x~@oZw#7'
- option_name: AUTH_SALT
  value: 'O;4uq817 CSs3-ZAUY>e%#xxxxx<:u~=Is4d6:CI3io;aL<h]+x~;S_fc3E oEB1_'
- option_name: SECURE_AUTH_SALT
  value: 'nF94Rasp-0iaxxxxxm:|e82*M9R!y>% b68[oN|?_&4MRbl.)n8uB-ph|*qIPq|e'
- option_name: LOGGED_IN_SALT
  value: '&Ah^OIb<`xxxxx+lKV=zFER_^`+gA%.UWCIy|fJ+RfKiYKBP^&,[|%6K<%C[eU]n'
- option_name: NONCE_SALT
  value: 'ZiKejG|xxxxx k3>nr)~AN5?*hd!aO-)E^fR^^!_PR1n[oq{??F`,NQmdfE2Mj:`'

Zip up your application to make it ready for deployment.

Do NOT start from the parent directory. The zip should start from the WordPress root directory. On Linux I used the this command from the main WordPress directory where wp-config.php lives:
zip -r ../wordpress-site-for-elb.zip .

Create The Elastic Beanstalk Environment

Login to the AWS Console.

Go to Elastic Beanstalk.

Go to ELB Create New Application.

 

AWS ELB application info
AWS ELB application info

Select Create Web Server.

AWS ELB Web Server Environment
AWS ELB Web Server Environment

Select the default permissions (I didn’t have a choice here).

AWS ELB Permissions
AWS ELB Permissions

Set the Environment to PHP and Load Balancing, auto scaling.

AWS ELB Environment Type
AWS ELB Environment Type

Upload your .zip file you created above as the source for the application.
Leave Deployment Limits at their defaults.
As a side note, this will create an application that you can later user for other environments, making it easy to launch new sites with their own RDS and Cloudfront settings but using the same WordPress setup.

AWS ELB Application Version
AWS ELB Application Version

Set your new Environment Name.
If your application name was unique you can use the default.
If your application name is “WordPress” it is likely in use on ELB, try something more unique.

AWS ELB Environment Name
AWS ELB Environment Name

Tell ELB to create an RDS instance for you.
I chose not to put his in a VPC, which is the default.
The guide I linked to above, shows a non-VPC, but then gives instructions on a VPC deployment.   This caused issues.
Some instance sizes for both RDS and the EC2 instance ELB creates will ONLY run in a VPC (anything with a “t” level).
You will need to choose the larger “m-size” instances for RDS and EC2 otherwise the ELB setup will fail after 15-20 minutes of “spinning its wheels”.

AWS ELB Create RDS not in VPC
AWS ELB Create RDS not in VPC

Set your configuration details.

Choose an instance type of m*, I chose m3.medium the first time around, but m1.small should suffice for a small WP site.

Select an EC2 key pair to be able to connect with SSH. If you did not create one on your MAIN AWS login, got the the IAM panel and do that now. Save the private key on your local ox and make a backup of it.

The email address is not required, I like to know if the environment changed especially if I did not change it.

Set the application health check URL to
HTTP:80/readme.html

Uncheck rolling updates.

Defaults for the rest will work.

AWS ELB Configuration Details
AWS ELB Configuration Details

You can set some tags for the environment, but it is not necessary. Supposedly they help in reporting on account usage, but I’m not that far along yet.

AWS ELB Tags
AWS ELB Tags

Setup your RDS instance.
Again, choose an m* instance as the t* instances will not boot unless you are in a VPC.
If you choose the wrong instance ELB will “sit and spin” for something that seems to be a decade, before booting to “gray state” which is AWS terminology for half-ass and useless.
If you cannot tell, this was the most frustrating part of the setup as I tried SEVERAL different instance classes.    Each time the ELB would hang and then take forever to delete.

Enter your DB username and password.
They will be auto-configured by the wp-config.php hack you made earlier.I do recommend, however, saving these somewhere in case you need to connect to MySQL remotely.  I hosed my host and siteurl and needed to go to my local dev box, fire up MySQL command line, and update the wp_options table after I booted my application in ELB.    Having the username/password for the DB is helpful for that type of thing.

AWS ELB RDS Config
AWS ELB RDS Config

Review your settings, launch and wait.

Reviewing ELB Settings

When you are done your Elastic Beanstalk should look something like this:

AWS ELB Web Tier Final Config
AWS ELB Web Tier Final Config
AWS ELB Data and Network Final Config
AWS ELB Data and Network Final Config

Useful Resources

Deploying WordPress with AWS Elastic Beanstalk – single or multiple zone, fully scalable, cached.

Deploying a WP install with git on ELB – single zone and may not scale.

 

Posted on

Getting Started With SLPDev Cent 7 Vagrant Box

This article is for the Store Locator Plus development team that is helping maintain the Store Locator Plus plugin family.   It is a basic outline on how to get the SLPDev Cent 7 virtual machine, a private distribution given to qualified developers, personalized and ready for use to start working on the Store Locator Plus plugins.

Get The Box

The first step is to get the latest SLPDev Cent 7 Vagrant box image from Charleston Software Associates.   The image is approximately 2GB and requires a working Vagrant installation on your laptop or desktop computer along with a working install of VirtualBox.   You will use the provided image to create a new VirtualBox machine with the a self-contained GUI development environment for the Store Locator Plus WordPress plugin.

What Is On The Box

The box is configured according to the CentOS 7 WP Dev Kit specification with a few additions.    An ssh key has been configured to provide easy access to the repositories. The WordPress installation has been completed with a site title “SLPDev” and an admin user login of vagrant / vagrant.   You get to the WordPress site by opening Firefox via Applications/Favorites and surfing to http://localhost/.

All of the current Store Locator Plus plugins are installed via the BitBucket git repositories including:

  • Store Locator Plus
  • Contact Extender
  • Directory Builder
  • Enhanced Map
  • Enhanced Results
  • Enhanced Search
  • Event Location Manager (in development, debaat/CSA)
  • Janitor
  • Location Extender
  • Pro Pack
  • Real Estate Extender (in development, aknight/CSA)
  • Social Media Extender (debaat)
  • Store Pages
  • User Managed Locations (debaat)
  • Widgets

The Selenium IDE test suite is installed in the vagrant home directory as is the WordPress Dev Kit with the Store Locator Plus publication scripts.

Personalize The Box

Before starting development you will want to change several identifiers so your code updates can be attributed to you.    You will need to run SmartGit and enter your BitBucket username and password credentials to access the repositories.    You will also want to configure git to set your username and email as the default commit author.

Git / SmartGit Update

Run the following commands from the Linux command line terminal (Applications / Favorites):


git config --global user.email 'your@email.com'

git config --global user.name 'Your Name'

The next thing you should do for SLP development is open the SmartGit repository and pull (rebase, not merge as the default mode) and fetch the latest updates for any plugins you are going to work on.

Posted on

Creating A Self Contained WordPress GUI Dev Box

WordPress CentOS 7 Banner

After a few days of doing a whole lot of nothing as I recover from surgery I decided that it was time to start doing some computer hacking.   I’m not nearly as tired today as I have been.    The post-surgery cobwebs are mostly gone and I’m thinking clearly enough to do some productive computer work.   I’m still not 100% so staying away from code for another day may not be a bad idea.   I figure creating an upgraded WordPress Development Kit box has no chance of wreaking havoc on my published WordPress plugins.   Today is going to be an “upgrade my virtual development environment” day.

The task at hand?   Upgrade my WordPress Development Kit Vagrant box.

The original box runs a fully self-contained GUI development box on CentOS 6.5 running a local version of WordPress 3.9 with some PHP developer tools like NetBeans at-the-ready.     This makes it easy to “pack up and move” my entire development and testing system and bring it to any laptop that can run VirtualBox.     This also gives me a solid baseline to get new developers started with the same exact tools and user experience which expedites the process of getting new people online with helping code the WordPress plugins.

Today I am upgrading the WP Dev Kit Box to run on CentOS 7.     CentOS 7 has an upgraded desktop GUI and, more importantly, is virtualization-aware with built-in OS-level tools that detect when it is running as a VirtualBox guest.   It should make for a more seamless experience when going between the virtual guest box and the host system.   I’ve also moved my development over to phpStorm versus NetBeans so an eval copy of phpStorm will be installed along with the latest free version of NetBeans.   SmartGit and grunt will remain as part of the plugin development toolkit.    WordPress will be the latest production version of 4.x.

Guest System Specifications

  • CentOS 7 with GUI
  • 2GB RAM
  • 20GB dynamic HDD
  • 4 CPU cores
  • 1x 32MB video at 1440 x 1050 resolution
  • no audio
  • USB 2 support
  • NAT networking
  • Bidirectional clipboard.
  • Bidirectional drag-and-drop.
  • Port Forwarding localhost (127.0.0.1) port 4567 to guest port 22 (ssh).
  • Standard vagrant user credentials.

Augmenting Minimal Setup

The box is installed using minimal install for CentOS 7.  The following elements have been updated to get to a standard WordPress plugin development environment:

  • vim installed
  • gvim installed
  • vagrant user added to sudoers file
  • GNOME desktop added
  • Graphical Admin Tools added
  • GUI runlevel (5) set as default
  • Install VirtualBox guest additions
  • Run the software update for all packages to 19-Oct-2014
  • Added the vagrant “base box public key” to the authorized keys file (allows vagrant scripts to run commands)

CentOS 7 GUI Base Box

The above configuration, without the developer tools like NetBeans, phpStorm, SmartGit, and Grunt nor the WordPress install is published as the “CentOS 7 GUI Base” box on VagrantCloud.   It is this foundation on which the WP Dev Kit Box will be built.

CentOS 7 WP Dev Kit Box

Now that the base GUI box is built it is time to layer on the WordPress system and development tools.  The following items are added to the mix:

  • Install PHP and PHP MySQL library.
  • Start httpd web server at boot (note: CentOS 7 uses the systemctl app to configure startup apps)
  • Install MariaDB (public domain MySQL branch) and set to start at boot.
  • Disable SELinux
  • Create the WordPress database.
    • database name: wordpress
    • database user: wpuser
    • database password: wppasswd
  • Install WordPress 4.0 and activate up to the “Enter site title, admin username, admin password” stage.
  • Install git
  • Install SmartGit 6.0.7 eval
  • Install NetBeans 8.0.1 HTML & PHP configuration
  • Install phpStorm 8.0.1 eval
  • Add Selenium IDE with GoTo lib, LastPass, and FireBug extensions to Firefox.

The WP Dev Kit box is also available in the Vagrant Cloud

 

Posted on

Creating A CentOS GUI Vagrant Base Box

CentOS 6.5 Vagrant Login Banner

While playing with PuPHPet and Vagrant I realized my needs are specific enough to warrant building my own Vagrant Base Box.    My process is outlined below.

Setup VirtualBox Hardware

Start VirtualBox and build a new guest “hardware” profile:

  • Base Memory: 2048MB
  • Processors: 2
  • Boot Order: CD/DVD , Hard Disk
  • Acceleration: VT-x/AMD-V , Nested Paging , PAE/NX
  • Display: 32MB Video Memory , 3D Acceleration
  • Network: Intel PRO/1000 MT Desktop (NAT)
  • Drive: SATA with 20GB pre-allocated fixed disk
  • CD/DVD : IDE Secondary Master Empty
  • No USB, Audio, or Shared Folders
VirtualBox CentOS 6.5 GUI Base Box
VirtualBox CentOS 6.5 GUI Base Box

Base Box “Unbest” Practice

These base settings do not fall within the Vagrant Base Box best practices, however I need something a bit different than the typical Vagrant box configuration which is why I am building my own.   I build my boxes with a full GUI which enables me to spin up the virtual environment, login to the GUI, and have my entire development environment in a self-contained portable setting.    There are “lightweight” ways to accomplish this but I do have my reasons for building out my WordPress development environment this way which has been outlined in previous posts.

Adding the Operating System

Now that I have the base box setup it is time to layer on the CentOS 6.5 operating system.   I setup my box for the English language with a time zone of New York (United States EST, UTC+5), no kernel dump abilities, full drive allocated to the operating system.     It is built as a “Desktop” server which gives me the full GUI login which makes it easier to setup my GUI dev environment further on down the road.  It does add some GUI apps I don’t need very often but it is nice to have things like a simple GUI text editor and GUI system management tools for the rare cases when I want them and am too lazy to jump out to my host box to do the work.

Per Vagrant standards the box profile is setup with the root password of “vagrant” and with a base user for daily use with an username and password also set to “vagrant”.

After a couple of reboots the system is ready for a GUI login, but not quite ready for full production.

CentOS 6.5 Login Screen
CentOS 6.5 Login Screen

Adding VirtualBox Guest Additions

One of the first things to do with a VirtualBox install running a GUI is to get VirtualBox Guest Additions installed.  It helps the guest communicate with the host in a more efficient manner which greatly improves the display and the mouse tracking.  Without it the mouse lag in the guest is horrid and is likely responsible for at least 300 of the 3,000 missing hair follicles on my big bald head.

While this SHOULD be a simple operation, the CentOS desktop installation makes it a multi-step process.   Selecting “insert Guest Additions CD” from the VirtualBox server menu after starting up the new box will mount the disk.   It will prompt to autorun the disk and then ask for the root user credentials.    The shell script starts running through the Guest Additions setup but it always falls while building the main Guest Additions module.     The reason is that kernel build kits are needed and they are not installed by default.    I will outline the typical user process here as a point of reference, though most often the first commands I run to fix the issue are those listed at the end of this section.  I’ve done this enough times to know what happens and don’t usually execute the autorun until AFTER I setup the kernel build kit.  You may want to do the same.

Here is what the output looks like after a default CentOS desktop install followed by an autorun of the Guest Additions CD:

Guest Additions Fail on CentOS
This is what happens when you don’t have Kernel build tools setup and try to run Guest Additions on VirtualBox.

[box type=”info” style=”rounded”]Mouse tracking driving you crazy? Toggle to a command line screen on any Linux box with CTRL-ALT-F2. Toggle back to the GUI with CTRL-ALT-F1.[/box]

With the mouse tracking driving me nuts I toggle over to the text console with ctrl-alt-F1 and login as root on there.   You can learn what broke the Guest Addition install by going to the log files:

more vboxxadd-install.log

The typical CentOS desktop build fails the Guest Additions install with this log:

/tmp/vobx.0/Makefile.include.header:97: *** Error: unable to find the sources of your current Linux kernel. Specify KERN_DIR= and run Make again. Stop.<br />Creating user for the Guest Additions.<br />Creating udev rule for the Guest Additions kernel module.<br />

With Guest Additions disabled and the VirtualBox not fully configured it is time to do some basic maintenance and get the kernel build environment available for Guest Additions.  Since I am logged in as root via the console I can start by getting yum updated, however the network connection does not always start up before Guest Additions is available.    The steps for getting the kernel dev in place:

Turn on the network interface eth0 (zero not oh) running:

ifup eth0

Make sure all of the installed software is updated to the latest revision:

yum update

Install the Linux kernel development files which are needed for the Guest Additions installation:

yum install kernel-devel

Install the development tool kit including compilers and other items needed to Guest Additions to hook into the kernel:

yum groupinstall "Development Tools"

Once you have the updates installed reboot the system with a shutdown -r now command while logged in as root.

The Guest Additions CD can now be mounted and autorun without error.

After running Guest Additions, reboot the server.

Turn On The Network At Boot

Now that the GUI is running and the mouse is tracking I can log in as the vagrant user and turn on the network connections.   Login, go  to System / Preferences / Network Connections on the main menu.    Check off “Connect Automatically” on the System eth0 connection.

Now the network will be enabled on boot.   That’s useful.

CentOS 6.5 Turn On Network At Boot
CentOS 6.5 turning on the network at boot.

Provide SSH Insecure Keypair To Vagrant

Best practices for Vagrant base boxes is to add an insecure keypair to the vagrant user.   While logged in as vagrant go to Applications/Systems Tools/Terminal to get to the command line.   Go the .ssh subdirectory and create the authorized_keys file by copying the public key from the Vagrant keypair repository into the authorized_keys file.

I use vim and copy the keypair content and paste it into the file.  You can use cat or other tools as well to get the content into the file.  Make sure not to introduce new whitespace in the middle of the key or it will not work.

Change the permissions of the authorized_keys file by using chmod, permission settings are very important for the file:

chmod 0600 authorized_keys 

Give Unrestricted Super Powers To Vagrant

Most users expect the vagrant login to have unrestricted access to all system commands. This is handled via the sudo application. CentOS restricts access by default and requires some updates to get it working per Vagrant best practices. Log back in to the command line console as root and edit the sudo file.

visudo

This brings up the vim editor with the sudo config file. Find the requiretty line and comment it out by adding a # before it. Then add the following line to the bottom of the file:

vagrant ALL=(ALL) NOPASSWD: ALL

Logout of the vagrant and root sessions and log back in as vagrant from the GUI. You should be able to open a terminal and run any sudo commands without a password prompt. You should also be able to run sudo commands “remotely” via the ssh connection to the system.

Make SSH Faster When DNS Is Not Available

If the host and/or virtual box cannot connect to the Internet the SSH access into the Vagrant virtual box will be slow.   Editing the sshd_config file and turning off DNS lookups will fix that.   Now that you have “vagrant super powers” you can do this by logging in as the vagrant user and opening the terminal:

sudo vim /etc/ssh/sshd_config

Add this line to the bottom of the file:

UseDNS no

Host To Guest SSH Access

Connecting from the host system to the guest system WITHOUT using the graphical login or console takes a couple of extra steps. To test the SSH connection I go back to my favorite SSH tool, PuTTY.     Before testing the connection the port forwarding needs to be setup on VirtualBox Manager.

  • Go to the new system listed on the VirtualBox Manager.
  • Right-click and select Settings.
  • Select Network.
  • Click the Port Forwarding button.
  • Add the following rule:
    • Name: SSH Local To Guest
    • Protocol: TCP
    • Host IP: 127.0.0.1
    • Host Port: 4567
    • Guest IP: leave this blank
    • Guest Port: 22

Save the settings.   Open PuTTY and connect to hostname 127.0.0.1 and change the port to be 4567.   You should get a login prompt.   Login with user vagrant.

VirtualBox SSH Port Forwarding
VirtualBox SSH port forwarding for Vagrant.

The issue with logging in with the vagrant private key file is that PuTTY only supports the proprietary PuTTY Private Key format.    You can download puttygen to convert the Vagrant private key file to the PuTTY Private Key file format (click to download the converted OpenSSH key in PPK format).

To use SSH keys in PuTTY, start a new session, enter 127.0.01 as the host and 4567 as the port, then set the PuTTY Private Key:

  • Click on “connection / SSH” in the left side menu to expand that selection.
  • Click on “Auth”.
  • Under Authentication parameters browse to your saved PPK file in the “Private key file for authentication” box.
Setting PuTTY Vagrant PPK
Setting PuTTY Vagrant PPK files.

Now you can connect with PuTTY and login by simply supplying a username.   This tells us that the remote vagrant command line should be able to execute all of the scripted setup commands without any issues.

Building A Box

Now that the basic system is in place it is time to “build the box”.   Vagrant has a command for doing this and if you’ve read my previous articles on setting up Vagrant you will know that I have a Windows command line shortcut that runs in my WP Development Kit folder.   With Vagrant already installed building a box is a one-line command.   I only need my machine name, which I’ve shorted to “CentOS6.5 GUI Base Box”.  Start up the Windows command line and run this:

vagrant package --base "CentOS6.5 GUI Base Box"

It will run for a while and eventually create a packaged Vagrant box ready for distribution.    By default the file will be named package.box.    I’ve renamed mine to centos6_5-gui-base.box for distribution purposes.   You can find it on my Vagrant Cloud account.

You can learn more about the box-building process via the Vagrant Creating A Base Box page.

Launching The Box

To launch the new box hosted on Vagrant Cloud I go to my local folder and execute these commands:

Download the image (stored on my Google Drive account) using Vagrant Cloud as a proxy:

vagrant box add charlestonsw/centos6.5-gui-base-box 

Create the vagrantfile that assists in the box startup command sequence:

vagrant init charlestonsw/centos6.5-gui-base-box

Start the box on VirtualBox:

vagrant up

By default, Vagrant starts boxes in headless mode, meaning no active console.   I want the GUI login so I shut down the box and find the vagrantfile to add the GUI startup line.    The command is already in the file and only needs a few lines to be uncommented to allow a GUI startup with a console.    Edit the vagrantfile and look for these lines:

config.vm.provider "virtualbox" do |v|
v.gui = true
end

There are few other comments in the default vagrantfile, you can leave the limits tweaks commented.  You will end up with a vagrantfile section that looks like this:


# Provider-specific configuration so you can fine-tune various
 # backing providers for Vagrant. These expose provider-specific options.
 # Example for VirtualBox:
 #
 config.vm.provider "virtualbox" do |vb|
 # Don't boot with headless mode
 vb.gui = true

 # # Use VBoxManage to customize the VM. For example to change memory:
 # vb.customize ["modifyvm", :id, "--memory", "1024"]
 end

Save the file and restart the box with the vagrant up box.

That’s it… a new Vagrant box.   Now on to the system tweaks to get my WP Dev Kit setup.

Posted on

Automated Virtual Box Creation V1.0 Notes

PuPHPet Banner

If you read my previous article,  WordPress Workflow : Automated Virtual Box Creation , you have an idea of what I am trying to accomplish with improving my WordPress development work flow.    The short version, I want to be able to create a fresh install of a virtual machine that has my entire development system intact with minimal input on my part.    The idea is to run a few commands, wait for the installs and updates, and be coding on a “clean” machine shortly after.    Once I get my own work flow updated I will also be able to share my scripts and tools via a git repository with the remote developers that are now working on Store Locator Plus add-on packs and hopefully simplify their development efforts or at least get all of us on a similar baseline of tools to improve efficiency in our efforts.

Here are my notes from the first virtual development box efforts via PuPHPet, Vagrant, and Puppet.    This build was done with recent “off-the-shelf” versions of each of these tools and using a base configuration with a handful of options from the PuPHPet site.

Headless Configuration

The VirtualBox machine appears to be created as a “headless” box, meaning no monitor or other display device is active.   I will need to tweak that as I work “on the box” with GUI development tools.    I know that I can install all of my development tools on my host system and read/write from a shared directory to get all of my work onto the virtual machine, but that is not my methodology.    Having worked with a team of developers I know all too well that eventually the host hardware will die.   A laptop will need to be sent off for repair.   Guess what happens?   You lose half-a-day, or more, setting up a new host with a whole new install of development tools.

The better solution, for my work flow, is to keep as much of the development environment “self contained” within the virtual box as possible.   This way when I backup my virtual disk image I get EVERYTHING I need in an all-in-one restore point.   I can also replicate and share my EXACT environment to any location in the world and be fully  “up and running” in the time it takes to pull down a 20GB install file.  In today’s world of super-fast Internet that is less of an issue than individually pulling down and installing a half-dozen working tools and hoping they are all configured properly.

What does this all mean?    I need to figure out how to get the PuPHPet base configuration tweaked so I can start up right from the VirtualBox console with a full Linux console available.  I’ll likely need to update Puppet as well to make sure it pulls down the Desktop package on CentOS.

I wonder if I can submit a build profile via a git pull request to PuPHPet.

Out-Of-Box Video Memory Too Low

The first hurdle with configuring a “login box” with monitor support will be adjusting the video RAM.   My laptop has 4GB of dedicated video RAM on a Quadro K3100M GPU.   It can handle a few virtual monitors and has PLENTY of room for more video RAM.   Tweaking the default video configuration is in order.

Since Vagrant “spins up” the box when running the vagrant up command the initial fix starts by sending an ACPI shutdown request to the system.     Testing the video RAM concept is easy.   Get to the VirtualBox GUI, right-click the box and select properties.   Adjust the video RAM to 32MB and turn on 3D accelerator (it makes the GUI desktop happy) and restart.

Looks like I can now get direct console login.  Nice!

PuPHPet Virtual Box with Active Console
PuPHPet Virtual Box with Active Console

Access Credentials

The second issue, which I realized after seeing the login prompt, is that I have NO IDEA what the login credentials are for the system.   This doesn’t matter much when you read/write the shared folders on your host to update the server and only “surf to” the box on port 8080 or SSH in with a pre-shared key, but for console login a username and password are kind of important.   And I have no clue what the default is configured as.  Time for some research.   First stop?  The vagrantfile that built the beast.

Buried within that vagrantfile, which looks just like Ruby syntax (I’m fairly certain it is Ruby code), is a user name “vagrant”.    My first guess?  Username: vagrant, password: vagrant.     Looks like that worked just fine.    Now I have a console login that “gets me around”, but it is not an elevated permissions user level such as root.   However, a simple sudo su – resolves that issue granting me full “keys to the kingdom”.

[box type=”info” size=”large” style=”rounded”]Vagrant Boxes Credentials are username vagrant, password vagrant[/box]

A good start.   Now to wreak some havoc to see what is on this box and where so I can start crafting some Puppet rule changes.   Before I get started I want to get a GUI desktop on here.

GUI Desktop

To get a GUI desktop on CentOS you typically run the yum package installer with yum groupinstall Desktop.    A visit under sudo su and executing that command gets yum going and pulling down the full X11/Gnome desktop environment.

A quick reboot with shutdown -r now from the root command line should bring up the desktop this time around… but clearly I missed a step as I still have a console login.  Most likely a missing startx command or something similar in the boot sequence of init.d.

A basic startx & from the command line after logging back in as vagrant/vagrant and my GUI desktop is in place, so clearly I need to turn on the GUI login/boot loader.

Tweaking PuPHPet Box Parameters

Now that I know what needs to change I need to go and create that environment via the PuPHPet/Vagrant/Puppet files so I can skip the manual tweaking process.   After some digging I found the config.yaml file.    When you use PuPHPet this file will be put in the .zip download you receive at the end of the PuPHPet process.   It is in the <boxid>/puphpet/ directory.

PuPHPet config.yaml
PuPHPet config.yaml

While some of the box parameters can be adjusted in these files, it appears much of the hardware cannot be manipulated.  There is a site called “Vagrant Cloud” that has multiple boxes that can be configured.   To switch boxes you can edit the config.yaml file and replace the box_url line to point to one of the other variants that may be closer to your configuration.  Since I don’t see one that is close to my needs it looks like I will have to build my own box profile to be hosted in the cloud.   That is content for another article.

 

Posted on

WordPress Workflow : Automated Virtual Box Creation

PuPHPet Vagrant Puppet Banner

I am into my first full day back after WordCamp Atlanta (#wcatl) and have caught up on most of my inbox, Twitter, and Facebook communications.   As I head into a new week of WordPress plugin production I decided now is as good a time as any to update my work flow.

I learned a lot of new things at WordCamp and if there is one thing I’ve learned from past experience it is DO NOT WAIT.   I find the longer I take to start implementing an idea the less chance I have of executing.

My first WordCamp Atlanta 2014 work flow improvement starts right at the base level.   Setting up a clean local development box.   I had started this process last week by manually configuring a baseline CentOS box and was about to setup MySQL, PHP, and all the other goodies by hand.  That was before I learned more about exactly what Vagrant can do.   I had heard of Vagrant but did not fully internalize how it can help me.  Not until this past weekend, that is.

My Work Environment

Before I outline my experience with the process I will share my plugin development work environment.

  • Host System: Windows 8.1 64-bit on an HP Zbook laptop with 16GB of RAM with a 600GB SATA drive
  • Guest System: CentOS 6.5 (latest build) with 8GB RAM on an Oracle VirtualBox virtual machine
    • Linux Kernel 2.6.32-431
    • PHP v5.4.23
    • MySQL v 14.14 dist 5.5.35
  • Dev Took Kit: NetBeans, SmartGit, Apigen and phpDoc, MySQL command line, vim
HP Zbook Windows 411
My Development System laptop config.

While that is my TYPICAL development environment, every-so-often I swap something out such as the MySQL version or PHP version and it is a HUGE PAIN.    This is where Vagrant should help.  I can spin up different virtual boxes such as a single-monitor versus three-monitor configuration when I am on the road or a box with a different version of PHP.     At least that is the theory anyway.   For now I want to focus on getting a “clean” CentOS 6.5 build with my core applications running so I can get back to releasing the Store Locator Plus Enhanced Results add-on pack this week.

Getting Started With Vagrant

The Rockin’ Local Development With Vagrant talk that Russel Fair gave on Saturday had me a bit worried as he was clearly on the OS/X host and the examples looked great from a command line standpoint.  Being a Linux geek I love command line, but I am not about to run virtual development boxes in in a VirtualBox guest.   Seems like a Pandora’s box to me… or at least a Russian doll that will surely slow down performance.   Instead I want to make sure I have Vagrant running on my Windows 8.1 bare metal host.    That is very much against my “full dev environment in a self-contained and portable virtual environment” standard, but one “helper tool” with configurations backed up to my remote Bitbucket repository shouldn’t be too bad, as long as I don’t make it a habit to put dev workflow tools on my host box. Yes, Vagrant does have a Windows installer and I’m fairly certain I won’t need to be running command-line windows to make stuff work.   If I’m running Windows I expect native apps to be fully configurable via the GUI.  Worst case I may need to open a text editor to tweak some files, but no command line please.

Here is the process for a Windows 8.1 install.

  • Download Vagrant.
  • Install needs to be run as admin and requires a system reboot.
  • Ok… it did something… but what?   No icons on the desktop or task bar or … well… anywhere that I can find!

Well… sadly it turns out that Vagrant appears to be a command line only port of the Linux/OSX variants.    No desktop icons, no GUI interface.   I get it.  Doing that is the fast and easy process, but to engage people on the Microsoft desktop you really do need a GUI.    Yes, I’m geek enough to do this and figure it out.   I can also run git command line with no problem but I am FAR more efficient with things like the SmartGit GUI interface.

Maybe I’m not a real geek, but I don’t think using command line and keyboard interaction as the ONLY method for interacting with a computer makes you a real techie.    There is a reason I use a graphical IDE instead of vim these days.    I can do a majority of my work with vim, but it is FAR more efficient to use the GUI elements of my code editor.

Note to Vagrant: if you are doing a windows port at least drop a shortcut icon on the desktop and/or task bar and setup a Windows installer.   Phase 2: consider building a GUI interface on top of the command line system.

It looks like Vagrant is a lower-level command line tool.   It will definitely still have its place, but much like git, this is a too on which other “helpers” need to be added to make my workflow truly efficient.  Time to see what other tools are out there.

Kinda GUI Vagrant : PuPHPet

Luckily some other code geeks seem to like the idea of  GUI configuration system and guess what?   Someone created a tool called PuPHPet (which I also saw referenced at WordCamp so it must be cool)  and even wrote an article about Vagrant and Puppet.   Puppet is a “add on”, called a provisioner,  to setup the guest software environment.

PuPHPet is an online form-based system that builds the text-file configuration scripts that are needed by Vagrant to build and configure your Virtualbox (or VMWare) servers.   It is fairly solid for building a WordPress development environment, but it does mean reverting back to CentOS 6.4 as CentOS 6.5 build scripts are not online.     Though I am sure I can tweak that line of the config files and fix that, but that takes me one-step away from the “point and click” operation I am looking for.

Either way, PuPHPet, is very cool and definitely worth playing with if you are going to be doing any WordPress-centric Vagrant work.

PuPHPet Intro Page
The PuPHPet online configuration tool for creating Vagrant + Puppet config files.

 

Puppet Makes Vagrant and PuPHPet Smarter

Now that I have Vagrant installed and I discovered PuPHPet I feel like I am getting closer to a “spin me up a new virtual dev box, destroy-as-desired, repeat” configuration.  The first part of my workflow improvement process.   BUT…. I need one more thing to take care of it seems… get Puppet installed.   I managed to wade through the documentation (and a few videos) to find the Windows installers.

Based on what is coming up in the install window it looks like the installer will roll out some Apache libs, ruby, and the windows kits that help ruby run on a windows box.

Puppet Install Licenses
The Puppet installer on Windows.

Again, much like Vagrant, Puppet completes the installation with little hint of what it has done.    Puppet is another command line utility that runs at a lower-level to configure the server environments.   It will need some of the “special sauce” to facilitate its use.     A little bit of digging has shown that the Puppet files are all installed under the C:\Program Files (x86)\Puppet Labs folder.    On Windows 8.1 the “Start Menu” is MIA, so the documentation about finding shortcuts there won’t help you.    Apparently those shortcuts are links to HTML doc pages and some basic Windows shell scripts (aka Batch Files) so nothing critical appears to have gone missing.

The two files that are referenced most often are the puppet and facter scripts, so we’ll want to keep track of those.   I’ll create a new folder under My Documents called “WP Development Kit” where I can start dumping things that will help me managed my Windows hosted virtual development environment for WordPress. While I’m at it I will put some links in there for Vagrant and get my PuPHPet files all into a single reference point.

WP Dev Kit Directory
The start of my WP Dev Kit directory. Makes finding my PuPHPet, Vagrant, and Puppet files easier.

Now to get all these command line programs to do my bidding.

Getting It Up

After a few hours or reading, downloading, installing, reading some more, and chasing my son around the house as the “brain eating dad-zombie”, I am ready to try to make it all do something for me.    Apparently I need to use something called a “command line”.  On Windows 8.1.

I’m giving in with the hopes that this small foray into the 1980’s world of command line system administration will yield great benefits that will soon make me forget that DOS still exists under all these fancy icons and windows.   Off to the “black screen of despair”, on of the lesser-known Windows brethren of the “blue screen of death”.     Though Windows 8 tries very hard to hide the underpinnings of the operating system, a recent Windows 8 patch and part of Windows 8.1 since “birth” is the ever-useful Windows-x keyboard shortcut.   If you don’t know this one, you should.   Hold down the Windows key and press x.   You will get a Windows pop-up menu that will allow you to select, among many other things, the Command Prompt application.

If you right-click on the “do you really want to go down this rabbit hole” confirmation box that comes up with the Command Prompt (admin) program you will see that it is running C:\Windows\system32\cmd.exe.     This will be useful for creating a shortcut link that will allow me to not only be in command mode but also to be in the “source” directory of my PuPHPet file set.    I’m going to create a shortcut to that application in my new WP Development Kit directory along with some new parameters:

  • Search for cmd.exe and find the one in the Windows\system32 directory.
  • Right-click and drag the file over to my WP Development Kit folder, selecting “create shortcuts here” when I drop it.
  • My shortcut to cmd.exe is put in place, but needs tweaking…
  • Right-click the shortcut and set the “Start in” to my full WP Development Kit folder.

Now I can double-click the command prompt shortcut in my WP Development Kit folder and not need to change directory to a full path or “up and down the directory tree” to get to my configuration environment.

Running Vagrant andn Puppet via PuPHPet Scripts
Running Vagrant andn Puppet via PuPHPet Scripts

A few key presses later and I’ve managed to change to my downloaded PuPHPet directory and execute the “vagrant up” command.   Gears starting whirring, download counters started ticking, and it appears the PuPHPet/Vagrant/Puppet trio are working together to make something happen.  At the very least it is downloading a bunch of stuff from far away lands and filling up my hard drive.   Hopefully with useful Virtualbox disk images and applications required to get things fired up for my new WordPress dev box.

We’ll see…

Link Summary

Posted on

VMWare Mounting Windows Host Folder On CentOS Guest

VMWare WorkstationAfter a failed upgrade of VMWare-Tools on VMWare Workstation 7.1.5, I ended up with a shared folder that would not mount automatically. After a bit of digging on the Internet I found the solution.  Here are my brief notes on how to manually mount a shared Windows host folder.

  1. Make sure you have the host folder shared and always enabled or enabled until next power off.

     

    VMWare Share Folder
    VMWare Share Folder
  2. Run the vmware mount client:
    /usr/bin/vmware-hgfsclient

    It will return the name of the shared host folder, it was named “Documents” in my case.

  3. Mount the host folder:
    mount -t vmhgfs .host:/Documents /mnt/hgfs

That’s it.  Hope that helps others that may have lost their auto-mounted windows host folders.