Posted on

Creating A CentOS GUI Vagrant Base Box

CentOS 6.5 Vagrant Login Banner

While playing with PuPHPet and Vagrant I realized my needs are specific enough to warrant building my own Vagrant Base Box.    My process is outlined below.

Setup VirtualBox Hardware

Start VirtualBox and build a new guest “hardware” profile:

  • Base Memory: 2048MB
  • Processors: 2
  • Boot Order: CD/DVD , Hard Disk
  • Acceleration: VT-x/AMD-V , Nested Paging , PAE/NX
  • Display: 32MB Video Memory , 3D Acceleration
  • Network: Intel PRO/1000 MT Desktop (NAT)
  • Drive: SATA with 20GB pre-allocated fixed disk
  • CD/DVD : IDE Secondary Master Empty
  • No USB, Audio, or Shared Folders
VirtualBox CentOS 6.5 GUI Base Box
VirtualBox CentOS 6.5 GUI Base Box

Base Box “Unbest” Practice

These base settings do not fall within the Vagrant Base Box best practices, however I need something a bit different than the typical Vagrant box configuration which is why I am building my own.   I build my boxes with a full GUI which enables me to spin up the virtual environment, login to the GUI, and have my entire development environment in a self-contained portable setting.    There are “lightweight” ways to accomplish this but I do have my reasons for building out my WordPress development environment this way which has been outlined in previous posts.

Adding the Operating System

Now that I have the base box setup it is time to layer on the CentOS 6.5 operating system.   I setup my box for the English language with a time zone of New York (United States EST, UTC+5), no kernel dump abilities, full drive allocated to the operating system.     It is built as a “Desktop” server which gives me the full GUI login which makes it easier to setup my GUI dev environment further on down the road.  It does add some GUI apps I don’t need very often but it is nice to have things like a simple GUI text editor and GUI system management tools for the rare cases when I want them and am too lazy to jump out to my host box to do the work.

Per Vagrant standards the box profile is setup with the root password of “vagrant” and with a base user for daily use with an username and password also set to “vagrant”.

After a couple of reboots the system is ready for a GUI login, but not quite ready for full production.

CentOS 6.5 Login Screen
CentOS 6.5 Login Screen

Adding VirtualBox Guest Additions

One of the first things to do with a VirtualBox install running a GUI is to get VirtualBox Guest Additions installed.  It helps the guest communicate with the host in a more efficient manner which greatly improves the display and the mouse tracking.  Without it the mouse lag in the guest is horrid and is likely responsible for at least 300 of the 3,000 missing hair follicles on my big bald head.

While this SHOULD be a simple operation, the CentOS desktop installation makes it a multi-step process.   Selecting “insert Guest Additions CD” from the VirtualBox server menu after starting up the new box will mount the disk.   It will prompt to autorun the disk and then ask for the root user credentials.    The shell script starts running through the Guest Additions setup but it always falls while building the main Guest Additions module.     The reason is that kernel build kits are needed and they are not installed by default.    I will outline the typical user process here as a point of reference, though most often the first commands I run to fix the issue are those listed at the end of this section.  I’ve done this enough times to know what happens and don’t usually execute the autorun until AFTER I setup the kernel build kit.  You may want to do the same.

Here is what the output looks like after a default CentOS desktop install followed by an autorun of the Guest Additions CD:

Guest Additions Fail on CentOS
This is what happens when you don’t have Kernel build tools setup and try to run Guest Additions on VirtualBox.

[box type=”info” style=”rounded”]Mouse tracking driving you crazy? Toggle to a command line screen on any Linux box with CTRL-ALT-F2. Toggle back to the GUI with CTRL-ALT-F1.[/box]

With the mouse tracking driving me nuts I toggle over to the text console with ctrl-alt-F1 and login as root on there.   You can learn what broke the Guest Addition install by going to the log files:

more vboxxadd-install.log

The typical CentOS desktop build fails the Guest Additions install with this log:

/tmp/vobx.0/Makefile.include.header:97: *** Error: unable to find the sources of your current Linux kernel. Specify KERN_DIR= and run Make again. Stop.<br />Creating user for the Guest Additions.<br />Creating udev rule for the Guest Additions kernel module.<br />

With Guest Additions disabled and the VirtualBox not fully configured it is time to do some basic maintenance and get the kernel build environment available for Guest Additions.  Since I am logged in as root via the console I can start by getting yum updated, however the network connection does not always start up before Guest Additions is available.    The steps for getting the kernel dev in place:

Turn on the network interface eth0 (zero not oh) running:

ifup eth0

Make sure all of the installed software is updated to the latest revision:

yum update

Install the Linux kernel development files which are needed for the Guest Additions installation:

yum install kernel-devel

Install the development tool kit including compilers and other items needed to Guest Additions to hook into the kernel:

yum groupinstall "Development Tools"

Once you have the updates installed reboot the system with a shutdown -r now command while logged in as root.

The Guest Additions CD can now be mounted and autorun without error.

After running Guest Additions, reboot the server.

Turn On The Network At Boot

Now that the GUI is running and the mouse is tracking I can log in as the vagrant user and turn on the network connections.   Login, go  to System / Preferences / Network Connections on the main menu.    Check off “Connect Automatically” on the System eth0 connection.

Now the network will be enabled on boot.   That’s useful.

CentOS 6.5 Turn On Network At Boot
CentOS 6.5 turning on the network at boot.

Provide SSH Insecure Keypair To Vagrant

Best practices for Vagrant base boxes is to add an insecure keypair to the vagrant user.   While logged in as vagrant go to Applications/Systems Tools/Terminal to get to the command line.   Go the .ssh subdirectory and create the authorized_keys file by copying the public key from the Vagrant keypair repository into the authorized_keys file.

I use vim and copy the keypair content and paste it into the file.  You can use cat or other tools as well to get the content into the file.  Make sure not to introduce new whitespace in the middle of the key or it will not work.

Change the permissions of the authorized_keys file by using chmod, permission settings are very important for the file:

chmod 0600 authorized_keys 

Give Unrestricted Super Powers To Vagrant

Most users expect the vagrant login to have unrestricted access to all system commands. This is handled via the sudo application. CentOS restricts access by default and requires some updates to get it working per Vagrant best practices. Log back in to the command line console as root and edit the sudo file.

visudo

This brings up the vim editor with the sudo config file. Find the requiretty line and comment it out by adding a # before it. Then add the following line to the bottom of the file:

vagrant ALL=(ALL) NOPASSWD: ALL

Logout of the vagrant and root sessions and log back in as vagrant from the GUI. You should be able to open a terminal and run any sudo commands without a password prompt. You should also be able to run sudo commands “remotely” via the ssh connection to the system.

Make SSH Faster When DNS Is Not Available

If the host and/or virtual box cannot connect to the Internet the SSH access into the Vagrant virtual box will be slow.   Editing the sshd_config file and turning off DNS lookups will fix that.   Now that you have “vagrant super powers” you can do this by logging in as the vagrant user and opening the terminal:

sudo vim /etc/ssh/sshd_config

Add this line to the bottom of the file:

UseDNS no

Host To Guest SSH Access

Connecting from the host system to the guest system WITHOUT using the graphical login or console takes a couple of extra steps. To test the SSH connection I go back to my favorite SSH tool, PuTTY.     Before testing the connection the port forwarding needs to be setup on VirtualBox Manager.

  • Go to the new system listed on the VirtualBox Manager.
  • Right-click and select Settings.
  • Select Network.
  • Click the Port Forwarding button.
  • Add the following rule:
    • Name: SSH Local To Guest
    • Protocol: TCP
    • Host IP: 127.0.0.1
    • Host Port: 4567
    • Guest IP: leave this blank
    • Guest Port: 22

Save the settings.   Open PuTTY and connect to hostname 127.0.0.1 and change the port to be 4567.   You should get a login prompt.   Login with user vagrant.

VirtualBox SSH Port Forwarding
VirtualBox SSH port forwarding for Vagrant.

The issue with logging in with the vagrant private key file is that PuTTY only supports the proprietary PuTTY Private Key format.    You can download puttygen to convert the Vagrant private key file to the PuTTY Private Key file format (click to download the converted OpenSSH key in PPK format).

To use SSH keys in PuTTY, start a new session, enter 127.0.01 as the host and 4567 as the port, then set the PuTTY Private Key:

  • Click on “connection / SSH” in the left side menu to expand that selection.
  • Click on “Auth”.
  • Under Authentication parameters browse to your saved PPK file in the “Private key file for authentication” box.
Setting PuTTY Vagrant PPK
Setting PuTTY Vagrant PPK files.

Now you can connect with PuTTY and login by simply supplying a username.   This tells us that the remote vagrant command line should be able to execute all of the scripted setup commands without any issues.

Building A Box

Now that the basic system is in place it is time to “build the box”.   Vagrant has a command for doing this and if you’ve read my previous articles on setting up Vagrant you will know that I have a Windows command line shortcut that runs in my WP Development Kit folder.   With Vagrant already installed building a box is a one-line command.   I only need my machine name, which I’ve shorted to “CentOS6.5 GUI Base Box”.  Start up the Windows command line and run this:

vagrant package --base "CentOS6.5 GUI Base Box"

It will run for a while and eventually create a packaged Vagrant box ready for distribution.    By default the file will be named package.box.    I’ve renamed mine to centos6_5-gui-base.box for distribution purposes.   You can find it on my Vagrant Cloud account.

You can learn more about the box-building process via the Vagrant Creating A Base Box page.

Launching The Box

To launch the new box hosted on Vagrant Cloud I go to my local folder and execute these commands:

Download the image (stored on my Google Drive account) using Vagrant Cloud as a proxy:

vagrant box add charlestonsw/centos6.5-gui-base-box 

Create the vagrantfile that assists in the box startup command sequence:

vagrant init charlestonsw/centos6.5-gui-base-box

Start the box on VirtualBox:

vagrant up

By default, Vagrant starts boxes in headless mode, meaning no active console.   I want the GUI login so I shut down the box and find the vagrantfile to add the GUI startup line.    The command is already in the file and only needs a few lines to be uncommented to allow a GUI startup with a console.    Edit the vagrantfile and look for these lines:

config.vm.provider "virtualbox" do |v|
v.gui = true
end

There are few other comments in the default vagrantfile, you can leave the limits tweaks commented.  You will end up with a vagrantfile section that looks like this:


# Provider-specific configuration so you can fine-tune various
 # backing providers for Vagrant. These expose provider-specific options.
 # Example for VirtualBox:
 #
 config.vm.provider "virtualbox" do |vb|
 # Don't boot with headless mode
 vb.gui = true

 # # Use VBoxManage to customize the VM. For example to change memory:
 # vb.customize ["modifyvm", :id, "--memory", "1024"]
 end

Save the file and restart the box with the vagrant up box.

That’s it… a new Vagrant box.   Now on to the system tweaks to get my WP Dev Kit setup.

Posted on

Automated Virtual Box Creation V1.0 Notes

PuPHPet Banner

If you read my previous article,  WordPress Workflow : Automated Virtual Box Creation , you have an idea of what I am trying to accomplish with improving my WordPress development work flow.    The short version, I want to be able to create a fresh install of a virtual machine that has my entire development system intact with minimal input on my part.    The idea is to run a few commands, wait for the installs and updates, and be coding on a “clean” machine shortly after.    Once I get my own work flow updated I will also be able to share my scripts and tools via a git repository with the remote developers that are now working on Store Locator Plus add-on packs and hopefully simplify their development efforts or at least get all of us on a similar baseline of tools to improve efficiency in our efforts.

Here are my notes from the first virtual development box efforts via PuPHPet, Vagrant, and Puppet.    This build was done with recent “off-the-shelf” versions of each of these tools and using a base configuration with a handful of options from the PuPHPet site.

Headless Configuration

The VirtualBox machine appears to be created as a “headless” box, meaning no monitor or other display device is active.   I will need to tweak that as I work “on the box” with GUI development tools.    I know that I can install all of my development tools on my host system and read/write from a shared directory to get all of my work onto the virtual machine, but that is not my methodology.    Having worked with a team of developers I know all too well that eventually the host hardware will die.   A laptop will need to be sent off for repair.   Guess what happens?   You lose half-a-day, or more, setting up a new host with a whole new install of development tools.

The better solution, for my work flow, is to keep as much of the development environment “self contained” within the virtual box as possible.   This way when I backup my virtual disk image I get EVERYTHING I need in an all-in-one restore point.   I can also replicate and share my EXACT environment to any location in the world and be fully  “up and running” in the time it takes to pull down a 20GB install file.  In today’s world of super-fast Internet that is less of an issue than individually pulling down and installing a half-dozen working tools and hoping they are all configured properly.

What does this all mean?    I need to figure out how to get the PuPHPet base configuration tweaked so I can start up right from the VirtualBox console with a full Linux console available.  I’ll likely need to update Puppet as well to make sure it pulls down the Desktop package on CentOS.

I wonder if I can submit a build profile via a git pull request to PuPHPet.

Out-Of-Box Video Memory Too Low

The first hurdle with configuring a “login box” with monitor support will be adjusting the video RAM.   My laptop has 4GB of dedicated video RAM on a Quadro K3100M GPU.   It can handle a few virtual monitors and has PLENTY of room for more video RAM.   Tweaking the default video configuration is in order.

Since Vagrant “spins up” the box when running the vagrant up command the initial fix starts by sending an ACPI shutdown request to the system.     Testing the video RAM concept is easy.   Get to the VirtualBox GUI, right-click the box and select properties.   Adjust the video RAM to 32MB and turn on 3D accelerator (it makes the GUI desktop happy) and restart.

Looks like I can now get direct console login.  Nice!

PuPHPet Virtual Box with Active Console
PuPHPet Virtual Box with Active Console

Access Credentials

The second issue, which I realized after seeing the login prompt, is that I have NO IDEA what the login credentials are for the system.   This doesn’t matter much when you read/write the shared folders on your host to update the server and only “surf to” the box on port 8080 or SSH in with a pre-shared key, but for console login a username and password are kind of important.   And I have no clue what the default is configured as.  Time for some research.   First stop?  The vagrantfile that built the beast.

Buried within that vagrantfile, which looks just like Ruby syntax (I’m fairly certain it is Ruby code), is a user name “vagrant”.    My first guess?  Username: vagrant, password: vagrant.     Looks like that worked just fine.    Now I have a console login that “gets me around”, but it is not an elevated permissions user level such as root.   However, a simple sudo su – resolves that issue granting me full “keys to the kingdom”.

[box type=”info” size=”large” style=”rounded”]Vagrant Boxes Credentials are username vagrant, password vagrant[/box]

A good start.   Now to wreak some havoc to see what is on this box and where so I can start crafting some Puppet rule changes.   Before I get started I want to get a GUI desktop on here.

GUI Desktop

To get a GUI desktop on CentOS you typically run the yum package installer with yum groupinstall Desktop.    A visit under sudo su and executing that command gets yum going and pulling down the full X11/Gnome desktop environment.

A quick reboot with shutdown -r now from the root command line should bring up the desktop this time around… but clearly I missed a step as I still have a console login.  Most likely a missing startx command or something similar in the boot sequence of init.d.

A basic startx & from the command line after logging back in as vagrant/vagrant and my GUI desktop is in place, so clearly I need to turn on the GUI login/boot loader.

Tweaking PuPHPet Box Parameters

Now that I know what needs to change I need to go and create that environment via the PuPHPet/Vagrant/Puppet files so I can skip the manual tweaking process.   After some digging I found the config.yaml file.    When you use PuPHPet this file will be put in the .zip download you receive at the end of the PuPHPet process.   It is in the <boxid>/puphpet/ directory.

PuPHPet config.yaml
PuPHPet config.yaml

While some of the box parameters can be adjusted in these files, it appears much of the hardware cannot be manipulated.  There is a site called “Vagrant Cloud” that has multiple boxes that can be configured.   To switch boxes you can edit the config.yaml file and replace the box_url line to point to one of the other variants that may be closer to your configuration.  Since I don’t see one that is close to my needs it looks like I will have to build my own box profile to be hosted in the cloud.   That is content for another article.

 

Posted on

WordPress Workflow : Automated Virtual Box Creation

PuPHPet Vagrant Puppet Banner

I am into my first full day back after WordCamp Atlanta (#wcatl) and have caught up on most of my inbox, Twitter, and Facebook communications.   As I head into a new week of WordPress plugin production I decided now is as good a time as any to update my work flow.

I learned a lot of new things at WordCamp and if there is one thing I’ve learned from past experience it is DO NOT WAIT.   I find the longer I take to start implementing an idea the less chance I have of executing.

My first WordCamp Atlanta 2014 work flow improvement starts right at the base level.   Setting up a clean local development box.   I had started this process last week by manually configuring a baseline CentOS box and was about to setup MySQL, PHP, and all the other goodies by hand.  That was before I learned more about exactly what Vagrant can do.   I had heard of Vagrant but did not fully internalize how it can help me.  Not until this past weekend, that is.

My Work Environment

Before I outline my experience with the process I will share my plugin development work environment.

  • Host System: Windows 8.1 64-bit on an HP Zbook laptop with 16GB of RAM with a 600GB SATA drive
  • Guest System: CentOS 6.5 (latest build) with 8GB RAM on an Oracle VirtualBox virtual machine
    • Linux Kernel 2.6.32-431
    • PHP v5.4.23
    • MySQL v 14.14 dist 5.5.35
  • Dev Took Kit: NetBeans, SmartGit, Apigen and phpDoc, MySQL command line, vim
HP Zbook Windows 411
My Development System laptop config.

While that is my TYPICAL development environment, every-so-often I swap something out such as the MySQL version or PHP version and it is a HUGE PAIN.    This is where Vagrant should help.  I can spin up different virtual boxes such as a single-monitor versus three-monitor configuration when I am on the road or a box with a different version of PHP.     At least that is the theory anyway.   For now I want to focus on getting a “clean” CentOS 6.5 build with my core applications running so I can get back to releasing the Store Locator Plus Enhanced Results add-on pack this week.

Getting Started With Vagrant

The Rockin’ Local Development With Vagrant talk that Russel Fair gave on Saturday had me a bit worried as he was clearly on the OS/X host and the examples looked great from a command line standpoint.  Being a Linux geek I love command line, but I am not about to run virtual development boxes in in a VirtualBox guest.   Seems like a Pandora’s box to me… or at least a Russian doll that will surely slow down performance.   Instead I want to make sure I have Vagrant running on my Windows 8.1 bare metal host.    That is very much against my “full dev environment in a self-contained and portable virtual environment” standard, but one “helper tool” with configurations backed up to my remote Bitbucket repository shouldn’t be too bad, as long as I don’t make it a habit to put dev workflow tools on my host box. Yes, Vagrant does have a Windows installer and I’m fairly certain I won’t need to be running command-line windows to make stuff work.   If I’m running Windows I expect native apps to be fully configurable via the GUI.  Worst case I may need to open a text editor to tweak some files, but no command line please.

Here is the process for a Windows 8.1 install.

  • Download Vagrant.
  • Install needs to be run as admin and requires a system reboot.
  • Ok… it did something… but what?   No icons on the desktop or task bar or … well… anywhere that I can find!

Well… sadly it turns out that Vagrant appears to be a command line only port of the Linux/OSX variants.    No desktop icons, no GUI interface.   I get it.  Doing that is the fast and easy process, but to engage people on the Microsoft desktop you really do need a GUI.    Yes, I’m geek enough to do this and figure it out.   I can also run git command line with no problem but I am FAR more efficient with things like the SmartGit GUI interface.

Maybe I’m not a real geek, but I don’t think using command line and keyboard interaction as the ONLY method for interacting with a computer makes you a real techie.    There is a reason I use a graphical IDE instead of vim these days.    I can do a majority of my work with vim, but it is FAR more efficient to use the GUI elements of my code editor.

Note to Vagrant: if you are doing a windows port at least drop a shortcut icon on the desktop and/or task bar and setup a Windows installer.   Phase 2: consider building a GUI interface on top of the command line system.

It looks like Vagrant is a lower-level command line tool.   It will definitely still have its place, but much like git, this is a too on which other “helpers” need to be added to make my workflow truly efficient.  Time to see what other tools are out there.

Kinda GUI Vagrant : PuPHPet

Luckily some other code geeks seem to like the idea of  GUI configuration system and guess what?   Someone created a tool called PuPHPet (which I also saw referenced at WordCamp so it must be cool)  and even wrote an article about Vagrant and Puppet.   Puppet is a “add on”, called a provisioner,  to setup the guest software environment.

PuPHPet is an online form-based system that builds the text-file configuration scripts that are needed by Vagrant to build and configure your Virtualbox (or VMWare) servers.   It is fairly solid for building a WordPress development environment, but it does mean reverting back to CentOS 6.4 as CentOS 6.5 build scripts are not online.     Though I am sure I can tweak that line of the config files and fix that, but that takes me one-step away from the “point and click” operation I am looking for.

Either way, PuPHPet, is very cool and definitely worth playing with if you are going to be doing any WordPress-centric Vagrant work.

PuPHPet Intro Page
The PuPHPet online configuration tool for creating Vagrant + Puppet config files.

 

Puppet Makes Vagrant and PuPHPet Smarter

Now that I have Vagrant installed and I discovered PuPHPet I feel like I am getting closer to a “spin me up a new virtual dev box, destroy-as-desired, repeat” configuration.  The first part of my workflow improvement process.   BUT…. I need one more thing to take care of it seems… get Puppet installed.   I managed to wade through the documentation (and a few videos) to find the Windows installers.

Based on what is coming up in the install window it looks like the installer will roll out some Apache libs, ruby, and the windows kits that help ruby run on a windows box.

Puppet Install Licenses
The Puppet installer on Windows.

Again, much like Vagrant, Puppet completes the installation with little hint of what it has done.    Puppet is another command line utility that runs at a lower-level to configure the server environments.   It will need some of the “special sauce” to facilitate its use.     A little bit of digging has shown that the Puppet files are all installed under the C:\Program Files (x86)\Puppet Labs folder.    On Windows 8.1 the “Start Menu” is MIA, so the documentation about finding shortcuts there won’t help you.    Apparently those shortcuts are links to HTML doc pages and some basic Windows shell scripts (aka Batch Files) so nothing critical appears to have gone missing.

The two files that are referenced most often are the puppet and facter scripts, so we’ll want to keep track of those.   I’ll create a new folder under My Documents called “WP Development Kit” where I can start dumping things that will help me managed my Windows hosted virtual development environment for WordPress. While I’m at it I will put some links in there for Vagrant and get my PuPHPet files all into a single reference point.

WP Dev Kit Directory
The start of my WP Dev Kit directory. Makes finding my PuPHPet, Vagrant, and Puppet files easier.

Now to get all these command line programs to do my bidding.

Getting It Up

After a few hours or reading, downloading, installing, reading some more, and chasing my son around the house as the “brain eating dad-zombie”, I am ready to try to make it all do something for me.    Apparently I need to use something called a “command line”.  On Windows 8.1.

I’m giving in with the hopes that this small foray into the 1980’s world of command line system administration will yield great benefits that will soon make me forget that DOS still exists under all these fancy icons and windows.   Off to the “black screen of despair”, on of the lesser-known Windows brethren of the “blue screen of death”.     Though Windows 8 tries very hard to hide the underpinnings of the operating system, a recent Windows 8 patch and part of Windows 8.1 since “birth” is the ever-useful Windows-x keyboard shortcut.   If you don’t know this one, you should.   Hold down the Windows key and press x.   You will get a Windows pop-up menu that will allow you to select, among many other things, the Command Prompt application.

If you right-click on the “do you really want to go down this rabbit hole” confirmation box that comes up with the Command Prompt (admin) program you will see that it is running C:\Windows\system32\cmd.exe.     This will be useful for creating a shortcut link that will allow me to not only be in command mode but also to be in the “source” directory of my PuPHPet file set.    I’m going to create a shortcut to that application in my new WP Development Kit directory along with some new parameters:

  • Search for cmd.exe and find the one in the Windows\system32 directory.
  • Right-click and drag the file over to my WP Development Kit folder, selecting “create shortcuts here” when I drop it.
  • My shortcut to cmd.exe is put in place, but needs tweaking…
  • Right-click the shortcut and set the “Start in” to my full WP Development Kit folder.

Now I can double-click the command prompt shortcut in my WP Development Kit folder and not need to change directory to a full path or “up and down the directory tree” to get to my configuration environment.

Running Vagrant andn Puppet via PuPHPet Scripts
Running Vagrant andn Puppet via PuPHPet Scripts

A few key presses later and I’ve managed to change to my downloaded PuPHPet directory and execute the “vagrant up” command.   Gears starting whirring, download counters started ticking, and it appears the PuPHPet/Vagrant/Puppet trio are working together to make something happen.  At the very least it is downloading a bunch of stuff from far away lands and filling up my hard drive.   Hopefully with useful Virtualbox disk images and applications required to get things fired up for my new WordPress dev box.

We’ll see…

Link Summary