Posted on

Windows Azure Virtual Machines, Not Ready For Prime Time

Just last month, Microsoft announced that their Windows Azure Virtual Machines were no longer considered a pre-release service.  In other words, that was the official notification from Microsoft that they feel their Virtual Machines offering is ready for enterprise class deployments.   In fact they even offer uptime guarantees if you employ certain round-robin and/or load balancing deployments that help mitigate the downtime in your cloud environment.

Essentially the Virtual Machines offering on Windows Azure equates to a virtual dedicated server that you would employ from most hosting companies.  The only different with the Windows Azure platform, like most cloud-based offerings, is that you need to serve as your own system admin.   This is not web hosting for business owners but for tech geeks.    In other words, it works perfect for guys like me.

Or so I thought.

Different Shades of White

As I learned tonight, there are differences between the various cloud offerings that are not easy to tease out of the hundreds of pages of online documentation touting how awesome a service provider’s cloud services are.   Sure, there are the metrics.  You can compare instance sizes in terms of disk space, CPU, and bandwidth.   You can comparing pricing and the relative costs of operating your server on each of the cloud platforms.    You can even get the background information on the company providing the virtualized environment, getting some clue (though never a clear picture) of where the servers are physically located, how many servers they have, how secure the environment is, and more.

At the end of the day they all look very similar.  Sure there are discrete elements you can point to on each comparison spreadsheet you throw together, but in the end the differences are relatively minor.   They pricing is similar.   The network and server room build-outs are similar.   The support offerings look similar.     When all is said-and-done you end up making a choice based on price, the reputation of the company, the quality of the online documentation, and the overall user interface experience (UX) that is presented during your research.

After a lot of research, and with quite a bit of experience with Amazon Web Services, all the cloud based offerings were very similar.   Different shades of white.     In the end I decided to try the Microsoft Windows Azure offering.    Microsoft has a good reputation in the tech world, they are not going anywhere, and as a Microsoft Bizspark member I also have preview access and discount services.

My decision to go against the recommendations I’ve been making to my clients for years, “Amazon was one of the first, constantly innovates, and is the leader in the space”, was flawed.    Yes, I tested and evaluated the options for months before making the move.   But it takes an unusual event to truly test the mettle of any service provider.

Breaking A Server

After following the advice of a Microsoft employee that was presented in a Windows Azure forum about Linux servers, I managed to reset the Windows Azure Linux Agent (or WALinuxAgent) application.    No, I did not do this on a whim.   I needed to install a GUI application on the server and followed the instructions presented.  It turns out that Microsoft has deployed a custom application that allows their Azure management interface to “talk” to the Linux server.  That same application DISABLES the basic NetworkManager package on CentOS.  To install any kind of GUI applications or interface you must disable WALinuxAgent, enable NetworkManager, install, disable NetworkManager, then re-enable WALinuxAgent.  The only problem with the instructions that are published in several places is they omit a very important step.  While connected with elevated privileges (sudo or su) you must DISABLE the WALinuxAgent (waagent) provisioning so that it does not employ the Windows Azure proprietary security model on top of your installation.  If you do not do this  and you log out of that elevated privs session y ou will NEVER have access to an elevated privs account again.

Needless to say, you cannot keep an enterprise level server running in this state.  Eventually you need to install updates and patches for security or other reasons.

As I would learn, there is ZERO support on recovering from this situation.

Support versus support

In the years of working with Amazon Web Services and hosting a number of cloud deployments on their platform, I had come accustomed to being able to gain access to support personnel that actually TRY to help you out.   They often go above-and-beyond what is required by contract and try to either get you back on track through their own efforts of at least provide you with enough research and information that you can recover from any issues you have with limited effort.    Amazon support services can be pricey, but having access to not just the level one but also higher level techs is an invaluable resource.

The bottom line is that Microsoft offers NO support services for their Linux images, even those they provide as “sanctioned images”, beyond making sure the ORIGINAL image is stable and that the virtual machine did not crash.    Not only do they not have any apparent means to elevate support tickets, as it turns out there is NO SUPPORT if you are running a Linux image.

Clearly Microsoft does not put this “front and center” on ANY of their Windows Azure literature.  In fact, just the opposite.  Microsoft has made an extended effort in all their “before the purchase” propaganda to try and make it sound like they EMBRACE Linux.   They go out of their way to make you feel like Linux is a welcome member of their family and that they work closely with multiple vendors to ensure a top-quality experience.

Until you have a problem.   At which point they wash their hands, as is evident in this support response along with a link to the Knowledgebase article saying “Linux.  Not our problem.”:

Hello Lance, I understand your concerns and frustration, but Microsoft does not offer technical support for CentOS or any other Linux OS at this time.

 Please, review guidelines for the Linux support on Windows Azure Virtual Machines: http://support.microsoft.com/kb/2805216

No Azure Support
No Azure Support

Other Issues

While the lack of support and the inability to regain privileged user access to my server is the primary concern that has me on the path of choosing a new hosting provider, there have been other issues as well.

A few times in the past several months the WordPress application has put Apache in a tailspin.  This consumes the memory on the server.   While that is not necessarily an issue with Windows Azure, the fact that the “restart virtual image” process DOES NOT WORK at least 50% of the time IS a big issue.   Windows Azure is apparently overly-reliant on that dreaded WALinuxAgent on the server.   If it does not response, because memory is over-allocated for example, the server will not reboot.   The only thing you can do is press the restart button, wait 15 minutes to see if it happened to get enough memory to catch the restart command, and try again.  Ouch.

The Azure interface is also not as nice as I first thought.   While better than the original UX at Amazon Web Services, it is overly simplistic in some places and downright confusing in others.  Try looking at your bill.  Or your subscription status.   You end up jumping between seemingly dis-jointed sites.    Forget about online support forums.  Somehow you end up in the MSDN network, far removed from your cloud portal.    I often find myself with a dozen windows open so I can keep track of where I was or what I need to reference, lest I lose my original navigation path and have to start over.   Not too mention the number of times that this site-to-site hand-off fails and your login is suddenly deemed “invalid” mid-session.

Azure Session Amensia
Azure Session Amensia

Moving Servers

So once again, I find myself looking for a new hosting provider. Luckily I recently made the move to Windows Azure and not only have VaultPress available to make it easy to relocate the WordPress site but also Crash Plan Pro to get all the “auxiliary” installation “cruft” moved along with it.

Where will I go?

In my mind there are only two choices for an expandable cloud deployment running Linux boxes. Amazon Web Services or Rackspace. I’ll likely end up with Amazon again, but who knows… maybe it is time to try the legendary support at Rackspace once again. We’ll see. Stay tuned.

Posted on

Hosting WordPress

I get a lot of questions about where to host a WordPress site.   While I’ve not found the “perfect host for all people”, I have learned a few things about who NOT to use , who I use, and who I *think* will be good to use based on your needs.

Let’s start with who to stay away from:

GoDaddy

DO NOT host with GoDaddy.

Besides my personal issues with their support of national policies that hamper an open Internet, they also have notable technical issues.    Just last fall they mis-configured a router and took tens-of-thousands of businesses offline for several days.  No, it was not Anonymous as first reported.  It was incompetence.  Even if you were not hosting at GoDaddy but had names served by the GoDaddy DNS service your site could have been impacted. Mine site was offline for several days.

The bad part was not that the sites went offline.  That happens.  It shouldn’t, but it does.   The thing that made GoDaddy suck beyond normal suck-itude, was the fact that after several attempts to contact them they ignored ALL communication.  No offer of a credit for the down time.  Nothing other than a blanket generic email saying “our stuff broke, we fixed it”.   Thanks GoDaddy.  My site, as well as thousands of others lost hundreds, if not thousands, of dollars in revenue and your only response was a generic bulk email saying “my bad”.

Even more troublesome is the fact that I’ve been doing business with GoDaddy for over a decade, was a reseller for years, and brought them hundreds of name service and hosting clients over the years.  They can’t even take 2 seconds to respond with a personal email.  Sad.

Enough about the stories of how bad their service is.  The big issue and the main reason I do not recommend them for hosting is the fact that in 8-of-8 paid support requests where the client was having issues and was hosted at GoDaddy, we traced the problem to being hosted on GoDaddy in EVER CASE.   Permissions are configured differently on different servers.  IP addresses are shared en-masse which makes geocoding lookups essentially useless.  Servers time out when overloading, breaking the AJAX listener.

In short, if you want your WordPress stuff to work, do not host on GoDaddy.

LiquidWeb

Do not host with Liquidweb.

I used them for years.  I rented, and still do, a dedicated server there.   I have used their virtual private server and have brought many clients to Liquidweb.  For years their service and prices were above par.    In the past 4 years it has been getting worse every year.

3 years ago, they crashed my dedicated server with a hard fault.  It took them 5 days to get it back online, for a multi-million-dollar software consulting firm.  They had a team working on it, which was good, but it was obvious their claims of “warm server” and “4 hour maximum down time” were false.   They had to order new hardware, wait for it to arrive, configure it, then move our stuff.   After all that the new server was NOT configured the same way which incurred weeks of “oh, that’s broke too”.

This past fall they crashed a new VPS server that was hosting my account.   It also crashed several client accounts.    All the sites on that server were offline for days.   They eventually got it fixed and I was given access to a top-level support rep, but they never did offer any form of compensation for the down time.    Again, the newly configured server was not configured the same way as the old server and stuff never worked right after that.    When I finally showed them that their server was not limiting or allocating resources they told me “your site is too big for the server”.  Really?   I moved it and the new server, which is smaller, runs at less than 10% maximum CPU usage, 25% peak memory usage, and 1% disk I/O usage.

They also made access to any real support basically impossible.   They put tickets in a generic pool and let any tech resolve them. Sometimes you get a guy with a clue, most times not.    I should not be educating my server admin on how to admin a server.

Microsoft Azure

www.windowsazure.com

This is who I use today.    I have several virtual machines running there.    I like the simple interface much more than the Amazon Web Services interface.  It is also slightly less expensive than Amazon services.   However you must be a tech geek (or know one) to use these services.  It is much like running your own server.   If you are not a server admin this is not for you.

If you ARE a server admin, or have on on staff, then you may qualify for Microsoft Bizspark.  This will give you free (or near-free) Azure services for several years.   You can also scale up or down the server as needed with relative ease.    If you are comfortable configuring your base operating system (I use CentOS), installing PHP, MySQL, WordPress and the other components, and managing security then Azure is a fully flexible and expandable platform for a WordPress site.

This type of setup is only for uber geeks or companies that employ them.

ClickHost

www.clickhost.com

I have not used ClickHost myself, however I spoke to many people at WordCamp Atlanta and the general work about ClickHost was that they get WordPress hosting.   They seem like nice people and do seem to go the extra mile to make sure you will be taken care of.    They give you a pre-configured hosting account with the WordPress goodies installed.  Even better, they are very affordable.  A basic setup can cost you as little as $50/year.

For my clients that are cost-aware I will be recommending ClickHost.

RackSpace

www.rackspace.com

If you want a site that never crashes, use RackSpace.  You will pay top-dollar but they have very responsive support and know how to manage servers.  I’ve not used them personally but I know several clients that have used them in the past.  Their support is top-notch and they know their stuff (or have access to someone that does).   They are not cheap, but if you want high performance and high reliability this is a good option.    I’m not familiar with their newer virtualized offerings, which are lower costs, but I have to imagine they are good enough to carry the RackSpace name and reliability image.

Posted on

Setting up local/staging/live

I ran into some issues setting up a local, staging and live copy of a WordPress site. Sticking everything in the repo works as expected, and there is a configuration file (wp-config.php) that can have each installation’s individual settings. The problem is that the base URL for the site is stored in the database.If you just use mysqldump to export the database, and save that in the repo, you will be exporting the current site’s URL. For instance, all permalinks will look like “http://localhost/your/dev/site“. When you restore that backup in a different location (IE staging or live), everything will break.

The secret is to override the database settings entirely, using the following lines in your wp-config.php file:

define('WP_HOME','http://your.local.site.com');
define('WP_SITEURL','http://your.local.site.com');

These two lines should be added to each installation’s local configuration file, along with that installation’s database settings, etc.

These settings seem to override everything else, completely disabling that configuration setting in the wp-admin panel. Permalinks use that URL as their base. CSS files use that as their base. It seems to fix everything, from what I can tell. I’m not sure if there will be issues with mod_rewrite /SEO-friendly URLs, once I enable them. I’ll post to this thread if I run into any issues.

See http://codex.wordpress.org/Changing The_Site_URL for more details. If you read down the rest of the page, you see that there are several other more complicated solutions that require manual intervention.