Posted on

HTTPS On Amazon Linux With LetsEncrypt

Internet2

In order to provide faster and more secure connections to the Store Locator Web service we have added https support through Sucuri.   Adding https will allow us to take advantage of SPDY and HTTP2 which are the latest improvements to web connection technology.   There are many reasons to get your servers onto full https support.   As we learned it isn’t a one-click operation, but without too much additional effort you can get your servers running on Amazon Linux with a secured connection.   Here are the cheat sheet notes based on our experience.

EC2 Server Rules

With EC2 you will want to make sure you set your security group rules to allow incoming connections on port 443.  By default no ports are open, you already added port 80 for web support.   Make sure you go back and add port 443 as an open inbound rule.

Apache SSL Support

Next you need to configure the Apache web server to handle SSL connections.   The easiest way to get started is to install the mod_ssl library which will create the necessary ssl.conf file in /etc/httpd/conf.d/ssl.conf and turn on the port 443 listener.


# sudo service httpd stop
# sudo yum update -y
# sudo yum install -y mod24_ssl

Get Your Let’s Encrypt Certificate

This is more of a challenge if you don’t know where to start. Part of the issue is Amazon Linux runs Python 2.6 and Let’s Enrypt likes Python 2.7. Luckily there has been progress on getting this working so you can cheat a bit.

# git clone https://github.com/letsencrypt/letsencrypt
# cd letsencrypt
# git checkout amazonlinux
# sudo ./letsencrypt-auto --agree-dev-preview --server https://acme-v01.api.letsencrypt.org/directory certonly -d yourdomain.name -d www.yourdomain.name -v --debug

You may get some warnings and other messages but eventually you will get an ANSI-mode dialogue screen (welcome to 1985) that walks you through accepting terms and the certification. Answer the questions and accept your way to a new cert.

Your certs will be placed in /etc/letsencrypt/live/ , remember this path as you will need it later.

Update SSL.conf

Go to the /etc/httpd/conf.d directory and edit the ssl.conf file.

Look for these 3 directives and change them to point to the cert.pem, privkey.pem, and chain.pem file.

SSLCertificateFile
SSLCertificateKeyFile
SSLCertificateChainFile

Restart Apache & Get Secure

No restart apache and check by surfing to https:///

# service httpd start

You may need to update various setting on your web apps especially if you use .htaccess to rewrite URLS with http or https.

Posted on

AWS gMail Relay Setup

SMTP Relay Banner

After moving to a new AWS server I discovered that my mail configuration files were not configured as part of my backup service on my old server. In addition my new server is using sendmail instead of postfix for mail services. That mean re-learning and re-discovering how to setup mail relay through gmail.

Why Relay?

Cloud servers tend to be blacklisted. Sure enough, my IP address on the new server is on the Spamhaus PBL list. While Amazon allows for elastic IP addresses, a quasi-permanent IP address that acts like a static IP, which can be added to the whitelist on the Spamhaus PBL it is not the best option. Servers change, especially in the cloud. I find the best option is to route email through a trusted email service. I use Google Business Apps email accounts and have one setup just for this purpose. Now to configure sendmail to re-route all outbound mail from my server to my gmail account.

Configuring Amazon Linux

Here are my cheat-sheet notes about getting an Amazon Linux (RHEL flavor of Linux) box to use the default sendmail to push content through gmail.

Install packages needed.

# sudo su -
# yum install cyrus-sasl ca-certificates sendmail make

Create your certificates

This is needed for the TLS authentication.

</p>
# cd /etc/pki/tls/certs
# make sendmail.pem
# cd /etc/mail
# mkdir certs
# chmod 700 certs
# cd certs
# cp /etc/pki/tls/certs/ca-bundle.crt /etc/mail/certs/ca-bundle.crt
# cp /etc/pki/tls/certs/sendmail.pem /etc/mail/certs/sendmail.pm

Setup your authinfo file

The AuthInfo entries start with the relay server host name and port.

U = the AWS server user that will be the source of the email.

I = your gmail user name, if using business apps it is likely @yourdomain.com not @gmail.com

P = your gmail email password

M = the method of authentication, PLAIN will suffice

# cd /etc/mail
# vim gmail-auth

AuthInfo:smtp-relay.gmail.com "U:ec2-user" "I:your-gmail-addy@gmail.com" "P:yourpassword" "M:PLAIN"
AuthInfo:smtp-relay.gmail.com "U:apache" "I:your-gmail-addy@gmail.com" "P:yourpassword" "M:PLAIN"
AuthInfo:smtp-relay.gmail.com:587 "U:ec2-user" "I:your-gmail-addy@gmail.com" "P:yourpassword" "M:PLAIN"
AuthInfo:smtp-relay.gmail.com:587 "U:apache" "I:your-gmail-addy@gmail.com" "P:yourpassword" "M:PLAIN"

# chmod 600 gmail-auth
# makemap -r hash gmail-auth < gmail-auth

Configure Sendmail

Edit the sendmail.mc file and run make to turn it into a sendmail.cf configuration file.  Look for each of the entries noted in the sendmail.mc comments.  Uncomment the entries and/or change them as noted.    A couple of new lines will need to be added to the sendmail.mc file.   I add the new lines just before the MAILER(smpt)dnl line at the end of the file.

Most of these exist throughout the file and are commented out.   I uncommented the lines and modified them as needed so they appear near the comment blocks that explain what is going on:

# vim /etc/mail/sendmail.mc
define(`SMART_HOST', `smtp-relay.gmail.com')dnl
define(`confAUTH_OPTIONS', `A p')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confCACERT_PATH', `/etc/mail/certs')dnl
define(`confCACERT', `/etc/mail/certs/ca-bundle.crt')dnl
define(`confSERVER_CERT', `/etc/mail/certs/sendmail.pem')dnl
define(`confSERVER_KEY', `/etc/mail/certs/sendmail.pem')dnl

Add these lines to the end of sendmail.mc just above the first MAILER()dnl entries:

</p>
<p style="padding-left: 30px;">define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl</p>
<p style="padding-left: 30px;">define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl</p>
<p style="padding-left: 30px;">FEATURE(`authinfo',`hash -o /etc/mail/gmail-auth.db')dnl</p>
<p style="padding-left: 30px;">

If you are using business apps you may need these settings to make the email come from your domain and to pass authentication based on your Gmail relay settings.    These are also in sendmail.mc:

MASQUERADE_AS(`charlestonsw.com')dnl
FEATURE(masquerade_envelope)dnl
FEATURE(masquerade_entire_domain)dnl
MASQUERADE_DOMAIN(localhost)dnl
MASQUERADE_DOMAIN(localhost.localdomain)dnl
MASQUERADE_DOMAIN(charlestonsw.com)dnl

Make the configuration-helper into a sendmail.mc file and restart sendmail:

# make
# service sendmail restart

Configure Gmail Services

This is for business apps users, you need to turn on relay.

Go to “manage this domain” for your business apps account.

Go to “Google Apps”.

Click on “Gmail”.

Click “advanced settings”.

Find the “SMTP relay service” entry.    Add a  new entry.

Only addresses in my domain, require SMTP, require TLS all need to be selected.

Give it a name.

Save.

Save again.

Posted on

PHP Pretty Print XML

I have been working on MoneyPress : Amazon Edition to get it updated for the latest API release and bring it into the Charleston Software Associates stable of products.  Along the way I found myself needing to debug the XML being returned from the Amazon Product API.   Here is a quick trick for doing that.

...
$returnedXML = $result['body']
$xmlDoc = DOMDocument::loadXML($result['body']);
$xmlDoc->formatOutput=true;
print '<pre>' . htmlentities($xmlDoc->sveXML()).'</pre>';
...


Posted on

Kindle Fire A Quick Review

Kindle FireThe Kindle Fire arrived earlier than expected, showing up at the door of Cyber Sprocket’s Headquarters in warm sunny Charleston yesterday afternoon.   I’ve only had 24 hours to play with it, but with plenty of experience with other mobile and tablet devices I have already formed some opinions & have decided what Fire’s purpose in life is.

Go Green

The first thing I noticed is the packaging.  Every single consumer product company should take cues from Amazon in this regard.  The Kindle Fire comes in Amazon’s “frustration free packaging”, which is something they offer on a number of products.  It truly is frustration free.   A simple cardboard box.  Inside is the Kindle Fire nicely cradled in a raised cardboard bumper wrapped in a thin plastic protective covering that is easily opened by lifting the lightly-glued flap.  Underneath is the power cord with a simple cardboard loop that reminded me of a napkin ring for hobos or Occupy Your Town folks.

Aside from the fact that you don’t need to drive to the local fire station and borrow their Jaws Of Life to open the product package, the other BIG benefit is that it is earth friendly.   I respect that.   I’m not overly “earthy-crunchy”, but if there is a simple, elegant, and low cost way to get the job done then why not do it?   The entire package contents other than that one very small piece of protective plastic is 100% recyclable cardboard.   Even better is that cardboard breaks down in the environment, so for those of you that don’t recycle the chances of something blowing into one of our local Charleston marshes and staying there for 3,297.5 years is zero.

Extra points for being green and being smart about it.

 The Geek Stuff

Ok, enough with the “go green” and packaging comments.   Now on to the device…

Form Factor

The form factor is perfect, IMO.  The 7″ screen is the perfect size for my hands, yet large enough for the screen to be usable.  The 10″ Toshiba Thrive tablet is nice, but simply too heavy and too big to make reading a book comfortable without propping it up in the carrying case.   The Kindle Fire, on the other hand, is perfectly comfortable to hold and light enough to not strain your wrist while doing so.   I watched the first 20 minutes of Ip Man last night without even realizing it had been that long, holding the Kindle Fire in my hand the entire time.    5 minutes of reading a book on the Thrive and I have to put it down or prop it up on something.

The Screen

The screen is nice.  It seems brighter and crisper than the Toshiba Thrive and the HTC Incredible.  I would say it is on par with the iPod Touch 4th gen that we recently got in stock.   It is not even close to the original Kindle eInk based devices as far as readability in bright light, but with a completely different technology behind it that is to be expected.  Color eInk would be awesome, but my guess is we’ll be waiting another year or two for that “surprise announcement” from Amazon.   In the meantime this display can compete with any other “glass” on the market for this type of device.

The video processor is decent enough, but does not seem to be on par with the nVidia Tegra in the Toshiba Thrive.  I did notice some artifacts when watching Ip Man, which do not appear on the HP HD monitor on the laptop or on the Thrive even though the bandwidth was the same in all 3 cases (in other words the artifacts were not caused by buffering).  Still, good enough to no distract from the experience.

The Apps

Here is where things are less than perfect for the Kindle Fire.   Amazon has made a concerted effort to ensure they can capitalize on any & all content that they can offer via Amazon.  That means you can only get apps through their app store (unless you hack the Fire).  Much to our surprise there are a number of apps that I don’t see as competing with Amazon that are NOT available via the marketplace.

Many of my favorite apps are missing, and one of the most important – the email app, is far inferior to the built-in Google Mail app on the Thrive or the Incredible.   That hurts.   I was really hoping this smaller, lighter device could take over for the Thrive as my go-to mobile device.   The built-in email app wouldn’t even recognize the settings for my Google Mail Business Apps account.    So no email for me on that device other than via the mobile web browser.  That will work in a pinch but is far less pleasant an experience than native Google Mail available on a full-fledged Android device.

Some of the other apps that are missing that I use regularly include direct competitors to the Amazon digital library.  Things like the USA Today mobile app, which is FREE on most other Android devices has been explicitly removed from the Amazon version of the Android Marketplace.   Sames goes for the CNN app.   That is a price you pay for getting a device that Amazon is selling AT A LOSS.   Rumor has it that Amazon is losing $50/device on the Kindle Fire and expects to make up the difference over time through the Kindle Store apps, like their paid subscription to USA Today for example.

Not sure I agree with that move.  I’d rather have paid $50 more for the device & had full access to the entire Droid Marketplace.   Amazon could still capitalize on the Kindle Fire experience by having a slicker interface that is better integrated with the device for their native Amazon content.   In other words, keep the full marketplace and leave the rest of the UX alone.

Speaking of User Experience

One place the Kindle Fire does excel is the user experience.  It is different from Apple, but very much on par with most of what Apple has done.  The interface is slick, refined, and generally trouble free.  Most elements are intuitive, though I would prefer a “hard button” to get back to the home screen and/or find the main menu  when in a running app.   You can get used to the screen swipes but that is sometimes not intuitive.  I know I’d be getting support calls from various family members if they owned this device asking “how do I get out of this movie?”.   That aside the general animations, graphics, and interface elements are very well designed and easy to navigate.

The connection with Amazon services is also very well done.   The integration is managed so well on all levels that you don’t even notice all the cool technical trickery going on behind the scenes.  They’ve even gone the extra mile in a few places, like when I first turn on the device… I put in my WEP key for the wireless network and the device did everything else.  It picked up my Amazon account  & synced the device with the account I used to purchase it from with NO INPUT on my part.   That was nice.   My eBooks where readily available and the movie that we watched via our Amazon Prime account was right there, remembering where we paused it the night before on our living room TV.   That is pretty slick, though not unique to the Kindle Fire; the “remember where I left off” feature is standard on all Amazon media devices.

Overall Impression

The Kindle Fire is a nice device.  It is a clean package, nice user interface, and has most of the tech widgets you’ll need for mobile interaction.  However it is clearly designed to be a consumer entertainment device and not a business tool.  Music, books, movies and the like are all very well done and easy to use when combined with the Amazon services.  As a general around-the-house, surf the web, check a few simple web based email messages, and read a book it is great.     But it is NOT a mobile workstation in the same way the Toshiba Thrive is.

The Amazon Kindle Fire is well designed and a great value, but it is not a laptop replacement nor a mobile phone replacement.  There is no camera, no 3G/4G cell network access, no expansions slots or device connectors other than the mini-usb charging port.   That is not necessarily a bad thing.  It keeps the device simple, light, and focused on its task which is a home entertainment and media device.  For that it is perfect.

For me, I will continue to use the Toshiba Thrive as my go-to device as I am  very much a power user.   The Thrive can serve as a mini laptop replacement.  Perfect for business meetings, presentations, and even the occasional movie or book.  However the weight & size will probably ensure the Kindle retains a place in my backpack as my on-the-go digital library for reference books when I’m not “fully engaged” in the business world… like on vacation this week at Disney, where the Kindle will be the perfect pool-side companion.

 

 

 

Posted on

Backing Up A Linux Directory To The Cloud

We use Amazon S3 to backup a myriad of directories and data dumps from our local development and public live servers.  The storage is cheap, easily accessible, and is in a remote third party location with decent resilience.  The storage is secure unless you share your bucket information and key files with a third party.

In this article we explore the task of backing up a Linux directory via the command line to an S3 bucket.   This article assumes you’ve signed up for Amazon Web Services (AWS) and have S3 capabilities enabled on your account.  That can all be done via the simple web interface at Amazon.

Step 1 : Get s3tools Installed

The easiest way to interface with Amazon from the command line is to install the open source s3tools application toolkit from the web.  You can get the toolkit from http://www.s3tools.org/.  If you are on a Redhat based distribution you can create the yum repo file and simply to a yum install.  For all other distributions you’ll need to fetch and build from source (actually running python setup.py install) after you download.

Once you have s3cmd installed you will need to configure it.  Run the following command (not you will need your access key and secret key from your Amazon AWS account):
s3cmd --configure

Step 2 : Create A Simple Backup Script

Go to the directory you wish to backup and create the following script named backthisup.sh:

#!/bin/sh
SITENAME='mysite'
# Create a tarzip of the directory
echo 'Making tarzip of this directory...'
tar cvz --exclude backup.tgz -f backup.tgz ./*
# Make the s3 bucket (ignored if already there)
echo 'Create bucket if it is not there...'
s3cmd mb s3://backup.$SITENAME
# Put that tarzip we just made on s3
echo 'Storing files on s3...'
s3cmd put backup.tgz s3://backup.$SITENAME

Note that this is a simple backup script.  It tarzips the current directory and then pushes it to the s3 bucket.  This is good for a quick backup but not the best solution for ongoing repeated backups.  The reason is that most of the time you will want to perform a differential backup, only putting the stuff that is changed or newly created into the s3 bucket. AWS charges you for every put and get operation and for bandwidth.  Granted the fees are low, but every penny counts.

Next Steps : Differential Backups

If you don’t want to always push all your files to the server every time you run the script you can do a differential backup.   This is easily accomplished with S3Tools by using the sync instead of the push command.   We leave that to a future article.