What a n00b! (s3)

Deploying Your Nikola Site to S3 (and CloudFront)

Wyatt — Wed, 03 Jul 2013 05:26:35 GMT

A few weeks ago, I made a post about moving my site into S3 and a few things unexpected issues that I ran into. What I didn't mention was my migration to Nikola from WordPress and then my actual deployment into S3. I've outlined most of the deployment steps I went through. A basic understanding of AWS services (specifically S3 and CloudFront) is assumed, but I try to be as helpful as I can.

S3 Setup

This is fairly well documented in the S3 documentation, but to host a website you first need to create a bucket with the name of the hostname you want to use. For me, I have two buckets: whatan00b.com and www.whatan00b.com (more on the two buckets later). Once the buckets are created, you'll want to enable website hosting on them. In the S3 interface, if you click on Properties for the bucket, expand the "Static Website Hosting" bit. Check the box for "Enable website hosting". The default document root of index.html is perfectly fine. Error Document is something you'll likely want to come back to later.

If you created two buckets like I did, you'll want to do that step only on the primary bucket you want to host the website through. For the other bucket (for me, whatan00b.com), instead check the radio button for "Redirect requests to another host name". You'll want to fill out the hostname you used in the other bucket to redirect requests and save it.

"Pretty URLs"

This is somewhat related to the migration from WordPress, but I really didn't want to have URLs that pointed directly to the static HTML file, I wanted to hit the directory index to view pages. This was mainly to avoid adding redirects in S3. This is easily doable, I just like the cleaner look of URLs. If you left index.html as the document root as index.html in the S3 setup step, things should be fine on the S3 side.

On the Nikola side, I used the PRETTY_URLS option and set it to True in config.py. At the time of this writing, I had to install Nikola from the master branch on Github to make this work (see example) . It appears this feature will be released in 5.4.5. Enabling this option creates a directory for each blog post, with an index.html inside. Note that S3 will redirect and add a trailing slash if the user does not add one themselves (this was fine for me).

Deploying the files!

I have been using Fabric for other projects as of late, so I actually skipped the deployment bit altogether inside of Nikola. The deployment piece inside of Nikola currently only allows users to specify a list of commands that make up their deployment anyway (I assume this is generally an rsync to a list of servers or something along those lines for most folks). This is probably easily extendable, but I'm already in the habit of doing a 'fab deploy' when I am ready to push out code anyway, so I chose to stick with Fabric and ignore the Nikola deployment bit altogether for now. You could easily write the same type of function and add a "fab deploy" to the deployment section of your Nikola config file. You can view my deployment script over on Bitbucket (and the whole site for that matter!). It's probably not the most efficient file sync function ever written, but it's mine and it works for me.

If you don't want to go through all of this, there are lots of FTP / SFTP / whatever clients out there that support S3. You can also easily use something like Cyberduck on a Mac or Windows to push the files to your bucket.

CloudFront Setup

The CloudFront setup was one that bit me. This is something that I mentioned in my previous post, but I'll reiterate it again. For this setup, don't set CloudFront to use the S3 bucket as its source. Set the origin to be a "Custom Origin", using the S3 bucket website hosting endpoint as the URL. This can be found in the bucket properties section you looked at in the S3 setup section. If you don't do this, the site index will work, but lower directory indexes won't work.

Once the distribution is created and origins set, you should be able to hit the origin URL given in the AWS console. Once you are happy with the way things look, you can add an alias on the CloudFront distribution with the hostname you want to use for your site. After that's complete, you're just a CNAME away from the site being hosted on CloudFront. I happen to use Route53 as my DNS provider, so I just added an alias for the CloudFront distribution and things work nicely.

Adventures in Static Site Hosting on S3

Wyatt — Tue, 11 Jun 2013 07:13:50 GMT

I recently switched this site from WordPress to be static pages stored on Amazon S3. I had significantly more problems than I anticipated with hosting a simple static site. This is my story.

Why static?

Every static site generator out there has a section header on their front page posing essentially the same question. While I don't particularly want to duplicate this list, I did want to spend a moment and speak to my own motivation for doing so.

No server needed! - I've been using WordPress basically since I started blogging, so about 4 years on and off. During that time, I had to find a place to host my blog that had PHP and MySQL installed. I moved from shared hosting to hopping AWS Free Tier accounts for a while, which has been getting old for a while now. I chose S3 specifically because it was a service that is far more available than I could accomplish by myself and is far simpler to use. Plus, the cost of a little bit of storage in S3 is stupidly cheap compared with running an EC2 instance or in a dedicated or shared host at any provider.
Security - With a CMS, it's very important to keep your systems patched. You have to worry about things at a platform level (for me: OS, PHP, Apache, MySQL) as well as the application layer (and don't forget about all the plugins!). With static hosting via S3, I have nothing to worry about except keeping my AWS credentials safe. Reducing this exposure was important to me as I don't have time to keep up on vulnerabilities at all these layers.

I'm sure there are plenty of other benefits, but I was already sold.

Pitfalls

As I said earlier, I hit quite a few more snags (and used a few more workarounds) than I anticipated when initially evaluating S3. Below are the major ones I encountered, hopefully I can help someone avoid these in the future.

Lack of CloudFront aliases in Route53 - This was a bit of an issue for me as I used to host my blog off the domain apex (whatan00b.com). I had to add a 'www.' in front to make this work properly using a CNAME. Of course, less than 24 hours after I post this I get an email in my inbox saying this feature was released.. :)
Lack of solid on-the-fly compression support - I greatly misunderstood this one at first. S3 does support serving gzip-compressed content, however you have to upload and link directly to the gzipped files. You essentially just gzip your HTML, CSS, etc files and serve them up directly, hoping that every single client that connects supports gzip. This probably isn't a terribly unsafe assumption with browsers, but I am a little worried about crawlers, not wanting to cause issues with SEO, etc. It may be a non-issue, but certainly something to consider carefully. For now I have opted to leave things plain.
CloudFront does not forgive. CloudFront does not forget. Expect CloudFront. - CloudFront can cache things for a while, which is of course a big part of why it's awesome and useful. Unfortunately, if you cache badness, badness lives on for a while. In my case, I changed the site to use "www." and changed the S3 bucket redirects in the correct order, but I did forget to change the origin for my CloudFront distribution to use the new bucket without a redirect. This caused infinite redirect loops for the site at various places throughout the world, but not all. It was impossible for me to catch without a tool like Pingdom. This took a couple of invalidation requests, but seems to have quieted down for me now.
Don't set CloudFront to front-end your S3 bucket directly - CloudFront does have the option to use an S3 bucket as the origin when setting up an origin. This works great if you aren't concerned about needing the index option or error document. To get both of those to work properly, use the endpoint URL from the Static Website Hosting setup section for the S3 bucket instead.

Performance

I use Pingdom for monitoring my website (and would highly recommend it), and saw some weird results when switching to the static site hosted on S3. The average response time globally went from ~500ms with WordPress on a single EC2 micro instance to ~700ms with S3. I would probably blame lack of gzip on this, but didn't take the time to properly investigate. CloudFront far more than made up for the performance difference. After the switch, the response time hangs out just above 150ms for a worldwide average.

Cost

I've been running the site on S3 + CloudFront for just shy of 10 days and so far have racked up a giant bill of 28 cents. Around half of that was due to a stupid amount of PUTs I generated when experimenting with deployment tools.. Oops. The pricing can't possibly compete with trying to run cloud systems on my own, and performance (and price!) is far better than any shared provider I have found.