April 1, 2011

Start to Finish - Serving Django with uWSGI/Nginx on EC2

I'm all over that free EC2 micro instance. Free application hosting with root access? Count me in.

I won't walk you through the Amazon signup part; if I could feel my way through that, you can too.  Plus, I can't remember most of it. So, this guide will pick up as soon as you first ssh into your shiny new free EC2 instance.

Set up security

Amazon makes this easy, but you should still take it upon yourself to setup iptables or something like it. I'm not the one to tell you about that; you can learn lots from Google.

Installing packages

I'm an ubuntu guy myself, although I can't claim to be so for any reason other than marketing.  There's not much different between the Red Hat that is the default at EC2, and Ubuntu, but the one thing that is different, and that you will need to know, is that aptitude/apt-get is replaced by yum. Also, the *-dev packages from debian repositories tend to be called *-devel here instead.

Run the following command to get the packages that you're going to need later.  There might be a few things in here that are already installed, but yum will know that.

yum install python26 python26-devel make automake nginx mysql mysql-server mysql-devel gcc gcc-c++ python-setuptools

Setting up a virtualenv

You use virtualenv, don't you? Well, I'm not going to teach you a different way, so you do now. You'll thank me later.

First, get it using easy_install:

easy_install virtualenv

Then, pick where you want to store your django app.  I like /opt/apps myself.

$ sudo mkdir /opt/apps
$ sudo chown ec2-user:ec2-user /opt/apps
$ cd /opt/apps
$ virtualenv --no-site-packages <your_app_name>-env
$ cd <your_app_name>-env
$ source bin/activate

There you go.  You should see something like "(<your_app_name>-env)" before the command prompt, indicating that you are operating within the virtual environment you just created.

Install python packages

We love installing packages. What python package you need is up to you, but if you're reading this, you'll want at least django.  I always recommend south (http://south.aeracode.org/), as well.

pip install django south

Get your app in there.

In a directory in your virtualenv (I usually do /opt/apps/<your_app_name>-env/site), use your SCM of choice to pull down your code from wherever it is. yum has git, mercurial and (ugh) subversion if you need them.

You can read all about my preferences for organizing your project in this post.

Test it out

Do what you need to to get django up and running - I usually call "./manage.py runserver" repeatedly, and do whatever it tells me I need to do.  Eventually, you should actually get the server running. You'll have to add a bit of flourish to make runserver serve to external IPs:

./manage.py runserver

This will make your server accessible at <horribly_long_aws_hostname>:8000.  Go there in a browser and test it out (having opened the port in the AWS control panel, of course.

Create a WSGI file for Django

Once you've confirmed that your app is installed and working, you'll want to create a wsgi configuration file. This is a normal python file which sets up the application to be served.

It should look something like this:

SITE_DIR = '/opt/apps/<your_app_name>-env/site'
import site

import os
import sys

os.environ['DJANGO_SETTINGS_MODULE'] = '<Your django project >.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Save that somewhere, and remember the path to it and its name. From here on let's pretend you saved it as:


Set up uWSGI

If you noticed that we didn't install uwsgi before, good eye, and good for you for actually reading the package list I posted instead of blind copy-pasting.

EDIT: Apparently you can now (or always could, I don't know) use pip to install uwsgi. That will be easier than this. You'll want to install it as root though; you'll need the binary:

$ sudo pip install uwsgi

If you still want to compile it yourself, keep reading. Otherwise, skip to "Daemonize uWSGI" below.

Yes, you will want to compile your own uwsgi.  Luckily, this is made easy on you because you installed all the build packages before.  Just get the package, use make to compile it

$ cd ~; 
$ wget http://projects.unbit.it/downloads/uwsgi-<check the latest version, don't be lazy>.tar.gz         
$ tar -xvvf uwsgi-*.tar.gz
$ cd uwsgi-*
$ make

Then, shuffle it off into /usr/local/lib.  That way, it's somewere someone might actually look for it. Make a link to /usr/local/bin so you can call it from the command line, if you so choose.

$ sudo mv ~/uwsgi* /usr/local/lib/
$ sudo ln -s /usr/local/lib/uwsgi /usr/local/bin/uwsgi

Daemonize uWSGI

EC2 has upstart, which will let us make a handy-dandy script to spin it up automatically. Make yourself a file called /etc/init/uwsgi.conf (you'll need root for that), and put the following in it:

# file: /etc/init/uwsgi.conf 
description "uWSGI server" 
start on runlevel [2345]
stop on runlevel [!2345]

exec /usr/local/bin/uwsgi \ 
--home /opt/apps/<your_app_name>-env/site/ \ 
--socket /tmp/uwsgi_<your_app_name>.sock \ 
--chmod-socket \ 
--module myapp_wsgi \ 
--pythonpath /opt/apps/<your_app_name>-env/site/wsgi \  
-H /opt/apps/<your_app_name>-env

Note: Depending on your setup, you might need to remove the '\'s from the exec command and put it on one line. I think I might have effed with the shell and not mentioned it before.

Lets look at the options we called uwsgi with:

  • --home The home directory for uwsgi to use
  • --socket The path to a socket file. We'll give this to Nginx later.
  • --chmod-socket Set the permissions on the socket so we can use it.
  • --module The wsgi configuration file's name, without extension. So, myapp_wsgi, not myapp_wsgi.py
  • --pythonpath The directory in which the wsgi configuration file resides.
  • -H The path to the virtualenv to use with uwsgi.

When all that is done, you can spin it up:

$ sudo start uwsgi

Configure Nginx

First, we'll make a file called /etc/nginx/uwsgi_params. Put the following in it:

uwsgi_param  QUERY_STRING       $query_string;
uwsgi_param  REQUEST_METHOD     $request_method;
uwsgi_param  CONTENT_TYPE       $content_type;
uwsgi_param  CONTENT_LENGTH     $content_length;

uwsgi_param  REQUEST_URI        $request_uri;
uwsgi_param  PATH_INFO          $document_uri;
uwsgi_param  DOCUMENT_ROOT      $document_root;
uwsgi_param  SERVER_PROTOCOL    $server_protocol;

uwsgi_param  REMOTE_ADDR        $remote_addr;
uwsgi_param  REMOTE_PORT        $remote_port;
uwsgi_param  SERVER_PORT        $server_port;
uwsgi_param  SERVER_NAME        $server_name;

I'm an apache convert, so I feel more comfortable with a sites-available directory to hold my individual site confs, and a sites-enabled directory to keep just the enabled one. I also like to store those files in /opt/nginx/sites-(available|enabled). So, make those directories.

$ mkdir /opt/nginx; mkdir /opt/nginx/{sites-available,sites-enabled}

Then, add the following line within the http section of /etc/nginx/nginx.conf, right after the line that reads "include /etc/nginx/conf.d/*.conf"

include /opt/nginx/sites-enabled/*;

Of course, you could also put your site configuration files in /etc/nginx/conf.d/*.conf like it wanted. Wherever you decide to put your configuration file, it should look a bit like this:

        server_name www..com .com;
        access_log /path/to/access/log.log;
        error_log /path/to/error/log.log;

        location /site_media {
                # Point this wherever the static files for your django app are stored (i.e. MEDIA_ROOT)
                alias /opt/apps/<your_app_name>-env/site/media;

        location / {
            uwsgi_pass   unix:///tmp/uwsgi_<your_app_name>.sock;
            include        uwsgi_params;


Finally, if you took the sites-available/sites-enabled route, remember to put the site in sites-enabled and restart nginx.

$ ln -s /opt/nginx/sites-available/<your_conf_file> /opt/nginx/sites-enabled/<your_conf_file>
$ sudo /etc/init.d/nginx restart