Deploying a Clojure web app with Pallet
The lost documentation
Pallet is a Clojure DevOps platform/library designed to automate blah blah blah. We all know what these things are supposed to do, the question is how they do it, and Pallet takes the agreeable tack of just being a Clojure library that you can run in a repl and do devops stuff.
That’s the great part of Pallet. The bad part of Pallet is that the documentation is sorely incomplete, at least for people like me who just want to get up and running with some modest project. I’m hoping to provide in this post not only an overview of my (relatively narrow) use case, but explain some key Pallet concepts at a level that the official docs, well, just don’t bother to.
The other bad part of Pallet is that it’s not really under active development
and parts of it are outdated.
Its site talks a big game about its ability to automate the allocation of
server resources, which it does this using JClouds.
This probably works great if you’re on AWS, but when I tried to get it
working with DigitalOcean, the version of JClouds that supports the DO v2
API that is now the only usable one was too new for Pallet, so none of that
worked out. So instead, this post will disregard resource allocation and just
use the node-list provider, which is where you just tell Pallet where to
find your servers that you procured yourself. This way, we can stick with the
solid parts of Pallet that work on decades-old infrastructure (SSH) and
probably won’t go bad so fast.
The more interesting part of Pallet is how you describe the tasks you want it to do, and that’s what I’m going to focus on. First, here’s a quick hierarchy of Pallet things, presented as a useful glossary that you’ll need to understand the rest of the post anyhow.
The smallest glossary possible
- An action is a thing you do on a server. Install a package, create a file, run a script, etc.
- A plan is a collection/sequence of actions. It looks a lot like a function.
- A phase is a collection of actions and/or plans, tagged with some identifier that hopefully describes what those actions do (e.g. “:install”).
- A server spec is a collection of phases, which perform tasks
relevant a particular component of an overall deployment. For example,
a server spec for nginx may contain an
:installphase to install nginx, a:configurephase to update the configuration, and a:runphase to actually start the service. - A crate is what Pallet calls a plugin or library, or the server spec invariably contained within, because presumably the metaphor was just to cute to pass up.
- A group spec is a collection of server specs (and perhaps some custom phases) describing a full deployment to a server.
That right there should give you more of a clue about how to Pallet than I managed to get from scouring the docs. You can combine actions into phases into server specs into group specs, and build your final configuration that way.
Note that phases can be more or less whatever you want, except that a :settings
phase is always automatically run before whatever other phases you’re executing.
Writing a Server Spec for Boot
Here’s a little starter Pallet example. My project was written with Boot, and since I used boot-ragtime for migrations I wanted to have a working boot install on the server.
Pallet lets you abstract this very simply. Here’s the entire implementation
of my boot crate:
(ns ops.boot
(:require
[pallet.api :as api]
[pallet.actions :refer [exec-script]]
[pallet.crate :refer [defplan]]
))
(defplan install-boot
[]
(exec-script
("wget" "https://github.com/boot-clj/boot-bin/releases/download/latest/boot.sh"))
(exec-script ("mv" "boot.sh" "boot"))
(exec-script ("chmod" "a+x" "boot"))
(exec-script ("sudo" "mv" "boot" "/usr/local/bin")))
(defn server-spec
[settings & {:keys [] :as options}]
(api/server-spec
:phases {:install (api/plan-fn
(install-boot))}
:default-phases [:install]))
This is short but probably dense if you’ve never tried Pallet before. First,
each of the exec-script calls is an action. These are our basic building blocks, remember?
Here, we download boot with wget, and execute the string of commands recommended
by boot right in their install docs:
$ mv boot.sh boot && chmod a+x boot && sudo mv boot /usr/local/bin
Note that, if an action fails, the whole thing fails and the server is left in a
half-configured state, which may or may not matter depending on what you’re doing. This
makes the &&s implicit.
The beauty of automating this stuff is, of course, repeatability; I nuked my server image at
least twice while stumbling through all this, and lost no real progress as a result.
(If exec-script gives you the willies, you can do the same with exec-script*, which
executes a string that you provide.)
Having defined a plan, we now create a server spec with a single phase: install boot. A server spec is a thing that we can bolt on to our final configuration to provide some ability or another.
Note also that the server spec accepts some arguments (settings and options, ugh),
and the install-boot plan function hs the capacity to do so too.
You could, for example, accept as an argument the bin directory you want to install
boot to, and pass that all the way along to the final exec-script in the plan.
A more complex service
On to the actual example. Here, I’ll be defining a custom group spec for my app. Here’s what it needs:
- Boot
- Java (obviously)
- Postgres 9.3 (plus configuration)
- Nginx (plus configuration)
- Upstart configuration (for automatic stop/starting)
As you might be expecting, each of these can more-or-less be expressed as a server spec. In fact, the Java, Nginx, and Upstart ones are available as crates: java-crate, upstart-crate, nginx-crate. Note that the nginx crate is not from the Pallet project; the Pallet one has not been updated to support either newer nginx or newer Pallet versions, so we’re using this fork (which thankfully has its own jar on clojars).
Postgres has a crate too, but it is also for an old Pallet version. But rolling our own might be another instructive experience, so let’s look at that:
Configuring Postgres
Here, we really just needed to install Postgres and create a user and database. I chose to do this as so:
(require
'[pallet.api :as api]
'[pallet.action :refer [with-action-options]]
'[pallet.actions :as actions :refer [exec-script]])
(defn db-server-spec
[{:keys [dbname user password] :as settings} & {:keys [] :as options}]
(api/server-spec
:phases {:install (api/plan-fn
(actions/package "postgresql-9.3")
(with-action-options {:sudo-user "postgres"}
(let [q (str "CREATE ROLE "
user
" WITH LOGIN PASSWORD '"
password
"'")]
(exec-script ("psql" postgres -c ~(str "\"" q "\"")))
(exec-script ("createdb" ~dbname -O ~user)))))
}))
Here, we’ve defined our one :install phase inline.
- The
packageaction will use the system’s default package manager (aptitude, since I’m running ubuntu) and get the package requested. - The
with-action-optionscontext wrapper will modify how the enclosed actions are executed, in this case running them as thepostgresuser that gets automatically created with the installation of the postgresql-9.3 package. - I construct a query to create the role, because I couldn’t be bothered to learn
how to deal with the interactive password dialog
createuserwould require. Note that I don’t quote thepostgres -cin theexec-scriptcall, and also that I unquote the call tostrthat I use to wrap the query I built in quotes. It turns out that exec-script will stringify whatever you pass to it unless you unquote it with ~. - Finally, I create the database with the user I just created as the owner.
Configuring Nginx
We don’t need to make our own server-spec for this, but we do need to construct a settings map to pass to the nginx server spec to create our config file. They chose to create a DSL that represents an NGINX config file as a Clojure data structure, which is weird but also handy. Here’s mine:
(def nginx-settings
{:sites
[{:action :enable
:name "default.site"
:upstreams
[{:lines
[{:server "127.0.0.1:8080"}
{:keepalive 32}]
:name "http_backend"}]
:servers
[{:access-log ["/var/log/nginx/app.access.log"]
:locations
[{:path "/"
:proxy-pass "http://http_backend"
:proxy-http-version "1.1"
:proxy-set-header
[{:Connection "\"\""},
{:X-Forwarded-For
"$proxy_add_x_forwarded_for"}
{:Host "$http_host"}]}]}]}]})
Configuring Upstart
This required some serious source code referencing (I used the riemann crate for reference), but I managed to figure out how to set this up.
First: you need to call the pallet.crate.service/supervisor-config function
with an identifier and the settings you want to use. Here’s my settings
function:
(defplan app-server-settings
[& {:keys [user project-dir service-name run-command db-name db-user db-password]}]
(let [settings {:version "0.1.0"
:user user
:group user
:owner user
:run-command run-command
:chdir project-dir
:service-name service-name
:supervisor :upstart
:db-name db-name
:db-user db-user
:db-password db-password
}]
(service/supervisor-config :app-server settings {})))
Next, you need to define a method for the pallet.crate.service/supervisor-config-map
multimethod, setting up your upstart job:
(defmethod supervisor-config-map [:app-server :upstart]
[_ {:keys [run-command service-name user chdir db-name db-user db-password] :as settings} options]
{:service-name service-name
:exec run-command
:chdir chdir
:setuid user
:env [(str "DATABASE_NAME=" db-name)
(str "DATABASE_USER=" db-user)
(str "DATABASE_PASSWORD=" db-password)]})
Here, I set up my job to provide some environment variables, and run a particular command in a particular directory as a particular user. So it sounds like we have some configuration to do to make sure those directories and users and commands exist.
Setting up the app
Here’s the setup I used. I create a user with a home directory because boot
needs a place to install itself, and use some commands from the git crate
to create a bare repository, and a checkout of said repository, for that user.
(defplan app-server-install
[& {:keys [user repo-dir project-dir] :as settings}]
; Init user
(actions/user user
:system false
:home (str "/home/" user)
:shell "/bin/bash"
:action :create)
(actions/directory (str "/home/" user) :owner user)
; Init repo
(git/git "init" "--bare" repo-dir)
(exec-script ("chown" -R ~(str user ":" user) ~repo-dir))
(actions/directory project-dir :owner user :group user)
(with-action-options {:script-dir project-dir :sudo-user user}
(git/clone repo-dir :checkout-dir ".")))
This way, I can use git to push my project, then run my :deploy phase to
pull any changes, run migrations or do whatever else, and then restart the
upstart job. In fact, here’s the plan that does just that:
(defplan app-server-deploy
[& {:keys [project-dir user db-name db-user db-password service-name] :as settings}]
(with-action-options {:script-dir project-dir
:sudo-user user
:script-env {"DATABASE_USER" db-user
"DATABASE_NAME" db-name
"DATABASE_PASSWORD" db-password}}
(git/pull :branch "master" :remote "origin")
(exec-script ("boot" ragtime -m)))
(service/service {:supervisor :upstart
:service-name service-name}
{:action :restart}))
Note the env variables, present so that boot can find them and configure ragtime to access the database.
That’s all we need for our app spec:
(defn app-server-spec
[{:keys [repo-dir project-dir user service-name run-command
db-name db-user db-password] :as settings}
& {:keys [] :as options}]
(api/server-spec
:phases {:settings (api/plan-fn (app-server-settings
:user user
:project-dir project-dir
:run-command run-command
:service-name service-name
:db-name db-name
:db-user db-user
:db-password db-password
))
:install (api/plan-fn (app-server-install
:project-dir project-dir
:repo-dir repo-dir
:user user
))
:configure (api/plan-fn
(service/service {:supervisor :upstart
:service-name service-name}
{:action :enable}))
:run (api/plan-fn
(service/service {:supervisor :upstart
:service-name service-name}
{:action :start}))
:deploy (api/plan-fn (app-server-deploy
:project-dir project-dir
:user user))
}))
Note the two new phases, :configure and :run, which just update
upstart. Also, this is a good time to mention that :configure is the
“default” phase, run if nothing else is mentioned.
Pulling it together
Now, all that’s left is to create the master group spec that will
fully a configure a server with all these different aspects we’ve defined.
We do that by passing a healthy bunch of server specs into the :extends
key of the group spec. Here’s how that looks:
(defn app-group
[& {:keys [repo-dir project-dir user service-name run-command
db-name db-user db-password group-name]}]
(api/group-spec
group-name
:extends [(upstart/server-spec {:service-dir "/etc/init"
:bin-dir "/usr/bin"
})
(git/server-spec {})
(java/server-spec {:version [7] :os :linux})
(nginx/nginx nginx-settings)
(boot/server-spec {})
(db-server-spec {:dbname db-name
:user db-user
:password db-password})
(app-server-spec {:repo-dir repo-dir
:project-dir project-dir
:user user
:service-name service-name
:run-command run-command
:db-name db-name
:db-user db-user
:db-password db-password
})
]
:phases {
:test (api/plan-fn (exec-script ("touch" "pallet.txt")))
}))
Here, we instantiate all those specs I mentioned before, as well as git and java, and the two custom specs we made for setting up postgres and running the app. Finally, it’s time to try a deploy. (Actually, IRL you will probably be doing this in a REPL and piecemeal.)
Running your spec
First, a few items of configuration:
(require
'[pallet.core.user :refer [make-user]]
'[pallet.compute :refer [instantiate-provider]]
'[pallet.compute.node-list :as nl])
(def root-user
(make-user "root"
{:private-key-path "/path/to/.ssh/id_rsa"
:public-key-path "/path/to/.ssh/id_rsa.pub"}))
(def my-nodelist
(instantiate-provider
"node-list"
:node-list [(nl/make-node "example.com" "example" "example.com" :ubuntu
:os-version "14.04")
]))
We hand Pallet our SSH keys so it can connect to the server as us, and instantiate
our “compute provider”, which is what Pallet (well, jclouds really) calls a server that is not a “blob store”.
The node-list provider is just a list of nodes, which you can make with the make-node function.
The arguments that expects are: name group-name ip os-family, followed by any key-value
arguments you’d like to throw on. Note that I used a host name as the IP; this is apparently
totally fine. The name and group-name won’t be used much by us, except that the group-name should
match the group-name you provided to your group spec, but if you had a bunch
of servers that you wanted to configure the same way you could give them the same group
name and apply the group spec to that group.
Pallet provides two functions that apply specs to nodes: converge, and lift. They’re mostly
the same, except converge is allowed to create or destroy nodes and lift isn’t. Since node-list
configuration makes that a non-issue, we’ll just use lift.
lift takes, at the least, a group spec, a provider (our node list), a user (the root user), and a :phase argument.
Here, I’ll show you:
(def my-group (app-group :group-name "example" :all "that" :other "stuff" :too "..."))
(api/lift my-group
:compute my-nodelist
:user root-user
:phase :install)
When you’re running stuff, you’ll just vary the phase (usually :install -> :configure -> :deploy -> :run or some such) and that will be it.
Setting up your ops project
I can’t recommend the Pallet lein template. It’s all old and janky. I used boot with these dependencies and everything went ok:
[[org.clojure/clojure "1.7.0" :scope "provided"]
[environ "1.0.1"]
[com.palletops/pallet "0.8.2"]
[com.palletops/pallet-jclouds "1.7.3"]
[org.apache.jclouds/jclouds-all "1.7.3"]
[org.apache.jclouds.driver/jclouds-sshj "1.7.3"]
[com.palletops/upstart-crate "0.8.0-alpha.2"]
[com.palletops/git-crate "0.8.0-alpha.2"]
[com.palletops/java-crate "0.8.0-beta.4"]
[org.clojars.strad/nginx-crate "0.8.6"]]
Alright, that’s all the advice I can give. I hope you have an easier time of it than I did. Good luck!