June 15, 2015

Ripping off Circular.io in Clojure

Introducing Cljcular? Cicljure? Circulure. Yeah, let's do that.

Circular.io is a clone of Buffer, or at least the Buffer of about 5 years ago. Back then, all buffer did was let you schedule posts to Twitter, to be sent at regular intervals throughout the day. It was on HN about 3 years back, and has been running since then. And people really use it! They’ve done a great job with their little app, and the fact that it’s open-source means that for the purposes of this post, we can steal their frontend and just install a new backend.

But why would we do that? Well, there are a few reasons:

  • I’m basically a PHP racist.
  • Clojure (or the JVM anyhow) lends itself to long-running jobs like this, and I think it can do a better job implementing the same features in much less code.
  • I need something to write about, and I want to see how the final product compares.

Circular (despite the PHP thing) is a well-written application using a modern (as of 3 years ago) PHP microframework, so I thought it would make a good comparison, I know I’ve done this sort of post before, but this one is a lot more in-depth, and I’m actually making a real effort to shed my bias and consider both on their merits (and detriments).

At the time of this writing – that is, this exact paragraph – I haven’t actually started yet. So this post will be a more-or-less play-by-play representation of how I work, and it may be long and confusing. I wanted to be honest about what development is actually like, even when I had to go back and fix previous code. After that, I’ll assemble some discussion.

(Ed. You can skip to the discussion now if you like, or see the finished source code on Github).

Writing the backend

Ok, so let’s make a project directory. After about a minute of thought I settled on circulure, so: $ mkdir circulure. I’ve been on the boot train as of this week, so I’ll start with a build.boot file: $ gvim build.boot. I’ll copy most of the dependencies from my [recent project][classifieds], and also copy in at-at from redditlater, since I know that’s what I’m going to use for scheduling.

Here’s my boot.build at this point:

(set-env!
 :source-paths #{"src"}
 :dependencies '[;Tasks
                 [pandeiro/boot-http "0.6.3-SNAPSHOT"]

                 ; Clojure
                 [org.clojure/clojure "1.6.0"]
                 [org.clojure/core.async "0.1.346.0-17112a-alpha"]
                 [environ "1.0.0"]
                 [clojure.jdbc/clojure.jdbc-c3p0 "0.3.2"]
                 [compojure "1.1.5"]
                 [ring "1.1.0"]
                 [org.clojure/data.json "0.2.6"]
                 [http-kit "2.1.18"]
                 [overtone/at-at "1.2.0"]
                 [com.taoensso/timbre "3.4.0"]
                 [com.novemberain/monger "2.1.0"]
])

I hear circular uses Mongo, so I put Monger in there, too.

That’s enough for me to $ boot repl, so that I have something to connect to. I’ll also make a src/circulure directory and start editing core.clj.

I know I’ll need to mirror Circular’s routes, so let’s start with that. The circular api is pretty small and powered by Silex, so it should be straightforward to port to compojure.

I started out by scrolling through the index.php files and filling out the routes with fake handler functions:

(defroutes app-routes
  (GET "/posts" req (get-all-posts req))
  (POST "/posts" req (create-post req))
  (DELETE "/posts" req (delete-post req))
  (PUT "/posts/:id" req (update-post req))

  (POST "/times" req (update-times req))
  (POST "/upload" req (handle-upload req))
  (GET "/settings" req (get-settings req))
  (POST "/settings" req (update-settings req))
  (GET "/counter" req (get-counter req))
  )

The handlers were a bit same-y, so I decided to clean them up with a macro:


(defn json-response [data]

  {:body (json/write-str data)
   :status 200
   :headers {"Content-Type" "application/json"} })

(defmacro r [method route handler]
  `(~method ~route req# (json-response (~handler req#))))

(defroutes app-routes
  (GET / [] "HELLO")
  (r GET "/posts" get-all-posts)
  (r POST "/posts" create-post)
  (r DELETE "/posts" delete-post)
  (r PUT "/posts/:id" update-post)

  (r POST "/times" update-times)
  (r POST "/upload" handle-upload)
  (r GET "/settings" get-settings)
  (r POST "/settings" update-settings)
  (r GET "/counter" get-counter))

Great! Now we can start working on the functions. But, it occurs to me that we should probably get auth in sooner rather than later.

After some research, I settled on friend. At this point, I suffered a bit of a distraction getting way into friend, resulting in this post.

I also remembered how much I hate oauth.

I created a db module because I needed to reference it as part of the credential function.

(ns circulure.db
  (:require [monger.core :as mg]
            [monger.collection :as mongo]))

(def conn (mg/connect))
(def db (mg/get-db conn "circulure"))


(defn user-from-twitter-response
  [{:keys [oauth_token
           oauth_token_secret
           user_id
           screen_name]}]
  {:twitter_access_token oauth_token
   :twitter_access_token_secret oauth_token_secret
   :twitter_user_id user_id
   :twitter_screen_name screen_name})


(defn get-user-by-twitter-user-id [user-id]
  (mongo/find-one-as-map db "users" {:twitter_user_id user-id}))


(defn put-user! [user]
  (if (:_id user)
    (do
      (mongo/update db "users" {:_id (:_id user)} user)
      user)
    (mongo/insert-and-return db "users" user)))


(defn get-or-create-account-by-user [user]
  (if-let [account (mongo/find-one-as-map db "accounts" {:users [(:_id user)]})]
    account
    (do
      (mongo/insert db "accounts" {:users [(:_id user)]})
      (get-or-create-account-by-user user))))

Then, I hooked up the twitter workflow I wrote:


(defn credential-fn
  "Get the user by the token from mongo"
  [oauth-token-response]
  (let [user (or (db/get-user-by-twitter-user-id (:user_id  oauth-token-response))
                 (db/put-user! (db/user-from-twitter-response oauth-token-response)))
        account (db/get-or-create-account-by-user user)]
    {:identity (:twitter_user_id user) :user user :account account :roles #{::user}}))

(def friend-config
  {:workflows [(twitter-auth-workflow
                 {:consumer-key (:twitter-consumer-key env)
                  :consumer-secret (:twitter-consumer-secret env)
                  :oauth-callback-uri {:path "/oauth/twitter"
                                       :url "http://localhost:8080/oauth/twitter"}
                  :credential-fn credential-fn})]})



(def app
(-> #'app-routes 
    (wrap-user)
    (friend/authenticate friend-config)
    (wrap-keyword-params)
    (wrap-params)
    (wrap-session)
    ))

Next, I moved the API routes to a context where they could be authorized. Access, taken care of!



(defroutes api-routes
  
  (comment
  (r GET "/posts" get-all-posts)
  (r POST "/posts" create-post)
  (r DELETE "/posts" delete-post)
  (r PUT "/posts/:id" update-post)

  (r POST "/times" update-times)
  (r POST "/upload" handle-upload)
  (r GET "/settings" get-settings)
  (r POST "/settings" update-settings)
  (r GET "/counter" get-counter)) 
  )

(defroutes app-routes
  (GET "/" [] "HELLO")
  (context "/api/v1" req
      (friend/wrap-authorize api-routes #{::user}))
  )

I wrote a middleware that would put the user and account objects right on the request:

(defn wrap-user [handler]
  (fn [{{id-obj ::friend/identity} :session :as req}]
    (handler
      (if-let [id (-> id-obj :current)]
        (assoc req
               :user (-> id-obj :authentications (get id) :user)
               :account (-> id-obj :authentications (get id) :account))
        req))))

And now we can get right at the user object and start writing the endpoints.

GET /posts: get all the posts (by user).

;; db.clj

(defn get-posts [user]
  (mongo/find-maps db "posts" {:user (:_id user)}))


;; core.clj

(defn get-all-posts [{user :user :as req}]
  (db/get-posts user))

That was pretty easy. Everything is automatically jsonified. Upon testing we may have to clean things up for the frontend, but right now it seems ok.

POST /posts: Creating a post.

;; db.clj
(defn put-post! [post]
  (mongo/insert db "posts" post))


;; core.clj
(defn create-post [{user :user {:keys [picture time status]} :params :as req}]
  (db/put-post! {:user (:_id user)
                 :time (if (= time "now") 0 time)
                 :status status
                 :type (if picture "post_with_media" "post")
                 :picture picture}) )

DELETE /posts/:id: Deleting a post is even easier:

;; db.clj

(defn delete-post! [user post-id]
  (mongo/remove db "posts" {:user (:_id user)
                            :_id (ObjectId. post-id)}))

;; core.clj
(defn delete-post [{user :user {post-id :id} :params :as req}]
  (db/delete-post! user post-id))

PUT /post/:id: Updating post only has one function – to change the time to “now” (i.e. to post it immediately). This requires a few more db helpers:

;; db.clj
(defn put-queued-post! [post]
  (mongo/insert db "queue" post))

(defn move-to-queue [user post-id]
  (when-let [post (get-post user post-id)]
    (delete-post! user post-id)
    (put-queued-post! post)))

;; core.clj
(defn update-post [{user :user {post-id :id post-time :time} :params}]
  (if (= post-time "now")
    (db/move-to-queue user post-id)))

POST /times: The “/times” endpoint is a way to bulk-set the times of posts:

;; db.clj
(defn update-queue-time [user post post-time]
  (mongo/update db "posts" {:_id (ObjectId. (:_id post))
                            :user (:_id user)}
                {:$set {:time post-time}}))

;; core.clj
(defn update-times [{user :user {posts :posts post-time :time} :params}]
  (doall (for [post posts]
              (db/update-queue-time user post post-time))))

GET /settings: doesn’t do much:

(defn get-settings [{account :account}]
  (dissoc account :users))

POST /settings just updates the email address for the logged-in account:

;; db.clj
(defn update-account-email [account email]
  (mongo/update db "accounts" {:_id (:_id account)}
                (if email
                  {:$set {:email email}}
                  {:$unset {:email true}})))

;; core.clj
(defn update-settings [{account :account {email :email} :params}]
  (db/update-account-email account email))

GET /counter: Finally, the /counter endpoint just returns the count of the Post table:

;; db.clj
(defn post-count []
  (mongo/count db "posts"))

(defn get-counter []
  {:count (db/post-count)})

With all the endpoints written, now comes the part where we hook it up for real. I checked out the Circular project from Github into the resources subdirectory, and added a Compojure route to serve the static files:

  (route/resources "/" {:root "Circular"})

After that, it was just a matter of going through and cleaning up the stuff that didn’t match what the frontend expected.

Oauth had to be reworked, since much of it is done on the client side. I ended up writing a custom friend workflow for it:

(def consumer
  (oauth/make-consumer
    (:twitter-consumer-key env)
    (:twitter-consumer-secret env)
    "https://api.twitter.com/oauth/request_token"
    "https://api.twitter.com/oauth/access_token"
    "https://api.twitter.com/oauth/authorize"
    :hmac-sha1))

(defn circular-oauth-workflow
  "Handle OAuth responses as expected by the Circular frontend, based on params:

  ?start=1: obtain the request token and return the authorization url
  ?oauth_verifier=?: When the user returns from twitter, get the access_token
                     and authenticate the user with friend"
  [credential-fn]
  (fn [{params :params :as req}]

    (when (= (path-info req) "/api/oauth.php")
      (cond

        ; Get the request token and get the redirect url
        (:start params)
        (let [request-token (oauth/request-token
                              consumer
                              "http://localhost:8080/api/oauth.php")
              approval-url (oauth/user-approval-uri
                             consumer
                             (:oauth_token request-token))]
          {:body (json/write-str {:authurl approval-url})
           :status 200
           :headers {"Content-Type" "application/json"}
           :session (assoc (:session req) :oauth-request-token request-token)})

        ; Log the user in after getting an access token
        (:oauth_verifier params)
        (when-let [request-token (get-in req [:session :oauth-request-token])]
          (when-let [access-token (oauth/access-token
                                    consumer
                                    request-token
                                    (get-in req [:params :oauth_verifier]))]
            (make-auth (credential-fn access-token))))))))

I had to go back and ensure the methods that wrote to mongo, returned an object for serialization, or else the json-response method would complain about not being able to serialize the WriteResult object. I also had to update the json-response method to take care of ObjectId objects and turn :_id into :id, which I did using a helper:

(declare clean-object-ids)
(defn- clean-object-id [ acc [k v]]
  (assoc acc
         (if (= k :_id) :id k)
         (cond
           (instance? ObjectId v) (str v)
           (map? v) (clean-object-ids v)
           (coll? v) (map clean-object-ids v)
           :else v)))

(defn clean-object-ids [d]
  (cond
    (map? d) (reduce clean-object-id {} d )
    (coll? d) (map clean-object-ids d)
    :else d))

(defn json-response [data]
  {:body (json/write-str (clean-object-ids data))
   :status 200
   :headers {"Content-Type" "application/json"} })

I also wasn’t getting the profile photo from twitter. This seemed like as good a time as any to start using the twitter API, so I added [twitter-api "0.7.8"] to my dependencies and added a method to fill out the user object on first login:

(ns circulure.twitter
  (:require [twitter.oauth :refer [make-oauth-creds]]
            [twitter.api.restful :as twitter] 
            [environ.core :refer [env]]))


(defn user-creds [user]
  (make-oauth-creds
    (:twitter-consumer-key env) 
    (:twitter-consumer-secret env)
    (:twitter_access_token user)
    (:twitter_access_token_secret user)))

(defn get-user-info [user]
  (:body (twitter/users-show :oauth-creds (user-creds user) :params {:screen-name "adambard"})) 
  )


(defn fill-user-object [user]
  (let [user-info (get-user-info user)]
    (merge user
           (select-keys user-info [:profile_image_url :description]))))

And in db.clj:

(defn user-from-twitter-response
  [{:keys [oauth_token
           oauth_token_secret
           user_id
           screen_name]}]
  (tw/fill-user-object ; Added
    {:twitter_access_token oauth_token
     :twitter_access_token_secret oauth_token_secret
     :twitter_user_id user_id
     :twitter_screen_name screen_name}))

File uploads are the last thing to implement. I quickly discovered that I would need to get a library for this purpose; I settled on image-resizer, which seemed on-the-nose for what I wanted to do.

This part required me to write some functions that PHP has available already. In particular, PHP gives you file md5s out of the box. I declined to use any more libraries and instead just implemented md5-file using java classes. I also had to use the wrap-multipart-params middleware to enable file upload handling.

Unfortunately, file handling is a place where Clojure doesn’t have optimal facilities – there are probably library solutions to this, but otherwise you’re stuck with wrappers around Java’s idioms, which are a bit… verbose.

(ns circulure.core
  (:require
    ;...
    [ring.middleware.multipart-params :refer [wrap-multipart-params]]
    [image-resizer.format :refer [as-file]]
    [image-resizer.core :refer [resize]])
  (:import
    ;...
    javax.xml.bind.DatatypeConverter
    java.security.MessageDigest
    java.security.DigestInputStream))

; ...

(defn md5-file [file]
  (let [md (MessageDigest/getInstance "MD5")
        dis (DigestInputStream. (io/input-stream file) md)
        ]
    (while (not (= (.read dis) -1)))
    (-> (.digest md)
        (DatatypeConverter/printHexBinary)
        (.toUpperCase))))

(defn handle-upload [{{{filename :filename tempfile :tempfile} "userfile"} :multipart-params
                      {acct-id :_id} :account}]
  (.mkdir (io/file "resources/uploads")) ; For first run
  (.mkdir (io/file "resources/uploads/" (str acct-id)))
  (let [file-md5 (md5-file tempfile)
        dirname (str "uploads/" acct-id "/") 
        destfilename (str dirname file-md5 "-" filename)
        thumbnailfilepath (as-file (resize tempfile 100 100)
                                   (str "resources/" destfilename))
        thumbnailfilename (.getName (io/file thumbnailfilepath)) ]

    (.mkdir (io/file (str "resources/" dirname)))
    (io/copy tempfile (io/file (str "resources/" destfilename )))
    {:url (str "/" destfilename)
     :thumbnail  (str "/uploads/" acct-id "/" thumbnailfilename)}))

; ...

(defroutes api-routes
  ; ...
  (wrap-multipart-params (r POST "/upload" handle-upload)))

Finally, it’s time to write the part that actually posts to twitter, the scheduler. This is the part I thought would be easier in Clojure, and I don’t think I was wrong.

I decided to make use of the twitter library’s asynchronous request support, and combine it with some core.async action to provide a non-callback-based (external) API, and ensure that the scheduler could clean up properly without being blocked on requests.

;; twitter.clj
(defn async-callback [c]
  (AsyncSingleCallback.
    (fn [& args] (go (>! c :ok)))
    (fn [& args] (go (>! c :failure)))
    (fn [& args] (go (>! c :exception)))))


(defn update-status [user status]
  (let [c (chan)]
    (twitter/statuses-update :oauth-creds (user-creds user)
                             :params {:status status}
                             :callbacks (async-callback c))
    c))


;; scheduler.clj
(ns circulure.scheduler
  (:require [monger.core :as mg]
            [monger.collection :as mongo]
            [circulure.twitter :as tw]
            [circulure.db :refer [conn]]
            [clojure.core.async :refer [chan go <! >!]]
            ))

(def db (mg/get-db conn "circulure"))

(defn now []
  (int (/ (System/currentTimeMillis) 1000)))


;; DB Stuff

(defn get-due-posts []
  (mongo/find-maps db "posts" {:time {:$lte (now)}}))

(defn move-to-queue [post]
  (mongo/remove db "posts" {:_id (:_id post)})
  (mongo/insert db "queue" post))

(defn get-queued []
  (mongo/find-maps db "queue"))

(defn move-to-archive [queued-post]
  (mongo/remove db "queue" {:_id (:_id queued-post)})
  (mongo/insert db "archive" queued-post) )

(defn get-user [user-id]
  (mongo/find-one-as-map db "users" {:_id user-id}))


;; 1) Move stuff to queue

(defn prepare-queued-posts []
  (doall (for [post (get-due-posts)]
           (move-to-queue post))))


;; 2) Send stuff in queue and archive it

(defn send-queued-item [item]
  (when-let [user (get-user (:user item))]
    (let [c (tw/update-status (get-user (:user item)) (:status item))]
      (go
        (let [result (<! c)]
          (move-to-archive (assoc item :api-result result)))))))

(defn send-queued-posts []
  (doall (for [item (get-queued)]
           (send-queued-item item))))


(defn run-scheduler []
  (future ; Start a new thread
    (while true
      (prepare-queued-posts)
      (send-queued-posts)
      (Thread/sleep 60000) ; 1 minute
      )))

That all came together pretty straightforwardly. It will first move those posts that are ready to be posted from the “posts” collection to the “queued” collection, then send all the posts in the queue. The tw/update-status function actually returns a clojure core.async channel, which we then asynchronously read out of to trigger the archiving after the posting is complete.

I didn’t end up using at-at after all; it just introduces too much trouble when it comes to deleting or editing posts, and we’re aiming for parity with Circular anyhow. Relatedly, it doesn’t retry failed posts, but neither does Circular though, so I’m happy to settle.

Upon further thought, there’s a possibility of something being posted twice this way if processing takes longer than 60 seconds. Twitter prevents identical statuses, but just to be safe:

;; scheduler.clj

(def currently-processing (atom #{}))

(defn archive-item [item]
  (move-to-archive item)
  (swap! currently-processing disj (:_id item)))

(defn send-queued-posts []
  (doall (for [item (get-queued)]
           (when-not (@currently-processing (:_id item)) ; Still a small race condition: acceptable
             (swap! currently-processing conj (:_id item))
             (send-queued-item item)))))

Now, nothing will attempt to double-post.

Finally, there remains the problem of packing it all up. Luckily, boot makes this pretty easy, although I had to take care of some uberjar exclusions myself:

;;core.clj
(ns circulure.core
  (:gen-class) ; Add gen-class
)

;...

;; Provide a main method
(defn -main []
  (run-jetty #'app {:port (Integer/parseInt (:port env "8080")) :join? false}))


;; build.boot
(deftask package []
(comp
  (aot :all true)
  (pom :project 'circulure :version "0.1.0-STANDALONE")
  (uber :exclude #{#"(?i)^META-INF/[^/]*.(MF|SF|RSA|DSA)$"
                   #"^((?i)META-INF)/.*pom.(properties|xml)$"
                   #"(?i)^META-INF/INDEX.LIST$"
                   #"(?i)^META-INF/LICENSE.*$"})
  (jar :main 'circulure.core)))

After running boot package, circulure-0.1.0-STANDALONE.jar appears in the target directory. Upon running it and going to port 8080, the app appears! Having it as a standalone jar is pretty cool – in theory anyone could download and run it (passing in their own twitter credentials as parameters), although it depends on having a mongo server running. If I’d thought ahead I would have used some embedded datastore instead. Ah, well.

Cleanup

After finishing all the functionality, I went around and cleaned some stuff up. Most of it’s boring – the main change was to take all the auth functions from core.clj and move it to auth.clj, which exports just a single function, wrap-auth, which is used to wrap the handler in place of the friend and wrap-user middlewares that were there before.

I made a helper function to coerce an object to a mongodb ObjectId, then went through db.clj and made sure every query involving an id value was wrapped in the helper.

Discussion

Design differences

A few aspects of the backend I wrote differ from the reference implementation:

  • Circular stores the entire user object on the post; Circulure just stores the id. This means potentially more lookups when the user is needed, but the only time this is the case is during the scheduler loop; the id is wholly sufficient for other purposes. In fact, Circular has to manually strip out the user object (save id) from its API responses in order not to reveal any API secrets.
  • Circulure’s queue runs on the same process as the webserver, just on a different thread. This makes code sharing a bit easier, but could be a performance issue under a lot (a lot) of load.
  • I made a point of separating DB operations into their own functions. This allows for the database to be swapped out if necessary with a minimum of fuss (although the semantics of the _id field still need to be accounted for in the application code)

Code Comparison

It’s hard to compare the two codebases directly, so I’ll just pull out what I consider representative samples of the strengths and weaknesses of each implementation. But first, here’s the line counts, because I know someone’s going to ask for it.

Clojure

PHP

However, the PHP files are a lot more generously spaced and contain extensive comments, which I did not bother with, so in reality it probably works out the same.

Where did which code win?

Given that Clojure is not designed specifically to be a language for building websites, I had to implement a few of things that PHP got for free, such as md5’ing files.

Let’s compare the file upload code:

$protected->post('/upload', function (Request $request) use ($app) {
	$file = $request->files->get('userfile');
	if ($file->isValid()) {
		$extension = $file->guessExtension();
		// Use MD5 to prevent collision between different pictures:
		$md5 = md5_file($file->getRealPath());
		
		$filename      = 'uploads/' . $app['account']['id'] . '/' . $md5 . '.' . $extension;
		$thumbnailname = 'uploads/' . $app['account']['id'] . '/' . $md5 . '.100x100' . '.' . $extension;
		
		$file = $file->move(__DIR__.'/../uploads/'.$app['account']['id'], $md5.'.'.$extension);
		
		// Create thumbnail:
		$simpleResize = new SimpleResize($file->getRealPath());
		$simpleResize->resizeImage(100, 100, 'crop');
		$simpleResize->saveImage(__DIR__.'/../'.$thumbnailname, 100);
		
		return $app->json(array('url' => APP_URL.$filename, 'thumbnail' => APP_URL.$thumbnailname));
	}
});
(defn md5-file [f]
  (let [md (MessageDigest/getInstance "MD5")
        dis (DigestInputStream. (io/input-stream f) md)
        ]
    (while (not (= (.read dis) -1)))
    (-> (.digest md)
        (DatatypeConverter/printHexBinary)
        (.toUpperCase))))

;; POST /upload
(defn handle-upload [{{{filename :filename tempfile :tempfile} "userfile"} :multipart-params
                      {acct-id :_id} :account}]
  (.mkdir (io/file "resources/uploads")) ; For first run
  (.mkdir (io/file "resources/uploads/" (str acct-id)))
  (let [file-md5      (md5-file tempfile)
        dirname       (str "uploads/" acct-id "/") 
        destfilename  (str dirname file-md5 "-" filename)
        thumbnailpath (as-file (resize tempfile 100 100)
                               (str "resources/" destfilename))
        thumbnailname (.getName (io/file thumbnailfilepath)) ]

    (io/copy tempfile (io/file (str "resources/" destfilename )))
    {:url (str "/" destfilename)
     :thumbnail  (str "/uploads/" acct-id "/" thumbnailfilename)}))

There’s no doubt that the PHP code is way ahead here, in both brevity and clarity.

In other cases, Clojure was often able to express operations in much less code – although perhaps this was because I had the benefit of starting from scratch with a defined spec:

$protected->post('/posts', function (Request $request) use ($app) {
	
	$post = $app['data'];
	
	// Check that this account really manages this user
	if (!array_key_exists($post['user'], $app['account']['users'])) {
		return new Response('Unauthorized', 401);
	}
	
	// Add user information:
	$m = new Mongo();
	$user = $m->circular->users->findOne(array('_id' => new MongoId($post['user'])));
	$post['user'] = $user;
	
	// Add Twitter request info:
	if (isset($post['picture'])) {
		$post['type'] = 'post_with_media';
	}
	else {
		$post['type'] = 'post';
	}
	
	// Nest status into `params`:
	$post['params'] = array('status' => $post['status']);
	unset($post['status']);
	// XXX: Apparently Backbone has poor support for nested attributes
	// @see http://stackoverflow.com/questions/6351271/backbone-js-get-and-set-nested-object-attribute
	
	
	$m = new Mongo();
	
	if (isset($post['time']) && $post['time'] == "now") {
		// If explicitly requested, send it right now through `queue`:
		$m->circular->queue->insert($post);
	}
	else {
		$m->circular->posts->insert($post);
	}
	
	// MongoId are assumed to be unique accross collections
	// @see http://stackoverflow.com/questions/5303869/mongodb-are-mongoids-unique-across-collections
	
	return $app->json(array('id' => (string) $post['_id']));
});

Vs.

;; core.clj
(defn create-post [{user :user {:keys [picture time status]} :params :as req}]
  (db/put-post! {:user (:_id user)
                 :time (if (= time "now") 0 time)
                 :status status
                 :type (if picture "post_with_media" "post")
                 :picture picture}))


;; db.clj
(defn put-post! [post]
  (if (:_id post)
    (do (mongo/update db "posts" {:_id (object-id (:_id post))} post)
        post)
    (mongo/insert-and-return db "posts" post)))

Whether this is a victory of the language or just the implementation is up for argument.

Other than that, the two are fairly equivalent. Silex is indeed a very nice framework to work with – the flexible ->before and ->after handlers on route sets are quite comparable to providing Ring middleware, which is one of my favorite features of Ring.

Performance

If anyone cares to benchmark the two, feel free to tell me about it. Silex is noted to have a performance penalty, but then again, there are many projects around that can speed up PHP remarkably.

Deployment

Usually, PHP has an edge on deployment, what with the FTP-friendly drop-your-files-here process it usually allows. But in this case, both of these implementations require enough access to a server to configure a long-running job (the Daemon in the case of PHP, and running the jar in the case of Clojure), and I don’t think either of these is much harder than the other.

I still think it’s pretty cool that you can just run a jar and have the whole thing spin up though.

Non-technical concerns

The most-cited objection to Clojure is to the syntax; reams and reams have been written about this, so I won’t go into too much detail in defense or agreement with this notion. I will say that, if left unchecked, Clojure code (or any lisp, really) can descend into a dense nest of s-expressions, especially given the iterative coding style that emerges from editor REPL integration. I’ve done my best to write clear code, but to just what extent “clear” is a relative term I’m not able to judge, being familiar with Clojure already. Then again, to paraphrase King Richard, just because you don’t speak German, doesn’t mean that German is incomprehensible.

Of course, I’ve said nothing about what can happen to PHP in careless hands, and I don’t think even the most enthusiastic PHP evangelist would disagree with the horrors of that avenue.

So that’s it for today, I hope you enjoyed it. You can take a gander at the full source on Github if you like.