June 13, 2013

Clojure.core: Batteries (almost) included

Today, I want to provide a guided tour through some of the many libraries available from the Clojure team that don’t come distributed with Clojure. Consider them Clojures standard library. Some came from old clojure.contrib libs, others are brand-new, but all are great.

Foreword

I spend a lot of time thinking about how to make Clojure more popular. After all, if there’s more code written in Clojure, there’s more need for Clojure developers, which means more opportunities to use my favorite language of late in production. One potential barrier to adoption – if only a psychological one – is that Clojure, as a Lisp-derived language, tends to eschew the batteries-included philosophy of other, more mainstream languages. Java, PHP, Python and Ruby come to mind as languages I’ve used that come with extensive standard libraries by default, and I’m sure C# and Perl and all the other ones I’ve never really dealt with.

Clojure’s default installation takes a different tack from these. Besides core, clojure includes only the following menagerie of utilities and helpers. Some of these are very useful indeed:

Other libraries included with clojure may be mostly only useful to the clojure.core team themselves, or people writing tools for clojure:

However, beyond the libraries distributed with clojure, there is a whole world of libraries maintained by the clojure team, mirroring many of the “batteries” from other languages. And thanks to Leiningen, these almost-first-class-members of the clojure experience are just an entry in :dependencies away.

First on our tour, we’ll check out 3 libraries available in the clojure.data namespace. Most projects won’t need all of these, but a good number will probably have occasion to use at least one.

data.csv

If a library doesn’t come with an easy way to deal with csv files, it probably doesn’t care about you and your petty problems. But Clojure loves you, baby, and clojure.data.csv is how you know it. (Actually, it’s Jonas Enlund who loves you, and who wrote this lib)

(require '[clojure.data.csv :as csv]
         '[clojure.java.io :as io])

(with-open [in-file (io/reader "in-file.csv")]
  (doall
    (csv/read-csv in-file)))

(with-open [out-file (io/writer "out-file.csv")]
  (csv/write-csv out-file
                 [["abc" "def"]
                  ["ghi" "jkl"]]))

data.xml

I mentioned above Rich Hickey’s effort, clojure.xml, but here’s it’s big brother. clojure.data.xml, maintained and likely written by Ryan Senior.

data.xml’s parsing utilities return records of the following description

    #clojure.data.xml.Element{:tag :foo,
                              :attrs {},
                              :content ...}

You can deal with these parsed records just like maps, as usual. Here’s an excerpt from some code I recently wrote to extract items from RSS feeds:


(defmulti get-items :tag)
(defmethod get-items :rss [doc]
  (->>
    doc
    :content
    first
    :content
    (filter-tag :item)
    (map :content)))

(defmethod get-items :feed [doc]
  (->>
    doc
    :content
    (filter-tag :entry)
    (map :content)))

(defmethod get-items :default [_] [])

You can construct XML documents by creating Elements yourself (using the (element [tag attrs content]) record definition), or you can use a Hiccup-esque syntax with sexp-as-element:

(sexp-as-element
  [:foo {:foo-attr "foo value"}
    [:bar {:bar-attr "bar value"}
      [:baz {} "The baz value"]]]))

Good stuff.

data.json

Like other JSON libraries for languages with literal hash maps, Stuart Sierra’s clojure.data.json is best thought of as a conversion layer between hash maps and JSON strings, although this library also provides stream-based i/o.

(json/read-str "{\"a\":1,\"b\":2}")
;;=> {"a" 1, "b" 2}

(json/write-str {:a 1 :b 2})
;;=> "{\"a\":1,\"b\":2}"

What more do you need to know? If you need to deal with json in clojure, you want clojure.data.json. There’s lots more functionality, so be sure to read the docs.

tools.macro

Adds a few syntactic sugars to macro definitions in clojure: macrolets, which are locally-scoped macros, and “symbol macros”.

(defsymbolmacro sym (+ a b))
(with-symbol-macros
  (+ 1 sym))
;; expands to (+ 1 (+ a b))

Some nice little utils. If your projects don’t already have half-baked implementations of these, you should include clojure.tools.macro before you do.

tools.logging

clojure.tools.logging, by Alex Taggart, “wraps”, i.e. implements, all of the big Java logging libraries at once. Write your code once, and pick your implementation later.

The logging functions are exactly what you’d expect:

(defn divide [x y]
  (try
    (info "dividing" x "by" y)
    (/ x y)
    (catch Exception ex
      (error ex "There was an error in calculation"))))

If you’re not using Timbre, you should be using clojure.tools.logging.

core.cache

I must admit, when I tried to use clojure.core.cache (written and maintained by Michael Fogus), I got a bit confused for a while before I realized that I had to be storing the output of the cache. Really, I don’t mind if the caching library I use is a bit stateful!

But, I’m still happy about core.cache, because it paves the way for…

core.memoize

This library (also Fogus) is great. You use it to wrap a function, and it pops an atom right in the metadata of the function. Your function calls are instantly memoized.

You can pick and choose your caching strategy as per core.cache, too:

    (require '[clojure.core.memoize :refer (memo-lu)])

    (def id (memo-lu #(do (Thread/sleep 5000) (identity %)) 3))

    (id 42)
    ; ... waits 5 seconds
    ;=> 42

    (id 42)
    ; instantly
    ;=> 42

Go read all about it.

core.match

David Nolen’s core.match is an alpha-quality attempt to bring pattern-matching to clojure. I prefer writing code to words, so let me explain it to you in the language of my people: Fizzbuzz!

(use '[clojure.core.match :only (match)])

(defn fizzbuzz
  "
  Print each number from 1 to 100, but for multiples of 3 print 'Fizz'
  instead, and for multiples of 5 print 'Buzz'. For multiples of both
  print 'FizzBuzz'
  "
  []
  (doseq [n (range 1 101)]
    (println
      (match [(mod n 3) (mod n 5)]
        [0 0] "FizzBuzz"
        [0 _] "Fizz"
        [_ 0] "Buzz"
        :else n))))

This library has a nice little writeup available for it, so I won’t belabour it.

I personally can’t wait for this to get out of alpha, and hopefully get absorbed into clojure.core

core.unify

The core.unify library, another Michael Fogus joint, is a library that brings unification to clojure.

It’s not a concept I was particularly familiar with before I started writing this, but I actually found the best examples in the comments section of the main library file from which the following examples are alternately inspired and outright stolen.

There are three primary functions exposed by core.unify.

(unify [expr1 expr2]) attempts to define what symbols (denoted by ?) it can given an expression.

(u/unify {:first '?first
          :last  '?last
          :genre :giallo}

         {:first "Dario"
          :last  "Argento"
          :genre :giallo})
;; => {?first "Dario" ?last "Argento"}

Here’s most of the table from that wikipedia article, translated into core.unify

(u/unify 'a 'a)                     ;; => {}
(u/unify 'a 'b)                     ;; => nil
(u/unify '?x '?x)                   ;; => {}
(u/unify 'a '?x)                    ;; => {?x a}
(u/unify '?x 'y)                    ;; => {?x y}
(u/unify '(f a ?x) '(f a b))        ;; => {?x b}
(u/unify '(f a) '(g a))             ;; => nil
(u/unify '(f ?x) '(f ?y))           ;; => {?x ?y}
(u/unify '(f ?x) '(g ?y))           ;; => nil
(u/unify '(f (g ?x)) '(f ?y))       ;; => {?y (g ?x)}
(u/unify '(f (g ?x) ?x) '(f ?y a))  ;; => {?y (g ?x) ?x a}
(u/unify '?x '(f ?x))               ;; (throws an exception)

(subst [expr bindings]) is the inverse of unify, in a way. Given an expression and a map of bindings, it returns the expression with the bindings substituted. Symbols can be nested:

(u/subst
  '[1 2 ?x ?y]
  '{?x [3 4 ?y 6] ?y 1})
;; => [1 2 [3 4 1 6] 1]

(unifier [expr1 expr2]) performs a unification and then a substitution, resulting in the most complete version of the provided expressions that can be formed.

(defn my-unifier [expr1 expr2]
  (u/subst expr1
  (u/unify expr1 expr2)))
(my-unifier '(f ?y) '(f (g ?x))) ;; => (f (g ?x))
(u/unifier '(f ?y) '(f (g ?x)))  ;; => (f (g ?x))

The primary motivation for writing unifiers seems to be the creation of type inference systems, but I’m sure there’s other cool stuff you can find to do with your newfound learning.

core.logic

This library, also written/maintained by David Nolen, clojure.core.logic is an implementation of both miniKanren and cKanren (pdf) in Clojure. You could think of it like core.unify, but fuller-featured.

It lets you write declarative statements, then run them against the logic engine to get a list of results satisfying the provided conditions.

Here’s a sample from the primer:

  (run* [q]
    (membero q [1 2 3])
    (membero q [3 4 5]))
  ;; => (3)

Of course there are better ways to expression set intersection, but there are tons of applications that clojure.core.logic can help express more concisely.

There is pretty good documentation available on the project’s wiki, but I prefer The Magical Island of Kanren as an intro (even if the syntax is mildly outdated).

Also, here’s a very recent blog post by the author that I found really helpful.

core.typed

Statically-typed functional programs have a nice habit of just working, once you get your types all sorted out. clojure.core.typed, written by Ambrose Bonnaire-Sergeant, lets you do the next best thing in Clojure, by adding type annotations to your functions.

Here’s how it works. First, write a function:

(ns example.typed
  (:require [clojure.core.typed :as t]))

(t/ann add [Number Number -> String])
(defn add [a b]
  (+ a b))

Then, call check-ns from somewhere within that namespace (I keep mine in a comment at the bottom of the file and run it with fireplace.vim):

(t/check-ns)

And, since I annotated that wrong, we’ll see some errors:

Type Error (example.typed:7:3) Return type of static
method clojure.lang.Numbers/add is java.lang.Number, expected java.lang.String.
in: (clojure.lang.Numbers/add a b)


Type Error (example.typed:6) Expected type: String
Actual: Number
in: (clojure.lang.Numbers/add a b)


Type Error (example.typed:6:1) Expected type: (Fn [Number Number -> String])
Actual: (Fn [Number Number -> Number])
in: (def add (fn* #))

You probably won’t annotate everything, but it’s nice to have core.typed around when you forsee there might be a typing problem, and it’s good to have your code checked anyhow.

core.contracts

I slacked off and left this out last night, but somebody called me out so now I’m back.

Fogus’s clojure.core.contracts is for people who want their functions to be even more uptight than can be had with static typing. It’s basically a replacement for starting your function with a bunch of assert calls. Here’s the example from the home page:

(use 'clojure.core.contracts)

    (def secure-doubler
      (with-constraints
        (fn [n] (* 2 n))
        (contract doubler
          "ensures doubling"
          [x] [number? => (= (* 2 x) %)]
          [x y] [(every? number? [x y])
                   =>
                 (= (* 2 (+ x y)) %)])))

    (secure-doubler 10)
    ;=> 20

As you can see, you can specify pre- and post- conditions, and even extend the function itself. Pretty cool.

Conclusion

That’s just a quick tour of some of the interesting libraries I found, but it’s far from exhaustive. There are a ton more, though, so be sure to check out the Clojure github repo to see what else exists.