What’s in a namespace?
Every well-behaved clojure source file starts with a namespace declaration.
ns macro, as we all know, is responsible for declaring the namespace
to which the definitions in the rest of the file belong, and generally also
includes some requirements and imports and whatnot. But today (at
Clojure/West, shoutout!) Stuart Sierra made a passing reference to
the internals of
ns during his talk that got me interested.
But what is a namespace really?
Some things in Clojure are implemented as Java classes of terrifying scope and, for reasons known and important only to the Man himself, using a perplexing and obscure formatting style. But, namespace is not one of those things, which is nice because otherwise the rest of this post would be Java source code and nobody wants to see that. (I’m perfectly comfortable with normal Java code, by the way, it’s just that clojure’s, for probably reasonable reasons, is not that).
No, you came to see some down and dirty Clojure internals, and that’s
what you’re going to get. It turns out that
ns is just a regular old
macro that expends to a bunch of regular old functions that just happen
to be how Clojure happens to load code. In other words, we can use our
macroexpand to peer (partway) down this particular
rabbit hole, which as you might have guessed is what is about to happen.
clojure.core’s code is about as transparent as the underlying
Java is opaque, so everything went better than expected for the purposes of this post.
The result of
(macroexpand '(ns namespaces.test)) is the following:
(do (clojure.core/in-ns (quote namespaces.test)) (clojure.core/with-loading-context (clojure.core/refer (quote clojure.core))) (if (.equals (quote namespaces.test) (quote clojure.core)) nil (do (clojure.core/dosync (clojure.core/commute (clojure.core/deref (var clojure.core/*loaded-libs*)) clojure.core/conj (quote namespaces.test))) nil)))
Hey, that’s not so bad! Inside that do, there are 3 things happening:
in-nsis called, setting the current value of the ns var in the
clojure.corenamespace. Stateful! Shame! But, there’s a rich tradition of lisps in general working this way behind it, and it saves adding an extra indentation level to every source file by including it in some
with-nsmacro, so I think we can let this slide. (Also, judging by the Java code, someone has a vendetta against excess indentation).
- Inside the
referthe clojure.core namespace. This is good, because we always want the functions in
clojure.coreto be handy. Also,
referturns out to be the root of all code-loading in clojure; more on this later.
- Finally, if the namespace in question is not
clojure.core, we append its name (as a symbol) to the
dosynctransaction as one does when working with refs.
“I hope he’s not about to go into excruciating detail about each of those steps,” I hear you psychically mutter. Too bad for you!
From the top:
in-ns is one of those things that Clojure defines in Java.
If I wasn’t clear before, I have no intention of delving beyond that barrier, but
from context it’s clear that
in-ns does what
ns would do if
dedicated to encapsulating the half-dozen different things involved in initializing
In other words,
in-ns is what you would be using if you were also using the
use functions in your code, instead of the
:use (p.s. don’t use
ns. In code form, this…
(in-ns 'namespaces.test) (require 'clojure.string)
… is about equivalent to:
(ns namespaces.test (:require clojure.string))
Well, except for all the other stuff that
Incidentally, while sternly recommending against it, the
in-ns extensively, sometimes spreading a namespace across files. But
who are we to judge?
The next thing that happens is that the
clojure.core namespace is
refer’d into the context of the namespace that was just declared the
current namespace in
in-ns above. This is important because we probably
want access to the functions in
refer ends up
reference method on the
*ns* (current namespace) var, which is
an instance of
clojure.lang.Namespace, which does a voodoo dance that results
in the relevant symbols being made available to us.
I didn’t explore too
deeply into what the undocumented
with-loading-context macro does,
accomplishes, because Java, but perhaps it has something to do with
how we’re able to use functions defined in
clojure.core to load
Or perhaps not?
Finally, the rigamarole surrounding the
*loaded-libs* refs is just there
load-lib doesn’t reload a loaded lib (er, namespace) unless specifically
asked to. More about
load-lib right now:
:import) into your
ns declaration just
adds a matching call to
import) to the
part of the namespace macro expansion.
Require and use turn out to be different flavors of the same thing:
(defn require "...docs..." [& args] (apply load-libs :require args)) (defn use "...docs..." [& args] (apply load-libs :require :use args))
load-libs is a collection of checks and whatnot wrapping a bunch of
load-lib, which we met above. In turn,
the namespace into a path and does some other stuff and then calls
on the path, while
load invokes the Java methd
clojure.lang.RT/load on it.
The eldritch magic contained therein in turn grabs the
file, compiles it, and puts all the top-level symbols it found into the current
import macro is a bit different, deferring to
import*, another magic
clojure symbol that’s only defined in Java. The details aren’t clear to me,
but I think it’s safe to assume that ultimately this just uses Java reflection
to load the class.
So what can we take away from this mutually enlightening experience?
nscall is actually not so special, until you get to the special parts.
- We could manually implement just about everything with
requiredirectly, but we shouldn’t.
- I should talk to someone about my underlying Java phobia.
That’s all for tonight!