Dec 17 2008Comments
This article covers Chapter 9, Variable Capture.
Macro Argument Capture
Macros and "normal" code are written in the same language, and can share access to names, data, and code, This is the source of their power, but can also cause subtle bugs. What happens if a macro caller and a macro implementer both try to use the same name? The macro can "capture" the name, leading to unintended consequences.
OL begins with an example of argument capture, a broken definition of
for. Here is a similar macro in Clojure:
(defmacro bad-for [[idx start stop] & body] `(loop [~idx ~start limit ~stop] (if (< ~idx ~stop) (do ~@body (recur (inc ~idx) limit)))))
The problem is the name
limit introduced inside the macro. If you call
bad-for after binding the name
limit, strange things will happen. What if you try:
(let [limit 5] (bad-for [i 1 10] (if (> i limit) (print i))))
Presumably the intent here is to print some numbers greater than five. But in many Lisps, this would print nothing, because the
bad-for macro invisibly binds
limit to ten.
Clojure catches this problem early, and fails with a descriptive error:
(let [limit 5] (bad-for [i 1 10] (if (> i limit) (println i)))) -> java.lang.Exception: Can't let qualified name: ol.chap-09/limit
Clojure makes it difficult to accidentally capture
limit, by resolving symbols into a namespace. The
limit inside the macro resolves to
ol.chap-09/limit, and there is no name collision.
Of course, you do not want your macros to use a shared global name either! What you really want is for macros to use their own guaranteed-unique names. Clojure provides this via auto-gensyms. Simple append
limit in the
bad-for example above, and you get
(defmacro good-for [[idx start stop] & body] `(loop [~idx ~start limit# ~stop] (if (< ~idx limit#) (do ~@body (recur (inc ~idx) limit#)))))
Now the macro will use a unique generated name like
limit__395, and callers can use
good-for as expected:
(let [limit 5] (good-for [i 1 10] (if (> i limit) (println i)))) 6 7 8 9
Another form of unintended capture is symbol capture, where a symbol in the macro unintentionally refers to a local binding in the environment. OL demonstrates the problem with this example:
w is a global collection of warnings that have occurred when using a library. In Clojure:
(def w (ref ))
This is different from the OL implementation because in Clojure data structures are immutable, and mutable things must be wrapped in a reference type that has explicit concurrency semantics. In the code above the
ref wraps the immutable
gripe macro adds a warning to
w, and returns
gripe is intended to be used when bailing out of a function called with bad arguments. In Clojure:
(defmacro gripe [warning] `(do (dosync (alter w conj ~warning)) nil))
Again, this is fairly different from OL because you must be explicit about mutable state. To update
w you must use a transaction (
dosync) and a specific kind of update function (such as
Third, there is a library function
sample-ratio that performs some kind of calculation, the details of which are irrelevant to the example.
sample-ratio also uses
gripe to warn and bailout for certain bad inputs. In Clojure:
(defn sample-ratio [v w] (let [vn (count v) wn (count w)] (if (or (< vn 2) (< wn 2)) (gripe "sample < 2") (/ vn wn))))
This is practically identical to the OL version, since there is no mutable state to (directly) deal with.
Since we are talking about symbol capture, you can probably guess the problem: What happens when the global
w for warnings collides with the local
w argument in
In Common Lisp, this sort of capture would cause the error message to be added to the wrong collection: the local samples
w instead of the global warnings
In Clojure, this just works. The global
w resolves into a namespace, and does not collide with the local one.
More Complex Macros
Clojure's namespaces and auto-gensyms take care of many common problems in macros, but what if you really want capture? You can capture symbols by unquoting them with the unquote character (
~, a tilde) and then requoting them with a non-resolving quote character (
', a single quote). For example, here is a bad version of
gripe that goes out of its way to do the wrong thing and capture
(defmacro bad-gripe [warning] `(do (dosync (alter ~'w conj ~warning)) nil))
I am not going to show more complex macros that really need this feature. My point here is to show that Clojure doesn't make macros safer by compromising their power. You can still do nasty things, you just have to be more deliberate about it.
Interestingly, Clojure protects you from
bad-gripe, even after you go to the trouble of introducing inappropriate symbol capture. Here is a
bad-sample-ratio that uses the buggy
(defn bad-sample-ratio [v w] (let [vn (count v) wn (count w)] (if (or (< vn 2) (< wn 2)) (bad-gripe "sample < 2") (/ vn wn))))
If you try to call
bad-sample-ratio with bad inputs,
bad-gripe will not be able to modify the wrong collection:
(bad-sample-ratio  ) -> java.lang.ClassCastException: clojure.lang.PersistentVector cannot\ be cast to clojure.lang.Ref
Now you see how having immutability as the default can protect you from bugs. The global
w is an explicitly mutable reference. But the local
w is an implicitly immutable vector. When
bad-gripe tries to update the wrong collection, it is thwarted by the fact that the collection is immutable.
Clojure makes simple macros easier and safer to write. The combination of namespace resolution and auto-gensyms prevents many irritating bugs.
Clojure still has the power to write more complex macros when you need it. With the right combination of unquoting and quoting, you can undo the safety net and write any kind of macro you want.
One final note: Because they are ported straight from Common Lisp, many of the examples here are not idiomatic Clojure. In Clojure most uses of imperative loops such as
good-for would be replaced by a more functional style. A good example of this is Clojure's own
for, which performs sequence comprehension.
- The sample code is available at http://github.com/stuarthalloway/onlisp-clojure.
If you find this series helpful, you might also like:
- 2008/12/17: initial version