Blog Posts tagged with: clojure

Jul 03 2009

Comments

Clojure in the Field

Clojure is getting a lot of positive buzz, but what is it like building and shipping a real application? In this talk, we will cover the good, the bad, and the ugly of commercial Clojure development, including:

  • BDD meets FP: how we adapted spec and test practices for functional code
  • To wrap or not to wrap: working with Java libraries
  • The learning curve: training a team of non-Clojurists on Clojure
  • What you will wish for: third-party libraries you should know about, plus some that don't exist yet
  • Good names or die: living without object context
  • Shipping it: how to manage staging and deployment of Clojure code

Experience with Clojure is useful but not a prerequisite; we will introduce the key concepts as we go.

May 27 2009

Comments

Rifle-Oriented Programming with Clojure

If you come to Clojure from an object-oriented background, you may not know where to start. It is sort of like looking at a rifle for the first time and asking "But where do I put the arrows?"

Clojure solves the traditional problems of OO (and then some!) but it does it in different ways. To learn how to translate your arrows (encapsulation, polymorphism, and inheritance) into bullets, check out my new article in the May issue of NFJS, the Magazine.

Also: I'll be spending the summer on the NFJS circuit, talking about Clojure, Git, and other good things. Come see us.

Apr 04 2009

Comments

Programming Clojure Beta 9 Is Out

Programming Clojure Beta 9 is now available. We are almost done, and most of the changes in this Beta are small.

What's new:

  • the notation conventions have changed to make console output and REPL results more distinct
  • a new subsection covering functions on vectors
  • a new subsection on sort functions
  • an example using map with more than one collection
  • an example using the :while option to a sequence comprehension
  • examples now use the new letfn form where appropriate
  • replicate is gone -- use repeat instead
  • an index!

I have also made the book more strict in discussions of laziness. Clojure sequences are lazy, but function evaluation is eager. In the Clojure community we often ignore this distinction, saying things like "iterate is lazy" when we really should say "iterate returns a lazy sequence." The book now uses the latter formulation.

To make sure you have the latest, greatest version of the sample code from the book, go and grab the github repo.

Thanks again to everyone who has been offering feedback. Keep the feedback coming!

Jan 31 2009

Comments

Clojure Interview on InfoQ

Werner Schuster over at InfoQ has interviewed me about Clojure. Check it out.

Jan 30 2009

Comments

Programming Clojure Beta 6 is Out

Programming Clojure Beta 6 is now available. What's new:

  • A new Chapter, "Functional Programming," exlains recursion, TCO, laziness, memoization, and trampolining in Clojure.
  • A new Section, "Creating Java Classes in Clojure," shows how to create and compile Java classes from Clojure.
  • The snake example has been moved into the concurrency chapter. The snake demonstrates how to divide your model layer into two parts: a functional model and a mutable model.
  • The STM example in the introduction now uses alter instead of commute, which allowed a race condition. Given the problem domain of the example, the race condition was acceptable. However, explaining this in the introductory chapter would have been distracting.
  • The lancet runonce function now uses locking instead of an an agent. Yes, you can do plain old-fashioned locking in Clojure! Agents are unsuitable for the kind of coordination lancet requires, because you cannot await an agent while inside another agent.

While I am talking about lancet: it now has its own repository. I have added integration with clojure.contrib.shell-out, so lancet can now call either Ant tasks or other applications. It is still far from being a replacement from Ant or Rake, but maybe your contributions can change that!

The book is now prose-complete, so if you have been waiting for the whole story, this is it. A number of readers have made suggestions for additional topics. If you have suggestions, please add them as comments to this post. For reasons of space and time, most new topics will appear in a series of articles on this blog, not in the book itself.

Clojure and clojure-contrib continue to evolve. To make sure you have the latest, greatest version of the sample code from the book, go and grab the github repo.

Thanks again to everyone who has been offering feedback. I have cleared almost 400 entries from the errata/suggestion page.  Keep the feedback coming!

Jan 13 2009

Comments

Programming Clojure Beta 5 is Out

Programming Clojure Beta 5 is now available. What's new:

A new chapter, Clojure in the Wild, with information on

I have rewritten Section 1.1 "Clojure Coding Quick Start." All dependent libraries (Clojure, clojure-contrib, etc.) are now prebuilt and included with the sample code for the book. This should make it much easier for people to get started.

I have rewritten Section 2.6 "Metadata" to clearly explain the difference between user metadata and compiler metadata.

Clojure and clojure-contrib continue to evolve. To make sure you have the latest, greatest version of the sample code from the book, go and grab the github repo.

Thanks to everyone who has been offering feedback. I have cleared over 300 entries from the errata/suggestion page.  Keep the feedback coming!

Dec 17 2008

Comments

On Lisp -> Clojure, Chapter 9

This article is part of a series describing a port of the samples from On Lisp (OL) to Clojure. You will probably want to read the intro first.

This article covers Chapter 9, Variable Capture.

Macro Argument Capture

Macros and "normal" code are written in the same language, and can share access to names, data, and code, This is the source of their power, but can also cause subtle bugs. What happens if a macro caller and a macro implementer both try to use the same name? The macro can "capture" the name, leading to unintended consequences.

OL begins with an example of argument capture, a broken definition of for. Here is a similar macro in Clojure:

  (defmacro bad-for [[idx start stop] & body]
    `(loop [~idx ~start limit ~stop]
       (if (< ~idx ~stop)
         (do
     ~@body
     (recur (inc ~idx) limit)))))

The problem is the name limit introduced inside the macro. If you call bad-for after binding the name limit, strange things will happen. What if you try:

  (let [limit 5] 
    (bad-for [i 1 10] 
      (if (> i limit) (print i))))

Presumably the intent here is to print some numbers greater than five. But in many Lisps, this would print nothing, because the bad-for macro invisibly binds limit to ten.

Clojure catches this problem early, and fails with a descriptive error:

  (let [limit 5] (bad-for [i 1 10] (if (> i limit) (println i))))
  -> java.lang.Exception: Can't let qualified name: ol.chap-09/limit

Clojure makes it difficult to accidentally capture limit, by resolving symbols into a namespace. The limit inside the macro resolves to ol.chap-09/limit, and there is no name collision.

Of course, you do not want your macros to use a shared global name either! What you really want is for macros to use their own guaranteed-unique names. Clojure provides this via auto-gensyms. Simple append # to limit in the bad-for example above, and you get good-for:

  (defmacro good-for [[idx start stop] & body]
    `(loop [~idx ~start limit# ~stop]
       (if (< ~idx limit#)
         (do
     ~@body
     (recur (inc ~idx) limit#)))))

Now the macro will use a unique generated name like limit__395, and callers can use good-for as expected:

  (let [limit 5] (good-for [i 1 10] (if (> i limit) (println i))))
  6
  7
  8
  9

Symbol Capture

Another form of unintended capture is symbol capture, where a symbol in the macro unintentionally refers to a local binding in the environment. OL demonstrates the problem with this example:

First, w is a global collection of warnings that have occurred when using a library. In Clojure:

  (def w (ref []))

This is different from the OL implementation because in Clojure data structures are immutable, and mutable things must be wrapped in a reference type that has explicit concurrency semantics. In the code above the ref wraps the immutable [].

Second, the gripe macro adds a warning to w, and returns nil. gripe is intended to be used when bailing out of a function called with bad arguments. In Clojure:

  (defmacro gripe [warning]
    `(do
       (dosync (alter w conj ~warning))
       nil))

Again, this is fairly different from OL because you must be explicit about mutable state. To update w you must use a transaction (dosync) and a specific kind of update function (such as alter).

Third, there is a library function sample-ratio that performs some kind of calculation, the details of which are irrelevant to the example. sample-ratio also uses gripe to warn and bailout for certain bad inputs. In Clojure:

  (defn sample-ratio [v w]
    (let [vn (count v) wn (count w)]
      (if (or (< vn 2) (< wn 2))
        (gripe "sample < 2")
        (/ vn wn))))

This is practically identical to the OL version, since there is no mutable state to (directly) deal with.

Since we are talking about symbol capture, you can probably guess the problem: What happens when the global w for warnings collides with the local w argument in sample-ratio?

In Common Lisp, this sort of capture would cause the error message to be added to the wrong collection: the local samples w instead of the global warnings w.

In Clojure, this just works. The global w resolves into a namespace, and does not collide with the local one.

More Complex Macros

Clojure's namespaces and auto-gensyms take care of many common problems in macros, but what if you really want capture? You can capture symbols by unquoting them with the unquote character (~, a tilde) and then requoting them with a non-resolving quote character (', a single quote). For example, here is a bad version of gripe that goes out of its way to do the wrong thing and capture w:

  (defmacro bad-gripe [warning]
    `(do
       (dosync (alter ~'w conj ~warning))
       nil))

I am not going to show more complex macros that really need this feature. My point here is to show that Clojure doesn't make macros safer by compromising their power. You can still do nasty things, you just have to be more deliberate about it.

Interestingly, Clojure protects you from bad-gripe, even after you go to the trouble of introducing inappropriate symbol capture. Here is a bad-sample-ratio that uses the buggy bad-gripe:

  (defn bad-sample-ratio [v w]
    (let [vn (count v) wn (count w)]
      (if (or (< vn 2) (< wn 2))
        (bad-gripe "sample < 2")
        (/ vn wn))))

If you try to call bad-sample-ratio with bad inputs, bad-gripe will not be able to modify the wrong collection:

  (bad-sample-ratio [] [])
  -> java.lang.ClassCastException: clojure.lang.PersistentVector cannot\
     be cast to clojure.lang.Ref

Now you see how having immutability as the default can protect you from bugs. The global w is an explicitly mutable reference. But the local w is an implicitly immutable vector. When bad-gripe tries to update the wrong collection, it is thwarted by the fact that the collection is immutable.

Wrapping up

Clojure makes simple macros easier and safer to write. The combination of namespace resolution and auto-gensyms prevents many irritating bugs.

Clojure still has the power to write more complex macros when you need it. With the right combination of unquoting and quoting, you can undo the safety net and write any kind of macro you want.

One final note: Because they are ported straight from Common Lisp, many of the examples here are not idiomatic Clojure. In Clojure most uses of imperative loops such as good-for would be replaced by a more functional style. A good example of this is Clojure's own for, which performs sequence comprehension.


Notes

Other Resources

If you find this series helpful, you might also like:

Revision history

  • 2008/12/17: initial version

Dec 12 2008

Comments

On Lisp -> Clojure, Chapter 7

This article is part of a series describing a port of the samples from On Lisp (OL) to Clojure. You will probably want to read the intro first.

This article covers Chapter 7, Macros.

A Few Simple Macros

OL begins with a simple nil! macro that sets something to nil. nil! is implemented as a macro in Common Lisp (CL) nil needs to generate a special form. Clojure puts much more careful boundaries around mutable state, so most Clojure data structures are not set-able at all. The few things that can be set are reference types, each with an explicit API and concurrency semantics.

Because setters go through an explicit API instead of a special form, the Clojure nil! does not need to be macro at all. Here is a nil! for Clojure atoms:

  (defn nil! [at]
    (swap! at (fn [_] nil)))

The swap! function is specific to atoms. Usage for nil! looks like:

  (def a (atom 10))
  (nil! a)
  @a
  -> nil  

The next interesting macro in OL is nif, which demonstrates the use of backquoting. One way to implement Clojure nif is:

  ((use '[clojure.contrib.fcase :only (case)])
  (defmacro nif [expr pos zer neg]
    `(case (Integer/signum ~expr) 
     -1 ~neg
     0 ~zer
     1 ~pos))

There are a few interesting differences from CL here:

  • Clojure unquoting uses ~ and ~@ instead of CL's , and ,@. This allows Clojure to treat commas as whitespace.
  • Clojure does not have a built-in signum, but it has access to all of Java, including Integer/signum.
  • Clojure's case is not part of core, and is provided by Clojure Contrib.

Defining Simple Macros

OL demonstrates the "fill in the blanks" approach to writing macros:

  • Write the desired expansion.
  • Write the desired macro invocation form.
  • Use backquoting to create a template based on the desired expansion.
  • Use unquoting to substitute forms from the macro invocation into the template.

As examples, OL uses our-when and our-while. The Clojure equivalents are:

  (defmacro our-when [test & body]
    `(if ~test
       (do
         ~@body)))
  (defmacro our-while [test & body]
    `(loop []
       (when ~test
         ~@body
         (recur))))

There is one interesting new thing here. Clojure' loop/recur is an explicit way to denote a self-tail-call so that Clojure can implement it with a non-stack-consuming iteration. (Clojure cannot optimize tail calls in a generic way due to limitations of the JVM.)

It is also worth noting that while loops are uncommon in Clojure. They rely on side effects that change the result of test, and most Clojure functions avoid side effects.

Destructuring in Macros

Both Clojure and CL support destructuring in macro definitions. The OL example of this is a when-bind macro. Here is a literal translation in Clojure:

  (defmacro when-bind [bindings & body]
    (let [[form tst] bindings]
      `(let [~form ~tst]
         (when ~form
     ~@body))))

The [form tst] is a destructuring bind. The first element of bindings binds to form, and the second element to tst. Usage looks like this:

  (when-bind [a (+ 1 2)] (println "a is" a))
  a is 3

Do not use the when-bind as defined above. Clojure provides a better version called when-let:

  ; from Clojure core
  (defmacro when-let
    [bindings & body]
    (if (vector? bindings)
      (let [[form tst] bindings]
        `(let [temp# ~tst]
           (when temp#
             (let [~form temp#]
               ~@body))))
      (throw (IllegalArgumentException.
               "when-let now requires a vector for its binding"))))

when-let adds two features not present in when-bind:

  • when-let requires that the binding form be a vector. This leads to the "arguments in square brackets" style that distinguishes Clojure from many Lisps.
  • when-let introduces a temporary binding temp# using Clojure's auto-gensym feature.

The temporary binding of temp# keeps the binding form from being expanded directly into the when, because some binding forms are not legal for evaluation. The following output shows the difference:

  (when-bind [[a & b] [1 2 3]] (println "b is" b))
  ->java.lang.Exception: Unable to resolve symbol: & in this context 
  (when-let [[a & b] [1 2 3]] (println "b is" b))
  -> b is (2 3)

If it is not clear to you why when-bind doesn't work, try calling macroexpand-1 on both the forms above.

Wrapping up

The concepts in OL Chapter 7 translate fairly directly from Common Lisp into Clojure. The bigger differences are choices of idiom. Many of the examples in Common Lisp presume mutable state. In the typical Clojure program these forms would be in the minority.


Notes

Revision history

  • 2008/12/12: initial version

Dec 12 2008

Comments

On Lisp -> Clojure

I am porting the examples from the macro chapters of Paul Graham's On Lisp (OL) to Clojure.

My ground rules are simple:

  • I am not going to port everything, just the code samples that interest me as I re-read On Lisp.
  • Where Paul introduced macro features in a planned progression, I plan to use whatever Clojure feature come to mind. So I may jump straight into more "advanced" topics.

Please do not assume that this port is a good introduction to Lisp! I am cherry-picking examples that are interesting to me from a Clojure perspective. If you want to learn Lisp, read OL. In fact, you should probably read the relevant chapters in OL first, no matter what.

The Series

Note: Fogus is also porting On Lisp to Clojure.

Other Stuff

If you find this series helpful, you might also like:

Talks

I am available to give conference talks on Clojure. Check the schedule for an event near you, or contact Relevance (info@thinkrelevance.com) to schedule an event.


Notes

Dec 01 2008

Comments

Living Lazy, Without Variables

Programmers coming to functional languages for the first time cannot imagine life without variables. I address this head-on in the Clojure book. In Section 2.7 (free download here), I port an imperative method from the Apache Commons Lang to Clojure. First the Java version:

  // From Apache Commons Lang, http://commons.apache.org/lang/
  public static int indexOfAny(String str, char[] searchChars) {
    if (isEmpty(str) || ArrayUtils.isEmpty(searchChars)) {
      return -1;
    }
    for (int i = 0; i < str.length(); i++) {
      char ch = str.charAt(i);
      for (int j = 0; j < searchChars.length; j++) {
        if (searchChars[j] == ch) {
            return i;
        } 
      }
    }
    return -1;
  }

And now the Clojure code. I have shown the supporting function indexed as well:

  (defn indexed [s] (map vector (iterate inc 0) s))
  (defn index-of-any [s chars]
    (some (fn [[idx char]] (if (get chars char) idx)) 
          (indexed s)))

There are many things I like about the Clojure version, but I want to focus on something I didn't mention already in the book. A reader thought the Clojure version did too much work:

...the [Java] version can be seen as *more efficient* when a match is found because scanning stops right there, whereas "indexed" constructs the whole list of pairs, regardless of whether or not a match WILL be found....

The reader's assumption is reasonable, but incorrect. Clojure's sequence library functions are generally lazy. So the call to indexed is really just a promise to generate indexes if they are actually needed.

To see this, create a logging-seq that writes to stdout every time it actually yields an element:

  (defn logging-seq [s]
    (if s
      (do (println "Iterating over " (first s))
    (lazy-cons (first s) (logging-seq (rest s))))))

Now, you can add logging-seq to indexed so that each element of indexed is of the form [index, element, logged-element].

  (defn indexed [s] (map vector (iterate inc 0) s (logging-seq s)))

Test the modified indexed function at the Clojure REPL:

  user=> (indexed "foo")
  Iterating over  f
  (Iterating over  o
  [0 \f \f] Iterating over  o
  [1 \o \o] [2 \o \o])

As you can see, the indexed sequence is only produced as needed. (At the REPL it is needed to print the return value.)

Finally, you can test indexed-of-any and see that Clojure only produces enough of the sequence to get an answer. For a match on the first character, it only goes to the first character:

  (index-of-any "foo" #{\f})
  Iterating over  f
  0

If there is no match, index-of-any has to traverse the entire string:

  (index-of-any "foo" #{\z})
  Iterating over  f
  Iterating over  o
  Iterating over  o
  nil

So give up on those variables, and live lazy!

Popular Tags