When (Not) to Write a Macro

The Solution in Search of a Problem

A few months ago I wrote an article called Syntactic Pipelines, about a style of programming (in Clojure) in which each function takes and returns a map with similar structure:

(defn subprocess-one [data]
  (let [{:keys [alpha beta]} data]
    (-> data
        (assoc :epsilon (compute-epsilon alpha))
        (update-in [:gamma] merge (compute-gamma beta)))))

;; ...

(defn large-process [input]
  (-> input

In that article, I defined a pair of macros that allow the preceding example to be written like this:

(defpipe subprocess-one [alpha beta]
  (return (:set :epsilon (compute-epsilon alpha))
          (:update :gamma merge (compute-gamma beta))))

(defpipeline large-process

I wanted to demonstrate the possibilities of using macros to build abstractions out of common syntactic patterns. My example, however, was poorly chosen.

The Problem with the Solution

Every choice we make while programming has an associated cost. In the case of macros, that cost is usually borne by the person reading or maintaining the code.

In the case of defpipe, the poor sap stuck maintaining my code (maybe my future self!) has to know that it defines a function that takes a single map argument, despite the fact that it looks like a function that takes multiple arguments. That’s readily apparent if you read the docstring, but the docstring still has to be read and understood before the code makes sense.

The return macro is even worse. First of all, the fact that return is only usable within defpipe hints at some hidden coupling between the two, which is exactly what it is. Secondly, the word return is commonly understood to mean an immediate exit from a function. Clojure does not support non-tail function returns, and my macro does not add them, so the name return is confusing.

Using return correctly requires that the user first understand the defpipe macro, then understand the “mini language” I have created in the body of return, and also know that return only works in tail position inside of defpipe.

Is it Worth It?

Confusion, lack of clarity, and time spent reading docs: Those are the costs. The benefits are comparatively meager. Using the macros, my example is shorter by a couple of lines, one let, and some destructuring.

In short, the costs outweigh the benefits. Code using the defpipe macro is actually worse than code without the macro because it requires more effort to read. That’s not to say that the pipeline pattern I’ve described isn’t useful: It is. But my macros haven’t improved on that pattern enough to be worth their cost.

That’s the crux of the argument about macros. Whenever you think about writing one, ask yourself, “Is it worth it?” Is the benefit provided by the macro – in brevity, clarity, or power – worth the cost, in time, for you or someone else to understand it later? If the answer is anything but a resounding “yes” then you probably shouldn’t be writing a macro.

Of course, the same question can (and should) be asked of any code we write. Macros are a special case because they are so powerful that the cost of maintaining them is higher than that of “normal” code. Functions and values have semantics that are specified by the language and universally understood; macros can define their own languages. Buyer beware.

I still got some value out of the original post as an intellectual exercise, but it’s not something I’m going to put to use in my production code.

2 Replies to “When (Not) to Write a Macro”

  1. Hey Stuart,

    I think you have a fair point – if the defpipe macro is used in one or two places. But if used pervasively throughout your system it’s completely ok. Any large system requires learning the semantics of its building blocks (if you’re lucky enough to work in a system with discernable building blocks. ugh).

    I’m not suggesting the defpipe macro as originally posted is ready for primetime. I’m saying it could still be a nice area to explore. For example, I did a little building on your defpipe macro. I adopted clojure’s require syntax to the pipe parameter declaration so you could write e.g.

    (defpipe [[request body headers uri]
    response :as resp
    (when (and ((complement nil?) resp)
    (authorized? headers))
    [resp (do-the-thing (query db-connection uri))
    flags [:assoc :secrets-included])))

    ;; a request context
    {:request {:body "blah"
    :headers {:http-basic-auth "authorized"
    :totally-a-real-header true}
    :uri "/secrets"}
    :db-connection #sqlite://localhost}

    I was specifically thinking about web development here – but it does not have to be. What’s neat is you can capture which pipes consume and produce which data – allowing for some interesting things.

    Not that I’m completely happy with this, but I think it has potential.

    tl;dr: there’s always more to explore

  2. My return value

    [resp (do-the-thing (query db-connection uri))
    flags [:assoc :secrets-included]]

    should probably be

    [resp (do-the-thing (query db-connection uri))
    flags #(assoc % :secrets-included]]

    i.e., if you return a function for a key it will be applied to the value of that key.

    Also, the parameter declaration can be extended to be better at reaching further nested maps.

Comments are closed.