Lifecycle Composition

In the first version, start and stop were side-effecting
procedures which returned a future or promise:

(defprotocol Lifecycle
  (start [this]
    "Begins operation of this component. Asynchronous, returns a
  promise on which the caller can block to wait until the component is
  started.")
  (stop [this]
    "Ceases operation of this component. Asynchronous, returns a
  promise on which the caller can block to wait until the component is
  stopped."))

The calling code could dereference the future to block until the
service had successfully started. For example, a database-access
component might look like this:

(defrecord Database [uri connection-atom]
  Lifecycle
  (start [_]
    (future (reset! connection-atom (connect uri))))
  (stop [_]
    (.close @connection-atom)
    (future (reset! connection-atom nil))))

(defn database [uri]
  (->Database uri (atom nil)))

My idea was that multiple services could be started in parallel if
they didn’t depend on one another, but in practice I always ended up
blocking on every call to start:

(defrecord System [database scheduler web]
  Lifecycle
  (start [_]
    (future
      @(start database)
      @(start scheduler)
      @(start web)))
  (stop [_]
    (future
      @(stop web)
      @(stop scheduler)
      @(stop database))))

(defn system [database-uri]
  (let [database (database database-uri)
        scheduler (scheduler)
        web (web-server database)]
    (->System database scheduler web)))

I decided to drop the requirement to return a promise from start/stop,
which meant that those functions became synchronous and had no return
value:

(defprotocol Lifecycle
  (start [this]
    "Begins operation of this component. Synchronous, does not return
  until the component is started.")
  (stop [this]
    "Ceases operation of this component. Synchronous, does not return
  until the component is stopped."))

This simplified the code calling start/stop, because I didn’t have to
worry about dereferencing any futures.

(defrecord System [database scheduler web]
  Lifecycle
  (start [_]
    (start database)
    (start scheduler)
    (start web))
  (stop [_]
    (stop web)
    (stop scheduler)
    (stop database)))

This also made it very clear that I was using start/stop only for
side-effects, forcing all of my components to contain mutable state.

Also, I had to manually place the calls to start/stop in the correct
order, to ensure that components were not started before other
components which depended on them.

I decided to try to make the component objects more like immutable
values by redefining start/stop to return updated versions of the
components:

(defprotocol Lifecycle
  (start [this]
    "Begins operation of this component. Synchronous, does not return
  until the component is started. Returns an updated version of this
  component.")
  (stop [this]
    "Ceases operation of this component. Synchronous, does not return
  until the component is stopped. Returns an updated version of this
  component."))

In this version, start and stop feel more like functions. They are
still not pure functions, because they will have to execute
side-effects such as connecting to a database or opening a web server
port, but at least they return something meaningful.

This version has the added benefit of removing some mutable state, at
the cost of making the start/stop implementations slightly more
complicated. Now these functions have to return a new instance of the
component record:

(defrecord Database [uri connection]
  Lifecycle
  (start [this]
    (assoc this :connection (connect uri)))
  (stop [this]
    (.close connection)
    (assoc this :connection nil)))

(defn database [uri]
  (->Database uri nil))

One interesting feature of this pattern is that the system record can
reduce over its own keys to start/stop all the components:

(defrecord System [database scheduler web]
  Lifecycle
  (start [this]
    (reduce (fn [system key]
              (update-in system [key] start))
            this
            ;; Keys are returned in the order they were declared.
            (keys this)))
  (stop [this]
    (reduce (fn [system key]
              (update-in system [key] stop))
            this
            ;; Reverse the order to stop.
            (reverse (keys this)))))

However, this relies on implementation behavior of Clojure records:
the keys function will return the keys in the order they are
declared in the record. I’m reluctant to rely on that undocumented
behavior, so instead I’ll declare the ordering explicitly:

(def component-order
  [:database :scheduler :web])

(defrecord System [database scheduler web]
  Lifecycle
  (start [this]
    (reduce (fn [system key]
              (update-in system [key] start))
            this
            component-order))
  (stop [this]
    (reduce (fn [system key]
              (update-in system [key] stop))
            this
            ;; Reverse the order to stop.
            (reverse component-order))))

I still don’t have good solution to specifying the order in which
components must be started/stopped. I’ve tried building a graph of
dependencies and computing the correct order. This would be similar to
what tools.namespace does with namespaces. But I haven’t been able to
come up with a good syntax for representing these relationships that
isn’t more cumbersome than just declaring them in order.

I’ve also tried using Prismatic’s Graph library or my own Flow library
to define the graph of dependency relationships, but neither of those
libraries can produce a structure that remembers the graph after it
has computed its output, so I have no way to recover the dependency
relationships after constructing the system object.

This technique is a form of dependency injection through constructors.
The choice of whether to make the individual components mutable,
stateful objects has an impact on how I can use them later on. In the
original version of this pattern using mutable objects, each component
gets stable references to other components it depends on. In the later
version using immutable data structures, each component gets
references to the constructed versions of other components it
depends on, but not the started versions of those components,
i.e. the values returned from start.

So far, this has not been a problem in the programs I write. For
example, a Datomic database connection is always recoverable from the
URI, so I don’t need to store it explicitly. But other components,
particularly components which rely on external state to function
properly, might need to be mutable so that their dependents can still
use them via the references the received in their constructors. I
could still have start and stop return new values, but they would
also have to modify some mutable state (such as a Ref or Atom) along
the way. As always, mutable objects muddy the distinction between
values and identities.

I’ve also experimented with variations of start and stop that
pass in other “started” components, but this was cumbersome and hard
to generalize through a single interface.

So I don’t have a perfect system. It works well enough for the
applications I’ve developed with it so far, and it facilitates my
REPL-driven development workflow, but I always have to adapt to
circumstance. Eliminating mutable state is generally a good thing,
but it can also be limiting, especially when you have to deal with
external state.