Comments on: Thoughts on Clojure Package Management

By: Stuart

Stuart — Tue, 01 Sep 2009 03:02:45 +0000

James wrote: “I’m hoping it’ll be unique enough to justify another re-invention of package management.”

I wish you luck, but I have my doubts that there’s anything unique left to be done here.

By: James Reeves

James Reeves — Tue, 01 Sep 2009 01:02:17 +0000

I agree that git submodules are rather useful, but I don’t think they’re a replacement for a good package manager. In my view, Clojure has reached the stage where there are enough third-party libraries that it would be useful to start thinking about package management.

I’m a little biased, though, as I’ve started working on a Clojure package manager called Capra (http://github.com/weavejester/capra). It’ll probably be a few months before the project is in a usable state, but I’m hoping it’ll be unique enough to justify another re-invention of package management.

By: Stuart

Stuart — Mon, 31 Aug 2009 21:33:14 +0000

Greg wrote: “we need to stop re-inventing package management for every single language and every single operating system distribution.”

I agree. But package/dependency management is one of those things that’s incredibly hard to get right. Witness all the attempts in the Linux world: rpm, yum, apt-get, yast, pkgtool, …. None of them is perfect. Then you’ve got one for every scripting language ever invented. Nix is pretty cool, but Unix-specific, and anything built for the JDK should work on any JDK platform.

Using Rubygems sounds … interesting. I’m not enthusiastic about Rubygem’s dependency resolution, and using it would require loading the entire JRuby runtime.

The long and short of it is, as Tim said, we’re stuck with Maven. Kudos to the first person who wraps Maven, including ALL the packaging and deployment features, in a simple Clojure interface.

By: Greg

Greg — Mon, 31 Aug 2009 19:53:39 +0000

Even if you don’t have “official production ready code” sharing it in a way that can resolve dependencies is very useful. IMHO we need to stop re-inventing package management for every single language and every single operating system distribution. We should all just pick something good like nix http://nixos.org/index.html
Speaking of re-use, I wonder if you can use rubygems given that it works with jRuby.

By: Tim Dysinger

Tim Dysinger — Mon, 31 Aug 2009 18:49:19 +0000

I should make that a little clearer “It took me an initial few days to figure out clojure + maven”. After that initial investment, new projects take only seconds to setup using “mvn archetype” and clojure-pom. They all build the exact same way and the dependency management is exact.

By: Phil

Phil — Mon, 31 Aug 2009 18:46:23 +0000

Stuart: yeah, if you can avoid using source repos directly then you’ll be in better shape, but sometimes you just don’t have an option given the state of libraries. Maybe if corkscrew or some other tool made it easier to package Clojure projects up as Maven artifacts we would see more libraries publishing them.

It’s true that you don’t get transitive dependencies with source repos; you need a full-fledged dependency system for that. But git can work with tags instead of arbitrary revisions, so you do have the option to use numbered releases if they are present.

The other problem is that lots of people see the word “maven” and run away screaming with their hair on fire. The corkscrew readme is pretty explicit about the fact that you don’t need to write any XML, but multiple people have told me that they simply stopped reading it as soon as they saw the word “maven”, so that sucks.

By: Tim Dysinger

Tim Dysinger — Mon, 31 Aug 2009 18:45:14 +0000

We use clojure-pom and maven. It took a couple hours to set up and it works. Introducing a new non-maven package manager into clojure is an uphill battle IMO.

By: Stuart

Stuart — Mon, 31 Aug 2009 16:50:37 +0000

Phil wrote: “As long as we can get people to quit including jar files in the lib/ directory of their repositories I’m happy. =)”

But that’s just it: everyone does this. Even big, BIG Java projects like Hadoop have JAR files in their version control systems. And Hadoop and Maven are both Apache projects! Automated dependency management for Java libraries has failed.

By: Stuart

Stuart — Mon, 31 Aug 2009 16:45:19 +0000

Sorry, Phil, I actually meant nothing ill by “shell-based.” I think Corkscrew is quite clever at what it does. My real issue is that it pulls directly from source code repositories. There’s no continuity if a project disappears from Github. (Rubygems has the same problem.) Also, there are no numbered releases or multi-level dependencies. As I wrote, that’s more a fault of the libraries themselves at the moment.

By: Phil

Phil — Mon, 31 Aug 2009 16:39:09 +0000

Though I think you’re right in general. I wrote Corkscrew because it was easy (it’s about 200 LOC), but sinking a lot of effort into a Clojure-specific solution probably wouldn’t offer too much benefit over the current state of the art.

As long as we can get people to quit including jar files in the lib/ directory of their repositories I’m happy. =)