Dependency Management First-Aid Kit

This article attempts to unravel some of the mysteries of dependency management with Maven and Maven-based tools.

Help, something’s missing!

Say you have a project named “my-new-project” which declares a dependency on version 3 of the “awesome-sauce” library by the Example.com corporation. You add the dependency to your pom.xml, project.clj, or whatever configuration file your build tool uses. You take a deep breath and start a build. And it fails!

If you’re using Maven 2, you see something like this:

[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Missing:
----------
1) com.example:awesome-sauce:jar:3.0.0

  Try downloading the file manually from the project website.

  Then, install it using the command: 
      mvn install:install-file -DgroupId=com.example -DartifactId=awesome-sauce -Dversion=3.0.0 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there: 
      mvn deploy:deploy-file -DgroupId=com.example -DartifactId=awesome-sauce -Dversion=3.0.0 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency: 
        1) my.group:my-new-project:jar:1.0.0-SNAPSHOT
        2) com.example:awesome-sauce:jar:3.0.0

----------
1 required artifact is missing.

for artifact: 
  my.group:my-new-project:jar:1.0.0-SNAPSHOT

from the specified remote repositories:
  central (http://repo1.maven.org/maven2),
  clojars (http://clojars.org/repo/)

Leiningen, which uses Maven 2 under the covers, produces similar output, but it mistakenly prints the current project name as org.apache.maven:super-pom:jar:2.0.

Maven 3 prints a less verbose (and less informative) error message, but the gist is the same.

What happened?

What is all this verbosity saying? Well, obviously, the build failed because something was missing. What was missing? Maven tells you:

Missing:
----------
1) com.example:awesome-sauce:jar:3.0.0

The JAR file for the project “awesome-sauce”, version 3.0.0, published in the “com.example” group, is missing. That just means Maven didn’t find it in any of the places it looked.

Where did it look? Maven tells you that too:

from the specified remote repositories:
  central (http://repo1.maven.org/maven2),
  clojars (http://clojars.org/repo/)

These are the public repositories where Maven searched for the file. Each repository has an ID (“central” and “clojars” in this case) and a URL. Both are specified in the configuration of:

1. Your project, in pom.xml or project.clj

2. Your build tool’s global configuration file

   • settings.xml for Maven
   • N/A for Leiningen

3. Your build tool’s built-in defaults

   • The Maven Central Repository for almost any Maven-based tool
   • Clojars for Leiningen

If you visit http://repo1.maven.org/maven2/com/example/awesome-sauce or http://clojars.org/repo/com/example/awesome-sauce in a browser you will see that those directories do not, in fact, exist.

Although it’s not listed, the first place Maven checks for a dependency is your local Maven repository. The local repository is just a big cache of everything Maven has downloaded in the past. It’s typically located at $HOME/.m2/repository.

What to do next

You have two options at this point:

1. Find a public repository containing “awesome-sauce”
2. Install “awesome-sauce” in your local repository

The first option is generally less work, and more repeatable if you ever build your project on another machine.

Finding a repository

Odds are, if the library you are looking for is free, open-source, and popular, it will already be in a public Maven repository somewhere. Start with the source: who released the library? Large organizations with a lot of open-source projects often host their own repositories, like Google and Codehaus. Failing that, search engines such as Mvnbrowser may help you find it.

Once you’ve found a repository, you need to add it to your build. For example, to add the Codehaus repository to a Maven project, add these lines to pom.xml inside the <project> tag:

<repositories>
  <repository>
    <id>codehaus</id>
    <name>Codehaus</name>
    <url>http://repository.codehaus.org/</url>
  </repository>
</repositories>

(You can pick your own <id> and <name>.)

For Leiningen, add the following lines inside the (defproject ...) block:

  :repositories {"codehaus" "http://repository.codehaus.org/"}

Installing locally

If the library you want is not available in any public repository, you’re not stuck, you just have to do a bit more work. You need to get the JAR file for the library, either by downloading it manually or building from source. Then you need to install that JAR file in your local Maven repository. That’s easy, because Maven has already told you exactly how to do it:

  Then, install it using the command: 
      mvn install:install-file -DgroupId=com.example -DartifactId=awesome-sauce \
 -Dversion=3.0.0 -Dpackaging=jar -Dfile=/path/to/file

Copy that command verbatim, changing only /path/to/file to the path to the library’s JAR file. Maven will copy the file to $HOME/.m2/repository/com/example/awesome-sauce/3.0.0/awesome-sauce-3.0.0.jar. The next time you build your project, Maven knows exactly where to find it.

Installing remotely

If you want others to be able to build your project without having to go through these manual steps, you need your own public Maven repository to which you can upload files. Hosting a Maven repository isn’t hard: all you need is a web server.

If you work with a team, consider setting up a shared repository for everyone to use. A repository manager such as Nexus can help you take care of user accounts and authentication.

If you publish open-source libraries, I strongly encourage you to get an account on Sonatype OSS, a free service provided by the makers of Nexus. Releasing your projects to Sonatype OSS gives them a path to get added to the Maven Central Repository. While the requirements for projects in Maven Central are more stringent than just tossing code into your own repository, it’s worth the effort. In Maven Central, your project will have greater visibility and will be easier for anyone in the world to use.

But what if I don’t want that dependency?

Maven dependencies are transitive: if your project depends on project X, which depends on projects Y and Z, then your build will try to download X, Y, and Z.

But sometimes projects declare dependencies that aren’t strictly necessary. Or they declare dependencies on something you want, but the wrong version. How can you avoid including those extra dependencies in your build?

Maven supports dependency exclusions for these cases. For example, suppose the “awesome-sauce” library declares a dependency on “com.example:stupidity:0.0.1”. You know that you don’t need “stupidity” in your project, so you want to prevent the build from including it. In pom.xml, you write:

<dependencies>
  <dependency>
    <groupId>com.example</groupId>
    <artifactId>awesome-sauce</artifactId>
    <version>3.0.0</version>
    <exclusions>
      <exclusion>
        <groupId>com.example</groupId>
        <artifactId>stupidity</artifactId>
      </exclusion>
    </exclusions> 
  </dependency>
</dependencies>

Or in Leiningen’s project.clj, you write:

  :dependencies [[com.example/awesome-sauce "3.0.0"
                  :exclusions [com.example/stupidity]]]

Note that once you start using exclusions, you’re on your own. It’s up to you to make sure you still have the correct versions of all the libraries your project needs.

On rare occasions, a project’s dependencies cannot be resolved at all. In particular, if you need two different versions of the same library with the same class names but incompatible APIs, you’re pretty much stuck. Time to refactor, or investigate multiple-Classloader schemes like OSGi. But that’s a whole ‘nother story.