Tuesday, January 26, 2010

Maven dependency heaven?

If you search Google for "maven dependency hell" you'll find many answers. It can be summarised to two types of root causes.

Transitive dependencies

The most usual one is a fact of transitive dependencies. Transitive dependencies are great but with them comes dependency resolution. Which in turns means that changing one declared dependency in your project can affect any part of your dependency tree. Most of the time this happens without the developers noticing at least until an issue comes up. This can be triggered by something as simple as changing a dependency's version.

Remote repositories

The other cause comes from a combination of inconsistent (remote) repositories and different build environments. If your different build environements, developement, continuous integration, release, talk to different repositories then you might get a different dependency tree. As a simple example, at work we have a repository where we deploy sources of libraries that don't provide them otherwise. This is great. However if someone deploys the source with a minimal POM depending on your build environment you might get the wrong POM. And this means a different dependency tree.

Continuous dependency management

I believe Continuous dependency management is the solution to this problem. Each time you build you have to know precisely what dependencies (all of them) are being used. This is why I developed the dependency tracker plugin. The existing dependency plugin offers a few goals – tree, analyze – to help with dependency management, but to me it fails short in two ways.

First it is difficult to assess what has changed from one tree to another. It is let up to the developers to do a mental diff. The dependency tracker plugin creates to this effect a text report that can and should be commited along side your project's source code. As it is a text report it is very easy to see changes between revisions using your VCS of choice or you IDE.

Second, it is a manual process. Developers have to actively use the plugin to understand and clean up their dependency tree. One idea of the dependency tracker is that it creates a text report that should commited alongside your source code. Then you can setup your project so that every build checks that its dependencies are the same as expected and fail if there is the slightest difference.

Using the plugin

The dependency tracker has three goals:

  • report
  • validate
  • checksum

The two first goals work together. I'll come back to the 'checksum' goal later. First you want to create a report. One report will be created for each module of your project in a dependencies.txt file. Its format is the following:

dependency|dependency trail|artifact checksum|pom checksum

Where dependency denotes the dependency we are talking about, the dependency trail is the information about how the dependency is reachable in the dependency tree and the checksums speak for themselves.

Then you just have to configure the plugin in your pom.xml to validate the report on each build. But don't forget to commit the report first.

<plugin>
  <groupId>com.atlassian.maven.plugins</groupId>
  <artifactId>maven-dependency-tracker-plugin</artifactId>
  <executions>
    <execution>
      <goals><goal>validate</goal></goals>
    </execution>
  </executions>
</plugin>

You're done. If any of your module's dependency changes unexpectidely, your build will fail.

The checksum is there to help calculate checksum of arbitrary files as done by the plugin in reports. Simply use that command:

mvn dependency-tracker:checksum -Dfile=/path/to/the/file

The plugin is available in Atlassian's public maven repository.

UPDATE

The source code of this plugin is now available on BitBucket at https://bitbucket.org/atlassian/maven-dependency-tracker-plugin. Feel free to fork…

7 comments:

  1. It seems to be a really interesting plugin. I'll give it a try a soon. Just FYI the link https://labs.atlassian.com/browse/MAP is protected. We cannot access to it without an account.

    ReplyDelete
  2. One Maven dependency problem I've been running into a lot lately is conflicting dependencies between test and production classes. This might happen when you have integration tests that exercise the production app using a simulated web client such as JWebUnit or HtmlUnit. These libraries have dependencies that may overlap or conflict with the libraries used by your application. Even though the tests run in a different process than your application, Maven has no way to specify different dependency versions for different scopes.

    In this case, the only solution I've found is to put your integration tests in a separate module than the artifact that they test. It is more overhead to get things set up this way, but I think it avoids a lot of pain.

    ReplyDelete
  3. @Arnaud Let me know how it goes, happy for any feedback. The project should now be accessible on Atlassian's labs.

    @Tim Yeah definitely an issue, not much for unit tests as they are supposed to run in the same jvm/process.
    The issue with integration tests is that the concept doesn't really exists in maven (apart from the phase) and therefore it always ends up being done on top of the unit test support which doesn't quite fit all the time.
    I agree with you having the integration tests in another module is easier and probably right for now…

    ReplyDelete
  4. I tried running v1.2.3 on a local project and it seems to work well for the dependencies it reports on, but it's not reporting on many of the dependencies.

    Does it ignore runtime scope or something by default? Are there any parameters that can be passed to it?

    Is the source available anywhere to browse online / checkout / submit patches to?

    ReplyDelete
    Replies
    1. What do you mean it's not reporting on many of the dependencies? It should report on all dependencies… If not that's probably a bug.

      I'm looking at making the source available at a better location, will try to keep you posted. Don't hesitate to ping me sam (at) leberrigaud (dot) org.

      Delete
    2. I've updated the post with the new BitBucket URL. You can checkout the source code here, fork and why not send a pull request.

      Delete
  5. This comment has been removed by the author.

    ReplyDelete