Techblog: Tools and Processes

Build Automation and Continuous Integration using Maven and Continuum

27 Dec 2006 (English)

This article describes some of the benefits of build automation and continuous integration and provides basic information on Apache Maven 2.

To build software means all steps that are necessary to transform source code into executable units for your target platform, including compiling, testing, creating documentation, packaging and so on. Automating the build process has many advantages on productivity and quality: Developers need to perform fewer manual steps and the build output is reproducible, no matter who performed the build. Errors can be avoided. The fact that builds can be performed at the push of a button makes it easier to test applications early and often on their target environments. That way, quality and productivity are increased as well. Barriers that prevent new developers from quickly becoming productive are teared down.

And then, automated builds make the idea of Continous Integration (CI) possible. Your CI server can check out the newest source changes from the source code repository and perform automated builds regularly (from nightly up to immediately, i.e. after each check-in). In case of any errors, the CI server informs the developers via email. As such feedback is delivered very soon after checking in an incompatible change, it is easy to find and fix the bug. And, your organization always has a current build for testing on the target environment available.

Basic concepts of Apache Maven 2

Many projects nowadays use build scripts based on Apache Ant for automating their builds. But the build subtasks that have to be solved are quite similar in many projects which is why also the build scripts often look similar. This is like reinventing the wheel again and again, so to speak. Furthermore, in complex projects, managing dependencies often is difficult - both managing inter-project-dependencies and managing dependencies to third-party libraries.

Maven tries to solve these difficulties using the following concepts:

Standardization instead of configuration:

Maven suggests a basic structure for projects (e.g. directories for sources, test files, resource files etc.). As long as a project adheres to this structure, very little configuration is needed for build automation. Basically one has to specify what kind of artifact should be produced (the so-called packaging, e.g. JAR, EAR, WAR, ...). Maven contains ready-to-use build processes for the typical artifacts that include all typical subtasks (e.g. generation, compilation, tests, packaging, ...).

Flexibility through POM and Plugins:

The main definition file for the build process of a project is the so-called "Project Object Model", in short POM or pom.xml. In this file, you can optionally customize every single subtask of your build, e.g. regarding custom directory structures or custom subtasks. Maven plugins can be called within your build, e.g. for generating a project website. The "antrun" plugins even allows for the execution of ant scripts within a build process.

The variety of ready-to-use Maven plugins is one of Maven's strengths. The most essential ones are provided by the Maven project itself. Other projects provide a bunch of additional plugins for different purposes. For example, it is very easy to integrate a test coverage analysis tests into your builds, using the "Cobertura" plugin.

Dependency management and repository:

In Maven-based projects you would not store the required JAR library files directly within your project. Instead you just declare all dependencies within your POM by specifying the artifact's name (e.g. log4j) and its version number (e.g. 1.2.8). All artifacts are saved in a central repository. Maven loads the required files into a local repository during the build execution (but only if this hasn't been done before). Afterwards the build can be performed locally.

Maven's dependency mechanism is very powerful due to its support for transitive dependencies. Example: Struts is dependent on commons-beanutils and commons-digester. Within your Struts-based web application, it is sufficient to declare a dependency to Struts. Maven loads commons-beanutils and commons-digester automatically.

Component overview

The following components are important for a Maven-based build framework within a company:

Local Maven installation:

Every developer needs to have a local Maven installation on his workstation. The local Maven installation is a command line tool, comparable to Ant. You need it for performing a build locally. Due to the fact that required JAR files are not stored within the project itself but must be loaded from the repository by Maven, even for editing the source code within an IDE like Eclipse, Maven is needed.

Local repository:

Maven stores all artifacts that have been downloaded from the repository (e.g. log4j.jar, ...) in a local directory structure, the so-called "local repository". That way, they can be used locally, e.g. as class-path entries for the Java compiler. Maven even stores its own plugins within the local repository.

Public repository on repo1.maven.org:

By default Maven tries to download all build-relevant artifacts from a public Internet repository on repo1.maven.org. While this is very convenient for many open-source projects, the direct access to this repository by each developer is not acceptable for many corporate organizations. (Some recommendations for corporate users follow later-on.)

Custom repositories:

You can also set up your own repositories which is very much recommended for corporate users (details follow later). A repository is just a directory structure, accessible through the file system or (recommended) through http or other protocols such as scp.

Continuous Integration (CI) Server:

Continuum is a Continuous Integration Server developed within Apache group's Maven project. It is a web application that can run builds regularly and automatically, send mails regarding build errors and so on. Use Continuum's web interface to conveniently define the auto build processes and always inquire the current build status. Continuum itself uses a local Maven installation including a local repository.

Further reading

In newer blog entries, you can find recommendations for corporate users.