The Agile practice of Continuous Integration is gaining more and more steam these days. It's quite evident why, people are trying to do more with less, and are looking for any opportunity to automate within their organizations. Software development teams are doing this by automating their build processes. Once the build has been automated, then Continuous Integration comes naturally because you can run the build more frequently, increase the amount of feedback your developers are getting, and introduce more items into the build like automated unit tests, automated code coverage utilities, and increase the value of the information that is produced by the automated build.

It's natural at this point when the build is doing so much for you to look at other ways that your Continuous Integration environment can be extended to interact with other areas of your software development lifecycle. In fact if you take a look over at the open source Hudson project, you'll see that the plugins that are available for Hudson tend to cluster into certain areas. Open Source is a great way to look at where the needs are within a community as it encourages people who have an itch to write some code to scratch their itch. If others also have the same issue then you'll tend to see certain clusters of activity around those areas of interest.

Looking at the list of plugins from Hudson, one of the large areas that you'll see people have clustered around is integration with Bug Tracking systems. They have developed plugins for Bugzilla, JIRA, Mantis, Trac, and several others specifically for the purpose of integrating Build information with Bug Tracking systems.

We're all for this level of integration, and for the awareness in general that Builds and Bugs are two systems that can and should work together. Software developers rely heavily on both of these systems, and our philosophy is that Builds and Bugs should be integrated from the ground up. Here are our top 5 reasons to integrate your Build and Bugs systems, and why we think you'll gain in efficiency, save time, and be more productive as you adopt these approaches.

  Bug records should display build tag  

When you are browsing through bugs, and you see a bug that is marked as "fixed" it should be easy and obvious which build this bug was fixed in. Many bug tracking systems have fields for this, but unless you integrate your build and bug tracking systems, this is left as an exercise to the developer to fill in. Which means they'll leave it blank, or they'll put something invalid in the field if it's difficult to track down which build the bug will be fixed in.

This activity should be handled automatically by the build system by scanning through all bugs in the system that match the build and when the build is complete, all matching bugs should be updated. In Zed Builds and Bugs, when a developer completes an assignment, the "Fixed In Build" field automatically defaults to "next". This keyword is used by the system so that it knows which bugs to update when the next build runs. It eliminates the usual errors associated with collecting this data.

  Completing a bug should be able to launch a build  

The very act of completing a bug, or an assignment on a task should be a point to which you can attach meaning. The idea of continuous integration is that the build system should be triggered by meaningful events to check that the code base is still in a working state. Your team may want this to be configured to run when developers commit their code. But you may also want to configure it to run when developers change the status of their bugs. For example, say there are 3 different changes to 3 different source code systems that comprise the real fix for a single bug. In this case it doesn't make sense to launch integration builds until everything has been finalized in each of the code systems. It makes a lot more sense to trigger the integration build, tests, and verifications only after the bug has been marked as fixed.

You can only do this reliably if your build system and your bug management system are tightly integrated and share events and triggers between them. Zed Builds and Bugs has a powerful event based rules system that lets you tie different parts of the system together in ways that make sense for your team and your workflow.

  Builds and Bugs should be linked and visible from the same system  

If you find yourself looking for all of the bugs fixed in a certain build, or for the build in which a certain bug is fixed, you either have to have powerful search capabilities, or a system that naturally links build and bug information together. It allows you to start from either side and easily drill down into the details of the other so that you can get a complete picture of what is happening in your system.

  Builds information should flow back to the bug tracking system  

Your build system is doing a great job at manipulating your code in many ways. It checks it out from source control, it arranges it properly in modules, it compiles it, tests it, and even performs static and runtime analysis of the code. There's a wealth of information that is output from the build system that by all means should be captured, displayed, and analyzed in its own right.

There are going to be parts of this information that should naturally be used to create bugs in your task system so that they can be further analyzed and followed-up on. In order to do this most efficiently, it should be up to the build system to communicate this information back to your bug tracking system.

Not that every failed build should open a bug record. Hopefully a failed build and the subsequent notification that everyone on your team receives is motivation enough to find and fix the underlying problem.

No, it is for the more interesting facts that the build system discovers that you'll want tracking records created. For example, if a new critical bug is discovered by findbugs, why not have your build system auto-create a real bug in your bug tracking system so that the real issue will be resolved. The developers don't have to manually scan the system to discover things that they should have been working on 3 days ago, the system should notify them now by opening a new bug record.

  Having Builds and Bugs in the same system reduces overhead  

Let's face it, we're always looking for ways to be more efficient, reduce overhead, save time, money, and do more with less. Having a single system that can accurately track and manage your bugs and tasks, while at the same time providing continuous integration builds for your software enhances the efficiency and quality of your day to day work. Your team has a single point to rally around, all of the information is in the same place, and everyone knows where to look to find the answers to what is going on.

Simply put, you save time and money by optimizing how you work with a single system. You get more done, and you do it more easily.

There are lots of other reasons to integrate your builds and bugs, but these are some of our top 5 reasons to do so. We'd love to hear from you about your own experiences with builds, bugs, and the integration of the two.

Zed Builds and Bugs Manager is proud to incorporate our Wiki engine from JSPWiki, an open source Wiki that is built on Java Server Pages technology. As such, it fits right in with the rest of Zed Builds and Bugs, as it is written in Java and is hosted perfectly by our Jetty embedded HTTP Server.

JSPWiki brings a lot to the table that we think has been overlooked by some of the reviews of available Open Source Wiki's. It is fully UTF-8 enabled right out of the box, has support for a wide variety of markup features, and one of the best features (in our opinion) is the easy way that you can create custom plugins for JSPWiki.

Creating Custom Plugins

JSPWiki makes creating custom plugins a breeze by providing the com.ecyrd.jspwiki.plugin.WikiPlugin interface. There's only one method to implement in the interface and that's the execute(WikiContext, Map) method.

The Map parameter gives you a name/value map of each of the arguments that were supplied to the plugin invocation so that you can tailor the output of your plugin to user customization. There are some handy access methods that you'll want to create to help you to pick out the parameters with appropriate data type conversions. For example:

boolean getBooleanParam(Map params, String name, boolean default_value){
  String value = (String)params.get(name);
  if(value == null || value.length() == 0){
    return default_value;
  }
  if(value.compareToIgnoreCase("true") == 0){
    return true;
  } else {
    return false;
}

That makes it easy to deal with pulling boolean values out of the parameter Map, and supplying a default value so that the user doesn't have to specify the parameter if they just want to go with the default values.

The WikiContext parameter also has some nice little gems that are of great use. One of them that we take significant advantage of is the access to the RenderingManager that is provided by the WikiContext. Simply do this:

context.getEngine().getRenderingManager();

to get the RenderingManager object. Why? Well, specifically in our case we allow users to display task and bug information inside their wiki pages. Since we don't want the user's entering or editing the information in two separate places, we've provided a JSPWiki plugin that does this. By having access to the RenderingManager we can allow the users to write their task description using Wiki Markup and when displayed in the wiki it will be appropriately formatted.

We give the user's 2 options for displaying task descriptions in the wiki. They can have the task contents formatted as Wiki Markup, or as a code block. Here's the snippet that does this and shows how easy it is to utilize the RenderingManager to handle the Wiki Markup processing:

if(description_included){ // has the user asked for a task description to be included?
  if(style.compareToIgnoreCase("wiki") == 0){ // user wants to use wiki markup for their task display
    sb.append( context.getEngine().getRenderingManager().getHTML( context, t.description );
  } else if(style.compareToIgnoreCase("code") == 0){ // user wants the task displayed as code
    sb.append("<pre>\n").append( t.description ).append( "</pre>\n");
  } else {
    ...
  }
}

Embedding JSPWiki

Embedding JSPWiki was a little tricky to get right, but worked out very well for our setup. We provide a minimum of 2 wiki's for each installation. The first is the Zed documentation wiki (a version of which you can see live on our web site) which is always a single instance. The second and beyond correspond to the setup of Zed itself. Zed can support multiple teams, with each team having its own build/task/discussion database, and separate wiki. Depending on how many separate teams the administrator wants to support via a single Zed server, they'll also get the same number of wiki instances.

Each JSPWiki instance points to the same set of libraries, so that functionality is always consistent, but each gets its own configuration file that is copied dynamically from a standard template that we ship. We do simple replacement on the standard template before bringing it live, and we supply a standard set of content to get each team started with their own Wiki.

Going Beyond JSPWiki

Since the Zed server is the core application, which launches the Jetty HTTP listener, and the JSPWiki web application contexts it is also possible to do centralized user management, password authentication, and to tie JSPWiki back into the containing Zed application in the user's web browser.

The Zed server handles users, groups, authentication, etc. To tie JSPWiki into this central administration, it was enough to extend the com.ecyrd.jspwiki.auth.login.AbstractLoginModule class as such:

public boolean login() throws LoginException {
  // The user has already been authenticated on the thread currently serving this HTTP request
  // Look up their information, and set the JSPWiki principal information properly
  long tid = Thread.currentThread().getId();
  String session = ThreadSessions.getInstance().getThreadSession(tid);
  if(session == null){
    return false;
  }
  SessionInfo si = SessionList.getInstance().getSession( session );

  Principal principal = new WikiPrincipal( si.fullname, WikiPrincipal.FULL_NAME);

  m_principals.add( new PrincipalWrapper( principal ) );

  // Now indicate that this new principal is authenticated
  m_principals.add( Role.AUTHENTICATED );
  m_principals.add( Role.ALL );
 
  // If login succeeds, commit these roles
  m_principalsToOverwrite.add( WikiPrincipal.GUEST );
  m_principalsToOverwrite.add( Role.ANONYMOUS );
  m_principalsToOverwrite.add( Role.ASSERTED );

  // If login fails, remove these roles
  m_principalsToRemove.add( Role.AUTHENTICATED );

  return true;
}

With Zed, you also don't have to worry about installing Tomcat, Jetty, or another application server yourself and handling the configuration required. It's all centralized and contained within a single installation process. Once Zed is up and running, JSPWiki is available right inside the box.

We're very happy to include JSPWiki in the Zed installation, as it is a key part of our overall offering and provides a professional, enterprise wiki that can be used by your entire software team. It is very well put together, and very functional. We'll be contributing as much back to the JSPWiki community as we can to help it to stay at the top of its game.

Usually when you build an application that is going to take advantage of an HTTP server like Jetty or Tomcat or any other HTTP server, you start with that application server as the foundation and build up your own application from there. Zed turns this paradigm on its head and uses Jetty the other way around.

Jetty is really a very flexible and adaptable server, and the folks over at Mortbay and WebTide who maintain Jetty have done a great job providing a standardized HTTP server that is excellently embeddable. We started with the details on Embedding Jetty and expanded on them as we built Zed. This post captures some of the things that we did as we followed this path.

As the Embedding Jetty guide talks about, it's really as easy as simply saying:

Server server = new Server(8080);
server.setHandler(handler);
server.start();

This will start your server listener on port 8080 with the appropriate handler. Allowing for configuration and customization becomes a little bit more involved. We needed two key components in addition to the standard setup: support for optional secure sockets, support for multiple web application contexts.

To implement secure socket listeners with Jetty, simply use their SocketConnector classes like this:

Server server = new Server(); // no port configuration by default
Connector connector;
if(m_use_ssl){
  SslSocketConnector ssc = new SslSocketConnector();
  ssc.setKeystore( keyStorePath );
  ssc.setPassword( keyStorePassword );
  ssc.setKeyPassword( keyPassword );
  connector = ssc;
} else {
  connector = new SocketConnector();
}

connector.setHost( local_ip_addr); // if required
connector.setPort( local_port ); // specify the port to listen on
server.setConnectors( new Connector[]{connector} );

Our configuration file allows the user to turn on/off secure sockets, and also provide the keystore, and passwords, so there are a few more details but this outlines a pretty easy example to follow.

Next we needed support for multiple web application contexts. The Zed Builds And Bugs server allows the administrator to set up a single server that will support multiple development teams, using multiple underlying databases to store each teams' saved data. This includes not only the continuous integration build details and bug data, but also has to include the Wiki documents for each team so that they don't collide.

We use the Jetty WebAppContext class to automatically create as many different Wiki applications as required to support the different team databases defined in the Zed configuration file. To start with, there is always the Zed documentation Wiki, which is set up like this:

WebAppContext wac = new WebAppContext();
wac.setContextPath("/doc_wiki");
wac.setWar("3rdParty/JSPWiki/"); // path to .war file or expanded existing webapp.
TreeMap main_wiki = new TreeMap();
main_wiki.put("jspwiki.propertyfile", "wikiconfig/doc_wiki.properties");
wac.setInitParams(main_wiki);

We then add the wac to the list of handlers for our server context and it is ready to go. We then repeat this process for every Team Wiki that needs to be configured and started based on our database configuration.

Once all of the handlers have been defined, here's how to add them to the server:

HandlerCollection handlers = new HandlerCollection();
handlers.setHandlers( handler_vector.toArray( new Handler[0] ));
server.setHandler( handlers );

And finally start the Jetty server with:

server.start();

Rationale

The reason for doing this with Zed is quite far-reaching. This allows us to create an HTTP listener that is just one among many listeners that the application supports. This means that Zed is capable of receiving not only HTTP request, but JMS messages, and any other type of request in order to invoke activities on the server. The central message processing routines don't care (and shouldn't care) where the original requests come from, they're just responsible for acting on and providing the core capabilities. Once a request has been processed, it is handed back to the proper listener to forward the response back to the caller in the manner that is appropriate for the listener's protocol.

This also allows the core Zed code to be listener independent. If we decide some day to swap Jetty for some other HTTP engine (which I doubt), we won't have to re-architect the entire application, just the listener that exposes Zed to the outside world.

Until that point, we're happy to be part of the Jetty family of users and supporters and wish the team over at WebTide continued success with Jetty.