Sun's Javadocs provide us with automatically generated, hyperlinked documentation of fields, methods, classes, interfaces, and inner classes. They are helpful in many ways, but you will soon develop a wish list that includes the ability to search for programming elements, the ability to access multiple APIs from one location, the ability to cross-reference relationships across APIs. The list goes on.
This article describes
ashkelon, an open source documentation system for Java that leverages Sun's Javadoc parser. When developing a UI for, say, 50 APIs, 500 packages, 10,000 classes, and 100,000 methods, how do you present mounds of information in a readily accessible and intelligible way? Patrick Chan's solution was to produce the Java Almanac, containing indices and cross-references to programming elements. But there are drawbacks to paper forms of referencing; for one thing, the Java Almanac covers only J2SE. And it's long and heavy.
The handful of techniques used repeatedly in
ashkelon's UI to address this problem are:
Hide information on the page using CSS
visibilityproperties, and toggle those properties dynamically (using various DHTML client-side "widgets" such as tabs, collapsible trees, and filtered tables).
Color code, for more natural filtering of information on the page.
Provide many ways to access the same information: via an index, browsing or searching, temporal navigation back to an element recently previewed, via a cross-reference, or another type of programmatic relationship such as a throws relationship.
A Database of Docs
One component of
ashkelon plugs in to Javadoc as a doclet to populate a relational database with Javadocs. The corresponding database schema for the Java language consists of tables with names such as
thrownexception. The schema can be updated and grow to include changes such as those introduced with J2SE 5.0 and aspects.
So what can you do with a database schema filled with Javadocs? For one thing, with
ashkelon you can populate multiple APIs into a single repository. You can do this incrementally, too. That is, you don't have to add all APIs in one run to "get your links." Let's look at how we might use
ashkelon to populate an API such as Hibernate into the database. Invoke the following command from the command line, in much the same way as you would use the Javadoc tool to produce HTML documentation:
~$ ashkelon add @hibernate.xml
To add another API (say
dom4j), you do this:
~$ ashkelon add @dom4j.xml
Now we can ask
ashkelon to list the APIs in its database:
~$ ashkelon list
which, with my local copy of
ashkelon running on my notebook computer today, produces:
Ashkelon: API: Ashkelon Ashkelon: API: Hibernate Ashkelon: API: J2SE Ashkelon: API: JTidy Ashkelon: API: JUnit Ashkelon: API: JiBX Ashkelon: API: Servlet 2.3 & JSP 1.2 Ashkelon: API: dom4j
I happen to be running my local copy of
ashkelonagainst PostgreSQL. The jdocs.org site chose to run
ashkelon with MySQL. (I've also recently added support for
mckoidb, a lightweight database written in Java.)
ashkelon's design decision to use a database gives it intimate knowledge of the code. It enables it to answer intelligent questions, such as: "list all methods that return objects of a certain type," "list all classes that implement a certain interface," "list all direct subclasses of a certain class," and so on.
More Than a Collection of Loosely Connected APIs
ashkelon makes a major effort to truly integrate the multiple APIs that reside in its repository.
Each time you add an API, a cross-referencing step takes place after the "store" step to update inter-API references between programming elements. Here's an example. Assume you already have the W3C's DOM API populated and then decide to add Hibernate. Look up this Hibernate method:
public org.w3c.dom.Document toGenericDOM() throws net.sf.hibernate.HibernateException
ashkelon's web-based UI, you'd find that the return type (
org.w3c.dom.Document) is hyperlinked.
Inter-API integration can also be seen in the cross-reference section of
ashkelon's UI. All classes and interfaces in the
ashkelon system have a cross-reference tab, shown in Figure 1.
Figure 1. GUI view of cross-linking
For example, you can ask
ashkelon to list all methods that return a
org.w3c.dom.Document. The results, shown in Figure 2, include methods from various APIs:
org.w3c.tidy, and of course,
Figure 2. Search for occurrences of a class
A third place where this integration is evident is in search results. A class search for "*Writer" will produce a listing that includes these classes or interfaces:
A Formal Definition for an API
Digging a little deeper, let's return to that
addcommand. We had to feed it an XML file: hibernate.xml ordom4j.xml. What's that all about? With the original Javadoc tool, you typically feed it the name of a file that contains the listing of packages that you want to document. Being a multi-API repository,
ashkelon needs to invent one more level up the containment hierarchy: the API.
These XML files are nothing more than information about the project you want to document. Those of you familiar with Maven are also familiar with its definition of a project XML file, known as the Project Object Model (POM). As of version 0.8.7,
ashkelon understands the Maven POM file format and can use these project.xmlfiles as input for populating APIs into its database repository. Here's how one populates
dom4j's API using itsproject.xml file:
~$ ashkelon add @project.xml
This makes the population of APIs trivial and effortless for projects that use Maven. Alternatively,
ashkelondefines its own XML file format for APIs. Below are the contents ofdom4j.xml.
<?xml version="1.0" ?> <api> <name>dom4j</name> <summarydescription>The flexible XML framework for Java</summarydescription> <description>dom4j is an easy to use, open source library for working with XML, XPath and XSLT on the Java platform using the Java Collections Framework and with full support for DOM, SAX and JAXP.</description> <publisher>SourceForge</publisher> <download_url>http://www.dom4j.org/</download_url> <release_date>2004-05-12T08:00:00</release_date> <version>1.4</version> <package>org.dom4j</package> <package>org.dom4j.bean</package> <package>org.dom4j.datatype</package> <package>org.dom4j.dom</package> <package>org.dom4j.dtd</package> <package>org.dom4j.io</package> <package>org.dom4j.io.aelfred</package> <package>org.dom4j.persistence</package> <package>org.dom4j.persistence.nativ</package> <package>org.dom4j.rule</package> <package>org.dom4j.rule.pattern</package> <package>org.dom4j.swing</package> <package>org.dom4j.tree</package> <package>org.dom4j.util</package> <package>org.dom4j.xpath</package> <package>org.dom4j.xpp</package> </api>
There's no need to write this XML file by hand. For one thing,
ashkelon comes "out of the box" with approximately 32 pre-written API XML files for various popular open source projects including Ant, J2SE, Hibernate,
dom4j, and many more. Secondly,
ashkelon comes with a script, calledapixml, that will automatically produce the XML file, directly from a Javadoc package-list file:
~/devel/dom4j-1.4/doc/javadoc $ apixml package-list
which produces this output (to
<?xml version="1.0" ?> <api> <name></name> <summarydescription></summarydescription> <description></description> <publisher></publisher> <download_url></download_url> <release_date>2001-07-03T08:00:00.000</release_date> <version></version> <package>org.dom4j</package> <package>org.dom4j.bean</package> <package>org.dom4j.datatype</package> <package>org.dom4j.dom</package> <package>org.dom4j.dtd</package> <package>org.dom4j.io</package> <package>org.dom4j.io.aelfred</package> <package>org.dom4j.persistence</package> <package>org.dom4j.persistence.nativ</package> <package>org.dom4j.rule</package> <package>org.dom4j.rule.pattern</package> <package>org.dom4j.swing</package> <package>org.dom4j.tree</package> <package>org.dom4j.util</package> <package>org.dom4j.xpath</package> <package>org.dom4j.xpp</package> </api>
Finally, fill in the missing pieces (API name, description, publisher, etc.), which you can look up on the project's home page.
A Different User Interface?
There have been two user interfaces for Javadocs. The first was released with JDK1.0 and used images (then in fashion) for page section headers, such as "Classes" and "Interfaces." The veterans among you will recall the convex yellow underlines adorning these titles. The second UI was released with JDK1.1 seven years ago. That is, by large, the same UI we use today.
ashkelon's DHTML user interface to address a number of usability issues that personally impeded my productivity as a Java developer.
Problem 1: Too much scrolling To get from the top of
java.awt.Componentto the bottom, you must "page down" 117 times. Contrast this with
ashkelon's UI, which uses tabbed panes to lay out the information horizontally across the page, thus minimizing scrolling. Furthermore,
ashkelonscrolls only those blocks of information that need to scroll (not the entire page), so you don't lose sight of header, footer, and sidebar sections that are the means of navigating to other pages in the system.
Figure 3. Compact layout
Problem 2: Monochromatic The class listing in Figure 4 below uses color and style to indicate whether a list item is an interface, an exception, an abstract class, or a concrete class. The mind can digest style and color information much faster than it can distinguish among keywords.
Figure 4. Color cues
Similar cues within a class method listing, as shown in Figure 5, help you identify static methods (in bold), deprecated methods (with strike-through), and abstract methods (italicized). How many times have you made use of a deprecated method, only to discover after implementation that you didn't read the Javadocs carefully enough?
Figure 5. Method styling
Problem 3: Static
ashkelon's UI is built dynamically using JSPs. Yes, it also is dynamic in other ways: it sports a number of client-side, DHTML features (as in a dynamically collapsible inheritance tree, or dynamically filtered tables). But that's not the point. With a dynamic (as in JSP or PHP) site, you can build a user interface that is more flexible. You can implement search forms (direct access to information). Figure 6 shows the results for searching for all classes whose names begin with
ashkelonalso provides advanced search forms for either classes or methods.
Figure 6. Search Results
With a dynamic system, one can build a navigation trail to allow temporal access to information. A sample navigation trail is displayed in Figure 7, just below the menu bar (note that the color coding applies also to elements within the navigation trail).
Figure 7. Navigation trails
The schema can be extended to produce views that don't exist in Javadocs. Figure 8 is one example of how you might display author information.
Figure 8. Author information
One can produce statistical information about code using SQL
group bystatements. Figure 9 shows the results of a query that lists all of the packages in alphabetical order. Next to each package is the number of classes it contains, with all packages with more than fifty classes highlighted in yellow.
Figure 9. Displaying statistical information
ashkelon can benefit an individual developer, a team, and even the Java community. The individual developer can run
ashkelon on his or her local box to reference multiple APIs locally. More interestingly, the team can use
ashkelon to look up both internally developed code and APIs that the team is using on their current project. The
ashkelon database can be updated nightly with the latest snapshot of the code from CVS so that everyone has a reference to the latest code. Lastly, a public web site hosting a large number of APIs can create some terrific economies, and at the same time capture community contributions and improve itself. I've long wanted to see something like Patrick Chan's examplets married with Javadocs. In the early days,
ashkelon was called
dbdoc and was featured in the java.sun.com article "dbdoc: Persistence Pays." A public instance of
ashkelonran at dbdoc.org a few years ago, and could be resurrected. We would need hosting as well as volunteers to maintain the site, populate APIs, field users' requests for new APIs, and so on.
There are so many ways you can get involved. Adopt
ashkelon in your work environment. Download
ashkelon and, if you can, please donateto the project and/or join the project. With a half-dozen talented developers, this project might have what it takes to become an Apache project. If you're a graphic designer, you could help significantly by producing a formal project logo or graphics to enhance the
ashkelon UI. If you don't have time to become a developer, please participate in the discussions. Join the ashkelon-users or ashkelon-devs mailing lists.