Improve on Javadocs with ashkelon Blog

Version 2

    {cs.r.title}




              
                     

    Contents
    A Database of Docs
    More Than a Collection of Loosely Connected APIs
    A Formal Definition for an API
    A Different User Interface?
    Final Thoughts

    Sun's Javadocs provide us with automatically generated, hyperlinked documentation of fields, methods, classes, interfaces, and inner classes. They are helpful in many ways, but you will soon develop a wish list that includes the ability to search for programming elements, the ability to access multiple APIs from one location, the ability to cross-reference relationships across APIs. The list goes on.

    This article describes ashkelon, an open source documentation system for Java that leverages Sun's Javadoc parser. When developing a UI for, say, 50 APIs, 500 packages, 10,000 classes, and 100,000 methods, how do you present mounds of information in a readily accessible and intelligible way? Patrick Chan's solution was to produce the Java Almanac, containing indices and cross-references to programming elements. But there are drawbacks to paper forms of referencing; for one thing, the Java Almanac covers only J2SE. And it's long and heavy.

    The handful of techniques used repeatedly inashkelon's UI to address this problem are:

    1. Hide information on the page using CSS display andvisibility properties, and toggle those properties dynamically (using various DHTML client-side "widgets" such as tabs, collapsible trees, and filtered tables).

    2. Color code, for more natural filtering of information on the page.

    3. Minimize scrolling.

    4. Provide many ways to access the same information: via an index, browsing or searching, temporal navigation back to an element recently previewed, via a cross-reference, or another type of programmatic relationship such as a throws relationship.

    A Database of Docs

    One component of ashkelon plugs in to Javadoc as a doclet to populate a relational database with Javadocs. The corresponding database schema for the Java language consists of tables with names such as member,classtype, package, author,field, parameter, method, and thrownexception. The schema can be updated and grow to include changes such as those introduced with J2SE 5.0 and aspects.

    So what can you do with a database schema filled with Javadocs? For one thing, with ashkelon you can populate multiple APIs into a single repository. You can do this incrementally, too. That is, you don't have to add all APIs in one run to "get your links." Let's look at how we might use ashkelon to populate an API such as Hibernate into the database. Invoke the following command from the command line, in much the same way as you would use the Javadoc tool to produce HTML documentation:

    ~$ ashkelon add @hibernate.xml
    

    To add another API (say dom4j), you do this:

    ~$ ashkelon add @dom4j.xml
    

    Now we can ask ashkelon to list the APIs in its database:

    ~$ ashkelon list
    

    which, with my local copy of ashkelon running on my notebook computer today, produces:

    Ashkelon: API: Ashkelon Ashkelon: API: Hibernate Ashkelon: API: J2SE Ashkelon: API: JTidy Ashkelon: API: JUnit Ashkelon: API: JiBX Ashkelon: API: Servlet 2.3 & JSP 1.2 Ashkelon: API: dom4j
    

    I happen to be running my local copy of ashkelonagainst PostgreSQL. The jdocs.org site chose to runashkelon with MySQL. (I've also recently added support for mckoidb, a lightweight database written in Java.)

    ashkelon's design decision to use a database gives it intimate knowledge of the code. It enables it to answer intelligent questions, such as: "list all methods that return objects of a certain type," "list all classes that implement a certain interface," "list all direct subclasses of a certain class," and so on.

    More Than a Collection of Loosely Connected APIs

    ashkelon makes a major effort to truly integrate the multiple APIs that reside in its repository.

    Each time you add an API, a cross-referencing step takes place after the "store" step to update inter-API references between programming elements. Here's an example. Assume you already have the W3C's DOM API populated and then decide to add Hibernate. Look up this Hibernate method:

    public org.w3c.dom.Document toGenericDOM() throws net.sf.hibernate.HibernateException
    

    Using ashkelon's web-based UI, you'd find that the return type (org.w3c.dom.Document) is hyperlinked.

    Inter-API integration can also be seen in the cross-reference section of ashkelon's UI. All classes and interfaces in the ashkelon system have a cross-reference tab, shown in Figure 1.

      
    Figure 1. GUI view of cross-linking
    Figure 1. GUI view of cross-linking

    For example, you can ask ashkelon to list all methods that return a org.w3c.dom.Document. The results, shown in Figure 2, include methods from various APIs:hibernate, dom4j, javax.xml,org.w3c.tidy, and of course,org.w3c.xml.

      
    Figure 2. Search for occurrences of a class
    Figure 2. Search for occurrences of a class

    A third place where this integration is evident is in search results. A class search for "*Writer" will produce a listing that includes these classes or interfaces:org.jibx.runtime.IXMLWriter,org.dom4j.io.XMLWriter,javax.swing.text.html.HTMLWriter,java.io.Writer, andjavax.sql.RowSetWriter.

    A Formal Definition for an API

    Digging a little deeper, let's return to that addcommand. We had to feed it an XML file: hibernate.xml ordom4j.xml. What's that all about? With the original Javadoc tool, you typically feed it the name of a file that contains the listing of packages that you want to document. Being a multi-API repository, ashkelon needs to invent one more level up the containment hierarchy: the API.

    These XML files are nothing more than information about the project you want to document. Those of you familiar with Maven are also familiar with its definition of a project XML file, known as the Project Object Model (POM). As of version 0.8.7, ashkelon understands the Maven POM file format and can use these project.xmlfiles as input for populating APIs into its database repository. Here's how one populates dom4j's API using itsproject.xml file:

    ~$ ashkelon add @project.xml
    

    This makes the population of APIs trivial and effortless for projects that use Maven. Alternatively, ashkelondefines its own XML file format for APIs. Below are the contents ofdom4j.xml.

    <?xml version="1.0" ?> <api> <name>dom4j</name> <summarydescription>The flexible XML framework for Java</summarydescription> <description>dom4j is an easy to use, open source library for working with XML, XPath and XSLT on the Java platform using the Java Collections Framework and with full support for DOM, SAX and JAXP.</description> <publisher>SourceForge</publisher> <download_url>http://www.dom4j.org/</download_url> <release_date>2004-05-12T08:00:00</release_date> <version>1.4</version> <package>org.dom4j</package> <package>org.dom4j.bean</package> <package>org.dom4j.datatype</package> <package>org.dom4j.dom</package> <package>org.dom4j.dtd</package> <package>org.dom4j.io</package> <package>org.dom4j.io.aelfred</package> <package>org.dom4j.persistence</package> <package>org.dom4j.persistence.nativ</package> <package>org.dom4j.rule</package> <package>org.dom4j.rule.pattern</package> <package>org.dom4j.swing</package> <package>org.dom4j.tree</package> <package>org.dom4j.util</package> <package>org.dom4j.xpath</package> <package>org.dom4j.xpp</package> </api>
    

    There's no need to write this XML file by hand. For one thing,ashkelon comes "out of the box" with approximately 32 pre-written API XML files for various popular open source projects including Ant, J2SE, Hibernate, dom4j, and many more. Secondly, ashkelon comes with a script, calledapixml, that will automatically produce the XML file, directly from a Javadoc package-list file:

    ~/devel/dom4j-1.4/doc/javadoc $ apixml package-list
    

    which produces this output (to stdout):

    <?xml version="1.0" ?> <api> <name></name> <summarydescription></summarydescription> <description></description> <publisher></publisher> <download_url></download_url> <release_date>2001-07-03T08:00:00.000</release_date> <version></version> <package>org.dom4j</package> <package>org.dom4j.bean</package> <package>org.dom4j.datatype</package> <package>org.dom4j.dom</package> <package>org.dom4j.dtd</package> <package>org.dom4j.io</package> <package>org.dom4j.io.aelfred</package> <package>org.dom4j.persistence</package> <package>org.dom4j.persistence.nativ</package> <package>org.dom4j.rule</package> <package>org.dom4j.rule.pattern</package> <package>org.dom4j.swing</package> <package>org.dom4j.tree</package> <package>org.dom4j.util</package> <package>org.dom4j.xpath</package> <package>org.dom4j.xpp</package> </api>
    

    Finally, fill in the missing pieces (API name, description, publisher, etc.), which you can look up on the project's home page.

    A Different User Interface?

    There have been two user interfaces for Javadocs. The first was released with JDK1.0 and used images (then in fashion) for page section headers, such as "Classes" and "Interfaces." The veterans among you will recall the convex yellow underlines adorning these titles. The second UI was released with JDK1.1 seven years ago. That is, by large, the same UI we use today.

    I developed ashkelon's DHTML user interface to address a number of usability issues that personally impeded my productivity as a Java developer.

    • Problem 1: Too much scrolling To get from the top ofjava.awt.Component to the bottom, you must "page down" 117 times. Contrast this with ashkelon's UI, which uses tabbed panes to lay out the information horizontally across the page, thus minimizing scrolling. Furthermore,ashkelon scrolls only those blocks of information that need to scroll (not the entire page), so you don't lose sight of header, footer, and sidebar sections that are the means of navigating to other pages in the system.

        
      Figure 3. Compact layout.
      Figure 3. Compact layout
    • Problem 2: Monochromatic The class listing in Figure 4 below uses color and style to indicate whether a list item is an interface, an exception, an abstract class, or a concrete class. The mind can digest style and color information much faster than it can distinguish among keywords.

        
      Figure 4. Color cues
      Figure 4. Color cues

      Similar cues within a class method listing, as shown in Figure 5, help you identify static methods (in bold), deprecated methods (with strike-through), and abstract methods (italicized). How many times have you made use of a deprecated method, only to discover after implementation that you didn't read the Javadocs carefully enough?

        
      Figure 5. Method styling
      Figure 5. Method styling
    • Problem 3: Static ashkelon's UI is built dynamically using JSPs. Yes, it also is dynamic in other ways: it sports a number of client-side, DHTML features (as in a dynamically collapsible inheritance tree, or dynamically filtered tables). But that's not the point. With a dynamic (as in JSP or PHP) site, you can build a user interface that is more flexible. You can implement search forms (direct access to information). Figure 6 shows the results for searching for all classes whose names begin withString. ashkelon also provides advanced search forms for either classes or methods.

        
      Figure 6. Search Results
      Figure 6. Search Results

      With a dynamic system, one can build a navigation trail to allow temporal access to information. A sample navigation trail is displayed in Figure 7, just below the menu bar (note that the color coding applies also to elements within the navigation trail).

        
      Figure 7. Navigation trails
      Figure 7. Navigation trails

      The schema can be extended to produce views that don't exist in Javadocs. Figure 8 is one example of how you might display author information.

        
      Figure 8. Author information
      Figure 8. Author information

      One can produce statistical information about code using SQLgroup by statements. Figure 9 shows the results of a query that lists all of the packages in alphabetical order. Next to each package is the number of classes it contains, with all packages with more than fifty classes highlighted in yellow.

        
      Figure 9. Displaying statistical information
      Figure 9. Displaying statistical information

    Final Thoughts

    ashkelon can benefit an individual developer, a team, and even the Java community. The individual developer can runashkelon on his or her local box to reference multiple APIs locally. More interestingly, the team can useashkelon to look up both internally developed code and APIs that the team is using on their current project. Theashkelon database can be updated nightly with the latest snapshot of the code from CVS so that everyone has a reference to the latest code. Lastly, a public web site hosting a large number of APIs can create some terrific economies, and at the same time capture community contributions and improve itself. I've long wanted to see something like Patrick Chan's examplets married with Javadocs. In the early days, ashkelon was calleddbdoc and was featured in the java.sun.com article "dbdoc: Persistence Pays." A public instance of ashkelonran at dbdoc.org a few years ago, and could be resurrected. We would need hosting as well as volunteers to maintain the site, populate APIs, field users' requests for new APIs, and so on.

    There are so many ways you can get involved. Adoptashkelon in your work environment. Downloadashkelon and, if you can, please donateto the project and/or join the project. With a half-dozen talented developers, this project might have what it takes to become an Apache project. If you're a graphic designer, you could help significantly by producing a formal project logo or graphics to enhance the ashkelon UI. If you don't have time to become a developer, please participate in the discussions. Join the ashkelon-users or ashkelon-devs mailing lists.

      
    http://today.java.net/im/a.gif