This discussion is archived
10 Replies Latest reply: Apr 19, 2010 5:05 AM by 791266 RSS

Generating a AST from source-code

800406 Newbie
Currently Being Moderated
Hi,

Anyone know how to use the new Java Compiler API to generate a AST from source-code that I can walk?

http://java.sun.com/javase/6/docs/api/javax/tools/JavaCompiler.html

Thanks,
Gili
  • 1. Re: Generating a AST from source-code
    EJP Guru
    Currently Being Moderated
    http://forum.java.sun.com/thread.jspa?threadID=5148150
  • 2. Re: Generating a AST from source-code
    800406 Newbie
    Currently Being Moderated
    At first glance it looks like I need to use the experimental "Java Tree API" released in Java6: http://java.sun.com/javase/6/docs/jdk/api/javac/tree/index.html

    Specifically, it looks like one uses JavacTask.parse() to return a collection of Trees and somehow uses them to walk the source-code. I will investigate this further and report back here if I figure this out :)

    Gili
  • 3. Re: Generating a AST from source-code
    800595 Newbie
    Currently Being Moderated
    I am interested in this thread; if you come to any conclusions,. please share !!!
  • 4. Re: Generating a AST from source-code
    800406 Newbie
    Currently Being Moderated
    Hi Chad,

    The API seemed to work pretty well in general but there is little documentation and support. As far as I can tell no one is working on it anymore and that sets off big warning lights in my mind.

    One particular bug that annoyed me was http://bugs.sun.com/view_bug.do?bug_id=6570730

    I posted a question asking for feedback about this bug today: http://forums.java.net/jive/thread.jspa?threadID=36595 so we'll see how that goes.

    Gili
  • 5. Re: Generating a AST from source-code
    608410 Newbie
    Currently Being Moderated
    Gili,

    As I understand it, that API "fills in" the gaps of javax.lang.model.* API (JSR-269) which doesn't model any method/contructor/initializer bodies. (statements, expressions etc). If JSR-269 provides some functionality, then the Tree API does not need to.


    Because ClassTree represents a class which is modelled by JSR-269, you should use that API if the Tree API is insufficient.

    This untested code is the approach I'd use
    import com.sun.source.util.Trees;
    import javax.lang.model.element.Element;
    import javax.lang.model.element.ElementKind;
    
    ...
    
        ClassTree classOrInterfaceTree = ...;
        CompilationTask myCompilationTask = ...;
        Trees trees = Trees.instance(myCompilationTask);
        Element classOrInterfaceElement = trees.asElement(classOrInterfaceTree); // convert from Tree API to JSR-269
        switch(classOrInterfaceElement.getKind()) {
           case ElementKind.CLASS : ...
           case ElementKind.INTERFACE : ...
        }
    Although there may currently be no apparent active work on this API, it is the compiler internal API, used by javac, and netbeans, so don't stress too much about those warning lights.

    As a proof, I have just opened the latest BGGA closures proposal prototype, looked in the enclosed lib\javac.jar file, and down in the guts of that there is a class com.sun.source.tree.ClosureTree. It suggests that you pretty much have to maintain the tree API as you maintain the compiler, and I think you understand the importance of maintaining the compiler. :)


    I'll post a comment along these lines on that bug report.

    Bruce
  • 6. Re: Generating a AST from source-code
    800406 Newbie
    Currently Being Moderated
    Hey Bruce,

    It looks like the Tree API does not cover Annotation classes. Specifically, if I try to parse the following class:
    public @interface Annotation
    {
      boolean booleanProperty() default false;
    }
    the Tree API will lose the default value (i.e. it does not generate a Tree node for it). This is true for any kind of default value, whether for primitive types, enums or nested annotations. Any ideas?

    Thanks,
    Gili
  • 7. Re: Generating a AST from source-code
    800406 Newbie
    Currently Being Moderated
    My mistake, it looks like MethodTree captures the default value information. There seems to be a bug in MethodTree.toString() in that it does not output those default values.
  • 8. Re: Generating a AST from source-code
    800595 Newbie
    Currently Being Moderated
    cowwoc,
    in case you weren't aware, you can create a alerts with google (I didn't know about this ) and get mailed notices whenever something that matches (in the Google sense) your criteria is found by Google on the web.

    I did this for Tree API and java and some JSRs 199 269 296 ....
  • 9. Re: Generating a AST from source-code
    843810 Newbie
    Currently Being Moderated
    Hi All,

    can anyone have example of generating AST from source Code using Compiler Tree API

    Thanks in Advance.
  • 10. Re: Generating a AST from source-code
    791266 Explorer
    Currently Being Moderated
    >

    Don't resurrect old threads, and don't cross/multi-post your question. I'm locking this thread.

    Kaj