10 Replies Latest reply: Apr 19, 2010 7:05 AM by 791266 RSS

    Generating a AST from source-code


      Anyone know how to use the new Java Compiler API to generate a AST from source-code that I can walk?


        • 1. Re: Generating a AST from source-code
          • 2. Re: Generating a AST from source-code
            At first glance it looks like I need to use the experimental "Java Tree API" released in Java6: http://java.sun.com/javase/6/docs/jdk/api/javac/tree/index.html

            Specifically, it looks like one uses JavacTask.parse() to return a collection of Trees and somehow uses them to walk the source-code. I will investigate this further and report back here if I figure this out :)

            • 3. Re: Generating a AST from source-code
              I am interested in this thread; if you come to any conclusions,. please share !!!
              • 4. Re: Generating a AST from source-code
                Hi Chad,

                The API seemed to work pretty well in general but there is little documentation and support. As far as I can tell no one is working on it anymore and that sets off big warning lights in my mind.

                One particular bug that annoyed me was http://bugs.sun.com/view_bug.do?bug_id=6570730

                I posted a question asking for feedback about this bug today: http://forums.java.net/jive/thread.jspa?threadID=36595 so we'll see how that goes.

                • 5. Re: Generating a AST from source-code

                  As I understand it, that API "fills in" the gaps of javax.lang.model.* API (JSR-269) which doesn't model any method/contructor/initializer bodies. (statements, expressions etc). If JSR-269 provides some functionality, then the Tree API does not need to.

                  Because ClassTree represents a class which is modelled by JSR-269, you should use that API if the Tree API is insufficient.

                  This untested code is the approach I'd use
                  import com.sun.source.util.Trees;
                  import javax.lang.model.element.Element;
                  import javax.lang.model.element.ElementKind;
                      ClassTree classOrInterfaceTree = ...;
                      CompilationTask myCompilationTask = ...;
                      Trees trees = Trees.instance(myCompilationTask);
                      Element classOrInterfaceElement = trees.asElement(classOrInterfaceTree); // convert from Tree API to JSR-269
                      switch(classOrInterfaceElement.getKind()) {
                         case ElementKind.CLASS : ...
                         case ElementKind.INTERFACE : ...
                  Although there may currently be no apparent active work on this API, it is the compiler internal API, used by javac, and netbeans, so don't stress too much about those warning lights.

                  As a proof, I have just opened the latest BGGA closures proposal prototype, looked in the enclosed lib\javac.jar file, and down in the guts of that there is a class com.sun.source.tree.ClosureTree. It suggests that you pretty much have to maintain the tree API as you maintain the compiler, and I think you understand the importance of maintaining the compiler. :)

                  I'll post a comment along these lines on that bug report.

                  • 6. Re: Generating a AST from source-code
                    Hey Bruce,

                    It looks like the Tree API does not cover Annotation classes. Specifically, if I try to parse the following class:
                    public @interface Annotation
                      boolean booleanProperty() default false;
                    the Tree API will lose the default value (i.e. it does not generate a Tree node for it). This is true for any kind of default value, whether for primitive types, enums or nested annotations. Any ideas?

                    • 7. Re: Generating a AST from source-code
                      My mistake, it looks like MethodTree captures the default value information. There seems to be a bug in MethodTree.toString() in that it does not output those default values.
                      • 8. Re: Generating a AST from source-code
                        in case you weren't aware, you can create a alerts with google (I didn't know about this ) and get mailed notices whenever something that matches (in the Google sense) your criteria is found by Google on the web.

                        I did this for Tree API and java and some JSRs 199 269 296 ....
                        • 9. Re: Generating a AST from source-code
                          Hi All,

                          can anyone have example of generating AST from source Code using Compiler Tree API

                          Thanks in Advance.
                          • 10. Re: Generating a AST from source-code

                            Don't resurrect old threads, and don't cross/multi-post your question. I'm locking this thread.