This discussion is archived
1 2 Previous Next 15 Replies Latest reply: Jul 22, 2009 6:00 AM by 843793 RSS

How to access the *.java file corresponding to a TypeElement T?

843793 Newbie
Currently Being Moderated
Here is a problem:

How to access the *.java file corresponding to a TypeElement T from a AnnotationProcessor environment?

Let us say the hook method
public boolean process(Set<? extends TypeElement> annos, RoundEnvironment roundEnv)
is invoked with a TypeElement T such that T.getQualifiedname() = "a.b.c.X"

And the problem is how to locate the file that has the *.java source code for a.b.c.X?
That is the file that has the source code for a.b.c.X say "some/path/a/b/c/X.java"?

For simplicity, let us assume that TypeElement T corresponds to a top-level Java class.
  • 1. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Pinaki wrote:
    Here is a problem:

    How to access the *.java file corresponding to a TypeElement T from a AnnotationProcessor environment?

    Let us say the hook method
    public boolean process(Set<? extends TypeElement> annos, RoundEnvironment roundEnv)
    is invoked with a TypeElement T such that T.getQualifiedname() = "a.b.c.X"

    And the problem is how to locate the file that has the *.java source code for a.b.c.X?
    That is the file that has the source code for a.b.c.X say "some/path/a/b/c/X.java"?

    For simplicity, let us assume that TypeElement T corresponds to a top-level Java class.
    See the forum archives for the answer in this question, such as the recent thread "Compile time Annotation Processing."
  • 2. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    I just saw your response to the recent thread "Compile time Annotation Processing.". Thanks.

    Here is a slightly more elaborate problem description:


    In the problem context, an annotation processor dynamically generates a new java file say a.b.c.X_.java from the input type a.b.c.X is being compiled.
    I am being able to write a.b.c.X_.java relative a directory of my choice.
    Now I want that a.b.c.X_.java be written to the same directory where original a.b.c.X.java resides.

    So, probably, I am not being naughty i.e. not trying to mutate a.b.c.X.java -- but trying to find out where is its source code resides in the file system from a given Annotation Processing environment.
  • 3. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Pinaki wrote:
    I just saw your response to the recent thread "Compile time Annotation Processing.". Thanks.

    Here is a slightly more elaborate problem description:


    In the problem context, an annotation processor dynamically generates a new java file say a.b.c.X_.java from the input type a.b.c.X is being compiled.
    I am being able to write a.b.c.X_.java relative a directory of my choice.
    Now I want that a.b.c.X_.java be written to the same directory where original a.b.c.X.java resides.
    That is not something you as the annotation processor author should be asking about. That is something you the person configuring the javac environment should set up via the -d option or its equivalent.

    Additionally, IMO it is a serious configuration error to put generated files and input files, presumably tracked under version control, in the same directory. I strongly recommend using a separate output directory hierarchy.
  • 4. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    That is not something you as the annotation processor author should be asking about. That is something you the person configuring the javac environment should set up via the -d option or its equivalent.
    I strongly recommend using a separate output directory hierarchy.
    That is the way the current implementation is. The annotation processor takes a -Aout= <some directory path> to write generated output relative to a user-specified location (which defaults to the class output location).
    Additionally, IMO it is a serious configuration error to put generated files and input files, presumably tracked under version control, in the same directory.
    Interesting you said that. We are just running some "field trial" with these things and some users want them to be in the same directory of the original *.java files (especially when their source files are spread across many roots) .
    Who knows what the user wants?

    presumably tracked under version control, in the same directory.
    version control is another 'usability issues' that we are trying to get our heads around with this stuff. "To check-in or not?" - that seems to be the question. When presented these facilities without any cue, "the users" were divided -- some wanted them to be checked-in, some did not. The context where these generated files being used -- there exists some rationale to check them in a version control system.
  • 5. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Pinaki wrote:
    That is not something you as the annotation processor author should be asking about. That is something you the person configuring the javac environment should set up via the -d option or its equivalent.
    I strongly recommend using a separate output directory hierarchy.
    That is the way the current implementation is. The annotation processor takes a -Aout= <some directory path> to write generated output relative to a user-specified location (which defaults to the class output location).
    That is contrary to the design of the annotation processing facility. The intended use is for that information to be configured via options like "javac -d".
    Additionally, IMO it is a serious configuration error to put generated files and input files, presumably tracked under version control, in the same directory.
    Interesting you said that. We are just running some "field trial" with these things and some users want them to be in the same directory of the original *.java files (especially when their source files are spread across many roots) .
    Who knows what the user wants?
    The user does and the user is free to (mis)configure their environment however they like ;-) However, that is the user's option and the user has control of this via the javac command line. The annotation processor is not the proper place to configure this setting; see slide 7 of http://blogs.sun.com/darcy/resource/J1_2006-BOF-0606.pdf for some thoughts on different roles in annotation processing.
    presumably tracked under version control, in the same directory.
    version control is another 'usability issues' that we are trying to get our heads around with this stuff. "To check-in or not?" - that seems to be the question. When presented these facilities without any cue, "the users" were divided -- some wanted them to be checked-in, some did not. The context where these generated files being used -- there exists some rationale to check them in a version control system.
    The right answer depends on the circumstances, but IMO generally generated files should not be checked in under version control, especially if they are derived from other source files. Checking in generated files of this nature just creates the opportunity for them to get out of date with the originating files.
  • 6. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    You can use the javac-specific tree API (http://java.sun.com/javase/6/docs/jdk/api/javac/tree/index.html) to map from a javax.lang.model.Element to a AST representing the source for a method or constructor.
    Thanks for the pointer. Got something working along this line.

    Noticed the following in the JavaDoc of javax.tools.JavaFileManager (btw, the javax.annotation.* API is well-documented).
    JavaFileObject getJavaFileForOutput(Location location, String className, Kind kind, FileObject sibling) throws IOException;

    * <p>Optionally, this file manager might consider the sibling as
    * a hint for where to place the output. The exact semantics of
    * this hint is unspecified. Sun's compiler, javac, for
    * example, will place class files in the same directories as
    * originating source files unless a class file output directory
    * is provided. To facilitate this behavior, javac might provide
    * the originating source file as sibling when calling this
    * method.


    An issue report [1] has been created to capture our discussions as we are trying to understand the usability aspects when annotation processing generates source code.

    Thank you again for your help and valued comments.


    [1] https://issues.apache.org/jira/browse/OPENJPA-1187
  • 7. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Pinaki wrote:
    You can use the javac-specific tree API (http://java.sun.com/javase/6/docs/jdk/api/javac/tree/index.html) to map from a javax.lang.model.Element to a AST representing the source for a method or constructor.
    Thanks for the pointer. Got something working along this line.

    Noticed the following in the JavaDoc of javax.tools.JavaFileManager (btw, the javax.annotation.* API is well-documented).
    Thanks; myself and other JSR 269 expert group members wanted the specification to be clear.
    JavaFileObject getJavaFileForOutput(Location location, String className, Kind kind, FileObject sibling) throws IOException;

    * <p>Optionally, this file manager might consider the sibling as
    * a hint for where to place the output. The exact semantics of
    * this hint is unspecified. Sun's compiler, javac, for
    * example, will place class files in the same directories as
    * originating source files unless a class file output directory
    * is provided. To facilitate this behavior, javac might provide
    * the originating source file as sibling when calling this
    * method.


    An issue report [1] has been created to capture our discussions as we are trying to understand the usability aspects when annotation processing generates source code.

    Thank you again for your help and valued comments.


    [1] https://issues.apache.org/jira/browse/OPENJPA-1187
    Hope this discussion has helped; the envisioned division on responsibilities is further discussed starting in slide 33 of
    http://blogs.sun.com/darcy/resource/J1_2005-TS-7425.pdf.
    That presentation is about JDK 5's apt, but the issues are the same as with annotation processing in javac as of JDK 6.
  • 8. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Hi,

    javac (version javac 1.6.0_10-rc) selp page shows the following option:
    -s <directory> Specify where to place generated source files

    Now that looks promising and standard.
    But the question is how to access the value of javac -s option within a Annotation Processor environment? Or do we need to at all.

    Say the user invokes as
    $ javac -s some/out/dir a.b.X.java
    Will the compiler (or someone) set StandardLocation.SOURCE_OUTPUT to "some/out/dir" so that StandardJavaFileManager can access it within Annotation Processor environment?

    Because, the way I have interpreted is that "there is no standard way to access the javac options from Annotation Processor environment" -- (correct me if this assertion is wrong)

    But a simple experiment showed that
    StandardJavaFileManager fileMgr = ...;
    Iterable<? extends File> outputs = fileManager.getLocation(StandardLocation.SOURCE_OUTPUT);

    return null value for 'outputs' within the Annotation Processor when invoked as part of the following command:
    $ javac -s some/out/dir a.b.X.java


    Thanks for being patient with all sorts of questions :)
  • 9. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Pinaki wrote:
    Hi,

    javac (version javac 1.6.0_10-rc) selp page shows the following option:
    -s <directory> Specify where to place generated source files

    Now that looks promising and standard.
    The command line of javac is a fairly stable convention, but it is not standardized in the same sense of, say, the javax.annotation.processing API.
    But the question is how to access the value of javac -s option within a Annotation Processor environment? Or do we need to at all.
    The annotation processor should just use StandardLocation.SOURCE_OUTPUT as an argument to the file. Passing in a value for "javac -s" sets the value javac uses for StandardLocation.SOURCE_OUTPUT.
    Say the user invokes as
    $ javac -s some/out/dir a.b.X.java
    Will the compiler (or someone) set StandardLocation.SOURCE_OUTPUT to "some/out/dir" so that StandardJavaFileManager can access it within Annotation Processor environment?
    Correct.

    >
    Because, the way I have interpreted is that "there is no standard way to access the javac options from Annotation Processor environment" -- (correct me if this assertion is wrong)
    The javac options themselves are not standardized and are not meant to be directly accessible by annotation processors; however the effects of setting javac options may be reflected in a way visible or usable by the annotation processors. In javac, the "-Afoo" options are explicitly passed through to help configure annotation processors.
  • 10. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    This is actually what I was asking, rather than a tree representation of the source code. I am not planning on modifying the source code, but rather trigger compiler warnings based on certain behaviour being performed in the code.

    From the element itself is there a way to capture the full source path and file name without having to get element.asType().toString() and then splitting the foo.bar.Blah into foo/bar/Blah.java

    thanks

    Chris
  • 11. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Milesy wrote:
    This is actually what I was asking, rather than a tree representation of the source code. I am not planning on modifying the source code, but rather trigger compiler warnings based on certain behaviour being performed in the code.
    Such warnings are meant to be generated based on the tree representation.
  • 12. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Hi,

    Sorry, I actually want to produce compilation warnings and errors based on code formatting as well. This is just an investigation, so there is no need for any do or do nots. I am just trying to see if it can be done.

    Thank You again.
  • 13. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Let me summarize the responses received (thanks to all) and my own small experiments on the original question of this thread:
    How to access the T.java file corresponding to a TypeElement T?

    First, the bad news:
    1. there is no 'standard', spec-compliant way. The reason being there may not even be a .java file corresponding to a TypeElement (it may have been read from a .class bytestream).
    2. The typical path related options to javac compiler such as -classpath, -d, -sourcepath, -s are not accessible from Annotation Processor invocation context. That is to say,
    StandardJavaFileManager.getLocation(StandradLocation.SOURCE_OUTPUT)
    will not return the directory path value set in -s option for the javac compiler.
    This is what I see in a small experiment -- can someone confirm?

    The good news:
    1. If the .java file exists, and one does not care to pierce the "standard" API boundaries, then the file can be accessed via com.sun.source.util.* API as the sketch follows
           
           Trees trees = Trees.instance(env);
           TreePath path = trees.getPath(myTypeElement);
           JavaFileObject source = path.getCompilationUnit().getSourceFile();
           File sourceFile = new File(source.toUri().toURL().getPath());
  • 14. Re: How to access the *.java file corresponding to a TypeElement T?
    843793 Newbie
    Currently Being Moderated
    Hi,
    You wrote:
    "The annotation processor should just use StandardLocation.SOURCE_OUTPUT as an argument to the file. Passing in a value for "javac -s" sets the value javac uses for StandardLocation.SOURCE_OUTPUT."

    Can you please confirm or verify the about statement?
    Because this is contrary to the observation of a small experiment with a TestAnnotationProcessor.

        @Override
        public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
            System.err.println("StandardLocation as seen by " + this.getClass().getName());
            for (Location loc : StandardLocation.values())
                printLocation(loc);
            return true;
        }
       
        void printLocation(StandardLocation loc) {
            Iterable<? extends File> paths = fileManager.getLocation(loc);
            if (paths == null) {
                System.err.println(loc.getName() + ":" + paths);
                return;
            } else {
                System.err.println(loc.getName());
                for (File f : paths) {
                    System.err.println("\t" + f.getAbsolutePath());
                }
            }
        }
        
    shows the following output
     
    C:\projects\jsr269>C:\java\jdk1.6.0_10\bin\javac -cp bin -d bin -sourcepath src -s generated-src -processor jsr269.TestAnnotationProcessor src/org/acme/Foo.java 
    StandardLocation as seen by jsr269.TestAnnotationProcessor
    ANNOTATION_PROCESSOR_PATH:null
    SOURCE_PATH:null
    SOURCE_OUTPUT:null
    CLASS_OUTPUT:null
    CLASS_PATH
         C:\projects\jsr269\.
         ... // omiited
        
1 2 Previous Next