This discussion is archived
5 Replies Latest reply: Apr 30, 2005 6:29 AM by 843853 RSS

Mechanism to get rid of NullPointerException

843853 Newbie
Currently Being Moderated
Hi, I hope this is the right place to introduce my idea.
You all know these ugly NullPointerExceptions which occur occasionally. I think one major problem is, that there are two possibilities of error handling: If a method fails, it can throw an Exception, which is of course the clean way to handle it. The other possibility is to simply return null which is sometimes even a better way to handle an error, because its kind of expected behaviour.
Just think of methods like Map#get(Object key). If the Map doesn't contain the searched element, you wouldn't want it to throw something like ElementNotFoundException, null as result is really good and meaningfully. The only problem is that you, need to be reminded of the fact that it might return null so that you don't forget to handle null as a result.

With annotations, this problem could be solved very easily:
We only need a marker annotation to indicate that a method might return null:
public @interface ReturnNullAllowed {}
and another annotation to indicate, if a method parameter accepts null as value or not:
public @interface ParameterNullAllowed {
    String value(); //should be called parameterName, but value is a naming convention
}
If all methods would be decorated with these annotation, I think there aren't that many methods which use null as return value, mondern IDEs like NetBeans or Eclipse could easily check if all return statements of a method are valid, and if all method calls are secure. With these "extensions" its possible to write secure and highly reliable java code.

All this might sound very easy, but there is still one problem: To use these annotations and "extensions", the J2SE classes and methods have to be decorated, because this mechanism only works if all or at least most of the used method are decorated. But as I already wrote, for now, i just want to get some other opinions about my idea.

regards Andreas
  • 1. Re: Mechanism to get rid of NullPointerException
    dcminter Newbie
    Currently Being Moderated
    I have doubts about using the annotations as the mechanism for implementing this - my understanding is that they're for applying runtime annotation information, whereas this is plainly a compiletime feature that you're seeking.

    On the other hand, the basic idea is pretty interesting. It makes a very nice change to see someone suggesting a feature which would actually add some value to the language.

    So perhaps a suitable keyword when declaring types:
    public static final notnull String foo1 = "Foo"; // Compiles
    
    public static final notnull String foo2; // Compiler error because notnull reference uninitialized
    
    public static final notnull String foo3 = null; // Compiler error because null assigned to notnull reference
    
    public static final notnull Map mymap = new HashMap(); // Compiles
    
    public notnull String getString1() {
       return mymap.get("key"); 
    } // Compiler error, because Map.get not declared notnull
    
    public notnull String getString2() {
       return (notnull)mymap.get("key");
    } // Compiles, because we've claimed to know what we're
      // doing, but calling method may well NPE at runtime
    I wouldn't be averse to seeing something like this in the language (although I think my choice of keyword could be improved). It definitely wins on the grounds of catching likely errors at compiletime instead of runtime. On the other hand I don't know if it adds sufficient value to be worth the effort.

    Anyone see any terrible flaws in the scheme? And am I right in thinking that annotations wouldn't be the right way to handle this?

    Dave.
  • 2. Re: Mechanism to get rid of NullPointerException
    843853 Newbie
    Currently Being Moderated
    Hi Dave,
    this is also a nice suggestion, i even think the choice of keyword isn't that bad, because in most databases you can specify columns as notnull (afaik), so it sounds kind of familiar. Anyways, introducing a new keyword is really a big effort. So annotations migth be a good compromise. About your statement that annotations apply runtime information: Well I'm new to annotations, but I've seen an example of an annotation like this:
    ------------------------
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.METHOD)
    public @interface Test { }
    Note that the annotation type declaration is itself annotated. 
    Such annotations are called meta-annotations. The first (@Retention(RetentionPolicy.RUNTIME))
    indicates that annotations with this type are to be retained by the VM so they can be read reflectively at 
    runtime. The second (@Target(ElementType.METHOD)) indicates that this annotation type can be used 
    to annotate only method declarations. 
    --------------------------
    This is an example from: http://java.sun.com/j2se/1.5.0/docs/guide/language/annotations.html.

    In the JavaAPIDocs for RetentionPolicy: http://java.sun.com/j2se/1.5.0/docs/api/java/lang/annotation/RetentionPolicy.html
    there are three enum constants: RUNTIME, CLASS, SOURCE. So if I got it right, annotations can apply all kind of information.

    A keyword is of course the most clean way to handle it, and i really like this cast-like operation (notnull)mymap.get("key"), but annotations might be an easy and quick way to introduce this feature.

    Note:
    There is another good thing about the choice of your keyword, but which could be also done with annotations: My idea was to mark method which might return null. The notnull keyword marks methods which never return null. It is of course better to mark method which never return null, because of the simple fact that unmarked methods are "insecure" by default, which is much better than treading unmarked methods as "secure" (secure= will never return null). Marking methods this way, whether by keyword, or with annotations, is the better choice, because APIs that have not refactored to use the mechanism are simply treat as "unsafe" but still can be used normally. If unmarked methods would be treated as secure(=will never return null), but the APIs is not yet refactored, the compiler will make wrong assertions, because it will treat all methods as secure.
  • 3. Re: Classpath
    dcminter Newbie
    Currently Being Moderated

    Oh, and I think you owe Peter a duke for his trouble, because that's actually the right answer both from a purely technical point of view (no classpath really is the minimum possible) and a good practices point of view (no default classpath will cause fewest problems in the long run). And he was first too.

    Dave.
  • 4. Re: Classpath
    dcminter Newbie
    Currently Being Moderated
    It seems tabbed browsing can confuse the Java forums... please ignore the above.

    Dave.
  • 5. Re: Mechanism to get rid of NullPointerException
    843853 Newbie
    Currently Being Moderated
    I had a similar idea about two years ago. NullPointerExceptions are one of the most common problems for Java developers along with ClassCast Exceptions, mainly because these are caught at runtime even though their presence is indicative of flawed or non-robust programming logic. Java 5 has gone a long way to solving the ClassCastException issue with Generics, but a mechanism is still needed for dealing with unexpected nulls.

    NullPointerExceptions often occur when there are a chain of method calls and one unexpectedly returns null, e.g.
    object.method1().method2()
    will fail if method1 returns null, although this is a tempting way to program as a more robust alternative would span several lines and appear less readable at a glance. A mechanism is needed to flag where this is unsafe, i.e. null can be returned, and where this chaining is perfectly acceptable. My solution, which is very close to that stated earlier in this thread, is a notnull keyword.

    I envisage three main uses for this keyword as a modifier.

    Firstly class attributes and local variables can use notnull to denote that they never take null as a value. They don't have to be initialised at their point of declaration providing that they are initialised before their use (which Java checks for any way), and whatever they are initialised to must be a literal or some other expression marked as notnull. As Dave suggests, finals would have to be initialised at the point of declaration, but again that doesn't deviate from current Java practice.

    Secondly, method return types can be marked as notnull. The compiler would check that all return statements return values marked as notnull or non-null literals. This would mean that method chaining is fine providing that all but optionally the last method call are marked as notnull. If this isn't the case then the compiler will complain.

    Thirdly, parameters to a method can be flagged as notnull. As you can probably guess, at compile time all calls on the method would have their parameters checked to see if they could possibly be null, and compilation would fail with an error if there was such a mismatch. These parameters could then safely be used in the method, knowing that they can never be set to null, and thus that many operations involving them will cause the method to return a value which in turn could never be null. This is reminiscent of data tainting in languages such as Perl where potentially dangerous user inputs are flagged up and any interactions with program data subsequently mark that value as tainted.

    The notnull keyword could only apply to objects as it makes no sense applying it to basic types unless smehow coupled with autoboxing/unboxing.

    For example:
    public class SampleClass
         {
         private notnull String string1;
         private notnull String string2;
         private String string3;
         private static final notnull String string4 = "not null";
         private static final notnull String string5;
              // Compile error - string5 not initialised and final cannot be reassigned
         
         public static notnull String getSection (notnull String text, notnull String match)
              {
              int i = text.indexOf(match);
              if (i == -1)
                   return null; // Compile error - not allowed to return null
              return text.substring(i); // See comments below
              }
    
         public static void main (String[] args)
              {
              string1 = "This string is not null under any circumstances";
              string2 += string1; // Compile error - string2 not initialised
              string3 = "null";
              
              System.out.println(getSection("NullPointerException", "in")); // OK
              System.out.println(getSection(string4, "u")); // OK
              System.out.println(getSection(string1, string4)); // OK
              System.out.println(glueReplace(string1, string3)); // Fails - string3 could be null
              assert getSection(string1, string4) != null; // Always true for any notnull parameters
              }
         }
    In this example all parameters to getSection() must be notnull and the method is contracted to return a non-null value. There is, however, a problem. The String.substring() method isn't marked as returning a notnull and thus the method couldn't be guaranteed to return a non-null value and would thus fail at compile time. To rectify this String.substring() would have to be redefined to return a notnull-modified variable. This would also require that it takes in a notnull. But as the method returns null any way in the case of there being no match there is no way the method should be flagged as returning notnull.

    But this would potentially break existing code if a method requiring a notnull is passed in a value from an API that doesn't use notnull. Even if the method couldn't possibly return null this wouldn't be immediately apparent to the compiler. It should, however, be possible to use program logic similar to that used to check whether or not a variable has been initialised to see whether a non-notnull variable can be treated as notnull, e.g.
    String var = "Something or other"; // Could be null
    
    if (var == null)
         System.out.println("Null");
    else
         // Compiler knows that var is notnull here
         takesNotNullParameter(var); // OK
    takesNotNullParameter(var); // Compile error - here var could be null
    This is a simple example, and in theory the notnull logic system could extend to more complex expressions, the idea being that in any scope where a variable couldn't possibly be null it is assumed to be using the notnull modifier. There are potential problems, though, e.g.
    String var = "Not null";
    
    if (var != null) // var notnull implied
         {
         for (int i=0; i<2; i++)
              { // is var notnull here?
              takesNotNullParameter(var);
              var = null; // var is reassigned to null
              }
         }
    If simple block logic is used here then the for block will see var as notnull and the first method call will succeed, but var is then reassigned to null (which is allowed since there is no global notnull limit on var) and so the second method call (on the next iteration) will fail because it doesn't accept null parameters. There are probably ways around this, e.g. checking whether any in-block assignments evaluate to notnull.

    I don't think this would pose a problem with concurrent access to variables since assigning a null value is in effect removing the reference pointer and not modifying the underlying object, thus only affecting the local thread, though if this is a problem then NullPointerExceptions could still be thrown.

    Of course, some methods could never return null, thus the assert statement could be used in a similar way to the if statement in the previous example:
    String var1 = "Not null";
    String notnull var2;
    
    var1 = var1.toUpperCase(); // Doesn't return notnull
    assert var1 != null; // var1 implied notnull
    var2 = var1; // allowed because var1 can't be null
    If var1 was to become null and the assertion fails then an AssertionError will be thrown. This is stronger than current practice as it says the variable can't possibly be null and will throw an exception only if that's a false belief. NullPointerExceptions are usually thrown when the possibility of a null value hasn't been considered or has been overlooked. With this model the compiler will force the programmer to spot these potential oversights.

    The assertion method described above makes more robust code, but also makes it longer and prevents chaining of non-notnull methods, which can be very useful when dealing with convoluted APIs such as I/O streams. Thus I propose what I term as inline assertions, which is actually what Dave used in his getString2() return statement. Although it might be possible to extend it to other assertions, I see it using notnull as a cast, albeit to a modifier rather than a class, thus for a method a() which returns a potentially null value and a method b() that acts on a's return type we could normally say:
    a().b();
    but this will be flagged as a compiler error if notnull is enforced, or could throw a NullPointerException otherwise. If we know that in our context a() will never return null then we could instead write:
    ((notnull) a()).b();
    Thus as a() returns a variable that will never be null there will always be an object upon which b() can act. If this isn't the case then there is a programmer logic error and an AssertionError is thrown.

    Similarly, if a method takes a notnull but needs to be passed a variable that isn't flagged as such then the inline assertion could be used thus:
    takesNotNull((notnull) undecoratedVariable);
    Casting like this can look a little messy, but the meaning is still clear, and use of annotations would make the code less readable and possibly less flexible, though they can also have compile-time retention.

    It is important to note that assertions can be disabled, however. Perhaps a similar disabling mechanism could be used for notnull that requires assertions to be enabled as a prerequisite.

    The introduction of a notnull keyword could break programs where this is used as a member name. Existing APIs and programs would ideally have to be refactored to use this new keyword and it would be difficult to implement partially, but the actual work that needs doing should be relatively simple - deciding what can never be null. Good programs should already document where there is the possibility of a variable being null. The use of implied notnull modification and inline assertions would allow existing APIs to be used without modification, though accessing them wouldn't be as clean as APIs that do use notnull. At the bytecode level notnull elements could be flagged with meta data in a similar way to annotations, though it would seem messy to use annotations at the code level.

    While this may make Java slightly less accessible to new programmers and could be a bit of work to update existing code, I believe the improvement in code reliability and robustness will more than justify the work and develop the reputation of Java as a language for writing reliable software.

    As I say, I've had this idea for a while, and I'm not trying to take credit away from others who have been thinking along similar lines. It does overlap quite heavily with what other people here have said, but I wanted to convey my thoughts on the issue, so sorry for any duplication, and I hope it is of interest.