jcnarahari

1 post
 

by
Jaisimha Narahari
Date: 02/June/2010

 

    In the month of June 2009, I did an initial post in the openjdk.java.net-jdk7-dev
    mailing list with a request for what I feel are two very important changes
    required in the Java Platform.This resulted in a thread of discussions on
     the Reification of Generic Types, but lead to no conclusions.Truly, the
     time is really very much ripe for Reification of Generic Types.The Java
     Language and Platform Implementers should not really ignore it much longer.
   
Contents
 
 
    1. Two Most Important Changes Required in Java 
     
      1.1 The Requests for Change 
 
      1.2 Reason for the Requests 
           
    2. Why Reification of Generics was not taken up 
     
      2.1 Design Constraints in Section 2.5 of JSR14 
     
    3. Hindsight Wisdom: Solutions that could have been 
 
      3.1 Dual APIs: Generic and Non Generic Versions 
 
      3.2 Why the Dual API solution was not adopted 
 
      3.3 Generics right from Java 1.2 
 
    4. The Finally Adopted Solution in Java 5.0 
 
    5. Why Resurrect the Controversy 
 
      5.1 "NO-WIN" situation with No Reification 
 
      5.2 Reversal of "NO-WIN" situation with Reification 
 
    6. Conclusion 
 
      6.1 Realizing Reification of Generics 
 
      6.2 Work involved in the Implementation 
 
      6.3 Sign Off Plea 


    1. Two Most Important Changes Required in Java 
 
     
        In the month of June 2009, I did an initial post in the 
 
        openjdk.java.net-jdk7-dev mailing list -June 2009

        with a request for what I feel are two very important changes required in the Java
        Platform. This resulted in a thread of discussions on the Reification of Generic 
        Types, but lead to no conclusions.Truly, the time is really very much ripe
        for Reification of Generic Types.The Java Language and Platform Implementers 
        should not really ignore it much longer.
 
    1.1 The Requests for Change 
 
        My main request for changes/inclusion in Java 7 were as follows:

        From Java Version 7.0, going forward:

                    Remove the provision for Backward Compatibility(See Section 2.1 below).
                    In other words:

                          1. "Deprecate" Raw Types

                               2. Adopt "Reification" in the place of "Erasure" for Generic Types.

                              The consequence of this will render the "contrived" and "constricting" rules
                              for the usage of Generic Types in Java 5.0 (given in Section 1.2 below)
                              unnecessary.

                              As a result, all types provided by the language will behave uniformly, most
                              especially without violating OOP principles, in what is essentially an OOP
                              Language.   
        Legacy code can still make use of JDK up to Java version 6.0 for         inter-operability, with support removed from Java 7.0.     1.2 Reason for the Requests         The three contrived rules mentioned above introduced as restrictions in the usage         of Generics in Java 5.0, as a consequence of Erasure, are as follows:          (A) For the sake of determining Type Hierarchy of Generic Types, the Type              Parameters are not to be considered. That is,code such as the line below is              rejected by the compiler:                 List<Object> lo = new ArrayList<Integer>              even as pure OOPS principles allow an Integer to be recognized as an Object.              That is, in Java 5.0 Generics, List<Object>and List<Integer> are              treated as totally unrelated types by the above rule, not only violating the              purity of OOPS but also sacrificing type system robustness.          (B) Any kind of Array Creation involving Generic Types,such as List<E> [],              List<String> [], E[], etc also get forbidden, on account of the fact              that Arrays are "Reified" types, implying that they carry over their full              type info into the Runtime, and are therefore aware of the type of their              contained elements at runtime, in direct contrast to Generic Types, which,              because of Erasure,discard the knowledge of the type of their contained              elements (i.e.,Type Parameters) and are therefore  "non-reifiable" types.              [However, the expression E[] can be used, (without "new" used for its              creation), to specify an array of ANY REGULAR TYPE, not Generic Type.                  This syntax therefore is not really one to be associated with Generic Types,              even as E[] in itself represents a Generic Type whose Formal Type Parameter              'E' can only have Regular, Non-Generic Types as its Actual Type Parameter              instantiations].          (C) Exceptions cannot be Parameterized Types.         See Referenced Java Text     2. Why Reification of Generics was not taken up         As it became evident to me in the course of the thread of discussions at the         openjdk.java.net-jdk7-dev mailing list - June 2009,         as well as via the contents of a couple of blogs that I was able to home in on:         (Thanks to Neal Gafter, Sun MicroSystems for the blogs)         Neal Gafter's Blog/2004/09                                                                    and           Neal Gafter's Blog/2006/11,         the summary of reasons of why my requests #1 and #2 in Section 1.1 above could not         be immediately accepted is as given in Section 2.1 below:        2.1 Design Constraints in Section 2.5 of JSR14         (a) In JSR 14, which introduced Generics to Java, there were special                 Design Constraints in Section 2.5             that sought to provide support for "Raw Types" in the Java Collections API.             Reason One for these constraints was Backward Compatibility.             This meant that existing legacy code that made use of the Java Collections API             without specifying Type Parameters (Type Parameters are part of the newly             introduced Generics version of the API) could continue to do so.             That is, legacy code could continue to treat the Collection Classes as regular             types rather than as Generic Types. The classes so used are referred to as             "Raw Types".             Reason Two for the above constraints was Migration Compatibility.             Backward Compatibility as a facility also allowed Third Party API Library             providers who directly used the old Java API to continue to do so, without             any changes forced on the Third Party API Library product that they delivered             to the market.             By extension, this made it also easy for other Third Party API Library             providers who in turn used the API provided by the above "first line" of Third             party API suppliers: No changes were forced, also, to their "second line"             product that they delivered to the market.             For any chain of such Third Party API providers that existed, delivering in             effect, an "API Stack" to the market, and also for other product vendors who             used APIs from this "stack" at any level, Backward Compatibility ensured that             the introduction of Java Generics did not force any such suppliers of products             up the chain to change their products to accommodate Generics.             Migration Compatibility is the provision that ensured that those users who             wanted to migrate to the new Java APIs with Generics could do so, by using             the Generic classes with their Type Parameters specified, rather than as             "Raw Types" (without specifying Type Parameters, as used by the chain of             product suppliers mentioned above).             This, seemingly, is two problems solved with one stroke. The provision of             "dual" usage of one and the same Java API - one usage as "Raw Types", and the             other usage as "Generic Types", for two sets of users with differing             requirements.             The reasons given above look perfect, as a case AGAINST             "Deprecating Raw Types",my request #1 cited in Section 1.1.                        Because NOT DEPRECATING Raw Types allows old users to continue as before,             without facing "disruptive" changes to their product, even as migrating users             who want to use the new features could also do so.             But alas, there are problems with this approach, as evidenced by the (by now             well-documented), contrived rules in Generics Usage in Java 5.0., as given in             Section 1.2 above. These rules practically restrict the proper usage of             Generics by the migrating users who want to take advantage of the new             features.         (b) However, the issue could not be considered closed. Reification could not             simply be abandoned, with Generic Types receiving "slipshod" treatment             forever.             So it is that one finds "Reification of Generics" under:             Bug ID 5098163 in Sun's Bug database.             (Thanks to Alex Buckley, Sun Microsystems for the URL).             Rather than treat this as a Bug, the entry labels it "RFE", standing for             "Request for Enhancement".                                        Against "Evaluation", it is marked as being "Under discussion as a potential             feature in Java SE 7."             And against "Priority" it is marked "4-Low".             I could not find any other clause in this Bug database entry that indicated             further action taken. Nor could I find any "current status" clause, maybe on             account of the assigned low priority, as seen above.                    3. Hindsight Wisdom: Solutions that "could have been"          3.1 Dual APIs: Generic and Non Generic Versions         In hindsight, one alternative solution that could have been adopted at the time         of Java 5.0 release, with Generics, to overcome these restrictions, would have         been to provide two versions of the all Java APIs into which Generics was         introduced :         One with Generics, and one without (the old API as it existed before introducing         Generics).         The APIs with Generics would then not have been required to support their usage as         "Raw Types", and as a result the classes in those new APIs would have been         "Reified" Types rather than types, each of whose type info was erased at runtime         because of "Erasure".         Java 5.0 would then have not been required to specify any restrictions in the         usage of Generics for the migrating users, which is of course the way things         should really be.              Those users who were dependent on the old API could simply continue to use the         "undisturbed" old API, without any disruptions.                While this solution looks more elegant, it was anyway NOT ADOPTED.         It has the drawback that Java would end up possessing a pair of APIs for         performing the same function, and that might be quite a number of extra APIs.                       But if this solution had been adopted, the older version of each such API could         have been deprecated in course of time, as and when all of the old non-migrating         users could safely be deemed to have also migrated to the new API.          3.2 Why the Dual API solution was not adopted         The logic behind NOT ADOPTING this solution [of two versions for the same API],         as I gathered, is as follows:                       - Doing so would result in a versioning nightmare of APIs, as one progressed             up the product/API chain mentioned in Section 2.1 (a) above, starting with             two versions in the Java API, and geometrically multiplying up the            "API stack", provided by Third Party vendors.                - This would happen because of a permutation of API combinations possible to             get delivered by vendors of API products up the stack, for other vendors             dependent on their APIs, by mixing and matching Generic plus Non-Generic             versions of their individual API Libraries.         I am not sure if a complete survey was conducted with vendors of such nature to         arrive at the above conclusions, but on the face of it the logic above looks a         bit stretched, and possibly, just possibly, may not reflect ground realities         accurately.           The very concept of stack upon stack of Third Party API Libraries existing in the         market, a higher one in the stack dependent on a lower one, seems a bit         far-fetched.              Even granting that such stacks do exist, the instances of such situations are         bound to be more an exception rather than the rule.              In any case, Java 5.0 with Generics got released with one and the same API for         each API that provided Generic Types, allowing the use of the Generic Classes as         "Raw Types" as well as use of the same classes as Generic Types.         So it is a moot point now whether a one-API solution or a dual-API solution would         have been the better one.                       Therefore         the Design Constraints in Section 2.5 of JSR 14         used the need for Backward Compatibility and Migration Compatibility as detailed         in Section 2.1 (a) above as their basis, and made support for "Raw Types"         mandatory as a result.          3.3 Generics right from Java 1.2         Also in hindsight, this whole problem of Generics and Backward and Migration         Compatibility, could have been completely avoided if at the time of the release of         the Collection classes in Java, that occurred with Java 1.2, the API had been         released only with Generics rather than without, to start with.         After all, if we are talking of Lists, Queues, etc, it makes a lot of sense to be         able to distinguish between a List/Queue of "foo"s and a List/Queue of "bar"s.         But this is in hindsight, and again, a moot point.         By the time of release of Java 5.0, every piece of code written using these         Non-Generic Collection API classes had become "legacy" code that used the classes         as "Raw Types", code that did not care if a List was comprised of only fairies,         only ogres, or a mixture of both. This is definitely an unfortunate turn of events,         as it has turned out.         ALL OF THE FOREGOING POINTS THAT I HAVE MADE IS ONLY TO PUT THINGS IN PERSPECTIVE,         AND TO ESTABLISH THE CONTEXT FOR THIS BLOG.              THEY MUST NOT BE MISCONSTRUED AS A FAULT FINDING MISSION AGAINST SUN MICROSYSTEMS         (NOW PART OF ORACLE) OR ANY OTHER STAKEHOLDER.                     4. The Finally Adopted Solution in Java 5.0              To the credit of the people involved in the development and release of Java 5.0         Generics, as it came to light in the thread of discussions at the              openjdk.java.net-jdk7-dev mailing list -June 2009,         the experts at Sun Microsystems had tried to find the best-fit solution for         simultaneously satisfying the needs of both the migrating users (those wanting to         use Generics) and non-migrating users (those who wanted to stick to the old API         usage) by considering as many workarounds as possible and finally settled for what         actually got released as the Java 5.0 Generics APIs.              The Design Constraints in Section 2.5 of JSR 14         had to be satisfied, because Backward Compatibility could not be sacrificed,         overnight breaking legacy code, even if the reason behind not releasing a         separate set of Generic-only APIs,for "proper" Migration Compatibility,         that of a versioning nightmare in a stack of interdependent APIs, is questionable.         It just so happens that, FUNDAMENTALLY, the needs of the non-migrating legacy         code users (read usage of Generic Types as "Raw Types") and the needs of the         migrating users (which is ideally the usage of Generic Types without twists and         quirks, achievable only without their simultaneous usage as "Raw Types") are         MUTUALLY EXCLUSIVE requirements, even as only a SINGLE API for a single function         became  available.         If you satisfy the one, you cannot satisfy the other, with just one and the same         API.              That is, there can either be Erasure or there can be Reification, but not both         together for Generic Classes of one and the same API.              As result of the mutual exclusivity involved, the bias that became inevitable in         arriving at a solution had to tilt towards the legacy users. Because, as mentioned         above, support for millions of lines of legacy code cannot be dumped overnight.              So Erasure won the day, Reification had to be discarded, but at very good cost for         the migrating users.         To be fair, Java 5.0 also provides a "patch up" API in the classes provided by         java.util.Collections.checkedXXX classes, which try to reverse the effects of         Erasure by carrying over the type info of Type Parameters of Generic Types also         into the runtime, by separately providing reflection (".class") types of the         Type Parameters as part of the checkedXXX classes.              While this provision re-ensures type safety (to compensate for the loss of type         safety due to erasure), there is no compensation concerning the three "makeshift"         rules of Section 1.2 above that end up gravely restricting Java from having a very         robust, elegant Type System, wherein Parameterized Types could enjoy the same         status as any other type, without getting quirks and twists in their usage         imposed upon them.         So, in the status quo, all decisions concerning Java Generics as a feature have         been explained to satisfaction as seen above, specifically the controversial one         of choosing Erasure over Reification for Generic Types.         Having got all the reasons outlined above, as the reasons for why my requests #1         and #2 in Section 1.1 could not be immediately accepted, from the thread of         discussions ensuing from my initial post at              openjdk.java.net-jdk7-dev mailing list -June 2009,              and the contents of these blogs:              Neal Gafter's Blog/2004/09                                                                            and               Neal Gafter's Blog/2006/11,              the hopes of getting the requests incorporated in Java 7 had to be abandoned.          5. Why Resurrect the Controversy              So why am I re-posting the same requests in this java.net forum?                  The reason is:                 Changed circumstances of today, vis a vis the circumstance at the time of            JSR 14 development and the release of Generics with Java 5.0, as well as            a couple of more arguments, as given below.     As outlined in Section 3 above, the evolution of the Java Language has         undergone two possibly avoidable hitches in its evolution, concerning         Generics:                (a). Not introducing Collections API only with Generics in the Java 1.2                release            (b). Not opting for two mutually exclusive API Libraries with Java 5.0                (one with Generics, and one without), providing for Backward                Compatibility and Migration Compatibility, but ignoring any                possibility of a versioning nightmare of APIs in presumed API                stacks in the market, treating this last problem as not affecting                too many users.     Now, the consequence of the second "oversight" above is currently underway in         Java's evolution, in the form of having to live with Erasure, with Reification         of Generic Types pushed to the background on low priority.            Also, opting for two different APIs NOW is out of the question (as the stage         for that is now past), and having only a single API for one set of features as         a result makes one of the two (MUTUALLY EXCLUSIVE by nature) options - Erasure         and Reification - edge out the other.         In the event, Erasure has edged out Reification.     Erasure was certainly the concept to opt for, at the time of Java 5.0 release,         also as seen above.     But that was in 2004, the year of release of Java 5.0.Now it is 2010, and         nearly SIX FULL YEARS have elapsed, with Java 7 release expected in 2010.     Since the idea behind adopting Erasure was Backward Compatibility (not         breaking legacy code) and Migration Compatibility (allowing users to migrate         to Generics over a time period), and since in these days of Internet Time,         SIX YEARS is pretty considerable, the time to re-assess the need to handhold         legacy code users has arrived.     If Erasure had been a pretty harmless concept, Reification could be postponed         indefinitely, without greatly affecting the migrating users.          Alas, the reality is that Erasure continues to play havoc with the Type System         of the Language.          It has forced severe restrictions in the usage of Generic Types, as seen in         Section 1.2, even as the whole idea of introducing Generics in the first place         was Type System Robustness and its attendant advantages.
             So, Reification must now be adopted, since all users have now been given
       enough time to migrate.
       Legacy code must not any longer enjoy support in future versions of Java.
       And the way to ensure this is to DEPRECATE RAW TYPES.
    Since Reification and Erasure are MUTUALLY EXCLUSIVE, introducing Reification         will mean getting rid of Erasure. This will mean that Raw Types cannot be         supported any more.          In other words, the time has really come for the two requests in Section 1.1         to get incorporated into Java, thereby at one stroke getting rid of both the         "mishaps" that the Java language has been subjected to in its evolution.          So I re-emphasize the issue by reproducing those two requests of Section 1.1         here again:

                        Remove the provision for Backward Compatibility.(See Section 2.1 above).
                        In other words:

                            1. "Deprecate" Raw Types.

                            2. Adopt "Reification" in the place of "Erasure" for Generic Types.

                               The consequence of this will render the "contrived" and "constricting" rules for
                               the usage of Generic Types in Java 5.0 (given in Section 1.2 above)
                               unnecessary.

                               As a result, all types provided by the language will behave uniformly, most
                               especially without violating OOP principles, in what is essentially an OOP Language.                   
      5.1 "NO-WIN" situation with No Reification         If the above two requirements are put on hold, consider what Java users live         with:           (a)  Legacy Code users continue to prefer NOT to distinguish a List of Elves                from a List of Goblins, which is of course, very, very bad programming                practice, in an era in which Best Practices are sought to be adopted in                most endeavors in the industry.                    In cases where Legacy Code has been careful to take care of such                distinctions, it could have been achieved only by burdening the                programmer, who must take care to code diligently and make updates in                code maintenance even more diligently.            Features added to the Java Language, JSE, and JEE in their respective                evolutions have always tried to make the programmer's task easier, not                tougher. Where possible, Java has tried to eliminate scope for human                error.                So Java should maintain this tradition and not lose it.                Not breaking Legacy Code is really a NO-WIN situation for Legacy Code                users, for the reason that it fails to bring them in line with better                programming practices. They are forced to count on human infallibility                to achieve robust programming.                     (b)  Migrating users get Generics, but with the problems outlined. Type                Parameters of Generics are really full-fledged Types in their own                right.                What is supposed to provide Type System Robustness, in fact what has                been introduced for that very purpose, is made to lose its                "raison d'être" immediately, upon its very release.                 Not being able to use the full type information for determining class                hierarchy in an OOP Language, and not being able to create the most                fundamental secondary type of them all, the Array, of Generic Types,                is, without any doubt, NO-WIN situation also for the migrating users.                In CONTINUING TO TRY TO SIMULTANEOUSLY MEET THE NEEDS OF BOTH THE LEGACY USERS         AND THE MIGRATING USERS with one and the same API for one set of features,         unfortunately there is a NO_WIN/NO_WIN situation for either category of user,         as seen above.     5.2 Reversal of "NO-WIN" situation with Reification         If the two requests for change given above are incorporated, the situation is         directly reversed:           (a) Legacy Code users, even if their code is broken, adopt practices that are               appropriate, do not depend on programmers to take care of a basic fault               in Language Evolution that has unfortunately crept in, and therefore face               a WIN situation.            (b) Migrating Users get exactly what they should expect to get: Generic Types,               after all, are meant to make life easier, and by behaving like any other               regular type, without "constricting, strange" rules constraining their               usage, they WILL make life  easier, with Reification in place. A total WIN               situation for Migrating users.         In honoring the two requests for change, at this APT JUNCTURE in Java's         Evolution, a NO-WIN/NO-WIN situation for either category of user can be         converted to a WIN/WIN situation.          6. Conclusion         Pragmatically, since even at this juncture, the Deprecation of Raw Types         cannot be done also "overnight", Sun Microsystems must obviously take steps         to give appropriate prior warning about the deprecation to all users, using         the usual channels it has used earlier for same purpose.          6.1 Realizing Reification of Generics              THE CONSTRAINTS (C1/C2/C4) IN SECTION 2.5 OF JSR 14              MUST NOW BE TREATED AS NO LONGER RELEVANT.              THE PRIORITY IN Bug ID 5098163 in Sun's Bug database              FOR REIFICATION OF GENERICS MUST BE MADE "HIGH", from "LOW".          EVEN IF THIS CANNOT BE DONE IN JAVA 7, JAVA 8 MIGHT BE A TAD LATE IN THE DAY         TO REMOVE THESE QUIRKS IN JAVA, W.R.T. GENERICS.         HENCE AN INTERMEDIATE RELEASE, SAY JAVA 7.5, ONLY FOR SETTING RIGHT THE         CONTROVERSIAL GENERICS PROBLEM, BY HONORING THE TWO SPECIFIED REQUESTS FOR         CHANGE WOULD CERTAINLY BE A STEP IN THE RIGHT DIRECTION.          6.2 Work involved in the Implementation         THE CHANGES INVOLVED IN JAVAC, THE JVM AND THE API IMPLEMENTATIONS CANNOT BE         REALLY MONUMENTAL, ONCE RAW TYPES ARE DEPRECATED. (See Below).         A few of the major code changes that would be required would be as follows:               For Generic Types, Actual Type Parameters (Instantiated Formal Type         Parameters) must get treated as full-fledged Types in their own right.         All the code for the treatment of Types already exists in the code base, for         the Base Types of Generic Types and for Non-Generic Types. Actual Type         Parameters must now get similar treatment in Type Checks, Generic Type         Inheritance, etc.              Reification replacing Erasure will of course mean that the full type info of         a Generic Type (the Base Type plus all the Type Parameters, nested to any         level) will now get carried over to the runtime, and not exist only at compile         time.         This will then mean that the Actual Type Parameters of a Parameterized Type         will also be considered in determining Type Hierarchy, that arrays of Generic         Types can now get created, and if necessary, Exceptions could also be         Parameterized Types.         All three currently existing major restrictions in the usage of Generics will         disappear, as they rightly must.                Generic Type Inheritance will additionally require that the framed rules for it         be followed, this time with the Actual Type Parameters also used in         determining Type Hierarchy.         Since Formal Type Parameters of Generic Types can also be Nested, all         new/modified code that has these types in context will be required  to make         use of Recursion to handle Type Identity at each level of nesting.         Since all types encountered in Java will now be considered part of any Type         Hierarchy that exists in context, and java.lang.Object is at the root of all         class hierarchies, any "unknown" types can also get represented by         java.lang.Object.         This will make wildcard usage, such as the type List<?> redundant.              Even so, the Wildcard (<?>) (alone) will have to be retained for         Backward Compatibility with Java <= 7.0. So, the syntax or semantics of wildcards         need not get impacted in anyway.              The above are just the major issues involved that I can think of.                I don't make a claim that they in anyway represent the complete spec in summary         for the changes requested. But as seen above, most of what is needed for the new         treatment of Actual Type Parameters ALREADY EXISTS in the code base, which is why         the effort involved will not be a massive one.          6.3 Sign Off Plea              IN THE INTEREST OF THE MULTITUDE OF JAVA USERS, AND THE RIGHT PATH IN THE         EVOLUTION OF THE JAVA LANGUAGE, SUN MICROSYSTEMS CANNOT AFFORD TO OVERLOOK THIS         AND PUT THE GENERICS ISSUE ON THE BACK BURNER.    

Referenced Java Text

  • "Java In A Nutshell" - David Flanagan - 5th Edition -O'Reilly - © March 2005
    Indian Reprint Jan 2007 - Chapter 4 - pp.160-178

back to top

Jaisimha Narahari is based in India and has 11 years of experience in business
domain software development. Contact Author

Filter Blog

By date: