In the month of June 2009, I did an initial post in the openjdk.java.net-jdk7-dev
mailing list with a request for what I feel are two very important changes
required in the Java Platform.This resulted in a thread of discussions on
the Reification of Generic Types, but lead to no conclusions.Truly, the
time is really very much ripe for Reification of Generic Types.The Java
Language and Platform Implementers should not really ignore it much longer.
1. Two Most Important Changes Required in Java 1.1 The Requests for Change 1.2 Reason for the Requests 2. Why Reification of Generics was not taken up 2.1 Design Constraints in Section 2.5 of JSR14 3. Hindsight Wisdom: Solutions that could have been 3.1 Dual APIs: Generic and Non Generic Versions 3.2 Why the Dual API solution was not adopted 3.3 Generics right from Java 1.2 4. The Finally Adopted Solution in Java 5.0 5. Why Resurrect the Controversy 5.1 "NO-WIN" situation with No Reification 5.2 Reversal of "NO-WIN" situation with Reification 6. Conclusion 6.1 Realizing Reification of Generics 6.2 Work involved in the Implementation 6.3 Sign Off Plea 1. Two Most Important Changes Required in Java In the month of June 2009, I did an initial post in the openjdk.java.net-jdk7-dev mailing list -June 2009 with a request for what I feel are two very important changes required in the Java Platform. This resulted in a thread of discussions on the Reification of Generic Types, but lead to no conclusions.Truly, the time is really very much ripe for Reification of Generic Types.The Java Language and Platform Implementers should not really ignore it much longer. 1.1 The Requests for Change My main request for changes/inclusion in Java 7 were as follows: From Java Version 7.0, going forward:
Remove the provision for Backward Compatibility(See Section 2.1 below).
In other words:
1. "Deprecate" Raw Types
2. Adopt "Reification" in the place of "Erasure" for Generic Types.
The consequence of this will render the "contrived" and "constricting" rules
for the usage of Generic Types in Java 5.0 (given in Section 1.2 below)
As a result, all types provided by the language will behave uniformly, most
especially without violating OOP principles, in what is essentially an OOP
Language. Legacy code can still make use of JDK up to Java version 6.0 for inter-operability, with support removed from Java 7.0. 1.2 Reason for the Requests The three contrived rules mentioned above introduced as restrictions in the usage of Generics in Java 5.0, as a consequence of Erasure, are as follows: (A) For the sake of determining Type Hierarchy of Generic Types, the Type Parameters are not to be considered. That is,code such as the line below is rejected by the compiler: List<Object> lo = new ArrayList<Integer> even as pure OOPS principles allow an Integer to be recognized as an Object. That is, in Java 5.0 Generics, List<Object>and List<Integer> are treated as totally unrelated types by the above rule, not only violating the purity of OOPS but also sacrificing type system robustness. (B) Any kind of Array Creation involving Generic Types,such as List<E> , List<String> , E, etc also get forbidden, on account of the fact that Arrays are "Reified" types, implying that they carry over their full type info into the Runtime, and are therefore aware of the type of their contained elements at runtime, in direct contrast to Generic Types, which, because of Erasure,discard the knowledge of the type of their contained elements (i.e.,Type Parameters) and are therefore "non-reifiable" types. [However, the expression E can be used, (without "new" used for its creation), to specify an array of ANY REGULAR TYPE, not Generic Type. This syntax therefore is not really one to be associated with Generic Types, even as E in itself represents a Generic Type whose Formal Type Parameter 'E' can only have Regular, Non-Generic Types as its Actual Type Parameter instantiations]. (C) Exceptions cannot be Parameterized Types. See Referenced Java Text 2. Why Reification of Generics was not taken up As it became evident to me in the course of the thread of discussions at the openjdk.java.net-jdk7-dev mailing list - June 2009, as well as via the contents of a couple of blogs that I was able to home in on: (Thanks to Neal Gafter, Sun MicroSystems for the blogs) Neal Gafter's Blog/2004/09 and Neal Gafter's Blog/2006/11, the summary of reasons of why my requests #1 and #2 in Section 1.1 above could not be immediately accepted is as given in Section 2.1 below: 2.1 Design Constraints in Section 2.5 of JSR14 (a) In JSR 14, which introduced Generics to Java, there were special Design Constraints in Section 2.5 that sought to provide support for "Raw Types" in the Java Collections API. Reason One for these constraints was Backward Compatibility. This meant that existing legacy code that made use of the Java Collections API without specifying Type Parameters (Type Parameters are part of the newly introduced Generics version of the API) could continue to do so. That is, legacy code could continue to treat the Collection Classes as regular types rather than as Generic Types. The classes so used are referred to as "Raw Types". Reason Two for the above constraints was Migration Compatibility. Backward Compatibility as a facility also allowed Third Party API Library providers who directly used the old Java API to continue to do so, without any changes forced on the Third Party API Library product that they delivered to the market. By extension, this made it also easy for other Third Party API Library providers who in turn used the API provided by the above "first line" of Third party API suppliers: No changes were forced, also, to their "second line" product that they delivered to the market. For any chain of such Third Party API providers that existed, delivering in effect, an "API Stack" to the market, and also for other product vendors who used APIs from this "stack" at any level, Backward Compatibility ensured that the introduction of Java Generics did not force any such suppliers of products up the chain to change their products to accommodate Generics. Migration Compatibility is the provision that ensured that those users who wanted to migrate to the new Java APIs with Generics could do so, by using the Generic classes with their Type Parameters specified, rather than as "Raw Types" (without specifying Type Parameters, as used by the chain of product suppliers mentioned above). This, seemingly, is two problems solved with one stroke. The provision of "dual" usage of one and the same Java API - one usage as "Raw Types", and the other usage as "Generic Types", for two sets of users with differing requirements. The reasons given above look perfect, as a case AGAINST "Deprecating Raw Types",my request #1 cited in Section 1.1. Because NOT DEPRECATING Raw Types allows old users to continue as before, without facing "disruptive" changes to their product, even as migrating users who want to use the new features could also do so. But alas, there are problems with this approach, as evidenced by the (by now well-documented), contrived rules in Generics Usage in Java 5.0., as given in Section 1.2 above. These rules practically restrict the proper usage of Generics by the migrating users who want to take advantage of the new features. (b) However, the issue could not be considered closed. Reification could not simply be abandoned, with Generic Types receiving "slipshod" treatment forever. So it is that one finds "Reification of Generics" under: Bug ID 5098163 in Sun's Bug database. (Thanks to Alex Buckley, Sun Microsystems for the URL). Rather than treat this as a Bug, the entry labels it "RFE", standing for "Request for Enhancement". Against "Evaluation", it is marked as being "Under discussion as a potential feature in Java SE 7." And against "Priority" it is marked "4-Low". I could not find any other clause in this Bug database entry that indicated further action taken. Nor could I find any "current status" clause, maybe on account of the assigned low priority, as seen above. 3. Hindsight Wisdom: Solutions that "could have been" 3.1 Dual APIs: Generic and Non Generic Versions In hindsight, one alternative solution that could have been adopted at the time of Java 5.0 release, with Generics, to overcome these restrictions, would have been to provide two versions of the all Java APIs into which Generics was introduced : One with Generics, and one without (the old API as it existed before introducing Generics). The APIs with Generics would then not have been required to support their usage as "Raw Types", and as a result the classes in those new APIs would have been "Reified" Types rather than types, each of whose type info was erased at runtime because of "Erasure". Java 5.0 would then have not been required to specify any restrictions in the usage of Generics for the migrating users, which is of course the way things should really be. Those users who were dependent on the old API could simply continue to use the "undisturbed" old API, without any disruptions. While this solution looks more elegant, it was anyway NOT ADOPTED. It has the drawback that Java would end up possessing a pair of APIs for performing the same function, and that might be quite a number of extra APIs. But if this solution had been adopted, the older version of each such API could have been deprecated in course of time, as and when all of the old non-migrating users could safely be deemed to have also migrated to the new API. 3.2 Why the Dual API solution was not adopted The logic behind NOT ADOPTING this solution [of two versions for the same API], as I gathered, is as follows: - Doing so would result in a versioning nightmare of APIs, as one progressed up the product/API chain mentioned in Section 2.1 (a) above, starting with two versions in the Java API, and geometrically multiplying up the "API stack", provided by Third Party vendors. - This would happen because of a permutation of API combinations possible to get delivered by vendors of API products up the stack, for other vendors dependent on their APIs, by mixing and matching Generic plus Non-Generic versions of their individual API Libraries. I am not sure if a complete survey was conducted with vendors of such nature to arrive at the above conclusions, but on the face of it the logic above looks a bit stretched, and possibly, just possibly, may not reflect ground realities accurately. The very concept of stack upon stack of Third Party API Libraries existing in the market, a higher one in the stack dependent on a lower one, seems a bit far-fetched. Even granting that such stacks do exist, the instances of such situations are bound to be more an exception rather than the rule. In any case, Java 5.0 with Generics got released with one and the same API for each API that provided Generic Types, allowing the use of the Generic Classes as "Raw Types" as well as use of the same classes as Generic Types. So it is a moot point now whether a one-API solution or a dual-API solution would have been the better one. Therefore the Design Constraints in Section 2.5 of JSR 14 used the need for Backward Compatibility and Migration Compatibility as detailed in Section 2.1 (a) above as their basis, and made support for "Raw Types" mandatory as a result. 3.3 Generics right from Java 1.2 Also in hindsight, this whole problem of Generics and Backward and Migration Compatibility, could have been completely avoided if at the time of the release of the Collection classes in Java, that occurred with Java 1.2, the API had been released only with Generics rather than without, to start with. After all, if we are talking of Lists, Queues, etc, it makes a lot of sense to be able to distinguish between a List/Queue of "foo"s and a List/Queue of "bar"s. But this is in hindsight, and again, a moot point. By the time of release of Java 5.0, every piece of code written using these Non-Generic Collection API classes had become "legacy" code that used the classes as "Raw Types", code that did not care if a List was comprised of only fairies, only ogres, or a mixture of both. This is definitely an unfortunate turn of events, as it has turned out. ALL OF THE FOREGOING POINTS THAT I HAVE MADE IS ONLY TO PUT THINGS IN PERSPECTIVE, AND TO ESTABLISH THE CONTEXT FOR THIS BLOG. THEY MUST NOT BE MISCONSTRUED AS A FAULT FINDING MISSION AGAINST SUN MICROSYSTEMS (NOW PART OF ORACLE) OR ANY OTHER STAKEHOLDER. 4. The Finally Adopted Solution in Java 5.0 To the credit of the people involved in the development and release of Java 5.0 Generics, as it came to light in the thread of discussions at the openjdk.java.net-jdk7-dev mailing list -June 2009, the experts at Sun Microsystems had tried to find the best-fit solution for simultaneously satisfying the needs of both the migrating users (those wanting to use Generics) and non-migrating users (those who wanted to stick to the old API usage) by considering as many workarounds as possible and finally settled for what actually got released as the Java 5.0 Generics APIs. The Design Constraints in Section 2.5 of JSR 14 had to be satisfied, because Backward Compatibility could not be sacrificed, overnight breaking legacy code, even if the reason behind not releasing a separate set of Generic-only APIs,for "proper" Migration Compatibility, that of a versioning nightmare in a stack of interdependent APIs, is questionable. It just so happens that, FUNDAMENTALLY, the needs of the non-migrating legacy code users (read usage of Generic Types as "Raw Types") and the needs of the migrating users (which is ideally the usage of Generic Types without twists and quirks, achievable only without their simultaneous usage as "Raw Types") are MUTUALLY EXCLUSIVE requirements, even as only a SINGLE API for a single function became available. If you satisfy the one, you cannot satisfy the other, with just one and the same API. That is, there can either be Erasure or there can be Reification, but not both together for Generic Classes of one and the same API. As result of the mutual exclusivity involved, the bias that became inevitable in arriving at a solution had to tilt towards the legacy users. Because, as mentioned above, support for millions of lines of legacy code cannot be dumped overnight. So Erasure won the day, Reification had to be discarded, but at very good cost for the migrating users. To be fair, Java 5.0 also provides a "patch up" API in the classes provided by java.util.Collections.checkedXXX classes, which try to reverse the effects of Erasure by carrying over the type info of Type Parameters of Generic Types also into the runtime, by separately providing reflection (".class") types of the Type Parameters as part of the checkedXXX classes. While this provision re-ensures type safety (to compensate for the loss of type safety due to erasure), there is no compensation concerning the three "makeshift" rules of Section 1.2 above that end up gravely restricting Java from having a very robust, elegant Type System, wherein Parameterized Types could enjoy the same status as any other type, without getting quirks and twists in their usage imposed upon them. So, in the status quo, all decisions concerning Java Generics as a feature have been explained to satisfaction as seen above, specifically the controversial one of choosing Erasure over Reification for Generic Types. Having got all the reasons outlined above, as the reasons for why my requests #1 and #2 in Section 1.1 could not be immediately accepted, from the thread of discussions ensuing from my initial post at openjdk.java.net-jdk7-dev mailing list -June 2009, and the contents of these blogs: Neal Gafter's Blog/2004/09 and Neal Gafter's Blog/2006/11, the hopes of getting the requests incorporated in Java 7 had to be abandoned. 5. Why Resurrect the Controversy So why am I re-posting the same requests in this java.net forum? The reason is: Changed circumstances of today, vis a vis the circumstance at the time of JSR 14 development and the release of Generics with Java 5.0, as well as a couple of more arguments, as given below. As outlined in Section 3 above, the evolution of the Java Language has undergone two possibly avoidable hitches in its evolution, concerning Generics: (a). Not introducing Collections API only with Generics in the Java 1.2 release (b). Not opting for two mutually exclusive API Libraries with Java 5.0 (one with Generics, and one without), providing for Backward Compatibility and Migration Compatibility, but ignoring any possibility of a versioning nightmare of APIs in presumed API stacks in the market, treating this last problem as not affecting too many users. Now, the consequence of the second "oversight" above is currently underway in Java's evolution, in the form of having to live with Erasure, with Reification of Generic Types pushed to the background on low priority. Also, opting for two different APIs NOW is out of the question (as the stage for that is now past), and having only a single API for one set of features as a result makes one of the two (MUTUALLY EXCLUSIVE by nature) options - Erasure and Reification - edge out the other. In the event, Erasure has edged out Reification. Erasure was certainly the concept to opt for, at the time of Java 5.0 release, also as seen above. But that was in 2004, the year of release of Java 5.0.Now it is 2010, and nearly SIX FULL YEARS have elapsed, with Java 7 release expected in 2010. Since the idea behind adopting Erasure was Backward Compatibility (not breaking legacy code) and Migration Compatibility (allowing users to migrate to Generics over a time period), and since in these days of Internet Time, SIX YEARS is pretty considerable, the time to re-assess the need to handhold legacy code users has arrived. If Erasure had been a pretty harmless concept, Reification could be postponed indefinitely, without greatly affecting the migrating users. Alas, the reality is that Erasure continues to play havoc with the Type System of the Language. It has forced severe restrictions in the usage of Generic Types, as seen in Section 1.2, even as the whole idea of introducing Generics in the first place was Type System Robustness and its attendant advantages.
So, Reification must now be adopted, since all users have now been given
enough time to migrate.
Legacy code must not any longer enjoy support in future versions of Java.
And the way to ensure this is to DEPRECATE RAW TYPES. Since Reification and Erasure are MUTUALLY EXCLUSIVE, introducing Reification will mean getting rid of Erasure. This will mean that Raw Types cannot be supported any more. In other words, the time has really come for the two requests in Section 1.1 to get incorporated into Java, thereby at one stroke getting rid of both the "mishaps" that the Java language has been subjected to in its evolution. So I re-emphasize the issue by reproducing those two requests of Section 1.1 here again:
Remove the provision for Backward Compatibility.(See Section 2.1 above).
In other words:
1. "Deprecate" Raw Types.
2. Adopt "Reification" in the place of "Erasure" for Generic Types.
The consequence of this will render the "contrived" and "constricting" rules for
the usage of Generic Types in Java 5.0 (given in Section 1.2 above)
As a result, all types provided by the language will behave uniformly, most
especially without violating OOP principles, in what is essentially an OOP Language. 5.1 "NO-WIN" situation with No Reification If the above two requirements are put on hold, consider what Java users live with: (a) Legacy Code users continue to prefer NOT to distinguish a List of Elves from a List of Goblins, which is of course, very, very bad programming practice, in an era in which Best Practices are sought to be adopted in most endeavors in the industry. In cases where Legacy Code has been careful to take care of such distinctions, it could have been achieved only by burdening the programmer, who must take care to code diligently and make updates in code maintenance even more diligently. Features added to the Java Language, JSE, and JEE in their respective evolutions have always tried to make the programmer's task easier, not tougher. Where possible, Java has tried to eliminate scope for human error. So Java should maintain this tradition and not lose it. Not breaking Legacy Code is really a NO-WIN situation for Legacy Code users, for the reason that it fails to bring them in line with better programming practices. They are forced to count on human infallibility to achieve robust programming. (b) Migrating users get Generics, but with the problems outlined. Type Parameters of Generics are really full-fledged Types in their own right. What is supposed to provide Type System Robustness, in fact what has been introduced for that very purpose, is made to lose its "raison d'être" immediately, upon its very release. Not being able to use the full type information for determining class hierarchy in an OOP Language, and not being able to create the most fundamental secondary type of them all, the Array, of Generic Types, is, without any doubt, NO-WIN situation also for the migrating users. In CONTINUING TO TRY TO SIMULTANEOUSLY MEET THE NEEDS OF BOTH THE LEGACY USERS AND THE MIGRATING USERS with one and the same API for one set of features, unfortunately there is a NO_WIN/NO_WIN situation for either category of user, as seen above. 5.2 Reversal of "NO-WIN" situation with Reification If the two requests for change given above are incorporated, the situation is directly reversed: (a) Legacy Code users, even if their code is broken, adopt practices that are appropriate, do not depend on programmers to take care of a basic fault in Language Evolution that has unfortunately crept in, and therefore face a WIN situation. (b) Migrating Users get exactly what they should expect to get: Generic Types, after all, are meant to make life easier, and by behaving like any other regular type, without "constricting, strange" rules constraining their usage, they WILL make life easier, with Reification in place. A total WIN situation for Migrating users. In honoring the two requests for change, at this APT JUNCTURE in Java's Evolution, a NO-WIN/NO-WIN situation for either category of user can be converted to a WIN/WIN situation. 6. Conclusion Pragmatically, since even at this juncture, the Deprecation of Raw Types cannot be done also "overnight", Sun Microsystems must obviously take steps to give appropriate prior warning about the deprecation to all users, using the usual channels it has used earlier for same purpose. 6.1 Realizing Reification of Generics THE CONSTRAINTS (C1/C2/C4) IN SECTION 2.5 OF JSR 14 MUST NOW BE TREATED AS NO LONGER RELEVANT. THE PRIORITY IN Bug ID 5098163 in Sun's Bug database FOR REIFICATION OF GENERICS MUST BE MADE "HIGH", from "LOW". EVEN IF THIS CANNOT BE DONE IN JAVA 7, JAVA 8 MIGHT BE A TAD LATE IN THE DAY TO REMOVE THESE QUIRKS IN JAVA, W.R.T. GENERICS. HENCE AN INTERMEDIATE RELEASE, SAY JAVA 7.5, ONLY FOR SETTING RIGHT THE CONTROVERSIAL GENERICS PROBLEM, BY HONORING THE TWO SPECIFIED REQUESTS FOR CHANGE WOULD CERTAINLY BE A STEP IN THE RIGHT DIRECTION. 6.2 Work involved in the Implementation THE CHANGES INVOLVED IN JAVAC, THE JVM AND THE API IMPLEMENTATIONS CANNOT BE REALLY MONUMENTAL, ONCE RAW TYPES ARE DEPRECATED. (See Below). A few of the major code changes that would be required would be as follows: For Generic Types, Actual Type Parameters (Instantiated Formal Type Parameters) must get treated as full-fledged Types in their own right. All the code for the treatment of Types already exists in the code base, for the Base Types of Generic Types and for Non-Generic Types. Actual Type Parameters must now get similar treatment in Type Checks, Generic Type Inheritance, etc. Reification replacing Erasure will of course mean that the full type info of a Generic Type (the Base Type plus all the Type Parameters, nested to any level) will now get carried over to the runtime, and not exist only at compile time. This will then mean that the Actual Type Parameters of a Parameterized Type will also be considered in determining Type Hierarchy, that arrays of Generic Types can now get created, and if necessary, Exceptions could also be Parameterized Types. All three currently existing major restrictions in the usage of Generics will disappear, as they rightly must. Generic Type Inheritance will additionally require that the framed rules for it be followed, this time with the Actual Type Parameters also used in determining Type Hierarchy. Since Formal Type Parameters of Generic Types can also be Nested, all new/modified code that has these types in context will be required to make use of Recursion to handle Type Identity at each level of nesting. Since all types encountered in Java will now be considered part of any Type Hierarchy that exists in context, and java.lang.Object is at the root of all class hierarchies, any "unknown" types can also get represented by java.lang.Object. This will make wildcard usage, such as the type List<?> redundant. Even so, the Wildcard (<?>) (alone) will have to be retained for Backward Compatibility with Java <= 7.0. So, the syntax or semantics of wildcards need not get impacted in anyway. The above are just the major issues involved that I can think of. I don't make a claim that they in anyway represent the complete spec in summary for the changes requested. But as seen above, most of what is needed for the new treatment of Actual Type Parameters ALREADY EXISTS in the code base, which is why the effort involved will not be a massive one. 6.3 Sign Off Plea IN THE INTEREST OF THE MULTITUDE OF JAVA USERS, AND THE RIGHT PATH IN THE EVOLUTION OF THE JAVA LANGUAGE, SUN MICROSYSTEMS CANNOT AFFORD TO OVERLOOK THIS AND PUT THE GENERICS ISSUE ON THE BACK BURNER.
- "Java In A Nutshell" - David Flanagan - 5th Edition -O'Reilly - © March 2005
Indian Reprint Jan 2007 - Chapter 4 - pp.160-178
Jaisimha Narahari is based in India and has 11 years of experience in business
domain software development. Contact Author