1 2 Previous Next 17 Replies Latest reply: Apr 3, 2014 3:39 PM by jschellSomeoneStoleMyAlias RSS

    Warning, don't use Object == Object (in Java)

    DavidThi808

      In Java the following code can return false:

       

      IntegerPlus o1 = new IntegerPlus(1000);

      IntegerPlus o2 = o1;

      boolean b1 = o1 == o2;

      boolean b2 = o1.equals (o2);

       

      b1 & b2 will usually be true, but can be false. Yes, you are comparing two references to the same object and the equality test can fail. What I think is happening (the companies creating JVMs say virtually nothing about how the JVMs are implemented) is as follows:

       

      o1 & o2 are references to an object. That object can be moved in memory when the garbage collector runs. The == uses System.identityHashCode() to determine if the objects are identical. I believe this is the address of the object in memory at the time the object is created. So o1 is created & identiyHashCode is assigned to it, the object is moved, then it is assigned to a new reference o2 which then makes its own call to identityHashCode and has a different number.

       

      REMOVEDThe bottom line is - don't use == on objects.

       

      Update: I changed Equals() to equals() - sometime auto-capitalization is not your friend.

        • 1. Re: Warning, don't use Object == Object (in Java)
          TPD-Opitz

          Thanks for your post.

          It can't be stressed enough that comparing objects with == doesn't reliably work.

           

          But this

          I believe this is the address of the object in memory at the time the object is created.

          is simply wrong.

          A reference in java has nothing to do with the objects address in memory (neither its hashcode has...).

           

          You should do some more investigation on Java specs before propagating such incorrect statements.

           

          bye

          TPD

          • 2. Re: Warning, don't use Object == Object (in Java)
            baftos

            > The == uses System.identityHashCode() to determine if the objects are identical.

            Any reference? Assume you have two pointers in native code. Why would the VM prefer (p1->identityHashCode() == p2->identityHashCode()) over the simple, fast and reliable (p1 == p2)?

             

            > I believe this is the address of the object in memory at the time the object is created.

            A 32 bit integer cannot contain the address of the object in the memory of my 64 bit machine.

            • 3. Re: Warning, don't use Object == Object (in Java)
              rp0428
              In Java the following code can return false:

               

              IntegerPlus o1 = new IntegerPlus(1000);

              IntegerPlus o2 = o1;

              boolean b1 = o1 == o2;

              boolean b2 = o1.Equals (o2);

               

              b1 & b2 will usually be true, but can be false. Yes, you are comparing two references to the same object and the equality test can fail.

              First - IntegerPlus is NOT a standard Java class in any Java version I can find.

               

              So there is no way we can evaluate WHAT your custom code and methods should, or should not, be doing.

               

              You are also using a custom method named 'Equals'. There is NO such method in Java; the Java method name is 'equals' (all lowercase) as shown in the Java API for class Object.

              http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#equals(java.lang.Object)

               

              The use of a custom class and a custom method makes it rather difficult for anyone other than you to know just what your code is even doing. Your 'Equals' method may be doing something entirely unrelated to what the Java 'equals' method does and unrelated to what the '==' operator does.

               

              You should submit a bug to the jvm vendor if you have a use case where  'b1' in your example above does NOT evaluate to true. See the '==' operator definition and contract in The Java Language Specification (JLS):

              http://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.21.3

              15.21.3. Reference Equality Operators == and !=


              If the operands of an equality operator are both of either reference type or the null type, then the operation is object equality.


              It is a compile-time error if it is impossible to convert the type of either operand to the type of the other by a casting conversion (§5.5). The run-time values of the two operands would necessarily be unequal.


              At run time, the result of == is true if the operand values are both null or both refer to the same object or array; otherwise, the result is false.

              What I think is happening (the companies creating JVMs say virtually nothing about how the JVMs are implemented) is as follows:

               

              o1 & o2 are references to an object. That object can be moved in memory when the garbage collector runs. The == uses System.identityHashCode() to determine if the objects are identical. I believe this is the address of the object in memory at the time the object is created. So o1 is created & identiyHashCode is assigned to it, the object is moved, then it is assigned to a new reference o2 which then makes its own call to identityHashCode and has a different number.

              Again - if that WAS what is happening then the JVM has a bug: that violates contract required in the language for the equality operator. As others have stated the identity of an object can NOT be dependent on its physical location for the simple reason you stated: objects can be moved at will by a garbage collector.

              REMOVED. The bottom line is - don't use == on objects.

              Unfortunately that 'article' is short on fact and long on opinion and conjecture. You might want to consider having future articles reviewed technically.

              The == uses System.identityHashCode() to determine if the objects are identical.

              Provide your supporting references for that statement. Although the 'contract' for the '==' operator is defined by the JLS the actual implementation of that contract is left to the implementor's discretion.

               

              And 'identityHashCode' is nothing more than a reference to the object's base 'hashCode' method as shown in the API

              http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#identityHashCode(java.lang.Object)

              public static int identityHashCode(Object x)

              Returns the same hash code for the given object as would be returned by the default method hashCode(), whether or not the given object's class overrides hashCode(). The hash code for the null reference is zero.

              And this entire paragraph seems to be nothing more than opinion and conjecture:

              If you’re placing an object in a collection, I believe it can be even more volatile as the JVM stores objects differently (using less memory per object) in a collection to reduce memory usage. And collections are an obvious choice for relocation by the garbage collector as they tend to use a lot of memory. So placement in an array can make things even more worrisome.

              What is that 'can be even more volatile' belief based on? Post any supporting references.

               

              A collection of 'objects' is just a collection of 'object references'. What relationship do you think exists between the physical location of a 'collection' and the physical location of the 'objects' that are in that collection?

               

              Ever been to an old-fashioned library? Ever use the card catalog? That catalog is a 'collection' of book 'references'. Each entry has the actual location of a book. You can relocate the card catalog to be closer to the door or across the room but that won't change the location of the books being referenced.

               

              Surely you don't think that relocating a 'collection' involves moving ALL of the elements do you?

               

              You may find it of interest to reread the API for the Object class methods 'hashCode' and 'equals'.

              http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#hashCode()

              Returns a hash code value for the object. This method is supported for the benefit of hash tables such as those provided by HashMap.

              The general contract of hashCode is:

              • Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consistently return the same integer, provided no information used in equals comparisons on the object is modified. This integer need not remain consistent from one execution of an application to another execution of the same application.
              • If two objects are equal according to the equals(Object) method, then calling the hashCode method on each of the two objects must produce the same integer result.
              • It is not required that if two objects are unequal according to the equals(java.lang.Object) method, then calling the hashCode method on each of the two objects must produce distinct integer results. However, the programmer should be aware that producing distinct integer results for unequal objects may improve the performance of hash tables.

              As much as is reasonably practical, the hashCode method defined by class Object does return distinct integers for distinct objects. (This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.)

              Note that third bullet - two objects that are NOT equal can have the exact same hashCode value.

              • 4. Re: Warning, don't use Object == Object (in Java)
                DavidThi808

                I think this replies to all the points raised above:

                1. I do not know how the JVM handles ==. I looked a lot and could not find anything that provides this specific info. If any of you are aware of this info, please post a link. But without this info, all that remains is educated guesses.
                2. IntegerPlus is a generic class I created for the purpose of the article. I'm sorry if it wasn't clear that this is not a standard Java class.
                3. Sorry yes, equals, not Equals. It must have auto-capitalized it in my blog when I typed it. And I then copied that for the post above.
                4. For my article, and I think I was clear in it, I did not write an equals() override. I stated that if you override equals() then this is not an issue.
                5. Good point that it can't use an int for the object location on a 64-bit system. So it may (probably) be using the object address, but that is not the identityHashCode.
                6. We did submit this bug to IBM about 5 years ago. They acknowledged the bug and suggested we not use == as a workaround. AFAIK, they never fixed it.
                7. We did not submit it to Oracle when we hit it recently on the latest Java release because it occurs on one of our customers systems, running a very specific load test. Oracle is not set up to accept bug reports where the only way to reproduce it is to say they have to instrument a specific system at another location.
                8. I agree that the language spec says that == will work. That doesn't mean it does 100% of the time.

                Keep in mind that this code of ours runs millions of times without hitting this issue. So it's a bug in the JVM that is very very rarely hit. Which makes sense as if it was more commonly hit, it would have been fixed by now.

                • 5. Re: Warning, don't use Object == Object (in Java)
                  PhHein

                  DavidThi808 : stop promoting your blog, please. If you do it again your account can be locked.

                  • 6. Re: Warning, don't use Object == Object (in Java)
                    DavidThi808

                    I was not promoting my blog - I was trying to keep what I posted here to just the main part as most people don't want to read all the background. And for the full background yes it was my blog, but where else would I put the background info?

                     

                    Can you point me to the rules that list what is ok for linking to background detail and what is not? Can I post to StackOverflow posts I wrote that provide additional information? Can I post to ones co-workers wrote? Or is the rule here to not have links (which I find very useful in many questions)?

                     

                    ps - I've been posting in the Sun forums for years and this has never been an issue before (my profile appears to have lost all of my posting history but I'm not a newbie to the Java forums - I've been coding in Java since 1.1).

                    • 7. Re: Warning, don't use Object == Object (in Java)
                      rp0428
                      I think this replies to all the points raised above:

                      You have to remember that we can only respond based on what you post.

                       

                      I do not know how the JVM handles ==. I looked a lot and could not find anything that provides this specific info. If any of you are aware of this info, please post a link. But without this info, all that remains is educated guesses.

                      I agree - but that is NOT what you stated in your post. You didn't say you 'do not know' you said this:

                      The == uses System.identityHashCode() to determine if the objects are identical.

                      That is stated as if it were fact when any actual implementation will be vendor-specific.

                      IntegerPlus is a generic class I created for the purpose of the article. I'm sorry if it wasn't clear that this is not a standard Java class. 

                      The issue that arises when you use custom classes is no one has any idea what functionality has been implemented in that custom class. The API isn't published and there is NO info about what, if any, methods have been overridden or what 'hidden' functionality might be in the constructors.

                       

                      There is also likely to be an assumption that a custom class was used because a standard class won't reproduce the problem.

                      If the Integer class had been used all of that info would have been available.

                      Sorry yes, equals, not Equals. It must have auto-capitalized it in my blog when I typed it. And I then copied that for the post above. 

                      For my article, and I think I was clear in it, I did not write an equals() override. I stated that if you override equals() then this is not an issue.

                      Would it surprise you to know that the 'equals' method in the Object class does NOTHING but use '=='? See the source code:

                          public boolean equals(Object obj) {

                      return (this == obj);

                      Which comes full circle that if one doesn't know how the JVM handles '==' then you also can't know how the JVM handles the default 'equals'.

                       

                      Good point that it can't use an int for the object location on a 64-bit system. So it may (probably) be using the object address, but that is not the identityHashCode. 

                      We did submit this bug to IBM about 5 years ago. They acknowledged the bug and suggested we not use == as a workaround. AFAIK, they never fixed it. 

                       

                      We did not submit it to Oracle when we hit it recently on the latest Java release because it occurs on one of our customers systems, running a very specific load test. Oracle is not set up to accept bug reports where the only way to reproduce it is to say they have to instrument a specific system at another location. 

                       

                      I agree that the language spec says that == will work. That doesn't mean it does 100% of the time.

                      ALL of that is what I said above.

                       

                      You found a bug. You now say that five years ago you KNEW it was a but and even reported it as a bug.

                       

                      But you still created both a forum thread and a blog article ala 'Chicken Little':

                       

                      Warning, don't use Object == Object (in Java)

                       

                      That gives the impression that you have discovered some odd, inner workings of Java when it is just a rarely occurring bug. The bug may even be in the specific vendor's JVM that you are using and may not be occuring in other JVMs at all.

                       

                      There is nothing wrong with presenting a use case that shows a problem/issue and how you were able to diagnose the problem and implement a workaround.

                       

                      The problem is misrepresenting the facts and misleading people into thinking that there is something inherently wrong with 'Object == Object' when there isn't and you KNEW it was just a bug.

                       

                      That doesn't do a lot for your credibility. Thanks for posting though - I hope others have learned something from the experience.

                      • 8. Re: Warning, don't use Object == Object (in Java)
                        DavidThi808

                        Hi;

                         

                        The reason I posted this was because this bug hit us again recently and I figured since it is still there I should let others know. Would it have been better to not say anything and then others would hit the same problem under heavy load? It's a really hard bug to find because you don't expect == to be the problem. It was only because of our earlier experience that I turned to seeing if this was the problem after only an hour or so of looking at other possibilities. The first time took us a couple of weeks.

                         

                        If this helps others avoid this problem, then the post is worth it. And as long as this bug exists, yes there is something inherently wrong with Object == Object.

                        • 9. Re: Warning, don't use Object == Object (in Java)
                          TPD-Opitz

                          DavidThi808 wrote:

                          The reason I posted this was because this bug hit us

                          Once more: This is not a bug (in the JVM)!

                          It is your misunderstanding of equality.

                          == is true if the references on both sides of this operator point to the same object.

                          Whereas two different objects are never the same but can be equal (as defined by the classes equals() method).

                           

                          When working with String objects comparing with == works in many cases because the JVM does a some effort to reduce the number of String objects hanging around.  But you cannot rely on it!

                           

                          bye

                          TPD

                          • 10. Re: Warning, don't use Object == Object (in Java)
                            baftos

                            Anyway, I decided to figure out somehow what object references are, at least in the Oracle VM. This VM option is a hint:

                              UseCompressedOops. Enables the use of compressed pointers (object references represented as 32 bit offsets instead of 64-bit pointers) for optimized 64-bit performance with Java heap sizes less than 32gb.

                            (from Java HotSpot VM Options).

                            Looks like they are usually pointers or derived from pointers and not hash codes (limited to int size) of any kind.

                            Then I de-assemled an int comparaison and a reference comparison. The opcodes are different: if_acmpne (a=address?) and if_icmpne(i=int). The JVM spec defines if_acmpne as:

                            jump if two object references are not equal

                            I assume this usually means the 2 pointers are not equal.

                            All this does not mean that there is no bug in such and such VM. It just clarifies, I think, the fact that identityHashCode() is not involved. An obvious bug would be to allow if_acmpne to be interrupted by garbage collection. I don't think for a moment this is the case, but maybe a more insidious bug does exist.

                            • 11. Re: Warning, don't use Object == Object (in Java)
                              DavidThi808

                              baftos - thank you. Really good points and I think you're right that identityHashCode is not involved. But could it be that if:

                               

                              IntegerPlus o1 = new IntegerPlus(1000);

                              // memory management moves the o1 object in memory

                              IntegerPlus o2 = o1;

                              // memory management moves the o1 object in memory again

                              boolean b1 = o1 == o2;

                               

                              That one of the above memory management moves causes the pointers to no longer be equal? And o1/o2 are not direct pointers to the object. They can't be because then the memory manager could not move objects.

                               

                              TPD - Am I missing something? When the code is:

                              ntegerPlus o1 = new IntegerPlus(1000);

                              IntegerPlus o2 = o1;

                               

                              Aren't o1 & o2 pointing to the same object? How would that be considered different objects?

                               

                              thanks - dave

                              • 12. Re: Warning, don't use Object == Object (in Java)
                                rp0428

                                Looks like they are usually pointers or derived from pointers and not hash codes (limited to int size) of any kind.

                                Then I de-assemled an int comparaison and a reference comparison. The opcodes are different: if_acmpne (a=address?) and if_icmpne(i=int). The JVM spec defines if_acmpne as:

                                jump if two object references are not equal

                                I assume this usually means the 2 pointers are not equal.

                                It seems to me that you and OP are both missing the point.

                                 

                                Whether the 'reference values' used are 'pointers' or not isn't really the issue.

                                 

                                The only issue that matters is what those 'pointers' really point to. And, as I indicated before, that is going to be both vendor and implementation specific.

                                 

                                That is, even JVMs produced by the same vendor a 'pointer' may point to something totally different that represents a particular object.

                                 

                                The Java Virtual Machine Spec even says just that in so many words. See section 2.7 Representation of Objects

                                http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-2.html#jvms-2.7

                                The Java Virtual Machine does not mandate any particular internal structure for objects.

                                 

                                In some of Oracle’s implementations of the Java Virtual Machine, a reference to a class instance is a pointer to a handle that is itself a pair of pointers: one to a table containing the methods of the object and a pointer to the Class object that represents the type of the object, and the other to the memory allocated from the heap for the object data.

                                Read that second paragraph very carefully: 'a reference to a class instance is a pointer to a handle that is itself a pair of pointers'. That pretty explicitly says that, for those JVMs, a 'reference' (which is what would be used by that 'if_acmpne' instruction) is NOT a pointer to the memory on the heap for an object instance.

                                 

                                It is 'a pointer to a handle that is itself a pair of pointers'. That 'pair of pointers' could be anything that the vendor, in this case Oracle, wants it to be.

                                 

                                The value of that second 'pointer' could very well change because the object is physically moved in memory. But that does NOT suggest, imply, or mean that the actual 'reference pointer' itself either needs to change in value or will change in value.

                                 

                                I would hypothesize that one of the reasons Oracle adopted that 'pointer to pointers' approach was actually to either deal with or prevent exactly the type of bug OP is talking about. Oracle's approach breaks that dependency between an object reference and the actual object location. It is the logical equivalence of a 'surrogate key' in the Data Modeling nomenclature.

                                 

                                If other JVM vendors don't use a similar approach they could easily cause that bug being discussed.

                                • 13. Re: Warning, don't use Object == Object (in Java)
                                  rp0428
                                  That one of the above memory management moves causes the pointers to no longer be equal? And o1/o2 are not direct pointers to the object. They can't be because then the memory manager could not move objects.

                                  Simply NOT true!

                                   

                                  You are letting your opinions and/or biases cloud your judgement.

                                   

                                  See my reply to baftos above where I provide a quote from the JVM spec that explicitly shows how they do what you just said can't be done.

                                   

                                  The inventors of the Java Virtual Machine left implementation details out of the specification and at the discretion of the implementor.

                                   

                                  The JVM Spec EXPLICITLY states that. See Chapter 2. The Structure of The Java Virtual Machine

                                  http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-2.html#jvms-2.7

                                  Chapter 2. The Structure of the Java Virtual Machine

                                  This document specifies an abstract machine. It does not describe any particular implementation of the Java Virtual Machine.
                                  To implement the Java Virtual Machine correctly, you need only be able to read the class file format and correctly perform the operations specified therein. Implementation details that are not part of the Java Virtual Machine's specification would unnecessarily constrain the creativity of implementors. For example, the memory layout of run-time data areas, the garbage-collection algorithm used, and any internal optimization of the Java Virtual Machine instructions (for example, translating them into machine code) are left to the discretion of the implementor.

                                  .

                                  • 14. Re: Warning, don't use Object == Object (in Java)
                                    TPD-Opitz

                                    DavidThi808 wrote:

                                     

                                    TPD - Am I missing something? When the code is:

                                    ntegerPlus o1 = new IntegerPlus(1000);

                                    IntegerPlus o2 = o1;

                                     

                                    Aren't o1 & o2 pointing to the same object? How would that be considered different objects?

                                    OK, I missed that one.

                                     

                                    But on the other hand:

                                    When I use == for comparing Object it is in unit tests to ensure that a mock is correctly passed to another mock by the production code and I never faced a false negative so far. Meight by because I'm a lucky guy... ;o)

                                     

                                    bye

                                    TPD

                                    1 2 Previous Next