7 Replies Latest reply: Mar 10, 2008 3:07 AM by 627183 RSS

    Visited Objects Optimization

    627183
      Hello.

      I have an optimization suggestion. I have detected in je-3.2.68 that making persistent a @Entity object related to many @Persistent objects takes too much time. I can see profiling the code that there is a loop in VisitedObjects.getOffset that takes many CPU time. I have recoded the implementacion to use a hashcode and the time consumed reduces from 120s to 2s with my testing objects.

      I have splitted VisitedObjects into two classes, one for input and the other for output. One uses a hashmap for offsets and the other for objects instead of the array, so the memory usage should be more or less the same.

      I wonder if you agree with the changes. Parhaps you want to apply the patch for future releases. Please let me know.

      Kind regards.
        • 1. VisitedObjectsForInput
          627183
          package com.sleepycat.persist.impl;

          import java.util.HashMap;

          class VisitedObjectsForInput {

          private static final int INIT_LEN = 50;
          private HashMap<Integer,Object> ioMap;

          VisitedObjectsForInput() {
          ioMap = new HashMap<Integer,Object>(INIT_LEN);
          }

          void put(int offset, Object o) {
          ioMap.put(new Integer(offset), o);
          }

          Object getObject(int offset) {
          Object o = ioMap.get(new Integer(offset));
          return (o == null) ? null : o;
          }
          }
          • 2. Re: VisitedObjectsForOutput
            627183
            package com.sleepycat.persist.impl;

            import java.util.IdentityHashMap;

            class VisitedObjectsForOutput {

            private static final int INIT_LEN = 50;
            private IdentityHashMap<Object,Integer> oiMap;

            VisitedObjectsForOutput() {
            oiMap = new IdentityHashMap<Object,Integer>(INIT_LEN);
            }

            void put(Object o, int offset) {
            oiMap.put(o, new Integer(offset));
            }

            int getOffset(Object o) {
            Integer i = oiMap.get(o);
            return (i == null) ? -1 : i.intValue();
            }
            }
            • 3. RecordInput
              627183
              25c25
              < private VisitedObjects visited;
              ---
              private VisitedObjectsForInput visited;
              145c145
              < visited = new VisitedObjects();
              ---
              visited = new VisitedObjectsForInput();
              147,148c147,149
              < int visitedIndex =
              < visited.add(VisitedObjects.PROHIBIT_REF_OBJECT, visitedOffset);
              ---
              // int visitedIndex =
              // visited.add(VisitedObjects.PROHIBIT_REF_OBJECT, visitedOffset);
              visited.put(visitedOffset, VisitedObjects.PROHIBIT_REF_OBJECT);
              160c161,162
              < visited.setObject(visitedIndex, o);
              ---
              // visited.setObject(visitedIndex, o);
              visited.put(visitedOffset, o);
              168c170,171
              < visited.replaceObject(o, o2);
              ---
              // visited.replaceObject(o, o2);
              visited.put(visitedOffset, o2);
              224c227
              < visited = new VisitedObjects();
              ---
              visited = new VisitedObjectsForInput();
              226c229,230
              < visited.add(o, VisitedObjects.PRI_KEY_VISITED_OFFSET);
              ---
              // visited.add(o, VisitedObjects.PRI_KEY_VISITED_OFFSET);
              visited.put(VisitedObjects.PRI_KEY_VISITED_OFFSET, o);
              • 4. RecordOutput
                627183
                25c25
                < private VisitedObjects visited;
                ---
                private VisitedObjectsForOutput visited;
                93c93
                < visited = new VisitedObjects();
                ---
                visited = new VisitedObjectsForOutput();
                97c97,98
                < int visitedIndex = visited.add(o, prohibitNestedRefs ?
                ---
                // int visitedIndex = visited.add(o, prohibitNestedRefs ?
                visited.put(o, prohibitNestedRefs ?
                107c108,109
                < visited.setOffset(visitedIndex, visitedOffset);
                ---
                // visited.setOffset(visitedIndex, visitedOffset);
                visited.put(o, visitedOffset);
                156c158
                < visited = new VisitedObjects();
                ---
                visited = new VisitedObjectsForOutput();
                158c160,161
                < visited.add(o, VisitedObjects.PRI_KEY_VISITED_OFFSET);
                ---
                // visited.add(o, VisitedObjects.PRI_KEY_VISITED_OFFSET);
                visited.put(o, VisitedObjects.PRI_KEY_VISITED_OFFSET);
                • 5. Comments
                  627183
                  Just some comments.

                  I have tested the results of getOffset and getObject in some testing objects and they seem to return the same values as the original ones. But I feel not so much confident in the input case because the line:

                  visited.put(VisitedObjects.PRI_KEY_VISITED_OFFSET, o);

                  It seems to be a little useless.

                  Also with the replacing line:
                  // visited.replaceObject(o, o2);
                  visited.put(visitedOffset, o2);
                  I am assuming that the replaced object has the same offset.

                  Please let me know if I am right or wrong.

                  King regards.
                  • 6. Re: Visited Objects Optimization
                    Greybird-Oracle
                    Hi,

                    Thank you very much for suggesting this optimization and especially for measuring the results! It will take us a few days to take a look at this and reply in detail.

                    In the test you ran (where the time went from 120s to 2s) how many embedded persistent objects were stored in the entity?

                    Thanks again,
                    --mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
                    • 7. Test size
                      627183
                      Hi Mark.

                      I make persistent just one entity with two major fields:

                      1. List<Line>
                      2. Map<String,Map<Integer,String[]>>

                      The first one has a size of 5021.
                      The second one has a size of 2+4+1+107+3+1+1+1479+1480+8+1+1+1+505+383+1+362+1049+1+3+383+1466+1013+369+3+2+1+144+162+1+1+1 = 48477.

                      Best.