6 Replies Latest reply: Dec 10, 2012 1:28 PM by 958563 RSS

    PofCollection.getChild

    958563
      I have a PofCollection.
      I extract a certain no. of elements from (getChild(index)).

      Looks like the elapsed time for this, increases based on the PofCollection size even though I am extracting the same no. of elements.

      Is this working as expected or I am doing something wrong ?
        • 1. Re: PofCollection.getChild
          robvarga
          955560 wrote:
          I have a PofCollection.
          I extract a certain no. of elements from (getChild(index)).

          Looks like the elapsed time for this, increases based on the PofCollection size even though I am extracting the same no. of elements.

          Is this working as expected or I am doing something wrong ?
          It actually increases proportionally to the value of index. What happens is that a PofCollection needs to read through and skip all preceeding elements before the specified element, as POF does not store pointers to elements in the collection or object sizes for each PofValue, hence it can only figure out where an element in the collection starts if it figures out where the preceeding element ends thus it needs to read through all preceeding elements. It is still more efficient than deserializing as it does not need to actually instantiate the skipped parts.

          This is why it is recommended to put to-be-extraced data to low POF property ids and the beginning of collections (if possible, of course).

          Best regards,

          Robert
          • 2. Re: PofCollection.getChild
            958563
            Thanx Rob,

            So looks like it always starts with offset 0 And there is now way to do like a bulk skip ahead.
            • 3. Re: PofCollection.getChild
              robvarga
              955560 wrote:
              Thanx Rob,

              So looks like it always starts with offset 0 And there is now way to do like a bulk skip ahead.
              I don't think there is a way to do a bulk skip without reading through the preceeding elements. The POF format does not contain forward pointers, and all relevant data types in the POF format except byte are variable length.

              Best regards,

              Robert
              • 4. Re: PofCollection.getChild
                958563
                Thanx again Rob,

                So in my PofCollection, i have POF Objects (say n elements ) which have POF fields of their own (say m POF Fields).
                It is a uniform collection.

                The read/skip is once per member of PofCollection --(n)
                or
                once per PofCollection member field --(n * m)
                • 5. Re: PofCollection.getChild
                  robvarga
                  955560 wrote:
                  Thanx again Rob,

                  So in my PofCollection, i have POF Objects (say n elements ) which have POF fields of their own (say m POF Fields).
                  It is a uniform collection.

                  The read/skip is once per member of PofCollection --(n)
                  or
                  once per PofCollection member field --(n * m)
                  You need to pay once to figure out the start of any PofValue within its parent PofValue. Both the attributes within a PofUserType, and a collection element within a PofCollection are PofValues in a parent PofValue.

                  So you need to pay once per collection element to figure out the start of that collection element within the collection (it is then cached in the PofValue). If you have multiple collection elements to navigate into, and you ask them in increasing index order, the PofCollection may be smart enough to optimize figuring out the start position for all requested collection elements with a total cost equalling to obtaining just the highest indexed element (by starting from the highest previously requested index).

                  Then you have to pay for figuring out the start of each requested attribute within each collection element user type, as the lengths of the field depend on the contained value. Again, increasing order may help optimization.

                  Best regards,

                  Robert
                  • 6. Re: PofCollection.getChild
                    958563
                    Hi Rob,

                    I am seeing something strange,

                    In my POF Collection I have ~400k elements.

                    First time I grab the first 1000 elements, it takes much longer than say the second time (for the first 1000 elements) etc.

                    I am using EntryProcessor to access the cache member.


                    Any ideas what could be going on ?

                    Thanx,