This content has been marked as final. Show 6 replies
955560 wrote:It actually increases proportionally to the value of index. What happens is that a PofCollection needs to read through and skip all preceeding elements before the specified element, as POF does not store pointers to elements in the collection or object sizes for each PofValue, hence it can only figure out where an element in the collection starts if it figures out where the preceeding element ends thus it needs to read through all preceeding elements. It is still more efficient than deserializing as it does not need to actually instantiate the skipped parts.
I have a PofCollection.
I extract a certain no. of elements from (getChild(index)).
Looks like the elapsed time for this, increases based on the PofCollection size even though I am extracting the same no. of elements.
Is this working as expected or I am doing something wrong ?
This is why it is recommended to put to-be-extraced data to low POF property ids and the beginning of collections (if possible, of course).
955560 wrote:I don't think there is a way to do a bulk skip without reading through the preceeding elements. The POF format does not contain forward pointers, and all relevant data types in the POF format except byte are variable length.
So looks like it always starts with offset 0 And there is now way to do like a bulk skip ahead.
955560 wrote:You need to pay once to figure out the start of any PofValue within its parent PofValue. Both the attributes within a PofUserType, and a collection element within a PofCollection are PofValues in a parent PofValue.
Thanx again Rob,
So in my PofCollection, i have POF Objects (say n elements ) which have POF fields of their own (say m POF Fields).
It is a uniform collection.
The read/skip is once per member of PofCollection --(n)
once per PofCollection member field --(n * m)
So you need to pay once per collection element to figure out the start of that collection element within the collection (it is then cached in the PofValue). If you have multiple collection elements to navigate into, and you ask them in increasing index order, the PofCollection may be smart enough to optimize figuring out the start position for all requested collection elements with a total cost equalling to obtaining just the highest indexed element (by starting from the highest previously requested index).
Then you have to pay for figuring out the start of each requested attribute within each collection element user type, as the lengths of the field depend on the contained value. Again, increasing order may help optimization.
I am seeing something strange,
In my POF Collection I have ~400k elements.
First time I grab the first 1000 elements, it takes much longer than say the second time (for the first 1000 elements) etc.
I am using EntryProcessor to access the cache member.
Any ideas what could be going on ?