2 Ответы Последний ответ: 29.03.2016 8:35, автор: 1336415

# Limit on bit vector size

Hello,

Is there any limits on bit vector size? Using vector.so it is impossible to get more than 536870912 results.

Imagine vector filled with integers with value 42. Let's search for values between 41 and 43 in arrays of size between 500E6 and 800E6:

\$ ./bin/dax-in-range 41 43 500000000

0.422279 seconds to find numbers between 41 and 43 using DAX. There are 500000000 such numbers

\$ ./bin/dax-in-range 41 43 600000000

0.579878 seconds to find numbers between 41 and 43 using DAX. There are 536870912 such numbers

\$ ./bin/dax-in-range 41 43 700000000

0.803640 seconds to find numbers between 41 and 43 using DAX. There are 536870912 such numbers

\$ ./bin/dax-in-range 41 43 800000000

1.014522 seconds to find numbers between 41 and 43 using DAX. There are 536870912 such numbers

\$

I'm searching using following calls:

bitVec = vector_in_range(valuesVec, &low, &high);

rescount = bit_vector_count(bitVec);

Is it DAX limitation or vector.so implementation?

• ###### 1. Re: Limit on bit vector size

I tried to reproduce the problem using two methods, with the same logic. Both gives me correct result.

1. I generate 700000000 numbers in memory, load that array into dax vector. Do the filter operation as the person give.

size_t nelem = 700000000;

int *content = (int *)malloc(nelem * sizeof(int));

for(size_t i = 0; i < nelem; i++) {

content[i] = 42;

}

vector valVec = vector_load_from_array(content, nelem, INTEGER, sizeof(int));

int low = 41, high = 43;

vector bitVec = vector_in_range(valVec, &low, &high);

int rescount = getVectorBitCount(bitVec);

printf("count: %d\n", rescount);

Output:

count: 700000000

2. I generate 700000000 numbers of 42 in a singleNumber.txt file and do the same logic above:

vector valVec = vector_load_from_file("./singleNumber.txt", INTEGER, sizeof(int));

int low = 41, high = 43;

vector bitVec = vector_in_range(valVec, &low, &high);

int rescount = getVectorBitCount(bitVec);

printf("count: %d\n", rescount);

Output:

count: 700000000

Thanks for checking out DAX.

• ###### 2. Re: Limit on bit vector size

Hello,

I tried to compile your example, but there is no getVectorBitCount() in vector.h:

\$ grep -i getVectorBitCount ./include/Vector.h

\$ grep -i Count ./include/Vector.h

int vector_elem_count(vector this_vector);

int bit_vector_count(vector bit_vector);

\$

I was using bit_vector_count() to count number of resulting bits.