DAX offloading isn't offloading cores?

1336415

Hello,

According to http://www.oracle.com/technetwork/server-storage/sun-sparc-enterprise/documentation/sparc-t7-m7-server-architecture-2702… "These engines can process 32 independent data streams, offloading the processor cores to do other work."

Let's try to pbind "yes > /dev/null" to all threads of single core and then launch vector_in_range() on one of the threads? One execution of vector_in_range() takes about 80ms (zone memory still not increased, case ID: 497386-1217697831), so we will run it in cycle. What we got is CPU time sharing:

   PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWP 

  5334 dglushe*  99 1.0 0.0 0.0 0.0 0.0 0.0 0.0   0  60 68K   0 yes/1

  5327 dglushe*  99 1.0 0.0 0.0 0.0 0.0 0.0 0.0   0  60 68K   0 yes/1

  5336 dglushe*  99 1.0 0.0 0.0 0.0 0.0 0.0 0.0   0  60 68K   0 yes/1

  5330 dglushe*  99 1.0 0.0 0.0 0.0 0.0 0.0 0.0   0  60 68K   0 yes/1

  5332 dglushe*  99 1.0 0.0 0.0 0.0 0.0 0.0 0.0   0  60 68K   0 yes/1

  5340 dglushe*  99 1.0 0.0 0.0 0.0 0.0 0.0 0.0   0  60 68K   0 yes/1

  5338 dglushe*  99 1.0 0.0 0.0 0.0 0.0 0.0 0.0   0  60 68K   0 yes/1

  5345 dglushe*  46  14 0.1 0.0 0.0 0.0 0.0  39   0  48 .1M   0 dax-in-range/1

  5342 dglushe*  39 0.4 0.0 0.0 0.0 0.0 0.0  61   0  49 24K   0 yes/1

Most of the user time dax-in-range spends in following stack:

libdax.so.1`dax_read_results+0x1c4

libdax_query.so.1`dax_query_execute+0x160

libdax_query.so.1`dax_scan+0xdb4

vector.so`dax_scan+0x12c

vector.so`vectorScanStream+0x1d8

vector.so`vectorFilter+0x418

vector.so`vectorInRange+0x2c

vector.so`vector_in_range+0x24

dax-in-range`main+0x1d0

dax-in-range`_start+0x108

Is it poor vector.so that uses DAX ineffectively or the offloading statement is incorrect? vector_in_range() steals time from "yes > /dev/null".

Thank you.

1336415

Oh, I see you shared dax.h. It seems that vector.so uses synchronous DAX calls, but dax.h states that there are asynchronous calls too:

/* 
    NAME: dax_post - family of functions that post asynchronous dax requests
    SYNOPSIS:
  */
..




dax_status_t dax_scan_range_post(dax_queue_t *queue, uint64_t flags,



    dax_vec_t *src, dax_vec_t *dst, dax_compare_t op, dax_int_t *lower,



    dax_int_t *upper, void *udata);

No more questions, thank you!

Yoda-Oracle

Hi,

Your Zone now has 16GB of RAM. Please let me know if that is sufficient

-Angelo

1336415

Thank you. Now memory limit is sufficient.

But CPU offloading still shows that while DAX doing its work - CPU core busy too.

With new memory limit vector_in_range() takes 0,8 seconds to complete. Let's run it in cycle on the same thread id with "yes > /dev/null" (using pbind). Again, we see that yes shares CPU time with dax-in-range:

   PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWP  

19860 dglushe* 99 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0  30 34K   0 yes/1

19868 dglushe* 99 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0  30 34K   0 yes/1

19864 dglushe* 99 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0  30 34K   0 yes/1

19866 dglushe* 99 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0  30 34K   0 yes/1

19857 dglushe* 99 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0  30 34K   0 yes/1

19862 dglushe* 99 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0  30 34K   0 yes/1

19870 dglushe* 99 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0  30 34K   0 yes/1

19880 dglushe* 50  15 0.1 0.0 0.0 0.0 0.0  35   1  23 .1M 0 dax-in-range/1

19850 dglushe*  34 0.3 0.0 0.0 0.0 0.0 0.0  65 0  24 10K   0 yes/1

1336415

Oh, I see you shared dax.h. It seems that vector.so uses synchronous DAX calls, but dax.h states that there are asynchronous calls too:

/* 
    NAME: dax_post - family of functions that post asynchronous dax requests
    SYNOPSIS:
  */
..




dax_status_t dax_scan_range_post(dax_queue_t *queue, uint64_t flags,



    dax_vec_t *src, dax_vec_t *dst, dax_compare_t op, dax_int_t *lower,



    dax_int_t *upper, void *udata);

No more questions, thank you!