Forum Stats

  • 3,770,683 Users
  • 2,253,153 Discussions
  • 7,875,539 Comments

Discussions

Asynchronous execution example

1336415
1336415 Member Posts: 12

Hello!

I have few questions:

- As far as I understand the DAX - caller process can offload some job to DAX while doing other things without multithreading. Is there an examples of such anychronous behavior?

- For the process, how the time DAX consumed counted? Is it counted as user time, system time or idle?

- What is memory limit for file with numbers? For big files I'm getting "(load_file) Cannot Allocate Memory for the input file".

Thank you!

Best Answer

  • Sarkar-Oracle
    Sarkar-Oracle Member Posts: 6
    edited Mar 16, 2016 7:25PM Accepted Answer

    > As far as I understand the DAX - caller process can offload some job to DAX while doing other things without multithreading. Is there an examples of such anychronous behavior?

    One can do that using low level library that comes with Solaris, not using the higher level vector library, which hides lot of low level details from developer. So far, we have not seen any need for application developers to write assynchronous programs. A DAX operation to scan one million numbers take 1 milli second. If you want to really do some thing in parallel, which do not need the result of DAX operation, why not use another thread to do that ? Note that CPU does not remain idle when DAX is doing its work, it may run another thread during that time anyway.

    >  What is memory limit for file with numbers? For big files I'm getting "(load_file) Cannot Allocate Memory for the input file".

    We have tested 10 GB files. How big is your input file ? One reason may be that there are not enough 4 MB pages due to other processes or some how memory has become too fragmented. How big file did you try ? How much memory your system had ?

Answers

  • Sarkar-Oracle
    Sarkar-Oracle Member Posts: 6
    edited Mar 16, 2016 7:25PM Accepted Answer

    > As far as I understand the DAX - caller process can offload some job to DAX while doing other things without multithreading. Is there an examples of such anychronous behavior?

    One can do that using low level library that comes with Solaris, not using the higher level vector library, which hides lot of low level details from developer. So far, we have not seen any need for application developers to write assynchronous programs. A DAX operation to scan one million numbers take 1 milli second. If you want to really do some thing in parallel, which do not need the result of DAX operation, why not use another thread to do that ? Note that CPU does not remain idle when DAX is doing its work, it may run another thread during that time anyway.

    >  What is memory limit for file with numbers? For big files I'm getting "(load_file) Cannot Allocate Memory for the input file".

    We have tested 10 GB files. How big is your input file ? One reason may be that there are not enough 4 MB pages due to other processes or some how memory has become too fragmented. How big file did you try ? How much memory your system had ?

  • 1336415
    1336415 Member Posts: 12
    edited Mar 17, 2016 10:08AM

    I agree that asynchronous operation is not needed in this case. Thank you.

    Regarding OOM issue - I was playing with Python's FindTopN and file 233MB in size. Interesting thing that from time to time OOM not occurs and all works fine.

    According to vmstat my zone has about 800GB free memory.

    prctl not showing any constraints which may cause out of memory errors. But after restarting nscd (which was about 230MB in RSS) OOM error gone. So, it looks like zone constraint. Can it be increased to 10GB please?

  • Vtatkar-Oracle
    Vtatkar-Oracle Member Posts: 7
    edited Mar 18, 2016 8:17PM

    Sure, we can do that (increase the memory to 10GB). Can you please send email using the Contact Us button on SWiSdev.oracle.com/DAX so we know who you are are as the user? [We can guess, but thats not very helpful, is it? :-) ]

    Thanx

  • Vtatkar-Oracle
    Vtatkar-Oracle Member Posts: 7
    edited Mar 23, 2016 6:45PM

    Per another reply, your memory has been bumped to 16GB. Hope that is now sufficient for you?

    Thanx

  • 1336415
    1336415 Member Posts: 12
    edited Mar 24, 2016 4:07AM

    Yes, thanks a lot!