This discussion is archived
1 2 Previous Next 24 Replies Latest reply: Sep 25, 2012 5:10 AM by rukbat RSS

/dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?

801904 Newbie
Currently Being Moderated
Hello

We are building a server to run Oracle 11g R2 database (11.2.0.3 x64) on Oracle Linux 6.2 with UEK R2.

Our preference is to use AMM to have Oracle 11g R2 manage memory. We may impose some minium SGA and PGA memory allocations but basically aim to use MEMORY_TARGET to manage overall memory.

By default Linux makes the size of /dev/shm ~50% of server physical RAM, as far as I can tell.

Here is the /etc/fstab entry created by the installation:

tmpfs /dev/shm tmpfs defaults 0 0

Given this Linux server will only run Oracle 11g R2 database and some monitoring software, almost application code will run on the server. The application code will run on the separate application server and is Java based.

Can I change the */etc/fstab* entry for /dev/shm to manually increase the size to take up ~80-90% of the server physical RAM ? Is it a good idea?

The server is 64-bit, the RAM = 64 GB, so I am thinking to manually make /dev/shm to be = ~55GB, leaving ~8GB for other purposes.
Right now it's about 32GB (50%?) if I leave the /dev/shm 'defaults' on.

many thanks
  • 1. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Dude! Guru
    Currently Being Moderated
    Physical memory for /dev/shm is only allocated when required and will be swapped to disk when necessary. If the system uses too much memory for /dev/shm then memory swapping will occur, which will have a negative impact on your Oracle database and system performance. The maximum size will therefore depend on your total amount of physical memory and particular system usage. I'm not aware of a recommendation. The default value of 50 % is probably based on the assumption that not many applications rely on it.

    How large will be your Oracle SGA?

    Oracle AMM will be easier to manage and is default, but if you plan to use any Oracle instance with more than 4 GB of RAM you should consider using kernel hugepages. Hugepages will divide your memory in 2 MB pages as opposed to the default 4 KB and thereby greatly reduce the memory required to manage your memory page table and increase performance. It will also guarantee, that SGA memory will not be swapped. Unfortunately however, kernel hugepages is not compatible with AMM.
  • 2. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    801904 Newbie
    Currently Being Moderated
    hello

    thanks for the reply.

    re "how large will SGA be"

    The server will be dedicated to Oracle 11g R2, the application will run separate server.

    The server will have 64 GB of RAM and I want to give as much of that RAM to Oracle SGA+PGA+etc as possible. By default the /dev/shm is 32GB leaving another ~25-30GB for PGA and O/S itself.
    Please correct me if I am wrong at this point?

    So the question is given 64-bit Oracle Linux 6.2 with UEK R2
    How big can I make the */dev/shm* and Oracle MEMORY_TARGET/MEMORY_MAX_TARGET, on such as 64 GB server?

    We prefer to avoid huge-pages because they are not compatible with AMM on Oracle 11.2.0.3.

    I have not seen any definitive performance studies showing me how much faster the system will perform with huge-pages so we are leaning towards AMM and setting some minimum SGA and PGA values as a baseline.
    If such studies do exists please provide a pointer, if possible.

    many thanks for your replies!
    cheers
    Yuri
  • 3. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Avi Miller Guru
    Currently Being Moderated
    yurib wrote:
    Can I change the */etc/fstab* entry for /dev/shm to manually increase the size to take up ~80-90% of the server physical RAM ? Is it a good idea?
    There is a bug in Red Hat Enterprise Linux and Oracle Linux 6 that means changes to /etc/fstab for /dev/shm are not seen during boot. If you want to change the size of /dev/shm, you need to manually remount it in /etc/rc.local. For example:
     # mount -o remount,size=50GB /dev/shm
    However, you have 64GB of RAM, so you should look at disabling AMM and using HugePages instead. It's way more efficient for any server with >4-8GB of RAM. HugePages don't use /dev/shm.

    EDITED: wrong command in example.

    Edited by: Avi Miller on May 31, 2012 12:55 PM
  • 4. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Avi Miller Guru
    Currently Being Moderated
    yurib wrote:
    I have not seen any definitive performance studies showing me how much faster the system will perform with huge-pages so we are leaning towards AMM and setting some minimum SGA and PGA values as a baseline.
    Our Director of Sustaining Engineering lists not configuring HugePages as the #1 cause of Database issues: http://event.on24.com/r.htm?e=452104&s=1&k=9292BFF32FC2A347E4771667EC0502D5&partnerref=OracleLinux_Linux_US
  • 5. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    801904 Newbie
    Currently Being Moderated
    thanks Avi

    is this bug present in 6.2 with UEK R2 also ?

    to make this /dev/shm change using umount - do I need to first shut down Oracle 11g R2 and then shut down Oracle Grid/ASM and then use umount?
    what is some other part of OS/app is using the /dev/shm? How do I find out and shut them down so I can re-configure /dev/shm?

    many thanks for your continued support!

    It seems that you and Dude are answering all my questions on this forum! 8^)
  • 6. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Avi Miller Guru
    Currently Being Moderated
    yurib wrote:
    is this bug present in 6.2 with UEK R2 also ?
    It's not a kernel bug, so it's present in all versions of OL6.
    to make this /dev/shm change using umount - do I need to first shut down Oracle 11g R2 and then shut down Oracle Grid/ASM and then use umount?
    I've edited my post - you need to use the mount command, not umount. My apologies for that. You don't need to shut anything down to change the size of /dev/shm.
    what is some other part of OS/app is using the /dev/shm? How do I find out and shut them down so I can re-configure /dev/shm?
    This is not required.
  • 7. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    801904 Newbie
    Currently Being Moderated
    MAGIC!!

    I just tried it and is WORKED with Oracle on-line.

    thank you very much!

    Now I need to read up on hugepages as per your and Dude advice, and Linux 6 and Oracle 11.2 implications....

    I found some printed help on hugepages in Oracle Linux/UNIX Admin Reference for 11.2 in Appendix G but its not very thick on details and spends most of the time on 32-bit Linux and Hugepages.

    back to the books now.... 8^)
  • 8. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Dude! Guru
    Currently Being Moderated
    Just in case, there was a previous thread about the problem with resizing /dev/shm under OL 6.

    The following should fix the problem. So you don't need to edit /etc/rc.local and remount /dev/shm.
    sudo cp /etc/rc.d/rc.sysinit /etc/rc.d/rc.sysinit.bck
    sudo sed -i 's/-f \/dev\/shm/\/dev\/shm/g' /etc/rc.d/rc.sysinit
    It simply removes the -f parameter when mounting /dev/shm in rc.sysinit. Then after a reboot /dev/shm should be sized according to /etc/fstab and you don't have to remount it.

    Re: Size option for /dev/shm in fstab is being ignored in 6.1
  • 9. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    801904 Newbie
    Currently Being Moderated
    thanks Dude

    I am trying to follow Avi's and your advise to get Hugepages 64-bit and Oracle 11.2.0.3 working together on Oracle Linux 6.2 UEK R2

    So far I failed, I cant get Oracle 11g R2 to use Hugepages.

    I cant get LOCK_SGA=true to work because it insists that MEMORY_MAX_TARGET=0 which Oracle 11.2.0.3 refuses to do.

    I turned off AMM using MEMORY_TARGET=0, using ASMM (SGA_%%%) and unable to crack it.

    I posted on Oracle Database forums but if you have it working, please chime in here or there....

    thanks
    Yuri
  • 10. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Dude! Guru
    Currently Being Moderated
    Why are you trying to set LOCK_SGA? Kernel hugepages cannot be swapped to disk.

    From what I understand, LOCK_SGA prevents SGA swapping and thereby applies to AMM or ASMM, but not when not using Hugepages.
  • 11. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    801904 Newbie
    Currently Being Moderated
    sorry Dude, I confused you, my apologies, forget LOCK_SGA please in this case.

    I cant get Hugepages to work with 11.2.0.3, period, without LOCK_SGA in the picture......

    hugepages requires memory_target and memory_max_target both = 0 and it can't be done because sga_max_size complains, its seems like a catch-22,
    I cant get the sequence of Oracle 11.2.0.3 init parameters to work with Hugepages. if I use use_huge_pages then it fails to start else it starts but does not use hugepages...

    thanks
  • 12. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Dude! Guru
    Currently Being Moderated
    kernel Hugepages is not compatible with AMM. You need to change from AMM to ASMM

    To do so set:

    MEMORY_TARGET=0
    SGA_TARGET=4G

    AMM=MEMORY_TARGET, ASMM=SGA_TARGET. You can set SGA_MAX_TARGET (default = SGA_TARGET) if you like, but remove MEMORY_MAX_TARGET or set it to 0.

    These are not Linux questions. I suggest to use the Database General forum, but you will most likely get flamed for not reading the documentation.
  • 13. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    801904 Newbie
    Currently Being Moderated
    thanks,

    I have read the doc (what little there is on this topic).

    I have asked on the database forum......

    just FYI - below is the proof:

    SQL> show parameter mem

    NAME                    TYPE     VALUE
    ------------------------------------ ----------- ------------------------------
    hi_shared_memory_address     integer     0
    memory_max_target          big integer 4G
    memory_target          big integer 0
    shared_memory_address     integer     0

    SQL> show parameter ga

    NAME                    TYPE     VALUE
    ------------------------------------ ----------- ------------------------------
    lock_sga               boolean     FALSE
    pga_aggregate_target          big integer 1600M
    pre_page_sga          boolean     FALSE
    sga_max_size          big integer 3G
    sga_target               big integer 1600M

    -----------------
    still does not work.

    And I cant change memory_max_target = 0 because I get error on startup:


    SQL> alter system set memory_max_target=0 scope=spfile;

    System altered.

    SQL> shutdown immediate;
    Database closed.
    Database dismounted.
    ORACLE instance shut down.
    SQL> startup;
    ORA-01078: failure in processing system parameters
    ORA-00843: Parameter not taking MEMORY_MAX_TARGET into account
    ORA-00849: SGA_TARGET 3221225472 cannot be set to more than MEMORY_MAX_TARGET 0.

    BUT if memory_max_target is > 0 then alert log says hugepages can not be used

    it feels like catch-22.....

    thanks

    Edited by: yurib on Jun 1, 2012 4:53 PM
  • 14. Re: /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    Dude! Guru
    Currently Being Moderated
    What happens if you remove memory_target and memory_max_target, instead of setting it to 0?

    I wrote "remove MEMORY_MAX_TARGET or set it to 0" in my previous reply, which works with normal memory pages, but apparently when using hugepages, the AMM parameters need to be removed.
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points