This discussion is archived
7 Replies Latest reply: Mar 17, 2013 4:55 PM by Dude! RSS

Get confused about how to calculate SHMMAX and SHMALL on Linux.

xport Newbie
Currently Being Moderated
Hi guys,

thanks for your time over here.

recently, I'd read the linux kernel header file '/usr/include/linux/shm.h' and seen something of the macro definitions for SHMMAX and SHMALL, it looks like these:

-----
/*
* SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
* be increased by sysctl
*/

#define SHMMAX 0x2000000          /* max shared seg size (bytes) */
#define SHMMIN 1               /* min shared seg size (bytes) */
#define SHMMNI 4096               /* max num of segs system wide */
#define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16))
#define SHMSEG SHMMNI               /* max shared segs per process */

-----

did you see the SHMALL = SHMMAX/getpagesize()*(SHMMNI/16)?

so if PAGE_SIZE was set to 4096, we can get SHMALL = SHMMAX/16 based on the above formular.

but I found acturally the system did NOT run as the macro definitions, because sysctl -a output display like these:

-----
kernel.shmmax = 1073741824
kernel.shmall = 262144
kernel.shmmni = 4096
-----

did you see that shmall had been set by SHMMAX/PAGE_SIZE in pages.

Finally, I'm getting confused why the acctually behavior was not like the kernel header files, and how to calculate the two kernel parameters in real word.

thanks for your help.
  • 1. Re: Get confused about how to calculate SHMMAX and SHMALL on Linux.
    Dude! Guru
    Currently Being Moderated
    SHMALL sets the total amount of shared memory pages that can be used system wide, in pages.

    SHMMAX is the maximum size of a single shared memory segment set in bytes.

    The SHMMAX parameter is a safeguard parameter that sets the upper limit of how much shared memory a process can possibly request. The maximum supported value of SHMMAX on a 32-bit system is 4 GB - 1 byte.

    SHMALL is the division of SHMMAX/PAGE_SIZE, e.g:. 1073741824/4096=262144.

    The default PAGE_SIZE in Linux, which also applies to /dev/shm (POSIX) shared memory is 4 KB (4096 bytes). With the introduction of the Linux kernel 2.6, the system can also be configured to use Huge Pages, which uses a page size of 2 MB (2048 kilobyte). Kernel Huge Pages, unlike standard shared memory pages are non-swapable and therefore set and reserved at system startup.

    Oracle 11g database uses Automatic Memory Management (AMM) by default, which relies on /dev/shm. /dev/shm (POSIX shared memory) uses 4 KB pages and is set to 50 % of phsyical memory (RAM), by default. For performance reasons, which can be very drastic depending on how much memory the system has to manage, any Oracle database that uses more than 4 GB of SGA should use kernel Huge Pages.
  • 2. Re: Get confused about how to calculate SHMMAX and SHMALL on Linux.
    xport Newbie
    Currently Being Moderated
    Yep..., but if we have run a box with 2.6.x kernel, the default PAGE_SIZE is supposed to be 4KB (4096 bytes), and then according to
    #define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16)), SHMALL should be (SHMMAX/4096*(4096/16)), means SHMALL = SHMMAX/16.

    but it seems some thing wrong, what am I wrong?
  • 3. Re: Get confused about how to calculate SHMMAX and SHMALL on Linux.
    Dude! Guru
    Currently Being Moderated
    Take a default Oracle Linux 6.4 installation for example:

    # cat /proc/sys/kernel/shmmni
    4096
    # cat /proc/sys/kernel/shmall
    4294967296
    # cat /proc/sys/kernel/shmmax
    68719476736

    The values are defined in /etc/sysctl.conf. Which formula was used?

    As far as I'm aware the values are set to the maximum values according to the platform and kernel architecture. Like previously mentioned, these are safeguard parameters to limit the demand for resources, for instance to prevent a bad process from eating up all shared memory.

    Perhaps your misunderstanding stems from thinking that the values have to be set based on some formula. While this is probably true for the minimum, it does not necessarily apply to the values used for the maximum. For instance, in order for the kernel to allow some process to request 1 GB of memory with a page size of 4 KB, SHMALL must be set to at least 262144 pages. However, it does not mean you cannot set it higher, for instance to disable any software based limits.

    Regarding the header file, it shows some formula, but it does not say what it is used for or how memory is going to be partitioned or constructed in the end.
  • 4. Re: Get confused about how to calculate SHMMAX and SHMALL on Linux.
    xport Newbie
    Currently Being Moderated
    Oh yes. Even the head file have some predefined rules for this, but it maybe not a global item.

    Thanks for your help.
  • 5. Re: Get confused about how to calculate SHMMAX and SHMALL on Linux.
    Dude! Guru
    Currently Being Moderated
    I searched for confirmation and found the following in the Performance Tuning Guide:
    https://linux.oracle.com/documentation/OL6/Red_Hat_Enterprise_Linux-6-Performance_Tuning_Guide-en-US.pdf

    Capacity Tuning:

    shmall: Defines the total amount of shared memory in bytes that can be used on the system at one time. The default value for machines with 64-bit architecture is 4294967296; for 32-bit architecture the default value is 268435456.

    shmmax: Defines the maximum shared memory segment allowed by the kernel, in bytes. The default value on machines with 64-bit architecture is 68719476736; for 32-bit architecture, the default value is 4294967295. Note, however, that the kernel supports values much larger than this.

    What surprises me is that shmall according to the above defines memory in bytes, not pages ?!

    The following might be related, but I cannot access it:
    https://access.redhat.com/knowledge/solutions/43525
  • 6. Re: Get confused about how to calculate SHMMAX and SHMALL on Linux.
    bigdelboy Pro
    Currently Being Moderated
    Dude wrote:

    What surprises me is that shmall according to the above defines memory in bytes, not pages ?!
    I felt that also that new recommendations (at leat for machine running oracle databases) appear to be bytes ... not pages.

    Yet I know of an OL 5.4 where shmall (specified in pages) is preventing "too many" oracle databases starting ....

    That didn't really use huge pages though ...

    I think the general recommendations seem to be to set it very high where it wont be a limiting factor.
  • 7. Re: Get confused about how to calculate SHMMAX and SHMALL on Linux.
    Dude! Guru
    Currently Being Moderated
    Perhaps it further helps to understand, that SHMMAX and SHMALL do not enable a process or system to use more shared memory than available. By default, POSIX shared memory (/dev/shm) is 50 % of RAM, using 4K pages, allocated on demand and can be swapped. Linux kernel Huge Pages are setup manually, use 2M pages, cannot be swapped and are reserved at system start up.

    As far as I'm aware, setting SHMMAX and SHMALL to the maximum possible has no negative affect other than allowing a process to be able to use all allocated shared memory resources. This is what most Oracle Database installations demand. To calculate specific limits probably makes sense if the systems runs other software that relies on shared memory to run.

    Shared memory is not the only memory resource a system requires, but it is important for the Oracle Database System Global Area (SGA). If there is insufficient shared memory, the Oracle database instance will not start. Setting the SHMMAX and SHMALL to maximum values therefore disables any possible resource constrains set by the kernel.

    I have not found specific documentation that confirms the above, but that is what I experienced so far.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points