1. Following information about memory lock is from TimesTen documentation:
On Solaris, Linux and Windows 64-bit systems, TimesTen enables applications that connect to a shared database to specify whether the real memory should be locked while the database is being loaded into memory or while the store is in memory. If the physical memory used for the database is locked, the operating system's virtual memory subsystem cannot borrow that memory for other uses. No part of the database is ever paged out but this could lead to memory shortages in a system that is under configured with RAM. While memory locking can improve database load performance, it may impede other applications on the same computer.
On Linux systems, set the groupname in the MemLock setting to be the same as the instance administrator in the /etc/security/limits.conf file. Set the value of MemLock to be at least as large as the TimesTen database shared memory segment.
2. Following is our persize and tempsize for two datastores (a and b) present in the system
3. We have set the TimesTen attribute MemoryLock=4 for our datastores.
4. My "max locked memory" is 64k
5. The system has 24GB of RAM.
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x5d0500ac 0 root 660 1048576 24
0x11040012 32769 root 660 8788663088 29 locked
0x12040012 65538 root 660 268435456 29
0x11040013 98307 root 660 9199704648 14 locked
0x12040013 131076 root 660 268435456 14
As TimesTen suggests "Set the value of MemLock to be at least as large as the TimesTen database shared memory segment". What is its significance?
What we will be lacking if we retain the "max locked memory" value to 64 which is default value for Linux?
It's a best practice recommendation to use MemoryLock=4 in you odbc.ini setting for your DSN(s). The reason for this is as described in the docs so that the TT shared memory segment is not a candidate to be swapped out of memory.
For this to happen the OS has to be configured to accommodate this locked shared memory segment. One of these OS parameters is memlock. If memlock is not large enough to accommodate the whole of the TT shared memory segment then the TT shared memory segment won't be able to be locked.
Your step 4 above is a little confusing. It looks like that is for the user root? What is the memlock value for the instance administrator, the user who starts and stops the TT daemon? I suspect it's a lot bigger as your output in step 6 implies there are 2 locked shared memory segments of about 8GB each which could be your TT databases? So it might be that your system is already configured to lock the TT segment?
Thanks a lot for your reply.
We are using 'root' user as the instance administrator and following is the ulimit -a output.
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 187095
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 2048
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Just wondering even though 64kb is used for 'max locked memory', we are able to lock the datastore.
And would like to know if we are missing anything here.
[12:03:01 root@S2PXDM $
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x5d0500ac 0 root 660 1048576 14
0x15040012 32769 root 660 8788663088 30 locked
0x16040012 65538 root 660 268435456 30
0x15040013 98307 root 660 9199704880 6 locked
0x16040013 131076 root 660 268435456 6
Why are you running TimesTen as root? That is against security best practice and is a deprecated configuration. We strongly discourage installing and running TimesTen as root and we may actually desupport this in a future release. There really is no legitimate reason nowadays to run TimesTen as root.
Normally, if your limit for locked memory is smaller than the total size of the datastore shared memory segment (which is given by PermSize+TempSize+LogBufMB+64, units are MB) then you will be unable to successfully load the datastore when MemoryLock=4 is specified.
My guess (I have not tested it) is that the limit on locked memory is not enforced for 'root' and so you are able to load the datastore successfulyl even though your configuration is incorrect. Yet another reason not to use root.
Thanks for the update.
Yes, we are planning to move to a non-root user as instance administrator.
As our application started using TimesTen from Release 5.x, we had some dependencies with our application which needed root privilege. And some monitoring tools which were developed also required root privilege. We are currently investigating the possibilities for moving to a non-root user. It is in our pipe line.
Another question we have is regarding the total space used for locking.
For example, Im having a datastore with PERMSIZE of 8GB, ie PERM_ALLOCATED_SIZE is 8GB
The actual datastore size used size is 3GB, ie PERM_IN_USE_SIZE is 3GB
If we have enabled memory lock (MemoryLock=4), is it that all the 8GB will be locked by TimesTen or only 3GB is locked by TimesTen and the remaining 5 GB is available for other application processes?
TimesTen always allocates the entire shared memory segment (PermSize + TempSize + LogBufMB + 64) up front regardless of whether it is locked or not. If we are told to lock the segment then we of course lock the entire segment. If your machine does not have enough memory to support locking the TT segment while still leaving memory for other applications then either your database is too large (i.e. larger than you need) or your machine has insufficient RAM. It is of course possible to run without MemoryLock but then you leave yourself exposed to performance impacts from virtual memory paging. We always recommend using MemoryLock=4 (or huge pages) for a production database.
You said >>>If we are told to lock the segment then we of course lock the entire segment. >>>>
By this, I understand that the whole 8GB is not permitted to use by any other process. Is it correct?
Will other processes will be able to access 5 GB from the 8GB, since PERM_IN_USE_SIZE is only 3GB
That question, as posed, does not really make sense. The 8GB is allocated as a single shared memory segment. Access to that segment is limited to TimesTen processes and applications. On a virtual memory OS such as Linux/Unix, Windows etc. at any moment of time only some of the memory making up this segment may actually be mapped to physical memory. This is the same as for any other type of memory in the system such as regular process memory. In general there is always memory oversubscription; the system can allocate more (virtual) memory than there is real physical memory available. The virtual memory management system tries to keep the most recently used pages of virtual memory in physical memory but by default it allows any and all pages to be paged out (saved to disk [or discarded if not modified] and the physical memory that backs the page allocated to a different virtual page for a different usage). Locking memory prevents the OS from paging it and hence ensures that all pages making up the locked region are always in physical memory and are accessible without any (potentially large) delay due to having to wait for the contents to be paged back in.
So without MemoryLock potentially *all* of the memory making up the datatsore (i.e. all 8 GB) is subject to paging in a completely unpredictable manner (i.e. it is 'available for use by other processes'). This can result in unpredictable performance and unexpected I/O load on the system depending on the overall levels of memory usage, memory access patterns and the specifics of the OS memory management algorithms. The database, like any other memory in the system, is at the mercy of the OS and the current level of 'demand' for memory and there is nothing we can do to control this. This is generally an undesirable situation hence our recommendation to use MemoryLock. Using MemoryLock 'fixes' the entire datastore (all 8 GB or whatever size it is defined as) into physical memory and disallows paging of any part of it. This provides consistent and predictable response times for anything accessing the database. This does of course make this 8 GB of physical memory 'dedicated' to the TimesTen database but that is a good thing. It will only be a problem if (a) your system is not properly configured or (b) you do not have enough physical memory to support the usage of the system. If either of these is the case then the solution is to rectify the underlying issue not to turn off memory lock!
I hope that clarifies.