This discussion is archived
10 Replies Latest reply: May 13, 2013 9:41 AM by 1008625 RSS

Memory leak issue in tuxedo server

985321 Newbie
Currently Being Moderated
Hi,

I am facing memory issues in our tuxedo application.One of the server memory utilization is gradually increasing after reboot. As a temporary solution , we are making restart of that particular server every week.

I have installed Valgrind tool.i am trying to call the service ( Around 800 services are part of application ) which i suspect causing memory leak through ud script. like below.

valgrind ud32 < servicename.ud

but it is not giving memory leak details in actual service instead it is giving details about ud.

Could you guys suggest how can we find memory leak in tuxedo services.


Thanks
Ramana
  • 1. Re: Memory leak issue in tuxedo server
    user9130018 Newbie
    Currently Being Moderated
    hello

    run valgrind on tuxedo server binary like this:

    tmshutdown -s <tux-server-binary>
    cd $APPDIR
    valgrind [...valgrind options...] ./<tux-server-binary> -C dom=domain -g <number> -i <number> ....

    grab actual server parametes from "ps -ef" command

    run your service for example 100 times, shutdown it gracefully with tmshutdown -s <tux-server-binary> and look in valgrind log file

    example:

    cd $APPDIR
    valgrind leak-check=full show-reachable=yes undef-value-errors=yes malloc-fill=0xfe free-fill=0xca alignment=256 trace-children=yes track-fds=yes time-stamp=yes log-file=logfile.out ./tuxserver -C dom=tuxdom -g 1100 -i 2200 -u bloo -U /some/path/log/ULOG -m 0 -A -r -- -other args
  • 2. Re: Memory leak issue in tuxedo server
    985321 Newbie
    Currently Being Moderated
    HI Sir,

    Thanks a lot for your response.

    I ran the comand as mentioned by you in your post. but i haven't shutdown the server bcoz it is prod environment.
    It logged huge output to the log file.

    Below is the summary of the logfile.

    ==00:00:00:05.383 23666== LEAK SUMMARY:
    ==00:00:00:05.383 23666== definitely lost: 7,105 bytes in 7 blocks
    ==00:00:00:05.383 23666== indirectly lost: 3,383 bytes in 18 blocks
    ==00:00:00:05.383 23666== possibly lost: 28,653 bytes in 9 blocks
    ==00:00:00:05.383 23666== still reachable: 529,286 bytes in 288 blocks
    ==00:00:00:05.383 23666== suppressed: 0 bytes in 0 blocks
    ==00:00:00:05.383 23666==
    ==00:00:00:05.383 23666== For counts of detected and suppressed errors, rerun with: -v
    ==00:00:00:05.383 23666== Use --track-origins=yes to see where uninitialised values come from
    ==00:00:00:05.383 23666== ERROR SUMMARY: 107 errors from 72 contexts (suppressed: 10 from 10)


    But I am not able to understand the output logfile. Could you please let me know how can we analyze the logfile.

    Thanks in advance.
  • 3. Re: Memory leak issue in tuxedo server
    MBriz Explorer
    Currently Being Moderated
    Hi R828,

    you can read http://valgrind.org/docs/manual/mc-manual.html#mc-manual.leaks

    Regards,
  • 4. Re: Memory leak issue in tuxedo server
    MBriz Explorer
    Currently Being Moderated
    Hi all,

    I have a problem when i Execute in my server:

    "valgrind --leak-check=full --show-reachable=yes --undef-value-errors=yes --malloc-fill=0xfe --free-fill=0xca --alignment=256 --trace-children=yes --track-fds=yes --time-stamp=yes --log-file=logfile.out ./tuxserver -C dom=tuxdom -g 1100 -i 2200 -u bloo -U /some/path/log/ULOG -m 0 -A -r -- -other args"

    The server not up, in the ulog i have these error:

    225830.XXXXX!?proc.14473.133477952.0: LIBTUX_CAT:1253: ERROR: shmat failure: errno = 22, key = 78391
    225830.XXXXX!?proc.14473.133477952.0: LIBTUX_CAT:340: ERROR: Wrong configuration file
    225830.XXXXX!?proc.14473.133477952.0: LIBTUX_CAT:248: ERROR: System init function failed, Uunixerr = : shmat: Success

    I want to start the server with valgrind to launch Load Test, but the server dont started. any suggestion?

    Thank in advanced

    Edited by: MBriz on 28-ene-2013 14:21
  • 5. Re: Memory leak issue in tuxedo server
    user9130018 Newbie
    Currently Being Moderated
    Hello R828

    this is what you are looking for:

    [...]
    ==00:00:02:58.740 23950== 10,800 bytes in 216 blocks are definitely lost in loss record 242 of 251
    ==00:00:02:58.740 23950== at 0x4027A68: *** <---- your source files
    ==00:00:02:58.740 23950== by 0x816985B: *** <---- your source files
    ==00:00:02:58.740 23950== by 0x80E47EF: *** <---- your source files
    ==00:00:02:58.740 23950== by 0x80D9C6D: *** <---- your source files
    ==00:00:02:58.740 23950== by 0x436BAB7: _tmsvcdsp (in /opt/bea/tuxedo11gR1/lib/libtux.so)
    ==00:00:02:58.740 23950== by 0x43969E5: _tmrunserver (in /opt/bea/tuxedo11gR1/lib/libtux.so)
    ==00:00:02:58.740 23950== by 0x436943C: _tmstartserver (in /opt/bea/tuxedo11gR1/lib/libtux.so)
    ==00:00:02:58.740 23950== by 0x80723AF: main (BS-5d76.c:160)
    [...]
    ==00:00:02:58.741 23950== LEAK SUMMARY:
    ==00:00:02:58.741 23950== definitely lost: 16,096 bytes in 221 blocks <--------- leak

    so I lost 221 blocks of memory and backtrace shown above was responsible for 216 of them and source of a problem
  • 6. Re: Memory leak issue in tuxedo server
    user9130018 Newbie
    Currently Being Moderated
    Hello MBriz

    first try to run it manually from APPDIR (without tmboot, just ./server-name <proper-args>). if you succeed you will be able to run it with valgrind
  • 7. Re: Memory leak issue in tuxedo server
    MBriz Explorer
    Currently Being Moderated
    Hi all,

    If i run in $APPDIR ./server-name <proper-args> the server start, but if i run" valgrind --tool=memcheck --leak-check=full --log-file=logfile.out./server-name <proper-args>) the server don´t start. I see in the logfile.out

    ==15036== Warning: client syscall shmat tried to modify addresses 0xc0af000-0xd0aefff

    Regardss,
  • 8. Re: Memory leak issue in tuxedo server
    MBriz Explorer
    Currently Being Moderated
    any idea?
  • 9. Re: Memory leak issue in tuxedo server
    user9130018 Newbie
    Currently Being Moderated
    Hello

    what is your platform and versions?

    could you paste more from terminal (whole sequence: tmshutdown -s <server>/tmboot -i <instance>/tmshutdown -i <instance>/valgrind run)

    also paste ulog from valgrind test (hiding sensitive data if you have to)
  • 10. Re: Memory leak issue in tuxedo server
    1008625 Newbie
    Currently Being Moderated
    Hi R828,

    We are also facing memory leak issues when upgrading our OS from HPUX 32 Bit to Red Hat 6.3 (64Bit) with Tuxedo 11.

    Could you please confirm your configuration. I didn't see it in the thread.

    Did you resolve your issues ? - if so could you please advise resolution in case it helps us / others.

    Regards & Thanks

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points