This discussion is archived
7 Replies Latest reply: Oct 5, 2012 8:31 AM by Branchbird - Pat RSS

High Memory utilization for a single project

SunilN Newbie
Currently Being Moderated
Hi All,
I am facing one issue with one of the projects configured. We have endeca installed on a Windows server and there are multiple projects setup. I am observing that dgraph.exe for a single project is taking as much as 80% of memory on server. I do see that the project is receiving around 300 to 350 requests every day which is slightly higher than other projects. I am required to manually restart the project before business starts and that releases the memory. I am not able to identify whats causing this issue. Any ideas on how to troubleshoot the issue?

Any suggestions will be of great help..

Thank You,
Sunil
  • 1. Re: High Memory utilization for a single project
    sabdelhalim Newbie
    Currently Being Moderated
    Hi Sunil,
    did you check what was the size of the index the Dgraph loads in memory (either the dgidx_output directory on ITL or dgraph_input on your MDEX ???) that should give you a hint about the minimum size for the dgraph, besides did you check in your appconfig for the running project how many threads does the dgraph run ? (dgraph default section), I suppose you can try and monitor memory usage by running sysload or other tools ...
    hope that helps
    regards
    Saleh
  • 2. Re: High Memory utilization for a single project
    PravinChikhale Explorer
    Currently Being Moderated
    Try identifying the queries that are taking too much of time. You can get these from Dgraph.reqlog or from http://<host>:<dgraph_port>/admin?op=stats page.
    Refer Details -> Most Expensive Queries

    Also refer Details ->Hotspots to get information like which type of queries are overloading server.

    After getting expensive queries, all you need to do is optimize it by removing unnecessary parameters.

    I am assuming hardware configuration is same as recommended by Endeca (MDEXInstallGuide.pdf)

    Ideal dgraph setup is you should have separate server for serving response for each project.

    HTH

    -- Pravin.
  • 3. Re: High Memory utilization for a single project
    PravinChikhale Explorer
    Currently Being Moderated
    Duplicate post.

    Edited by: Pravin Chikhale on Oct 4, 2012 7:13 PM
  • 4. Re: High Memory utilization for a single project
    Branchbird - Pat Journeyer
    Currently Being Moderated
    The three factors that have the most impact on a dgraph's memory footprint are:

    Size of the index on disk (i.e. how big is it)
    Size of the cache memory (set via the cmem flag)
    Are you running a lot of partial updates?

    If you could tell us how much memory you have, how big your index is and what your cmem is set to (or if you just left it as default), we can estimate how much memory that dgraph "should" be using.
  • 5. Re: High Memory utilization for a single project
    SunilN Newbie
    Currently Being Moderated
    Hi Branchbird ,

    Please find my comments below in bold

    Size of the index on disk (i.e. how big is it) : *511 MB*
    Size of the cache memory (set via the cmem flag): it is default
    Are you running a lot of partial updates? No i am not running partial update, only running full update once in a day

    I have 16 GB REM, the dgraph is taking more then 9 GB for this project.
    We have enabled the wildcard search for typeAhead feature.

    Any suggestions will be of great help..

    Thanks,
    Sunil
  • 6. Re: High Memory utilization for a single project
    sabdelhalim Newbie
    Currently Being Moderated
    Hi,
    complementary question : Did you check in your appconfig.xml at the default section for Dgraphs what is the default n° of threads (starting from MDEX version 6.1.4 default n° of threads is 2) ??
    what MDEX release are you running ?? I fail to understand how a 511Mb index can be turned into 9 Gb !!!
    regards
    Saleh
  • 7. Re: High Memory utilization for a single project
    Branchbird - Pat Journeyer
    Currently Being Moderated
    The number of threads will not have any impact on memory usage, however, I agree with Saleh that this doesn't make any sense.

    I would suggest the following:

    Grab the stats page XML (http://[your server name]:[your port]/admin?op=stats).
    Restart the dgraph and see how much memory is taken up on the initial load. Then, assuming the memory consumption is reasonable, issue a couple N=0, N=something queries to the index and track what's happening with how much memory is being taken up.

    If you're still at a normal level, it seems like there must be some process or "bad query" that causes the memory to spike. A possible culprit might be found on the "most expensive queries" section of the stats page but it's possible it won't be there as that only tracks queries that have completed, not queries that are still in process or taking forever.

    The only other explanation I can think of is a rogue "bulk export" query where a user is trying to (for example) export the entire index to excel or something like that. Does your application offer that functionality?

    Regards,

    Patrick Rafferty
    Branchbird

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points