This discussion is archived
1 2 Previous Next 15 Replies Latest reply: Aug 22, 2012 8:56 PM by 938489 RSS

Java total memory and memory usage increase when using mysql function?

938489 Newbie
Currently Being Moderated
We have a snippet of codes which does a select based on the mysql defined function.We have caught that once this portion is enabled then runtime.totalMemory() and used memory keep increasing. Below is the snippet of codes. The problem starts from String selectQuery4. I have tried this for debugging usr/java/jdk1.7.0_03/bin/jstat -gcutil 7525. I notice there is a change in the S0 and S1 values. Any idea what is this the problem with the increase?
double currentLong = Double.parseDouble(longitude);
double currentLat = Double.parseDouble(latitude); 

Statement stmt6 = null;
stmt6 = dbconn.createStatement(); 

String selectQuery3 = "Select geoFenceID,enterpriseID,clientID,geoFenceString,geoFenceType,geoFenceName,geoFenceDescription,geoFencePreference, geoFenceRadius From tblGeoFence Where ((enterpriseID="+enterpriseID+" And clientID=0) Or (enterpriseID="+enterpriseID+" And clientID="+clientID+")) And geoFenceStatus='a'"; 
ResultSet rs3 = stmt6.executeQuery(selectQuery3);

while(rs3.next()) {
    geoFenceID = rs3.getInt("geoFenceID");
    geoFenceType = rs3.getString("geoFenceType");
    geoFenceString = rs3.getString("geoFenceString");
    geoFenceRadius = rs3.getInt("geoFenceRadius");
    geoFencePreference = rs3.getString("geoFencePreference");
    String polygon  =  "GeomFromText('"+geoFenceString+"')";
    String point    =  "GeomFromText('POINT("+currentLong+"  "+currentLat+")')";

    Statement stmt7 = null;
    stmt7 = dbconn.createStatement();
    String selectQuery4 = "SELECT GISWithin("+point+","+polygon+") As geoFenceStatus";
    ResultSet rs4 = stmt7.executeQuery(selectQuery4);

    if(rs4.next()) {

        if(rs4.getInt("geoFenceStatus")==1) {

            geoFenceIDFound=geoFenceID;
            geoFenceName = rs3.getString("geoFenceName");
            geoFenceDescription = rs3.getString("geoFenceDescription");                                                     
            geoFencePreference = rs3.getString("geoFencePreference");   

            if(lastGeoFenceID==geoFenceID) {
                geoFenceInID = geoFenceID;                  
                break;
            } else {

                if(previousGeoFenceID==geoFenceID &&(previousTimeDifferenceInt<0 || geoFenceArriveTimeDifferenceInt <= 0)) //check the funny problem double entry back into the geo fence
                {
                    lastGeoFenceID=geoFenceID;
                    geoFenceInID = geoFenceID;
                    break;
                } else {
                    geoFenceAlertEmailMessage="\nGeo Fence Alert Arrival\nTime:"+dateTimer+"\nGeo Fence Name :"+geoFenceName+"\nGeo Fence Description :"+geoFenceDescription+"\nGeo Fence Preference:"+geoFencePreference;

                    if(lastGeoFenceEntryStatus.equals("") || lastGeoFenceEntryStatus.equals("Out")) {
                        geoFenceEntryStatus="In";
                    } else {
                        geoFenceEntryStatus="Out";
                    }
                }
            }
        }
    }
}

try {
    if ( rs4 != null ) {
        rs4.close();
    }                                        
    if ( stmt7!= null ) {
        stmt7.close();
    }
} catch(SQLException ex) {
    ex.printStackTrace(System.out);
}             
}           
try {
    if ( rs3 != null ) {
        rs3.close();
    }                              
    if ( stmt6!= null ){
        stmt6.close();
    }                              
} catch(SQLException ex) {
    ex.printStackTrace(System.out);
} 
     Codes used to print the memory stats.
int mb = 1024*1024;
System.out.println("##### Heap utilization statistics [MB] #####");

//Print used memory
System.out.println("Used Memory:" + (runtime.totalMemory() - runtime.freeMemory()) / mb);

//Print free memory
System.out.println("Free Memory:" + runtime.freeMemory() / mb);

//Print total available memory
System.out.println("Total Memory:" + runtime.totalMemory() / mb);

//Print Maximum available memory
System.out.println("Max Memory:" + runtime.maxMemory() / mb);
Edited by: 935486 on Aug 20, 2012 3:04 AM
  • 1. Re: Java total memory and memory usage increase when using mysql function?
    sabre150 Expert
    Currently Being Moderated
    935486 wrote:
    We have a snippet of codes which does a select based on the mysql defined function.We have caught that once this portion is enabled then runtime.totalMemory() and used memory keep increasing. Below is the snippet of codes. The problem starts from String selectQuery4. I have tried this for debugging usr/java/jdk1.7.0_03/bin/jstat -gcutil 7525. I notice there is a change in the S0 and S1 values. Any idea what is this the problem with the increase?
    I am not convinced there is a problem. Do you eventually experience an OOM exception? If not then the chances are there is not a problem.

    P.S. I do love your variable names. They really do make it easy to know what the objects they reference represent.

    Edited by: sabre150 on Aug 20, 2012 11:50 AM
  • 2. Re: Java total memory and memory usage increase when using mysql function?
    gimbal2 Guru
    Currently Being Moderated
    sabre150 wrote:
    I am not convinced there is a problem. Do you eventually experience an OOM exception? If not then the chances are there is not a problem.
    (the same thing that was said about the OP's previous thread on this very subject)
  • 3. Re: Java total memory and memory usage increase when using mysql function?
    sabre150 Expert
    Currently Being Moderated
    gimbal2 wrote:
    sabre150 wrote:
    I am not convinced there is a problem. Do you eventually experience an OOM exception? If not then the chances are there is not a problem.
    (the same thing that was said about the OP's previous thread on this very subject)
    :-) Interesting. I'm surprised the OP opened a new thread; I can't see how he could possibly have expected to get a different answer unless he thinks that we are all morons. But then if we are all morons then how can he trust any response even if it says that he positively has a memory problem.
  • 4. Re: Java total memory and memory usage increase when using mysql function?
    gimbal2 Guru
    Currently Being Moderated
    sabre150 wrote:
    gimbal2 wrote:
    sabre150 wrote:
    I am not convinced there is a problem. Do you eventually experience an OOM exception? If not then the chances are there is not a problem.
    (the same thing that was said about the OP's previous thread on this very subject)
    :-) Interesting. I'm surprised the OP opened a new thread; I can't see how he could possibly have expected to get a different answer unless he thinks that we are all morons. But then if we are all morons then how can he trust any response even if it says that he positively has a memory problem.
    I've asked that before in a similar case. Reply: I hoped other people would answer. Indeed, its very easy to silently add the "because the people I've been getting answers from are morons" in such cases.
  • 5. Re: Java total memory and memory usage increase when using mysql function?
    938489 Newbie
    Currently Being Moderated
    Dear Sabre,
    I have not reached the OOM but what worries is that the memory values keep increasing. So is that indicating something of memory overload will eventually take place? If it reached the OOM exception should we just add the extra maximum heap value?
  • 6. Re: Java total memory and memory usage increase when using mysql function?
    EJP Guru
    Currently Being Moderated
    It seem to me you could combine those two queries into one and get rid of the outer loop.

    The variables you are reading from the result set should be local variables.

    I never understand code where a variable is declared and initialized to null and then assigned in the very next statement.

    You should also be using prepared statements.
  • 7. Re: Java total memory and memory usage increase when using mysql function?
    sabre150 Expert
    Currently Being Moderated
    935486 wrote:
    Dear Sabre,
    I have not reached the OOM but what worries is that the memory values keep increasing. So is that indicating something of memory overload will eventually take place? If it reached the OOM exception should we just add the extra maximum heap value?
    Increasing the allocated memory is rarely a solution to OOM. The JVM will perform a garbage collection to try to free memory before it throws an OOM exception. If you get an OOM exception then you should first profile the application to see where the problem lies. The code you posted in this thread should not throw an OOM exception but you have only posted a small fragment.

    Now you can continue ignoring responses to your threads but eventually people will give up responding. There are other Java forums you can post in but you will get the same sort of responses. Your call.

    P.S. Please please please use more appropriate names for your variables.
  • 8. Re: Java total memory and memory usage increase when using mysql function?
    938489 Newbie
    Currently Being Moderated
    Dear Ejp,
    How do you suggest to combine the codes I dont see how to run both together? There are some variable where once set in this loop will be used outside the loop (this is just a small part of the code). Are you saying to remove this Statement stmt7 = null; and straight put this Statement stmt7 = dbconn.createStatement();? Yes will look into prepared statement next after able to solve this memory issue.
  • 9. Re: Java total memory and memory usage increase when using mysql function?
    Kayaman Guru
    Currently Being Moderated
    935486 wrote:
    Yes will look into prepared statement next after able to solve this memory issue.
    You should probably look into it now, considering that you're only thinking you have a memory issue.
  • 10. Re: Java total memory and memory usage increase when using mysql function?
    DarrylBurke Guru Moderator
    Currently Being Moderated
    Cross posted
    http://www.java-forums.org/advanced-java/62380-java-total-memory-memory-usage-increase-when-using-mysql-function.html
    http://stackoverflow.com/questions/12032845/java-total-memory-and-memory-usage-increase-when-using-mysql-function

    db
  • 11. Re: Java total memory and memory usage increase when using mysql function?
    938489 Newbie
    Currently Being Moderated
    Dear All,
    By taking all your good advices I have left the application to run for some time and nearly after 36 hours it stop. Surprisingly there is not OOM error generated where I google is it because I dont have this exception set
    catch (OutOfMemoryError e) {
    but I have other like SocketTimeoutException ex,IOException ex and Exception ex. So in addition to that what I monitor from this section of my codes

    Runtime runtime = Runtime.getRuntime();
    
                             
     int mb = 1024*1024;
     System.out.println("##### Heap utilization statistics [MB] #####");
             
     //Print used memory
     System.out.println("Used Memory:"+ (runtime.totalMemory() - runtime.freeMemory()) / mb);
                         
    //Print free memory
    System.out.println("Free Memory:"+ runtime.freeMemory() / mb);
                                 
    //Print total available memory
    System.out.println("Total Memory:" + runtime.totalMemory() / mb);
                         
    //Print Maximum available memory
    System.out.println("Max Memory:" + runtime.maxMemory() / mb);
    The final values I looked before it stop/crashed is
    ##### Heap utilization statistics [MB] #####
    Used Memory:239
    Free Memory:0
    Total Memory:239
    Max Memory:239
    So when I restart the initial values is as below.
    ##### Heap utilization statistics [MB] #####
    Used Memory:7
    Free Memory:8
    Total Memory:15
    Max Memory:239
    There after I notice at times there is a drop in the used memory but it keep increasing till it stopped. So is this a genuine case where I need to increase the heap or the other way around?
  • 12. Re: Java total memory and memory usage increase when using mysql function?
    Kayaman Guru
    Currently Being Moderated
    935486 wrote:
    is it because I dont have this exception set
    catch (OutOfMemoryError e) {
    No. You shouldn't need to catch Errors ever.
    So when I restart the initial values is as below.
    ##### Heap utilization statistics [MB] #####
    Used Memory:7
    Free Memory:8
    Total Memory:15
    Max Memory:239
    There after I notice at times there is a drop in the used memory but it keep increasing till it stopped. So is this a genuine case where I need to increase the heap or the other way around?
    What other way around? You're giving your program a tiny amount of memory and wondering if you have a memory leak. Give it 512M and run it for a few days and see what happens.

    I suspect you're putting unnecessary effort into trying to minimize the memory parameters here, how much does your computer have memory anyways?
  • 13. Re: Java total memory and memory usage increase when using mysql function?
    938489 Newbie
    Currently Being Moderated
    Dear Kayaman,
    We have 2 machines one is very small that is just 1Gb ram and another is 4GB. From my analysis I have already found which part is causing the memory to grow which is the code I posted in this thread. Indeed that is a quite a complex query but is run on the db server. So I shall take your advice and I will increase the memory accordingly. Is this the right way to increase it javac -J-Xmx512m appService.java? I am to learn what the signals of memory leaks?
  • 14. Re: Java total memory and memory usage increase when using mysql function?
    EJP Guru
    Currently Being Moderated
    No it isn't right. That just increases the memory available to the compiler. What you want is java -Xmx.
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points