This discussion is archived
4 Replies Latest reply: Oct 20, 2011 2:32 AM by 895396 RSS

Problem with GC

894042 Newbie
Currently Being Moderated
Dear all;

I write a simple code to force one of the tasks in my algorithm to work in parallel. Since the rate of creating threads are high and I create ExecutorService and a fixed pool of 5 repeatedly after a few iterations the system almost run out of memory. What is wrong with this code and how I can improve it? Remember the loop is running at list 500 times.


     for (int i = pop.length - 1; i >= 0 ; i--)
     {
          
          if (totalUnique > 5)
          {
          ExecutorService distanceExecutor = Executors.newFixedThreadPool(5);
          AntibodyDistanceChecker[] DCT = new AntibodyDistanceChecker[5];
          int step = (int) Math.floor(totalUnique / 5);
          DCT[0] = new AntibodyDistanceChecker("DCT0"+i,
                    pop,
                    Arrays.copyOfRange(bestSet, 0, step),
                    minAllowedDistance);

          DCT[1] = new AntibodyDistanceChecker("DCT1"+i,
                                                       pop[i],
                                                       Arrays.copyOfRange(bestSet, step , 2*step),
                                                       minAllowedDistance);

          DCT[2] = new AntibodyDistanceChecker("DCT2"+i,
                                                       pop[i],
                                                       Arrays.copyOfRange(bestSet, (2*step) , 3*step),
                                                       minAllowedDistance);

          DCT[3] = new AntibodyDistanceChecker("DCT3"+i,
                                                       pop[i],
                                                       Arrays.copyOfRange(bestSet, (3*step) , 4*step),
                                                       minAllowedDistance);

          DCT[4] = new AntibodyDistanceChecker("DCT4"+i,
                                                       pop[i],
                                                       Arrays.copyOfRange(bestSet, (4*step), totalUnique ),
                                                       minAllowedDistance);

          distanceExecutor.execute(DCT[0]);
          distanceExecutor.execute(DCT[1]);
          distanceExecutor.execute(DCT[2]);
          distanceExecutor.execute(DCT[3]);
          distanceExecutor.execute(DCT[4]);
          
               distanceExecutor.shutdown();
               while (!distanceExecutor.isTerminated()) {}

               for(int m = 0; m < DCT.length; m++)
               {
                    if(!DCT[m].getAcceptable())
                    {
                         isUnique = false;
                         break;
                    }
               }
               
               distanceExecutor = null;
               DCT = null;
          }else
               {
               for(int j = 0; j < totalUnique; j++)
               {
                    if (pop[i].HamingDistance(bestSet[j]) < minAllowedDistance)
                         {
                         //System.out.printf("Haming distance:\t%d\n" , pop[i].HamingDistance(bestSet[j]));
                         isUnique = false;
                         break;
                         }
               }
               }
          

          // check if the Ab reaches its lifespan limit or not. Also, check the feasibility of the solution.
     if (totalUnique < numBest && isUnique &&
          pop[i].getAge() < maxAntibodyLifeSpan &&
          pop[i].getAffinity(2) > 1 &&
          pop[i].getAffinity(2) < num_Attributes - 1)
     {
          bestSet[totalUnique] = pop[i].clone();
          totalUnique++;
     }
     isUnique = true;
     }

Edited by: 891039 on Oct 12, 2011 4:38 AM
  • 1. Re: Problem with GC
    796440 Guru
    Currently Being Moderated
    Not gonna read all that unformatted code, and even if I did, it's unlikely I'd be able to correctly guess where the memory problem is. That's why the gods invented profilers.

    However, it does seem rather pointless and wasteful to create a new ExecutorService each time through the loop. Create it once before the loop and just keep reusing it.
  • 2. Re: Problem with GC
    DrClap Expert
    Currently Being Moderated
    I believe it's also true that if you create more threads than the operating system can support, you get an exception whose name suggests you are out of memory. I don't remember exactly its name but no doubt the OP could tell us what the actual error message was if he or she was interested.
  • 3. Re: Problem with GC
    796440 Guru
    Currently Being Moderated
    DrClap wrote:
    I believe it's also true that if you create more threads than the operating system can support, you get an exception whose name suggests you are out of memory. I don't remember exactly its name but no doubt the OP could tell us what the actual error message was if he or she was interested.
    Right. It's an OutOfMemoryError with a message along the lines of "could not create native thread".
  • 4. Re: Problem with GC
    895396 Newbie
    Currently Being Moderated
    Problem is because of your continuous Fixed thread creation within the loop.
    Number of threads/process can be created per user/session is limited in almost all platforms.

    Bring out Executors.newFixedThreadPool(5); to outside of the for loop. (Many be done only once)
    Now it retains only 5 threads for the service.
    Tasks will be queued up if there is no active threads available to serve the new incoming tasks. So there will be wait over there

    If your task/objects are short lived and if you are not sure how many tasks may arrive go for Cached Thread Pool.
    This fine tunes the pool with unused threads going away from the pool and new threads created during the run time in case no active threads are present

    Edited by: 892393 on Oct 20, 2011 2:28 AM

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points