6 Replies Latest reply on Dec 9, 2010 7:27 PM by AJYADAV

    How to Run 50 Parallel Processes simultaneously in FDM?

    AJYADAV
      Hi,
      This is from FDM admin Guide "Because FDM supports parallel processing of batch files, multiple files (up to 50) can run simultaneously." We tried running 50 parallel processes but FDM only creates 1 parent and 20 parallel child processes.

      I am trying to find out a way to run 50 parallel proceses at the same time. I know i can run multiple batches to come up to this number but wanted to know if we can increase the number of child processes from 20 to 50.

      We have the lngParallelProcessCount = 50 in our parallel batch processing script.

      Thanks, AJ
        • 1. Re: How to Run 50 Parallel Processes simultaneously in FDM?
          WPaffhausen
          Hello,

          How many files are you placing in the OpenBatches Folder?

          Thank you,
          • 2. Re: How to Run 50 Parallel Processes simultaneously in FDM?
            AJYADAV
            We have 400+ locations and placing 400+ files in openbatch. The POV are all unique, there is no overlapping. After running the batch script these 400 files are distributed to 20 parallel processes. So each parallel process has 20 files each. 20 parallel Process * 20 files = 400 data load files.
            What i want to do is to spawn 50 parallel process with 8 files each. 50 paralle process * 8 files = 400 data load files.

            Thanks, AJ
            • 3. Re: How to Run 50 Parallel Processes simultaneously in FDM?
              TonyScalese
              I believe the actual number of parallel process is a multiple of how many processors you have on the FDM server. Wayne should be able to confirm,
              • 4. Re: How to Run 50 Parallel Processes simultaneously in FDM?
                AJYADAV
                we have 2 environments dev and stage. Dev having 2 processors and stage having 4 processors but still in both the instances only max 20 parallel processes are created. So i don't see any correlation between no of processors versus parallel batch processess.
                • 5. Re: How to Run 50 Parallel Processes simultaneously in FDM?
                  WPaffhausen
                  AJ,

                  A few things:

                  Tony brings up a good point ... the Max # of processes leveraged really shouldn't exceed the # of processors available. This will ensure that 100% of processor is provided to that task. If you have 4 processors, and tell it 8 tasks; then only 50% of each processor is dedicated to that task. Which means it can take more time to process the files than actually intended. While this calculation is squishy ... it is just a rough outline.

                  On the topic of not being able to exceed 20 processes ... if I remember correctly this was a defect that existed in the software. While it let you select 50 ... it never really spawned past 20. This could be because of the number of DCOM ports left open or the available # of processes the OS can handle.

                  If you are sure your script is set appropriately for 50 ... and that all the files are unique; then it might be something you will want to open with Oracle Support. BUT ... keep in mind that they might tell you that setting the value to 20 (for a box that only has 2 or 4 CPUs) might not be able to fly with them ... but that is a battle you can fight.

                  Hopefully this helps,
                  • 6. Re: How to Run 50 Parallel Processes simultaneously in FDM?
                    AJYADAV
                    I have got the reply to the Oracle SR i have opened and the reply is

                    "The FDM guide and workbench are incorrect with the setting of 50. The limit on processes is 20 according the the code for this versions of FDM. I will enter in a bug to have FDM code changed to allow 50 parallel processes."


                    I wanted some more feedback from the folks at otn thats why i started this thread.

                    Its the code in FDM that is limiting the processes to be only 20.

                    Thanks, AJ