5 Replies Latest reply: Aug 17, 2011 10:48 AM by Johnny_hunter RSS

    Predecessor Tasks

    aksarben
      I debating how to best implement a system that will call a number of Callable tasks, many of which should only be run if a specified predecessor task (also Callable ) finishes successfully. Specifically, I have several dozen overnight batch processes that update database tables; the majority of the tasks should execute only if its predecessor has successfully completed its own part of the table updates.

      I've looked at the ExecutorService interface, & while it has some of the features I need, it doesn't seem to include the concept of "predecessor" tasks. With some work, I could implement this functionality manually, but I'm wondering if there's some a concurrency class I've missed that already has this capability. Any suggestions would be appreciated!
        • 1. Re: Predecessor Tasks
          796440
          CountdownLatch, Semaphore, CyclicBarrier, or Exchanger, perhaps?
          • 2. Re: Predecessor Tasks
            aksarben
            Well, yeah, I briefly looked at those, and while I think I'd likely end up using one or more of those if I have to do this manually, I guess a better way to ask my question is: Do the standard Java libraries have something like the executor service that lets you specify dependencies between tasks in a collection? Basically, I'm trying to see if there's a way to avoid the work of a "roll your own" solution.
            • 3. Re: Predecessor Tasks
              796440
              aksarben wrote:
              Well, yeah, I briefly looked at those, and while I think I'd likely end up using one or more of those if I have to do this manually, I guess a better way to ask my question is: Do the standard Java libraries have something like the executor service that lets you specify dependencies between tasks in a collection?
              What, you mean like:
              service.execute(list); // executes tasks in the list sequentially, not starting task N+1 until task N is complete
              ?

              If so, there's nothing like that that I know of, and if you've searched through java.util.concurrent.*, then you know more than I what's there.

              I may be missing something, but requirements seem a bit thin to offer anything more concrete.

              Edited by: jverd on Feb 24, 2011 12:03 PM
              • 4. Re: Predecessor Tasks
                aksarben
                J2SE was some help, but I did end up having to "roll my own," to some extent. Here's what I came up, if if anyone's interested. I haven't tested it yet, but the intent is that one would extend DependentTask, then invoke lllaunchPredecessors() at the beginning of your call() method. If anyone sees any glaring flaws in this design, please feel free to shout out!
                import java.util.ArrayList;
                import java.util.List;
                import java.util.concurrent.Callable;
                import java.util.concurrent.ExecutorService;
                import java.util.concurrent.Executors;
                import java.util.concurrent.Future;
                import java.util.concurrent.RejectedExecutionException;
                
                //----------------------------------------------------------------------------------------------
                /**
                 * A task that requires one or more other tasks to finish successfully before it executes.
                 */
                public abstract class DependentTask implements Callable<PredecessorOutcome> {
                
                private static final String UNSUCCESSFUL_PREDECESSOR = "Unsuccessful predecessor: %s";
                
                private final String name;
                
                /** Tasks which must finish successfully before this task can execute. */
                private final List<Callable<PredecessorOutcome>> predecessors =
                     new ArrayList<Callable<PredecessorOutcome>>();
                
                //----------------------------------------------------------------------------------------------
                /**
                 * Constructor
                 * @param name Task name.
                 */
                public DependentTask(final String name) {
                     this.name = name;
                }
                //----------------------------------------------------------------------------------------------
                /**
                 * Adds a task to those that must complete successfully before this task can execute.
                 * @param task The task to add. <em>Notes:</em>
                 * <ol>
                 * <li>A task might be a predecessor of <em>multiple</em> other tasks. If the task should be
                 * idempotent (e.g., it must execute only once even if invoked multiple times), its
                 * {@link Callable#call()} method is responsible for enforcing this requirement.</li>
                 * <li>The {@link PredecessorOutcome#getTaskName()} should return a unique value for each
                 * predecessor task, so that if one of the predecessors fails, it is easily determined which one
                 * had a problem.</li>
                 * </ol>
                 */
                public void addPredecessor(final Callable<PredecessorOutcome> task) {
                     predecessors.add(task);
                }
                //----------------------------------------------------------------------------------------------
                /**
                 * Retrieves the task name.
                 * @return The name.
                 */
                public String getName() {
                     return name;
                }
                //----------------------------------------------------------------------------------------------
                /**
                 * Executes the predecessor tasks &amp; checks their results. Subclasses should invoke this
                 * function at the beginning of their {@link Callable#call()} method.
                 * @param maxThreads The maximum number of threads to use.
                 * @throws Exception If a predecessor task did not finish successfully.
                 */
                protected void launchPredecessors(final int maxThreads) throws Exception {
                
                     ExecutorService service = null;
                     try {
                          service = Executors.newFixedThreadPool(maxThreads);
                          final List<Future<PredecessorOutcome>> futures = service.invokeAll(predecessors);
                          for (final Future<PredecessorOutcome> future : futures) {
                               final PredecessorOutcome result = future.get();
                               if (result.getOutcome() != TaskOutcome.SUCCESSFUL) {
                                    assert result.getTaskName() != null;
                                    throw new RejectedExecutionException
                                         (String.format(UNSUCCESSFUL_PREDECESSOR, result));
                               }
                          }
                     }
                     finally {
                          if (service != null) {
                               service.shutdown();
                          }
                     }
                }
                //----------------------------------------------------------------------------------------------
                }
                The usual support stuff:
                /**
                 * Holds data on the outcome of a task which precedes one or more other tasks.
                */
                public class PredecessorOutcome {
                
                private TaskOutcome outcome;
                private String taskName;
                
                //----------------------------------------------------------------------------------------------
                /**
                 * Retrieves the outcome.
                 * @return the outcome
                 */
                public TaskOutcome getOutcome() {
                     return outcome;
                }
                //----------------------------------------------------------------------------------------------
                /**
                 * Retrieves the task name.
                 * @return The name.
                 */
                public String getTaskName() {
                     return taskName;
                }
                //----------------------------------------------------------------------------------------------
                /**
                 * Builds a string representation of the object.
                 * @return The string.
                 */
                @Override public String toString() {
                     return String.format("DEP_TASK_OUTCOME[name=%s,outcome=%s]", taskName, outcome);
                }
                //----------------------------------------------------------------------------------------------
                }
                /**
                 * Describes the result of a task&rsquo;s execution.
                 */
                public enum TaskOutcome {
                
                /** Task completed without error. */
                SUCCESSFUL,
                
                /** Task aborted due to an error. */
                FAILED,
                
                /**
                 * Task started, but did not complete due to being interrupted or canceled.
                 */
                INCOMPLETE,
                
                /** Task was not run, because it was canceled before starting, or because a required predecessor
                 * did finish successfully.
                 */
                DIDNT_RUN
                
                }
                • 5. Re: Predecessor Tasks
                  Johnny_hunter
                  there might be a possible solution that not related directly to Java. Use a scheduler cybermation. It differs from the normal cronjob in that the sebsequential tasks won't start until the earlier ones finished.

                  thanks, John