Hi. I'm new to submitting jobs through Grid Engine, and have skimmed users guides, googled, etc., without figuring this out and could use some advice.
I need to be able to submit jobs that will start up several distinct executables that need to run at the same time. If I was doing my work in an interactive shell, I'd just do something like this:
$ foo &
$ bar &
$ baz &
. . .and so they'd all start around the same time, run in the background, and complete around the same time. The executables talk to each other through the output files, and I don't need to do any more than that.
But from what I've read in users guides, I shouldn't invoke processes in the background in jobs submitted to GE; and that makes sense to me, since then the interpreter would reach the bottom of the script while the executables are still running. But submitting
won't work, because those are run sequentially.
The only thing I can think of to do is to separate each executable out into its own job submitted to the queue; but then I need to guarantee that they start executing at the same time.
Any advice would be very helpful. Thanks much.
try this :
( foo & bar & baz )
for example :
$ time ( sleep 5 & sleep 5 & sleep 5 )
$ time ( sleep 5 ; sleep 5 ; sleep 5 )
Hi, and thanks for the reply. That works; but my (perhaps incorrect) understanding is that one shouldn't do stuff like that, because it conflicts with Grid Engine's management of nodes/cores. Each of foo/bar/baz will require a certain number of cores (based on Open MP environment variables set just before their execution); I guess I somehow need to make sure that the total number of cores that foo, bar and baz will use all together is allocated to the parent job . . .