Process Scheduling

Let's say, I have 10 scripts that开发者_Go百科 I want to run regularly as cron jobs. However, I don't want all of them to run at the same time. I want only 2 of them running simultaneously.

One solution that I'm thinking of is create two script, put 5 statements on each of them, and them as separate entries in the crontab. However the solution seem very adhoc.

Is there existing unix tool to perform the task I mentioned above?

---

The jobs builtin can tell you how many child processes are running. Some simple shell scripting can accomplish this task:

MAX_JOBS=2

launch_when_not_busy()
{
    while [ $(jobs | wc -l) -ge $MAX_JOBS ]
    do
        # at least $MAX_JOBS are still running.
        sleep 1
    done
    "$@" &
}

launch_when_not_busy  bash job1.sh --args
launch_when_not_busy  bash jobTwo.sh 
launch_when_not_busy  bash job_three.sh 
...
wait

---

NOTE: As pointed out by mobrule, my original answer will not work because the wait builtin with no arguments waits for ALL children to finish. Hence the following 'parallelexec' script, which avoids polling at the cost of more child processes:

#!/bin/bash

N="$1"

I=0

{
        if [[ "$#" -le 1 ]]; then
                cat
        else
                while [[ "$#" -gt 1 ]]; do
                        echo "$2"

                        set -- "$1" "${@:3}"
                done
        fi
} | {
        d=$(mktemp -d /tmp/fifo.XXXXXXXX)

        mkfifo "$d"/fifo

        exec 3<>"$d"/fifo

        rm -rf "$d"

        while [[ "$I" -lt "$N" ]] && read C; do
                ($C; echo >&3) &

                let I++
        done

        while read C; do
                read -u 3

                ($C; echo >&3) &
        done
}

The first argument is the number of parallel jobs. If there are more, each one is run as a job, otherwise all commands to run are read from stdin line by line.

I use a named pipe (which is sent to oblivion as soon as the shell opens it) as a synchronization method. Since only single bytes are written there are no race condition issues that could complicate things.

---

GNU Parallel is designed for this kind of tasks:

sem -j2 do_stuff
sem -j2 do_other_stuff
sem -j2 do_third_stuff

do_third_stuff will only be run when either do_stuff or do_other_stuff has finished.

Watch the intro videos to learn more:

http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1