Re: sqlmaster "nowait" and "append" functionality?

[Ole Tange]

On Wed, Dec 7, 2016 at 12:34 PM, Ole Tange <ole@tange.dk> wrote:
> On Wed, Dec 7, 2016 at 6:46 AM, Andy Loftus <aloftus@gmail.com> wrote:
:
>> In general, I think the best approach is "keep it simple" and try to retain
>> as much flexibility as possible by putting the template in the command
>> column.
>
> When you submit a patch for this, please make sure the non-sql case
> does not get slower by this. As mentioned a lot of the optimization is
> depending on the command template does not change. And as a user I
> would find it very confusing, if the first entry in the command column
> is applied to all commands - especially if I had appended some
> commands with a template command that was different:
>
> parallel --sqlmaster $DBURL echo {%} ::: {1..10}
> parallel --sqlmaster $DBURL echo {} ::: {1..10}
> parallel --sqlworker $DBURL

Try the git version:

# Insert jobs in :my/tbl
# Exit when jobs are inserted (do not wait for them to be run)
$ parallel --sqlmaster :my/tbl echo {%} ::: {1..10}

# The command is GNU Parallel's internal representation of the command:
# Replacement strings are substituted by ¯<...¯>, and splitting spaces is ¯
# The reason for this is to be able to tell the difference between:
#   parallel echo "a  b" ::: foo
#   parallel echo  a  b  ::: foo
$ sql :my 'select * from tbl'
Seq     Host    Starttime       JobRuntime      Send    Receive
Exitval _Signal Command V1      Stdout  Stderr
1       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     1       NULL    NULL
2       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     2       NULL    NULL
3       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     3       NULL    NULL
4       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     4       NULL    NULL
5       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     5       NULL    NULL
6       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     6       NULL    NULL
7       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     7       NULL    NULL
8       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     8       NULL    NULL
9       NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     9       NULL    NULL
10      NULL    NULL    NULL    0       NULL    -1000   NULL    echo¯
¯<1 $_=$job->slot()¯>     10      NULL    NULL

# Start a worker
# Note: command and arguments are ignored, so no need to write any
$ parallel --sqlworker :my/tbl
1
2
3
4
5
6
7
8
1
2

# The finished table is the same as before
$ sql :my 'select * from tbl'
Seq     Host    Starttime       JobRuntime      Send    Receive
Exitval _Signal Command V1      Stdout  Stderr
1       :       1484295420.155  0.094   0       2       0       0
 echo 1  1       1\n
2       :       1484295420.167  0.113   0       2       0       0
 echo 2  2       2\n
3       :       1484295420.179  0.13    0       2       0       0
 echo 3  3       3\n
4       :       1484295420.19   0.134   0       2       0       0
 echo 4  4       4\n
5       :       1484295420.201  0.138   0       2       0       0
 echo 5  5       5\n
6       :       1484295420.214  0.141   0       2       0       0
 echo 6  6       6\n
7       :       1484295420.225  0.144   0       2       0       0
 echo 7  7       7\n
8       :       1484295420.235  0.151   0       2       0       0
 echo 8  8       8\n
9       :       1484295420.267  0.134   0       2       0       0
 echo 1  9       1\n
10      :       1484295420.299  0.118   0       2       0       0
 echo 2  10      2\n

Test it and give feed back.

git clone git://git.savannah.gnu.org/parallel.git


/Ole