bug#50701: cannot append or insert to empty file or stream

[Assaf Gordon]

tag 50701 notabug
close 50701
stop

Hello,

Thank you for providing clear reproducible example of the situation -
it makes troubleshooting much easier.

This is, however, not a bug - but the intended behavior:

On 2021-09-20 2:53 a.m., lexi hale wrote:
> there is an inconsistency in the handling of the a and i commands in
> sed. if the input stream or file immediately yields EOL, sed ignores
> the commands and produces an empty stream.

First, let's clarify what the files are:

>         $ echo -n >file line-1
>    $ touch empty

The file 'file' is not empty, it has one line.
This line just happens to be empty (i.e. no characters in the line 
before the newline character).

The file 'empty' is empty, it contains NO lines.

The above might seem obvious,
but the distinction is important for the next step:

>         $ sed -i aline-2 file

The sed command "aline-2", means append ("a") the text "line-2",
to *every* line.
The "every line" parts is due to the command "a" having no address
part (contrast with sed command "3aline-2" which would add "line-2" only 
to the third line).

If the input file has (any) lines, the command will be executed.
If the input does not have any lines, the command will not be executed.
And that is the behavior you are seeing.

> however, >
>   $ sed -i aline-1 empty >   $ cat empty>
> prints nothing,  instead of the expected result "line-1". >

Compare the situation to this (slightly more obvious) case:

    $ printf "%s\n" hello world > in1
    $ cat in1
    hello
    world

Add 'FOO' after every line:

    $ sed 'aFOO' in1
    hello
    FOO
    world
    FOO

Add 'FOO' after the second line:

    $ sed '2aFOO' in1
    hello
    world
    FOO

Add 'FOO' after the third line:

    $ sed '3aFOO' in1
    hello
    world

Since there is NO third line in the input, the command wasn't executed.

And similarly, since the 'empty' file does not have ANY lines,
the command (which is programmed to run on every line) is not executed.

> this is an extremely surprising behavior that limits the utility of sed
> when one cannot predict the contents of the file in question.  

I hope the explanation above makes this behavior less surprising.

> [...] it would be helpful to at least have a flag that would turn
> on consistent behavior for these commands.
>
> however, making the consistent behavior the new default might also fix
> a few extremely rare bugs in existing shell scripts :)

I humbly think that this behavior is consistent and does not require any
modification or flags - but other opinions are welcomed.

When integrating with shell scripts, edge-cases (e.g. empty input)
should probably be checked explicitly.

Few suggestion to check for empty files (contrived and not tested):

    if test -s INPUTFILE ; then
       # process the file
    else
       echo "FILE IS EMPTY"
    fi

    ---

    lines=$(wc -l < INPUTFILE)
    if test "$lines" -gt 0 ; then
       # process the file
    else
       echo "FILE IS EMPTY"
    fi

And lastly, since you are already using GNU sed extensions
(and not worrying about portability), you can use
the 'qNUM' command extension to exit with a specific code
if there was any input:

    sed -e 'aline-1'  -e '$q42' INPUTFILE > OUTPUTFILE
    exit_code=$?
    if test $exit_code -eq 42 ; then
       # file processed OK, wasn't empty
    elif test $exit_code -eq 0 ; then
       # file was empty
    else
       # another sed error (e.g. bad program, I/O error, etc)
    fi


As such, I'm marking this as "not a bug", but discussion can continue
by replying to this thread.

regards,
 - assaf