bug#30336: sed

[Assaf Gordon]

Hello,

On 2018-02-03 09:11 AM, Roger Adair wrote:
> when I execute the following command:
>
> $ sed 's/c$/7/g' test.data.asc
>
> where the contents of test.data.asc is:
>
> abc  abc
> 123456
>
> I get the following output:
>
> abc  abc
> 123456
>
> Whereas I would expect the following:
>
> abc  ab7
> 123456
>
> If I leave the $ out of the command "ab7 ab7" appears.
>
> What am I doing wrong?  Many thanks for reading my email.

My immediate guess would be that the file has dos/mac line endings
(e.g. CRLF "\r\n" instead of just line-feed "\n").

You can test it by using 'file', like so:

  $ file test.data.sac
  test.data.asc: ASCII text, with CRLF line terminators

Or by printing it's content like so:

  $ od -tcz test.data.asc
  0000000   a   b   c       a   b   c  \r  \n      >abc  abc..<


If you see only "\r" in the output - the file has MAC line-endings.
If you see both "\r\n" - the file has dos/windows line endings.

In unix, GNU sed "$" only matches "\n", and "\r" is just another character.


Some ways to work-around it:
1. Use "dos2unix" or "mac2unix" on the file (simplest and recommend).

2. Use the following to strip "\r" from the file
(assuming it also has "\n"):

    cat test.data.asc \
        | tr -d '\r' \
        | sed 's/c$/7/g'

If the file has only "\r", convert them to "\n":

    cat test.data.asc \
        | tr '\r' '\n' \
        | sed 's/c$/7/g'


3. Add sed command to remove "\r" before doing the
other substitution:

   sed -e 's/\r$//g' -e 's/c$/7/g' test.data.asc

4. Or modify the regex itself to optionally match "\r":

   sed 's/c\r*$/7/g' test.data.asc

---

If none of the above work, and you suspect another problem,
please provide a small fragment of your input file so we can reproduce 
the issue.

regards,
 - assaf