bug#22254: [PATCH] sed: do not elide an invalid byte in a substitution RHS

[Jim Meyering]

On Sun, Dec 27, 2015 at 10:24 PM, Jim Meyering <address@hidden> wrote:
> On Sun, Dec 27, 2015 at 10:02 PM, Jim Meyering <address@hidden> wrote:
>> Just after pushing the fix for that heap-clobbering bug,
>> http://debbugs.gnu.org/22127, I noticed there was another minor
>> problem: sed would silently elide any invalid byte specified in a
>> substitution replacement string.
>>
>> Here's a patch that's not quite complete (no NEWS entry, and I haven't
>> yet tried to determine when it was introduced).
>
> This one has a checkered past. I keep a decent sampling of old
> versions of sed around.
> Running one of the two tests that would be affected shows how
> treatment of an invalid byte in a multi-byte locale has differed over
> the years:
>
> $ for i in $(env ls -dv /p/p/sed-*); do printf '%-14s: ' $i; echo a
> |LC_ALL=ja_JP $i/bin/sed 's/a/b\U\xb2c/'|od -tx1|head -1; done|sed
> s/.....//
> sed-1.18 : 0000000 62 55 78 62 32 63 0a
> sed-2.05 : 0000000 62 55 78 62 32 63 0a
> sed-3.01 : 0000000 62 55 78 62 32 63 0a
> sed-3.02 : 0000000 62 55 78 62 32 63 0a
> sed-4.0a : 0000000 62 00 43 0a
> sed-4.0b : 0000000 62 00 43 0a
> sed-4.0c : 0000000 62 00 43 0a
> sed-4.0d : 0000000 62 00 43 0a
> sed-4.0e : 0000000 62 00 43 0a
> sed-4.0.6: 0000000 62 b2 43 0a
> sed-4.0.7: 0000000 62 b2 43 0a
> sed-4.0.8: 0000000 62 b2 43 0a
> sed-4.0.9: 0000000 62 b2 43 0a
> sed-4.1  : 0000000 62 00 43 0a
> sed-4.1c : 0000000 62 00 43 0a
> sed-4.1d : 0000000 62 00 43 0a
> sed-4.1e : 0000000 62 00 43 0a
> sed-4.1f : 0000000 62 00 43 0a
> sed-4.1.1: 0000000 62 00 43 0a
> sed-4.1.2: 0000000 62 00 43 0a
> sed-4.1.3: 0000000 62 00 43 0a
> sed-4.1.4: 0000000 62 00 43 0a
> sed-4.1.5: 0000000 62 00 43 0a
> sed-4.2  : 0000000 62 00 43 0a
> sed-4.2.1: 0000000 62 00 43 0a
> sed-4.2.2: 0000000 62 00 43 0a
>
> Now, it does what 4.0.6..4.0.9 did.

Here's a complete patch:

0001-sed-do-not-elide-an-invalid-byte-in-a-substitution-R.patch
Description: Text Data