[bug-diffutils] Re: Bug#570008: dpkg-source: doesn't work together with diffutils 1:2.9-1

[Jim Meyering]

Raphael Hertzog wrote:
> clone 570008 -1
> reassign -1 diffutils 1:2.9-1
> severity -1 serious
> retitle -1 non-backwards compatible change in diff output with binary files
> thanks
>
> On Mon, 15 Feb 2010, Andreas Hoenen wrote:
>> Building a binary package for dblatex 0.2.12-3 (which has been built and
>> distributed successfully before) fails after the diffutils upgrade 
>> 1:2.8.1-18 ->
>> 1:2.9-1, as dpkg-source fails on the changed diff output:
> [...]
>> dpkg-source: error: unknown line from diff -u on 
>> dblatex-0.2.12/examples/dblatex/example.pdf: `Files /dev/null and 
>> dblatex-0.2.12/examples/dblatex/example.pdf differ'
>> dpkg-buildpackage: error: dpkg-source -b dblatex-0.2.12 gave error exit 
>> status 2
>>
>> The problem is the changed diff output:
>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> dblatex-0.2.12$ diff --version | head -1
>> diff (GNU diffutils) 2.9
>> dblatex-0.2.12$ diff -u /dev/null examples/dblatex/example.pdf
>> Files /dev/null and examples/dblatex/example.pdf differ
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>
>> After downgrading package diffutils the output is (with dblatex bulding 
>> fine):
>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> dblatex-0.2.12$ diff --version | head -1
>> diff (GNU diffutils) 2.8.1
>> dblatex-0.2.12$ diff -u /dev/null examples/dblatex/example.pdf
>> Binary files /dev/null and examples/dblatex/example.pdf differ
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>
> Indeed:
> http://git.savannah.gnu.org/cgit/diffutils.git/commit/?id=a352f09806a8606b4bbec07048da6762ce7d9afa
>
> dpkg-source has been relying this specific output of diff
> since its inception (in the nineties) and I really don't agree with the

Hi Raphael,

I'm willing to revert the change, even though it has been nearly 8
years (albeit only in "alpha releases"), and there's a risk that
some application now requires the other/current-in-2.9 syntax.

> justification of the upstream change. We need a way as users to know
> that one of the file contains binary data, saying "Files differs"
> would lead me to ask myself "but then why don't you show me the difference??".
> The binary indication was fine.
...
> Paul or Jim, can you revert this change on the uptream side?

If no one objects, that's just what I'll do.
Paul replied privately that he too is ok with reverting that change.

> Santiago, can you revert this commit in the debian package of diffutils
> at least for now? Just revert the patch linked above.
>
> If upstream doesn't want to revert the change, we'll adapt dpkg-source
> but you will have to keep the patch until a fixed dpkg-dev is available in
> all distributions and/or add a Breaks on the dpkg-dev version that are
> not compatible.
>
> BTW, the change above dates back to 2002 according to git but it was first
> part of diffutils 2.8.4 (2004?) and we just switched from 2.8.1 to 2.9 in
> Debian sid (whoa, what happened Santiago?).
>
>> /usr/share/perl5/Dpkg/Source/Patch.pm needs the leading "Binary" to match the
>> regexp at line 111, without it the code falls through the switch into the 
>> error
>> case.
>
> Yes and also the is_binary() function in
> /usr/share/perl5/Dpkg/Source/Functions.pm uses "diff -u /dev/null
> <filetotest>" to detect whether diff would consider <filetotest> as a
> binary file.
>
> Questions for the diff upstream maintainers: what's the best way to call
> diff to detect if diff considers a file as binary?

Here's one way:

  some_file=/dev/zero
  $ echo |diff - $some_file
  Files - and /dev/zero differ
  [Exit 2]

Note that you'd have to check stderr for /.* differ$/ only when
that command exits with status 2.

> Can we rely on "File /dev/null and bar differ" to mean that bar is a
> binary file?

Yes, for diff's definition of "binary".