[Quilt-dev] Quilt support for binary files

[Mitch Davis]

Hello,

Thank you to everyone who looks after quilt!  I like quilt very much.

I'm using quilt to manage a series of patches which reflect the
changes I've made to a large tree of source.  One of those changes was
to replace some small .gif web graphics with new images.

Out of the box, quilt doesn't support changes to binary files.  So I
have modified quilt a little to add rudimentary support for binary
files.  We can do pushes, pops, and refreshes.

The changes extend the diff format a little to express binary changes
as the base64 encoding of the compressed output of xdelta.  While
xdelta does a good job of detecting small changes to a binary file, my
mods also work ok for files of at least several hundred k (for
example, a patch which adds a .tar.gz file).

My implementation works by wrapping the system diff and patch programs
(no mods to the quilt program itself).  For example, the diff wrapper
spawns the system diff, and if the system diff complains about
comparing binary files, the wrapper calls xdelta, bzip2 and recode to
produce something which is similar to diff format.  Similarly, the
patch wrapper walks through the provided patch; if it's binary, use
recode/bunzip2/xdelta to handle the patch data, otherwise pass the
patch through to the system patch program.

Here's an example of a patch (note the "function name" after the "@@"
of "base64"):

Index: nsp/src/html/html/defs/style5/images/ti_menu_help.gif
===================================================================
--- ../../nsp-370-test2/nsp/src/html/html/defs/style5/images/ti_menu_help.gif
+++ nsp/src/html/html/defs/style5/images/ti_menu_help.gif
@@ -1,7 +1,32 @@ base64
-JVhEWjAwNCUAAAAIABAAEAAAAAAAAAAAAAAAAAAAAAB0aV9tZW51X2hlbHAuZ2lmdGlfbWVudV9o
-ZWxwLmdpZh+LCAAAAAAAAAMBwAA//0dJRjg5YTMAGQCiAAC7u9vKyuOoqNHt7fWwsNXk5PH///+V
-lcch+QQAAAAAACwAAAAAMwAZAAADhXi63P4wykmrvTjrzbv/YChyQKEUwBEUrCAUwrKy6RgYihEc
-xRAELgNh0VvhbEcdz6QgCIkm51B0y+1YCyn0oKUKndcBYdhdHgBPr2F9XX/TPPdgpEpemVx4b0yv
-W81Nenh0VQdKWIFTgH2Fh2JkOnyIjHZxa2NuJpN9nJ2en6ChoqOkogkAOwedD4jAAAAAH4sIAAAA
-AAAAA2NgaGBmYGDoONXueWj+76b5iRWeiW7FUdIHGBkZeTQKEkuSMxRSEksSNbEpYGRgOMAIAG2C
-4e5CAAAAAAABFyVYRFowMDQl
+JVhEWjAwNCUAAAAIABAAEAAAAAAAAAAAAAAAAAAAAAB0aV9tZW51X2hlbHAuZ2lmdGlfbWVudV9o
+ZWxwLmdpZh+LCAAAAAAAAAPdk4tX0vcCwL8oOtQ0fKE23MAcva5b29zptXa9PdVqtXus23G1tc5O
+L7eEdOUjd0VvYbmSfMzuxAAhUUHFx0wnqSQoKIqPtmy+AEHe/n48FFGR2/6N+zmf8/kPPkcTjuze
+880l8DfgAMDtdq+vr7tcrrW1tdXV1ZWVFafTuby87HA4lpaWrI5lm8O5uLhot9sh+xJkX7TZbJBt
+0Wq1WiwWGIZ1tiUIgoyw1QhZFhYWzGazyWTSmRaMRqPBYNDr9TqdTmFcUBghrVY7qzePG20zOtP8
+/PyYwTZmsGo0mlE9rFarR/WWubm5ER08OW8Y0VlGdJBKpZJroeH5BaVSqVAohjXmIY1ZpjaNzemH
+1KbBOePwnH5AZZAq9eJpTfu0oXpC/3RC9/SVTiaTNY5MtsvGqSMaVu/wbencbakq500lquy/VGb3
+K7P6FVn9ysw+RVafIkM8kyGezRDN3BLNZIimb/ZOpwv/LG0Tcrnc2trampoaDoeT1Syurq4+3TF5
+uv01m81msVhMJjOd181gMOh0ei67qbKykkajUSuZ+xr/INSMEWrG3+OMEjjjBM7Ye9VjEazRiKqR
+CKY8gjEcwRgKpw+G02UnSuoC6fJA+nAgQx7+S19YhRhFG0LRhsMfi8B/ZWGlz8NKu8DjgbCS52El
+AlA+AH6WgDIJKBXvpTAw1A5Q1od52AZKRKD4BXgkBEXd4OFz8LAr9B4/9H4TeCAIpdSDwg5AaQ3J
+qwb/4YfkskF+A8hrCM5hgrx6dE4VIPOCsytBLi84kxacWQFyOL6ZT0BmVVBaCbhZCUjl6JR7AV/e
+DEhOD0hOA+eywdmbIIkI/r/AO5HIEOB+M8Mb/noCoNwAQmHj+QxhPip4+zkJn9lLQW9OpOlgqrgw
+/ONrU5ImVn8R7uB9bEIzW1oSta3unLT56WD5TnwVLaGleqhid1rxT9IWjpwed9qATWytGWV9nL/9
+fCL/5LufnWIlVib+Wvc792zLtd/vc1JYjQ+2n35xqK3kdcuV8brzg231k8+I01/9FlvbcP4EPY4c
+Jd5VNtud6+n3zvF2vrK3wC9/hS0tfvqPfMwX1pCB0m1n6j9KmZF1tGjljIOF7/j/mbT1ZGmQz0XR
+lfGtH+4DkicnOtvMk+1pxpmhTsfDWNZufLLGt6xoRlKyozuS0WHTjrCObewBUu4/my9Fvt7/vNMB
+PF4Uut892SVwInTj3K/lXftX3udfdCJSfxSsri6/hZ6Qd3e73QCDU+QiMVcUZC8Z2icmEYTj8tC+
+Zw8E+6ELDgT7oq703MVuOkXGhm6YUBZ4Yy+o7m2trlHdJzST/TZE952NKoz9RSLCb0tvF+FjRiY8
+imKjcGyqT8hV9aN4Qu23Hqnqiz3UU0f2qIuTME4kdcteksT3YADicFAAyvkC/yEOzvMM+NQi8jz6
+kYLg6S/apqWl/3GVE/YrDp5/cnvhqpZ+43NSfNAHISseERQBNQ55Bxt0lHUoGnewygdr7cf/EJuM
+pJEOKGqRjLvbDTXsn1fuxPhk1BnqeD8mG7iMM27hoyoi8mI0lhId14uP3UksqD+s2qtFij6PEzZJ
+CFxT82Ae85in+MiO+lbZmWwe7xNThFdjGeZA/7MHY/yURspxyyV/ihpX52VKpgo74b4dkMB6l+mH
+NM9zoS77UgU5VN+QKMZ/EktEYWNfMojC3ejv+CE3fEkFKOm1C03eb3/Ls/Rhb8ks4c+OEy2S6HKT
+Rfr2BVL8xtGvSKiN6NJA2Ovf6ChyAEWHIx/eE5eLwN3BpZDeOhRsFo7Gb6m3jyVUkPy8Dh+NWXwZ
+ufP7RQwFPErFUNaIvMiwwCAPr7/zPhXi084loXzgIJdwkpiPc0ylv/reMXVZV4yYuu0YNTsmb81O
+IwYivuxBoAmCY1Z8MtXlKUgowjnVW5KuOzWPL1/fs+m373Y6tQzK+x3zbPe0w/uLHTkSzObN+zt8
++f0NHn6tr66vmtvHG/piIiULq1C39nq2iX3ys2e6zcTAQ7PlnYo1m1yOc9lf/ivVtSFlyoqwT92C
+XIsvs90CuOPyusBfx2ns8t/VCq2vwv2zdwPtr1LdrmVto3t9zQG53WtpvTe843s8NsG5qOMMjzBJ
+j/demIz+iYHcEi/0TYbzwmUM710SYUAWNReBAPv+B/zfZXv0BgAAH4sIAAAAAAAAA2NgaGBmYGDo
+kDLen7hNfg5flSKP52225L1feBkZeTQKEkuSMxRSEksSNbEpYGRg+MILAKrQe3dCAAAAAAAGnCVY
+RFowMDQl

The changes are against quilt 0.45, although I'm considering bumping
that to 0.47.  I haven't tried "make check".

I don't think it's a particularly good solution: Doing it with
wrappers is a bit gross, the wrappers use temporary files, some quilt
commands I don't use don't work, and the use of eval can lead to
shellcode injection attacks.  So I'm not expecting it to be included
in quilt any time soon.  But as a proof of concept, it works.

My question is, while it's useful to me, is anyone else interested in
this?  Is there any need for quilt to support changes to binary files?

Thank you,

Mitch.