Re: [Mingw-cross-env-list] How to write .mk for libraries that have no source releases?

[Volker Grabsch]

Mark Brand <address@hidden> schrieb:
> On 11/22/2010 03:19 PM, DiAppoggio wrote:
>> I'd like to write a .mk for a library which is only git available (  
>> http://www.videolan.org/developers/libbluray.html ).
>> How can I write $(PKG)_URL variable and $(PKG)_UPDATE section?
>
> That's a good question that hasn't really come up yet. Since there  
> really is no package tarball, there should some kind of null $(PKG)_URL,  
> maybe pointing to /dev/null.

I strongly disagree with this, see below for explainations.

We'll have to find a proper solution to this.

One helpful detail is that the Git web interface allows for
downloading a certain version as tarball. See below for my
thoughts about using hashes as version numbers.

Another possibility would be to use Git directly to check
out those versions. However, this would destroy the simplicity
of the pkg/ directory, which would then have big subdirectory
structures. It would be a waste of space, compared to compressed
tarballs in pkg/.

So downloading compressed tarballs seems to be the best option,
but has an important drawback: At least with Gitweb, the created
tgz archives differ on every download. Note that the uncompressed
data (the tarball itself) does not differ. Also, the compressed
data itself remains identical. However, the added gzip timestamp
is always the download time instead of the changeset time - for
whatever reason. :-(  Those 4 bytes near the beginning the the
*.tgz archive always differ, which makes the checksum differ, too.
Here, several workarounds come to mind, but I can't think of a
really good solution.

> The commits from the git repo could be
> packaged using 'git format-patch -p'. The patch would be applied during  
> the build to an empty directory. The patch in src/qt-1-win32.patch is an  
> example of how this is done, the difference being that this patch is  
> applied to the package tarball contents instead of to an empty directory.

Doing that would defeat the whole purpose of keeping the sources
separate from the build rules.

Yes, there are patches in mingw-cross-env, but those are mostly either

    1) patches of important bugfixes that will be included in the
       next release of that library, or

    2) patches of ugly (but necessary) hacks regarding win32 / MinGW /
       cross compiling that won't ever be accepted by upstream.

In other words, the main goal for the patches included in mingw-cross-env
is to get rid of them, not to accumulate them!

Yes, there are patches which are or will soon become bigger than
the tarball they patch, but this only a necessary evil until we
have a better solution. The increasing size of our release tarballs
is alarming:

       17,127   mingw-cross-env-2.0.tar.gz
              ...
      526,260   mingw-cross-env-2.11.tar.gz
              ...
      639,063   mingw-cross-env-2.15.tar.gz
    1,953,049   mingw-cross-env-2.16.tar.gz

This drastic increase is not because we support so many
libraries, and it is also not because of the little test
C/C++ programs we include. The big patches are solely
responsible for that increase.

In the current development version, our 5 biggest patches are:

       17,449 openscenegraph-1-pkg-config.patch
      196,160 w32api-2-directx-additions-for-qt.patch
      266,277 libgpg_error-1-upstream.patch
      336,418 w32api-1-dinput-dsound.patch
    5,541,917 freetds-1-fastforward.patch

While the two big w32api patches are currently unavoidable,
the other 3 patches will hopefully vanish within the next
releases of openscenegraph / libgpg_error / freetds.

Without those, the compressed release tarball would shrink
by about 1.2 MB, back to less than a third of its current
size.

> Not sure what to do about $(PKG)_UPDATE for this case.

We could check for new Git commits using the web interface.
The Git hash would lead version numbers which are not strictly
increasing, but would still be usable for checking whether a
new version is available.

Also, not that there's nothing in mingw-cross-env which asserts
that version numbers are increasing. This is just coded into
some $(PKG)_UPDATE rules for packages whose versions aren't
ordered in a useful way on server side.

We don't even need the full Git hash. The tarballs have a
directory suffix that consists of 7 hex digits. So even in
a project with 100,000 commits, the chance that the latest
hash collides with some other hash is less than 0.0004 (0.04%),
which is good enough I guess. :-)

That way, the first 7 hex digits of the commit hash make
an ideal version number: can be used directly in the URL,
are exactly the version suffix part of the directory, and
thus also an ideal version suffix for the downloaded tarball,
and still shorter than certain other version numbers like
"0.9999beta" or "4.1.3-rc2".


Greets,
Volker

-- 
Volker Grabsch
---<<(())>>---