Re: [lmi] wxWidgets submodules and install_wx.make

[Vadim Zeitlin]

On Sun, 4 Feb 2018 23:19:46 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2018-02-02 17:11, Vadim Zeitlin wrote:
GC> > 
GC> >  As alluded to in a previous message, there is a problem with using
GC> > install_wx.make with the latest versions of wxWidgets because downloading
GC> > wxWidgets archive doesn't include the submodules and even though lmi
GC> > doesn't need all of them (and so anybody working on it would profit from
GC> > having a smaller local working copy not including libjpeg or libtiff), it
GC> > does need some of them (expat, libpng, zlib).
GC> 
GC> This used to work, so what has changed?

 Sorry, I can't believe I haven't actually stated this, but the change is
that wxWidgets used to just include the sources of 3rd party libraries in
its own repository, while now it uses Git submodules to track them. This
has a lot of advantages, the main one arguably being the ease of updating
the 3rd party libraries, which is quite important as security issues are
regularly found in some of them and so it's very nice to be able to update
them quickly whenever this happens.

GC> Is it the case that the dynamically-generated tarballs that we've
GC> used exclusively for the last several years formerly contained
GC> "submodules" like expat, but now exclude them? Is that the issue?
GC> Did github change its snapshotting process? Did wx restructure
GC> its repository?

 So the answer to the last question is "yes", and so the previous ones
don't really matter but, just to be 100% explicit, GitHub didn't change
anything, they never included submodules into the automatically generated
archives and still don't.

GC> At this point, 'install_wx.make' uses tarballs for historical reasons
GC> only; we can simply use 'git clone' once and something like 'git pull'
GC> subsequently (and we even know that those commands can be used across
GC> a severe corporate firewall). We don't need to validate downloads with
GC> md5sums because git's SHA1 self-validation is stronger. We don't need
GC> tools like wget, tar, etc.

 Yes, I, unsurprisingly, agree with all this. To play the devil's advocate,
the only possible drawback I see is that you could have previously placed
all the required tarballs/ZIPs on the local filesystem and then install
without needing to download anything, which wouldn't be now doable as
easily. But with Git bundles -- which you also already use -- it should
still be possible, if really needed.

GC> I've never used git submodules before, but initializing and updating
GC> them is easy with '--recurse-submodules', and I'm sure you can easily
GC> explain any extra commands required to build them.

 I don't think any other commands will be needed. At least not until
wxWidgets adds any new submodules, but we'll worry about this when it
happens (nothing is planned for the moment).

GC> Can we conveniently make wxPdfDoc behave as a submodule of wxWidgets,
GC> locally, so that updating and building it would become as automatic
GC> as building, say, libpng as a submodule?

 To do this would require maintaining our own wxWidgets repository which,
while possible, doesn't look desirable. It also doesn't really make much
sense as wxWidgets doesn't depend on wxPdfDoc and neither is wxPdfDoc part
of wxWidgets.

 It is lmi itself which depends on both of them, so it would make sense to
make wxWidgets and wxPdfDoc submodules of lmi repository. But I hesitate to
recommend that we start using submodules for it, they're not the most
intuitive part of Git and can sometimes be confusing and I'm not sure if
the benefits of using them really outweigh the need to learn about them.
But, FWIW, if I was working on my own project, I'd definitely include all
the dependencies I build as part of it as submodules, in order to make the
repository as self-contained as possible.

 Also, just to be clear, submodules don't change anything about building
anything. They are a purely Git thing and so just allow to check out and
update the dependencies more easily (and also allow them to differ for
different branches, but this is, again, not really an issue for lmi).

GC> I take it that our accustomed way of upgrading wx no longer works;

 It will work with the (upcoming) official 3.1.1 release, but it indeed no
longer works with arbitrary Git commits.

GC> yet we might need to upgrade at any time, so we'd better rewrite
GC> 'install_wx.make' pretty soon.

 Would you like me to do this or do you prefer to do it yourself?
VZ