Re: GeoIP database redistribution?

[Ludovic Courtès]

Efraim Flashner <address@hidden> skribis:

> On Tue, Jan 24, 2017 at 06:06:21AM +0000, Pjotr Prins wrote:
>> On Mon, Jan 23, 2017 at 07:17:12PM +0100, Marius Bakke wrote:
>> > ng0 <address@hidden> writes:
>> > 
>> > > I want to slowly package OONI (https://ooni.torproject.org/).
>> > > One of its dependencies, txtorcon, requires python-geoip which
>> > > depends on geoip-c-api. I've got both covered, but both of them
>> > > want (either to download or to be present) for tests (a/the)
>> > > legacy database file of maxmind.
>> > >
>> > > Question 1: Can we distribute the database in a source? I can't
>> > > access the homepage of maxmind for cloudflare reasons.
>> > 
>> > The database is distributed freely under cc-by-sa4.0:
>> > 
>> > https://dev.maxmind.com/geoip/legacy/geolite/#License
>> > 
>> > So packaging it should be fine. :)
>> 
>> This actually raises the issue of packaging large data files (we are
>> getting into TB's). Could there be a way Guix fetches external
>> datasets as part of the distribution? I think that if it is not
>> executable code and SHA values/pfff values match it would be safe to
>> do.
>> 
>
> The other thing is that guix downloads the files into ram and then
> writes them into the store, which can be a problem if the source is
> larger than the available memory.

It’s only if the file is added via the ‘add-to-store’ RPC, and not via
‘import-path’ or a substitute:

  https://bugs.gnu.org/23666

But yes, that can be a problem.

Ludo’.