Re: pypi URL change

[Ben Woodcroft]

On 25/04/16 08:26, Cyril Roelandt wrote:
> On 04/24/2016 09:27 AM, Ben Woodcroft wrote:
> > Hi,
> >
> > Today I tried using pypi-url to autogenerate a package that was only
> > released today. Unfortunately, it seems that the URL scheme has changed
> > and I was unable to download, though I believe this only affects newly
> > released package versions. I found this on the nix mailing list:
> > http://comments.gmane.org/gmane.linux.distributions.nixos/20158
> >
> > >     So, previously PyPI used URLs like :
> > > //packages//{python version}/{name[0]}/{name}/{filename}
> > >
> > >     Now it uses:
> > > //packages//{hash[:2]}/{hash[2:4]}/{hash[4:]}/{filename}
> > >     Where hash is blake2b(file_content, digest_size=32).hexdigest().lower()
> > Is there a blake2b implemented in Guile? In the meantime, the URL will have to 
> > be specified manually: 'guix import pypi ...' provides the correct and updated 
> > URL.
> The URL should always be correct, because the pypi importer can detect
> whether the URL build by pypi-uri is wrong (see make-pypi-sexp).
>
> Now, in order to handle this new pattern, maybe pypi-uri should:
>
> 1) try to compute the "new URI"
> 2) issue a "curl --head $uri" using some Guile magic
> 3) return the URI or compute the "old URI" depending on 2)
>
> WDYT?

Are suggesting that pypi-uri queries pypi each time it is called, even 
during a build? That seems a little wasteful.

I think there is actually 2 problems here.

1) 'guix refresh -u' updates the package with 'pypi-uri', not the URL it 
couldn't guess.
2) The new URLs are even uglier than the old one and a working pypi-uri 
would hide them from our eyes.

Fixing (1) should be easier and guards against future URL changes too, 
but fixing (2) would be good too.

I'm not familiar with Guile bindings, but there is a C library for 
blake2b so perhaps could simply bind to that?
https://github.com/BLAKE2/libb2

Thoughts?

Thanks,
ben