Re: [PATCH] import: pypi: read requirements from wheels.

[Cyril Roelandt]

On 01/24/2016 09:08 PM, Ludovic Courtès wrote:
> Cyril Roelandt <address@hidden> skribis:
> 
>> * guix/import/pypi.scm (latest-wheel-release): New function.
> 
> s/function/procedure/  :-)
> 
> Please also mention the changes in ‘guess-requirements’,
> ‘compute-inputs’, etc.
> 
> So do I get it right that pypi now provides packages both in Wheels and
> in “traditional” format, but that Wheels provides more info about
> dependencies?
> 
> IOW: What does this buy us?  :-)

When uploading a package to PyPI, one may provide a wheel in addition to
a tarball. The wheel is built by setuptools, and contains metadata,
which includes *all* the requirements specified by the authors.

It is much better than reading "requirements.txt", because this file is
not mandatory.

Since wheels may not be provided, we need to keep the old way of
retrieving dependencies, even though it is not extremely reliable.

> 
>> +(define (latest-wheel-release pypi-package)
>> +  "Return the url of the wheel for the latest release of pypi-package, of 
>> #f if
> 
> Line a bit long.  s/of/or/
> 
>> +  (define (read-wheel-metadata wheel-archive)
>> +    ;; Given WHEEL-ARCHIVE, a ZIP Python wheel archive, return the package's
>> +    ;; requirements.
>> +    (let* ((dirname (string-append
>> +                     (string-join
>> +                      (list-head
>> +                       (string-split (last (string-split wheel-url #\/))  
>> #\-) 2)
>> +                        "-")
> 
> "-" should be aligned with (list-head.
> 
> I would be best to turn this transformation into a top-level procedure,
> say ‘wheel-url->extracted-directory’.
> 
>> +                     ".dist-info"))
>> +           (json-file (string-append dirname "/metadata.json")))
>> +      (and (system* "unzip" "-q" wheel-archive json-file)
>> +           (dynamic-wind
>> +             (const #t)
>> +             (lambda ()
>> +               (call-with-input-file json-file
>> +                 (lambda (port)
>> +                   (let* ((metadata (json->scm port))
>> +                          (run_requires (hash-ref metadata "run_requires"))
>> +                          (requirements (hash-ref (list-ref run_requires 0)
>> +                                                  "requires")))
>> +                     (map (lambda (r)
>> +                            (python->package-name (clean-requirement r)))
>> +                          requirements)))))
>> +             (lambda ()
>> +               (delete-file json-file)
>> +               (rmdir dirname))))))
> 
> Eventually I wonder if we should do this in a derivation instead of
> hoping for ‘unzip’ & co. to be available there (“eventually”, because
> the problem is already present with the ‘requirements.txt’ thingie.)
> 

I think it is fine as is, seeing how it makes Python packaging much
easier :)

> Could you add a test in tests/pypi.scm?
> 

Will do!

Cyril.