--- Begin Message ---
|
Subject: |
Python modules installed by pip in virtualenv can't find shared objects. |
|
Date: |
Tue, 23 Jan 2018 11:49:59 +0000 |
When a python module needs to load a dynamic shared object, it looks in the
path provided by *LD_LIBRARY_PATH*(1), but guix doesn't modify this environment
variable to export the needed path for python.
* Backtrace
In my case, it's lightgbm (installed by pip), needs libgomp.so from gcc:lib for
openmp support. Here is the backtrace:
--8<---------------cut here---------------start------------->8---
Traceback (most recent call last):
File "ex1.py", line 5, in <module>
import lightgbm as lgb
File
"/home/fis/Workspace/tianchi/medical_treatment/lib/python3.5/site-packages/lightgbm/__init__.py",
line 8, in <module>
from .basic import Booster, Dataset
File
"/home/fis/Workspace/tianchi/medical_treatment/lib/python3.5/site-packages/lightgbm/basic.py",
line 32, in <module>
_LIB = _load_lib()
File
"/home/fis/Workspace/tianchi/medical_treatment/lib/python3.5/site-packages/lightgbm/basic.py",
line 27, in _load_lib
lib = ctypes.cdll.LoadLibrary(lib_path[0])
File
"/gnu/store/jb3n0bsdpkhvyb8y70jyr8fcx8fqssr9-python-3.5.3/lib/python3.5/ctypes/__init__.py",
line 425, in LoadLibrary
return self._dlltype(name)
File
"/gnu/store/jb3n0bsdpkhvyb8y70jyr8fcx8fqssr9-python-3.5.3/lib/python3.5/ctypes/__init__.py",
line 347, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libgomp.so.1: cannot open shared object file: No such file or directory
--8<---------------cut here---------------end--------------->8---
* Reproduce
To install lightgbm, simply use `pip install lightgbm`(in virtualenv).
Then in python shell:
import lightgbm as lgb
* Ad-hoc
export LD_LIBRARY_PATH=~/.guix-profile/lib:$LD_LIBRARY_PATH
[1]: https://stackoverflow.com/a/1100016
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#30229: Python modules installed by pip in virtualenv can't find shared objects. |
|
Date: |
Thu, 13 Jan 2022 15:59:42 +0100 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hi,
On Thu, 25 Nov 2021 at 00:39, zimoun <zimon.toutoune@gmail.com> wrote:
> On Tue, 23 Jan 2018 at 13:00, Ricardo Wurmus <rekado@elephly.net> wrote:
>
>>> When a python module needs to load a dynamic shared object, it looks in the
>>> path provided by *LD_LIBRARY_PATH*(1), but guix doesn't modify this
>>> environment
>>> variable to export the needed path for python.
>>
>> We cannot set this environment variable by default lest we break other
>> packages that may be installed. LD_LIBRARY_PATH is dangerous as it
>> tells the runtime linker to prefer libraries in the specified
>> directories.
>>
>> For Guix packages we use different means to embed store paths in
>> binaries, which are looked up at runtime. For binaries that’s achieved
>> with RUNPATH; for others we patch the sources to ensure that libraries
>> are not looked up merely by name but by absolute path.
>>
>> In your particular case (installing packages without Guix) I think the
>> best way is to manually set LD_LIBRARY_PATH on demand, or to set
>> LD_PRELOAD to the specific libraries that are required.
>>
>> In general, though, I recommend using Guix for package management and
>> development instead of virtualenv and pip.
>
> Regarding the improvements of ’guix import pypi’ since 2018, and because
> tweaking LD_LIBRARY_PATH is dangerous, I do not see what could be the
> next action to solve this.
>
> Therefore, I propose to close it. WDYT?
After 7 weeks of delay, I am closing.
Cheers,
simon
--- End Message ---