bug#10227: Python installation fails for Python 3

[Stefano Lattarini]

tags 10227 + patch
close 10227
thanks

On 11/23/2012 10:26 AM, Stefano Lattarini wrote:
> On 11/22/2012 11:35 PM, Roumen Petrov wrote:
>> Stefano Lattarini wrote:
>>> tags 10227 - moreinfo
>>> thanks
>>>
>>> On 11/21/2012 06:58 PM, Reuben Thomas wrote:
>>>> On 21 November 2012 13:41, Stefano Lattarini <address@hidden>wrote:
>>>>
>>>>> tags 10227 + moreinfo
>>>>> thanks
>>>>>
>>>>> Hi Roumen, Reuben.
>>>>>
>>>>> I'm going through old open bugs, and I've noticed this one.  Is the
>>>>> problem still present, after the recent updates to the python support?
>> [SNIP]
>>>>> on my Ubuntu 12.10 system). I'm sorry, I can't tell whether, as Roumen
>>>>> suggests, this is an Ubuntu packaging bug, or an automake bug.
>> It seems to me this is a feature as I found this one:
>>
>> http://patch-tracker.debian.org/patch/series/view/python2.6/2.6.6-8/distutils-install-layout.diff
>>
> The problem didn't lie in the Debian's use of 'dist-packages' instead
> of 'site-packages' (as you noticed, that was present in python 2.6
> already, and Automake still worked nicely); the issue is that, with
> Python 3 on Debian, the call to
> 
>     distutils.sysconfig.get_python_lib(0,0,prefix='/usr/local')
> 
> returned:
> 
>     /usr/local/lib/python3/dist-packages
> 
> that is *not* searched by default by the 'import' statement.
> 
>>> OK, thanks for explaining it once again.  I can now reproduce the same
>>> issue on Debian.  I think this is something we should try to work around,
>>> since we cannot have our installation rules broken by default on both
>>> Debian and Ubuntu ...
>>
>> Lets say platforms that use the "/posix_local/" installation scheme
>> instead wrong or broken.
>>
> I failed to parse this, sorry.  Care to rephrase?
> 
>>>> It seems to me that this is the sort of thing the pyconfigure people
>>>> ought to know about.
>>>>
>>> By peeking at their repository, I came up with the minimal patch below,
>>> which should do the trick.  Can you verify it works for you as well?
>>>
> 
>>> Signed-off-by: Stefano Lattarini <address@hidden>
>>> ---
>>>   [SNIP]
>>>
>>> @@ -122,7 +141,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>>        else
>>>          am_py_prefix=$prefix
>>>        fi
>>> -     am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import 
>>> sysconfig; 
>>> sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))" 
>>> 2>/dev/null`
>>> +     am_cv_python_pythondir=`$PYTHON -c "
>>> +$am_python_setup_sysconfig
>>> +if can_use_sysconfig:
>>> +    sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'})
>>> +else:
>>> +    from distutils import sysconfig
>>> +    sitedir = sysconfig.get_python_lib(0, 0, prefix='$am_py_prefix')
>>> +sys.stdout.write(sitedir)"`
>>
>> Tested with python build from master branch on  2012-11-13 - both paths
>> (sysconfig and distutils.sysconfig based) return same results.
>>
> Thanks for pointing that out.  I've given the code a try with a python 3.2
> installed from sources, and I can confirm this happens for me as well.  So
> no regression for that kind of setup, at least.
> 
> On the other hand, for a python 3.2 installed from Debian packages, the
> results are different:
> 
>     # Code with distutils.sysconfig
>     ${prefix}/lib/python3/dist-packages
> 
>     # Code with sysconfig
>     ${prefix}/lib/python3.2/site-packages
> 
> The second result is the correct one; in fact, the 'dist-packages'
> directory is meant as the location of python modules installed from
> Debian packages (reference: <http://wiki.debian.org/Python>), so we
> shouldn't need to install into such a directory; and when ${prefix}
> is /usr/local, the "${prefix}/lib/python3.2/site-packages" directory
> is searched by default by the 'import' statement.
> 
> At this point, I'm convinced my patch is quite correct.
> 
>>>        case $am_cv_python_pythondir in
>>>        $am_py_prefix*)
>>>          am__strip_prefix=`echo "$am_py_prefix" | sed 's|.|.|g'`
>>> @@ -157,7 +183,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>>        else
>>>          am_py_exec_prefix=$exec_prefix
>>>        fi
>>> -     am_cv_python_pyexecdir=`$PYTHON -c "import sys; from distutils import 
>>> sysconfig; 
>>> sys.stdout.write(sysconfig.get_python_lib(1,0,prefix='$am_py_exec_prefix'))"
>>>  2>/dev/null`
>>> +     am_cv_python_pyexecdir=`$PYTHON -c "
>>> +$am_python_setup_sysconfig
>>> +if can_use_sysconfig:
>>> +    sitedir = sysconfig.get_path('platlib', vars={'base':'$am_py_prefix'})
>>
>> Stefano, please use platbase as variable - see Lib/sysconfig.py:
>> ....
>> INSTALL_SCHEMES = {
>>     'posix_prefix': {
>> ...
>>         'purelib': '{base}/lib/python{py_version_short}/site-packages',
>>         'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
>> ...
>>
> Good catch!  I will fix this.
> 
>>> +else:
>>> +    from distutils import sysconfig
>>> +    sitedir = sysconfig.get_python_lib(1, 0, prefix='$am_py_prefix')
>>> +sys.stdout.write(sitedir)"`
>>>        case $am_cv_python_pyexecdir in
>>>        $am_py_exec_prefix*)
>>>          am__strip_prefix=`echo "$am_py_exec_prefix" | sed 's|.|.|g'`
>>
>> Also I think that python experts should confirm if patch is correct.
>>
> I'd love that.  Feel free to ping any GNU python expert about giving
> a look at this patch.
> 
I've pushed this patch, since it seems quite correct to me (and I
think we have correctly understood the underlying issue).  I'm ready
to revisit this change if anyone complains.

I'm marking this issue as "done".

Thanks,
  Stefano