Re: [Duplicity-talk] Unable to import 'kerberos'/'pydevd'

[Kenneth Loafman]

Hi ede,

I noticed one thing while working on this, after the commit of course.  😏  If the import is within a try/except block, then pylint does not check it.  What I need to do now is back out some of the noise and make sure that we use try/except around optional imports.  Wish that had been documented a bit better!

Essentially, a backend should only import if its class is initiated.  In dpbxbackend.py, global makes sure it's available through the entire module and try/except keeps pylint from trying to import them.  That and the except gives us the option to tell the user where to find the imports they need.

...Ken

On Wed, Dec 20, 2017 at 7:42 AM, edgar.soldin--- via Duplicity-talk <address@hidden> wrote:

hey Ken, Aaron,

those pylint setting comments are bothersome to add and maintain. sporadic backend authors will probably forget to add those as well.

wouldn't it be easier to configure pylint during runtime to ignore those. there seems to be a setting for the pylintrc.

[TYPECHECK]
ignored-modules = something

..ede

On 20.12.2017 14:07, Kenneth Loafman via Duplicity-talk wrote:
> Hi Aaron,
>
> I just committed a fix for pylint 1.8.1 that's cleaner than backing off to 1.75.
>
> Pay attention to dpbxbackend.py.  We may need to do more like that.
>
> ...Thanks,
> ...Ken
>
>
> On Tue, Dec 19, 2017 at 5:49 AM, Kenneth Loafman <address@hidden <mailto:address@hidden>> wrote:
>
>     Whoops, cancel that last msg!  Just noticed that I had not used 'tox -r' in a while and I was still on 1.7.5, not 1.8.1, of pylint under tox.  Looks like we need to back off to 1.7.5 until we can find a fix.
>
>
>     *With pylint 1.7.5:*
>
>
>    address@hidden:~/workspace/duplicity-src8$ tox -e code
>     GLOB sdist-make: /Users/ken/workspace/duplicity-src8/setup.py
>     code inst-nodeps: /Users/ken/workspace/duplicity-src8/.tox/dist/duplicity-0.0dev.tar.gz
>     code installed:
>     args==0.1.0,asn1crypto==0.23.0,astroid==1.5.3,attrs==17.3.0,Babel==2.5.1,backports.functools-lru-cache==1.4,bcrypt==3.1.4,boto==2.48.0,certifi==2017.11.5,cffi==1.11.2,chardet==3.0.4,clint==0.5.1,configparser==3.5.0,coverage==4.4.2,cryptography==2.1.4,debtcollector==1.19.0,dropbox==6.9.0,duplicity==0.0.dev0,enum34==1.1.6,fasteners==0.14.1,funcsigs==1.0.2,future==0.16.0,google-api-python-client==1.6.4,httplib2==0.10.3,humanize==0.5.1,idna==2.6,ip-associations-python-novaclient-ext==0.2,ipaddress==1.0.18,iso8601==0.1.12,isort==4.2.15,jottalib==0.5.1,keyring==10.5.0,keystoneauth1==3.3.0,lazy-object-proxy==1.3.1,lxml==4.1.1,mccabe==0.6.1,mock==2.0.0,monotonic==1.4,msgpack-python==0.4.8,netaddr==0.7.19,netifaces==0.10.6,oauth2client==4.1.2,os-diskconfig-python-novaclient-ext==0.1.3,os-networksv2-python-novaclient-ext==0.26,os-virtual-interfacesv2-python-novaclient-ext==0.20,oslo.config==5.1.0,oslo.i18n==3.19.0,oslo.serialization==2.22.0,oslo.utils==3.33.0,paramiko==2.4.0,pbr==3.1.1,pexpect==4.3.1,pluggy==0.6.0,prettytable==0.7.2,ptyprocess==0.5.2,py==1.5.2,pyasn1==0.4.2,pyasn1-modules==0.2.1,pycodestyle==2.3.1,pycparser==2.18,PyDrive==1.3.1,pylint==1.7.5,PyNaCl==1.2.1,pyparsing==2.2.0,pyrax==1.9.8,pytest==3.3.1,pytest-runner==3.0,python-cloudfiles==1.7.11,python-dateutil==2.6.1,python-keystoneclient==3.14.0,python-novaclient==2.27.0,pytz==2017.3,PyYAML==3.12,rackspace-auth-openstack==1.3,rackspace-novaclient==2.1,rax-default-network-flags-python-novaclient-ext==0.4.0,rax-scheduled-images-python-novaclient-ext==0.3.1,requests==2.18.4,requests-toolbelt==0.8.0,rfc3986==1.1.0,rsa==3.4.2,simplejson==3.13.2,singledispatch==3.4.0.3,six==1.11.0,stevedore==1.28.0,tox==2.9.1,typing==3.6.2,uritemplate==3.0.0,urllib3==1.22,virtualenv==15.1.0,wrapt==1.10.11
>     code runtests: PYTHONHASHSEED='2844061312'
>     code runtests: commands[0] | pytest testing/test_code.py
>     ================================================= test session starts ==================================================
>     platform darwin -- Python 2.7.14, pytest-3.3.1, py-1.5.2, pluggy-0.6.0
>     rootdir: /Users/ken/workspace/duplicity-src8, inifile: tox.ini
>     collected 3 items                                                                                                      
>
>     testing/test_code.py ...                                                                                         [100%]
>
>     ============================================== 3 passed in 55.03 seconds ===============================================
>     _______________________________________________________ summary ________________________________________________________
>       code: commands succeeded
>       congratulations :)
>
>     *
>     *
>     *With pylint 1.8.1:*

>
>
>    address@hidden:~/workspace/duplicity-src8$ tox -r -e code
>     GLOB sdist-make: /Users/ken/workspace/duplicity-src8/setup.py
>     code recreate: /Users/ken/workspace/duplicity-src8/.tox/code
>     code installdeps: -rrequirements.txt
>     code inst: /Users/ken/workspace/duplicity-src8/.tox/dist/duplicity-0.0dev.tar.gz
>     code installed:
>     args==0.1.0,asn1crypto==0.24.0,astroid==1.6.0,attrs==17.3.0,Babel==2.5.1,backports.functools-lru-cache==1.4,bcrypt==3.1.4,boto==2.48.0,certifi==2017.11.5,cffi==1.11.2,chardet==3.0.4,clint==0.5.1,configparser==3.5.0,coverage==4.4.2,cryptography==2.1.4,debtcollector==1.19.0,dropbox==6.9.0,duplicity==0.0.dev0,enum34==1.1.6,fasteners==0.14.1,funcsigs==1.0.2,future==0.16.0,google-api-python-client==1.6.4,httplib2==0.10.3,humanize==0.5.1,idna==2.6,ip-associations-python-novaclient-ext==0.2,ipaddress==1.0.19,iso8601==0.1.12,isort==4.2.15,jottalib==0.5.1,keyring==10.5.1,keystoneauth1==3.3.0,lazy-object-proxy==1.3.1,lxml==4.1.1,mccabe==0.6.1,mock==2.0.0,monotonic==1.4,msgpack-python==0.4.8,netaddr==0.7.19,netifaces==0.10.6,oauth2client==4.1.2,os-diskconfig-python-novaclient-ext==0.1.3,os-networksv2-python-novaclient-ext==0.26,os-virtual-interfacesv2-python-novaclient-ext==0.20,oslo.config==5.1.0,oslo.i18n==3.19.0,oslo.serialization==2.22.0,oslo.utils==3.33.0,paramiko==2.4.0,pbr==3.1.1,pexpect==4.3.1,pluggy==0.6.0,prettytable==0.7.2,ptyprocess==0.5.2,py==1.5.2,pyasn1==0.4.2,pyasn1-modules==0.2.1,pycodestyle==2.3.1,pycparser==2.18,PyDrive==1.3.1,pylint==1.8.1,PyNaCl==1.2.1,pyparsing==2.2.0,pyrax==1.9.8,pytest==3.3.1,pytest-runner==3.0,python-cloudfiles==1.7.11,python-dateutil==2.6.1,python-keystoneclient==3.14.0,python-novaclient==2.27.0,pytz==2017.3,PyYAML==3.12,rackspace-auth-openstack==1.3,rackspace-novaclient==2.1,rax-default-network-flags-python-novaclient-ext==0.4.0,rax-scheduled-images-python-novaclient-ext==0.3.1,requests==2.18.4,requests-toolbelt==0.8.0,rfc3986==1.1.0,rsa==3.4.2,simplejson==3.13.2,singledispatch==3.4.0.3,six==1.11.0,stevedore==1.28.0,tox==2.9.1,typing==3.6.2,uritemplate==3.0.0,urllib3==1.22,virtualenv==15.1.0,wrapt==1.10.11
>     code runtests: PYTHONHASHSEED='1763987639'
>     code runtests: commands[0] | pytest testing/test_code.py
>     ================================================= test session starts ==================================================
>     platform darwin -- Python 2.7.14, pytest-3.3.1, py-1.5.2, pluggy-0.6.0
>     rootdir: /Users/ken/workspace/duplicity-src8, inifile: tox.ini
>     collected 3 items                                                                                                      
>
>     testing/test_code.py ..F                                                                                         [100%]
>
>     ======================================================= FAILURES =======================================================
>     _________________________________________________ CodeTest.test_pylint _________________________________________________
>
>     self = <testing.test_code.CodeTest testMethod=test_pylint>
>
>         @skipCodeTest
>         def test_pylint(self):
>             """Pylint test (requires pylint to be installed to pass)"""
>             self.run_checker([
>                 "pylint",
>                 "-E",
>                 "--msg-template={msg_id}: {line}: {msg}",
>                 "--disable=E0203",  # Access to member before its definition line
>                 "--disable=E0602",  # Undefined variable
>                 "--disable=E0611",  # No name in module
>                 "--disable=E1101",  # Has no member
>                 "--disable=E1103",  # Maybe has no member
>                 "--disable=E0712",  # Catching an exception which doesn't inherit from BaseException
>                 "--ignore=_librsync.so",
>                 "--msg-template='{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}'",
>                 os.path.join(_top_dir, 'duplicity'),
>                 os.path.join(_top_dir, 'bin/duplicity'),
>                 os.path.join(_top_dir, 'bin/rdiffdir')],
>                 # Allow usage errors, older versions don't have
>                 # --msg-template
>     >           [0, 32])
>
>     test_code.py:87: 
>     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>     test_code.py:43: in run_checker
>         self.assertTrue(process.returncode in returncodes, output)
>     E   AssertionError: ************* Module duplicity.backends.webdavbackend
>     E   /Users/ken/workspace/duplicity-src8/duplicity/backends/webdavbackend.py:275: [E0401(import-error), WebDAVBackend.get_kerberos_authorization] Unable to import 'kerberos'
>     E   ************* Module duplicity
>     E   /Users/ken/workspace/duplicity-src8/bin/duplicity:74: [E0401(import-error), ] Unable to import 'pydevd'
>     ========================================= 1 failed, 2 passed in 57.26 seconds ==========================================
>     ERROR: InvocationError: '/Users/ken/workspace/duplicity-src8/.tox/code/bin/pytest testing/test_code.py'
>     _______________________________________________________ summary ________________________________________________________
>     ERROR:   code: commands failed
>
>

>     On Tue, Dec 19, 2017 at 5:25 AM, Kenneth Loafman <address@hidden <mailto:address@hidden>> wrote:
>
>         Can't really think of anything.  Maybe an old pylint?  I'm running:
>
>         pylint 1.8.1, 
>         astroid 1.6.0
>         Python 2.7.14
>
>         ...Thanks,
>         ...Ken
>
>
>         On Tue, Dec 19, 2017 at 4:59 AM, Aaron <address@hidden <mailto:address@hiddennz>> wrote:
>
>             Thanks Ken,
>
>             On 2017-12-18 23:05, Kenneth Loafman wrote:
>
>                 Pydevd is only used when debugging under Eclipse or LiClipse.  It's only imported if --pydevd is supplied.
>
>                 Kerberos should only be imported if the backend is instantiated.
>
>                 Sounds like something is moving imports.
>
>
>             It is the Pylint check, so I do not think it is moving imports so much as testing all the imports even if they are within conditionals/function defs.
>
>             Is there anything else that you can think of that you have installed that may be letting this import for you but not me?
>
>             Kind regards,
>
>             Aaron
>
>
>
>
>
>

> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>

_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk