Re: [Duplicity-talk] Duplicity + asymmetric encryption

[Marko Božiković]

On 09/09/2015 09:45, address@hidden wrote:
> On 09.09.2015 00:03, Marko Božiković wrote:
>> On 08.09.2015. 10:37, address@hidden wrote:
>>> On 08.09.2015 00:56, Marko Božiković wrote:
>>>> On 07.09.2015. 12:08, address@hidden wrote:
>>>>> On 07.09.2015 00:24, Marko Božiković wrote:
>>>>>> On 03.08.2015. 3:31, address@hidden wrote:
>>>>>>>
>>>>>>>
>>>>>>> On August 2, 2015 11:41:31 PM GMT+02:00, "Marko Božiković" 
>>>>>>> <address@hidden> wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I've started playing with Duplicity for my cloud backup needs and I
>>>>>>>> have a few
>>>>>>>> questions regarding asymmetric encryption with gpg. Apologies for a
>>>>>>>> long mail,
>>>>>>>> but I'm confused by Duplicity's behaviour.
>>>>>>>>
>>>>>>>> I'm using Debian Jessie, duply 1.9.2, duplicity 0.7.3 and gpg 2.1.6
>>>>>>>> (these are
>>>>>>>> installed from testing/experimental repos)
>>>>>>>>
>>>>>>>> I've created a symbolic link gpg->gpg2 in order to force duplicity to
>>>>>>>> use gpg2.
>>>>>>>>
>>>>>>>> I have a gpg2 keyring setup with a master 4096 bit key, three 2048 bit
>>>>>>>> subkeys
>>>>>>>> (for signing, encryption and authentication) for everyday use that are
>>>>>>>> stored
>>>>>>>> on a smart card and a newly created 2048 bit (S, E) subkey for
>>>>>>>> duplicity backup.
>>>>>>>>
>>>>>>>> I keep my secret master key offline. The keyring on my laptop only
>>>>>>>> contains
>>>>>>>> the master public key and stubs for secret subkeys that live on the
>>>>>>>> smartcard.
>>>>>>>> The keyring on my backup machine only contains the public master key
>>>>>>>> and the
>>>>>>>> duplicity backup subkey.
>>>>>>>>
>>>>>>>> Here are the relevant bits from my duply config (I've replaced my
>>>>>>>> backup key's
>>>>>>>> fingerprint with YYYYYYYY:
>>>>>>>>
>>>>>>>> GPG_KEY='YYYYYYYY'
>>>>>>>> GPG_OPTS='--pinentry-mode loopback'
>>>>>>>> VOLSIZE=250
>>>>>>>> DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "
>>>>>>>> DUPL_PARAMS="$DUPL_PARAMS --file-prefix-archive=backup-data-
>>>>>>>> --file-prefix-manifest=includes- --file-prefix-signature=signatures-"
>>>>>>>>
>>>>>>>> Gpg 2.1 always uses gpg-agent for secret key access and it doesn't set
>>>>>>>> GPG_AGENT_INFO env. variable. This is the output I get when I run duply
>>>>>>>> with
>>>>>>>> no GPG_AGENT_INFO set (YYYYYYYY is my backup key's fingerprint and
>>>>>>>> XXXXXXXX is
>>>>>>>> my master key fingerprint)
>>>>>>>>
>>>>>>>> -------------------------------
>>>>>>>> -- BEGIN no GPG_AGENT_INFO set
>>>>>>>> -------------------------------
>>>>>>>> Start duply v1.9.2, time is 2015-08-02 22:02:48.
>>>>>>>> Using profile '/home/bozho/.duply/photo'.
>>>>>>>> Using installed duplicity version 0.7.03, python 2.7.10, gpg 2.1.6
>>>>>>>> (Home:
>>>>>>>> ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan',
>>>>>>>> grep
>>>>>>>> '(GNU grep) 2.20', bash '4.3.30(1)-release (x86_64-pc-linux-gnu)'.
>>>>>>>> Autoset found secret key of first GPG_KEY entry 'YYYYYYYY' for signing.
>>>>>>>> Cannot use gpg-agent. GPG_AGENT_INFO not set.
>>>>>>>> Checking TEMP_DIR '/tmp' is a folder (OK)
>>>>>>>> Checking TEMP_DIR '/tmp' is writable (OK)
>>>>>>>> Enter passphrase: <MY SECRET KEY PASSWORD>
>>>>>>>> Test - Encrypt to 'YYYYYYYY' & Sign with 'YYYYYYYY' (OK)
>>>>>>>> Test - Decrypt (OK)
>>>>>>>> Test - Compare (OK)
>>>>>>>> Cleanup - Delete '/tmp/duply.1629.1438549369_*'(OK)
>>>>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>>>>
>>>>>>>> WARNING:
>>>>>>>>
>>>>>>>> Backup failed.
>>>>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>>>>> passphrase
>>>>>>>> given - skipped
>>>>>>>>
>>>>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>>>>
>>>>>>>> WARNING:
>>>>>>>>
>>>>>>>> Backup failed.
>>>>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>>>>> passphrase
>>>>>>>> given - skipped
>>>>>>>>
>>>>>>>>
>>>>>>>> --- Start running command PRE at 22:03:14.018 ---
>>>>>>>> Skipping n/a script '/home/bozho/.duply/photo/pre'.
>>>>>>>> --- Finished state OK at 22:03:14.122 - Runtime 00:00:00.103 ---
>>>>>>>>
>>>>>>>> --- Start running command BKP at 22:03:14.135 ---
>>>>>>>> Reading globbing filelist /home/bozho/.duply/photo/exclude
>>>>>>>> Local and Remote metadata are synchronized, no sync needed.
>>>>>>>> Last full backup date: none
>>>>>>>> GnuPG passphrase: <ANY PASSPHRASE>
>>>>>>>> GnuPG passphrase for signing key: <MY SECRET KEY PASSWORD>
>>>>>>>> No signatures found, switching to full backup.
>>>>>>>> -------------------------------
>>>>>>>> -- END no GPG_AGENT_INFO set
>>>>>>>> -------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>> Note that I get asked a passphrase three times:
>>>>>>>>
>>>>>>>> - "Enter passphrase", which requires my secret backup subkey password -
>>>>>>>> why is
>>>>>>>> that needed here?
>>>>>>>>
>>>>>>>> - "GnuPG passphrase:", which can be any password - I'm guessing that's
>>>>>>>> used as
>>>>>>>> the session key that actually encrypts the data and then gets encrypted
>>>>>>>> by my
>>>>>>>> public subkey. Why is this necessary? In OpenPGP, the session key
>>>>>>>> can/should
>>>>>>>> be a long random key - the recipient doesn't need to know it.
>>>>>>>>
>>>>>>>> - "GnuPG passphrase for signing key" - ok, this one is for signing,
>>>>>>>> since it's
>>>>>>>> enabled and I'm using the same key.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Now, if I restart gpg-agent in order to reset its caching and actually
>>>>>>>> export
>>>>>>>> GPG_AGENT_INFO, I get a slightly different output:
>>>>>>>>
>>>>>>>> -------------------------------
>>>>>>>> -- BEGIN GPG_AGENT_INFO set
>>>>>>>> -------------------------------
>>>>>>>> Start duply v1.9.2, time is 2015-08-02 22:12:42.
>>>>>>>> Using profile '/home/bozho/.duply/photo'.
>>>>>>>> Using installed duplicity version 0.7.03, python 2.7.10, gpg 2.1.6
>>>>>>>> (Home:
>>>>>>>> ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright(C) Michael D. Brennan',
>>>>>>>> grep
>>>>>>>> '(GNU grep) 2.20', bash '4.3.30(1)-release (x86_64-pc-linux-gnu)'.
>>>>>>>> Autoset found secret key of first GPG_KEY entry 'YYYYYYYY' for signing.
>>>>>>>> Autoenable use of gpg-agent. GPG_PW or GPG_PW_SIGN (enc != sign key)
>>>>>>>> not set.
>>>>>>>> Checking TEMP_DIR '/tmp' is a folder (OK)
>>>>>>>> Checking TEMP_DIR '/tmp' is writable (OK)
>>>>>>>> Enter passphrase: <MY SECRET KEY PASSWORD>
>>>>>>>> Test - Encrypt to 'YYYYYYYY' & Sign with 'YYYYYYYY' (OK)
>>>>>>>> Test - Decrypt (OK)
>>>>>>>> Test - Compare (OK)
>>>>>>>> Cleanup - Delete '/tmp/duply.4740.1438549962_*'(OK)
>>>>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>>>>
>>>>>>>> WARNING:
>>>>>>>>
>>>>>>>> Backup failed.
>>>>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>>>>> passphrase
>>>>>>>> given - skipped
>>>>>>>>
>>>>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>>>>
>>>>>>>> WARNING:
>>>>>>>>
>>>>>>>> Backup failed.
>>>>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>>>>> passphrase
>>>>>>>> given - skipped
>>>>>>>>
>>>>>>>>
>>>>>>>> --- Start running command PRE at 22:13:22.870 ---
>>>>>>>> Skipping n/a script '/home/bozho/.duply/photo/pre'.
>>>>>>>> --- Finished state OK at 22:13:22.956 - Runtime 00:00:00.085 ---
>>>>>>>>
>>>>>>>> --- Start running command BKP at 22:13:22.966 ---
>>>>>>>> Reading globbing filelist /home/bozho/.duply/photo/exclude
>>>>>>>> Local and Remote metadata are synchronized, no sync needed.
>>>>>>>> Last full backup date: none
>>>>>>>> No signatures found, switching to full backup.
>>>>>>>> -------------------------------
>>>>>>>> -- END GPG_AGENT_INFO set
>>>>>>>> -------------------------------
>>>>>>>>
>>>>>>>> Notice that I get asked for a password only once, at the beginning
>>>>>>>> ("Enter
>>>>>>>> passphrase") Why is that, and is the session key being randomly
>>>>>>>> generated in
>>>>>>>> this case or is my secret subkey password used as a session key (this
>>>>>>>> is a bad
>>>>>>>> idea)
>>>>>>>>
>>>>>>>> Thank you,
>>>>>>>
>>>>>>>
>>>>>>> probably 
>>>>>>>  https://sourceforge.net/p/ftplicity/feature-requests/36/
>>>>>>>
>>>>>>> try setting --use-agent manually in conf.
>>>>>>>
>>>>>>> ..ede/duply.net
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Sorry for a late reply.
>>>>>>
>>>>>> I tried using your suggestion and added --use-agent to DUPL_PARAMS, but 
>>>>>> it
>>>>>> didn't help.
>>>>>>
>>>>>> I've tried with and without setting GPG_AGENT_INFO variable, I get the 
>>>>>> same
>>>>>> output after being asked for the key password:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Start duply v1.9.2, time is 2015-09-05 22:45:50.
>>>>>> Using profile '/home/bozho/.duply/photo'.
>>>>>> Using installed duplicity version 0.7.03, python 2.7.10, gpg 2.1.6 (Home:
>>>>>> ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', 
>>>>>> grep
>>>>>> '(GNU grep) 2.21', bash '4.3.39(1)-release (x86_64-pc-linux-gnu)'.
>>>>>> Autoset found secret key of first GPG_KEY entry 'YYYYYYYY' for signing.
>>>>>> Autoenable use of gpg-agent. GPG_PW or GPG_PW_SIGN (enc != sign key) not 
>>>>>> set.
>>>>>> Checking TEMP_DIR '/tmp' is a folder (OK)
>>>>>> Checking TEMP_DIR '/tmp' is writable (OK)
>>>>>> Test - Encrypt to 'YYYYYYYY' & Sign with 'YYYYYYYY' (OK)
>>>>>> Test - Decrypt (OK)
>>>>>> Test - Compare (OK)
>>>>>> Cleanup - Delete '/tmp/duply.3034.1441489550_*'(OK)
>>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>>
>>>>>> WARNING:
>>>>>>
>>>>>> Backup failed.
>>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No passphrase
>>>>>> given - skipped
>>>>>>
>>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>>
>>>>>> WARNING:
>>>>>>
>>>>>> Backup failed.
>>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No passphrase
>>>>>> given - skipped
>>>>>>
>>>>>>
>>>>>> --- Start running command PRE at 22:45:54.331 ---
>>>>>> Skipping n/a script '/home/bozho/.duply/photo/pre'.
>>>>>> --- Finished state OK at 22:45:54.410 - Runtime 00:00:00.079 ---
>>>>>>
>>>>>> --- Start running command BKP at 22:45:54.419 ---
>>>>>> Reading globbing filelist /home/bozho/.duply/photo/exclude
>>>>>> Local and Remote metadata are synchronized, no sync needed.
>>>>>> Last full backup date: none
>>>>>> No signatures found, switching to full backup.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> However, the backup seems to proceed normally...
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>
>>>>> can i see your duply conf file? obfuscate private strings in it 
>>>>> beforehand.
>>>>
>>>> Here you go.
>>>>
>>>> Thank you for persevering with this :)
>>>>
>>>> Just as a note (I don't remember if I've mentioned this before), I moved my
>>>> /usr/bin/gpg to /usr/bin/gpg1 and created /usr/bin/gpg symlink to gpg2.
>>>>
>>>>
>>>
>>> i notice you do not set the gpg passphrase. this is the classic way to 
>>> provide it, the other being gpg-agent. 
>>>
>>> for testing purposes please try and set GPG_PW and see if it works as 
>>> expected. don't be confused that you need the gpg-agent parameters 
>>> (allow-loopback-pinentry in your .gnupg/gpg-agent.conf and 
>>> GPG_OPTS='--pinentry-mode loopback') for that to work as gpg-agent is now 
>>> (since gpg2.1) the "unified" passphrase input interface to gpg2.1.
>>
>> Ok, if I set GPG_PW, I get an output with no errors, but duplicity exports my
>> keys to files: gpgkey.YYYYYYYY.pub.asc and gpgkey.YYYYYYYY.sec.asc.
>>
>>> alternatively you can try the gpg-agent route. therefor you remove the 
>>> GPG_OPTS conf param and add DUPL_PARAMS="$DUPL_PARAMS --use-agent". i 
>>> didn't come around to implement a senseful route to deal with the gpg2.1 
>>> changes so for for duply.
>>
>> If I remove GPG_OPTS (loopback pin entry), I also get no errors and the keys
>> are exported to files.
>>
> 
> so both setups w/ and w/o gpg-agent work for you now?

Yeah, it seems that if there's a password specified, it will get used to
export the keys and if I want to use gpg-agent, I need to disable loopback
pinentry option.

>> Why is there a need to have keys exported in files? They are protected with a
>> password, but is there a practical need for them to be in files, rather than
>> requested from the agent when needed?
> 
> it's a convenience feature since the early duply days. this way your profile 
> folder contains everything necessary to restore your backup on a different 
> machine.

Ah, ok. Makes sense.

Thank you,
-- 
Marko