Re: [Qemu-devel] [PATCH V8 08/14] Introduce file lock for the block layer

[Michael S. Tsirkin]

On Wed, Sep 07, 2011 at 12:08:22PM -0400, Stefan Berger wrote:
> On 09/07/2011 11:16 AM, Michael S. Tsirkin wrote:
> >On Wed, Sep 07, 2011 at 11:06:42AM -0400, Stefan Berger wrote:
> >>On 09/07/2011 10:35 AM, Michael S. Tsirkin wrote:
> >>>On Wed, Sep 07, 2011 at 10:25:08AM -0400, Stefan Berger wrote:
> >>>>On 09/07/2011 10:10 AM, Michael S. Tsirkin wrote:
> >>>>>On Wed, Sep 07, 2011 at 09:56:52AM -0400, Stefan Berger wrote:
> >>>>>>On 09/07/2011 09:16 AM, Michael S. Tsirkin wrote:
> >>>>>>>On Wed, Sep 07, 2011 at 09:06:05AM -0400, Stefan Berger wrote:
> >>>>>>>>>>First: There are two ways to encrypt the data.
> >>>>>>>>>>
> >>>>>>>>>>One comes with the QCoW2 type of image and it comes for free. Set
> >>>>>>>>>>the encryption flag when creating the QCoW2 file and one has to
> >>>>>>>>>>provide a key to access the QCoW2. I found this mode problematic for
> >>>>>>>>>>users since it required me to go through the monitor every time I
> >>>>>>>>>>started the VM. Besides that the key is provided so late that all
> >>>>>>>>>>devices are already initialized and if the wrong key was provided
> >>>>>>>>>>the only thing the TPM can do is to go into shutdown mode since
> >>>>>>>>>>there is state on the QCoW2 but it cannot be decrypted. This also
> >>>>>>>>>>became problematic when doing migrations with libvirt for example
> >>>>>>>>>>and one was to have a wrong key/password installed on the target
> >>>>>>>>>>side -- graceful termination of the migration is impossible.
> >>>>>>>OK let's go back to this for a moment. Add a load
> >>>>>>>callback, access file there. On failure, return
> >>>>>>>an error. migration fails gracefully, and
> >>>>>>>management can retry, or migrate to another node,
> >>>>>>>or whatever.
> >>>>>>>
> >>>>>>>What's the problem exactly?
> >>>>>>>
> >>>>>>>
> >>>>>>The switch-over from source to destination already happened when the
> >>>>>>key is finally passed and you just won't be able to access the QCoW2
> >>>>>>in case the key was wrong.
> >>>>>This is exactly what happens with any kind of othe rmigration errror.
> >>>>>So fail migration, and source can get restarted if necessary.
> >>>>>
> >>>>I guess I wanted to improve on this and catch user errors.
> >>>>If we let migration fail then all you can do is try to terminate the
> >>>>VM on the destination and cold-start on the source.
> >>>No, normally if migration fails VM is not started on destination,
> >>>and it can just continue on source.
> >>>
> >>When I had tried this in conjunction with encrypted QCoW2 the
> >>switch-over already had happened and the source had died.
> >Giving continue command should bring it back.
> >
> On the source? Qemu on the source didn't exist anymore.

didn't exist? Well, fix your management to not kill
until destination starts running then.
Really, all other devices have the same problem,
we can't solve it 100% but yes, we could
make it more robust. But there appears to be
nothing TPM specific here.

> >>So a wrong key on the destination was fatal.
> >So it's a bug in the code then?
> >
> From what I saw, yes. Migration is not complete until the passwords
> had been entered. Though the requirement for a correct password
> wasn't there before because Qemu just couldn't know which password
> is correct since it doesn't know what content in a VM image is
> correct -- just using the wrong key gives you content but it's of
> course not understandable.

OK, we covered that on irc - the issue is that monitor
on destination is inactive until migration is complete.
Yes we need to fix that but no, it's not a tpm only
problem.

> >>>>>>Similar problems occur when you start a
> >>>>>>VM with an encrypted QCoW2 image. The monitor will prompt you for
> >>>>>>the password and then you start the VM and if the password was wrong
> >>>>>>the OS just won't be able to access the image.
> >>>>>>
> >>>>>>    Stefan
> >>>>>Why can't you verify the password?
> >>>>>
> >>>>I do verify the key/password in the TPM driver. If the driver cannot
> >>>>make sense of the contents of the QCoW2 due to wrong key I simply
> >>>>put the driver into failure mode. That's all I can do with encrypted
> >>>>QCoW2.
> >>>You can return error from init script which will make qemu exit.
> >>>
> >>I can return an error code when the front- and backend interfaces
> >>are initialized, but that happens really early and the encyrption
> >>key entered through the monitor is not available at this point.
> >>
> >>I also don't get a notification about when the key was entered. In
> >>case of QCoW2 encryption (and migration) I delay initialization
> >>until very late, basically when the VM accesses the tpm tis hardware
> >>emulation layer again (needs to be done this way I think to support
> >>block migration where I cannot even access the block device early on
> >>at all).
> >So it in the loadvm callback. This happens when guest is
> >stopped on source, so no need for locks.
> Two bigger cases here:
> 
> 1) Encryption key passed via command line:
>     - Migration with shared storage: When Qemu is initializing on
> the destination side I try to access the QCoW2 file. I do some basic
> tests to check whether a key was needed but none was given or
> whether some of the content could be read to confirm a valid key.
> This is done really early on during startup of Qemu on the
> destination side while or before actually the memory pages were
> transferred. Graceful termination was easily possible here.
>     - Migration using block migration: During initialization I only
> see an empty QCoW2 file (created by libvirt). I terminate at this
> point and do another initialization later on which basically comes
> down to initializing upon access of the TPM TIS interface. At this
> point graceful termination wasn't possible anymore. There may be a
> possibility to do this in the loadvm callback, assuming block
> migration at that point has already finished, which I am not quite
> sure. Though along with case 2) below this would then end up in 3
> different times for initialization of the emulation layer.
> 
> 2) QCoW2 encryption:
>     - This maps to the last case above. Also here graceful
> termination wasn't possible.
> 
> As for the loadvm callback: I have a note in my code that in case of
> QCoW2 encryption the key is not available, yet. So I even have to
> defer initialization further. In this case Qemu on the source
> machine will have terminated.
> 
>    Stefan

The point is to decrypt when you start running on dest.
At that point source is stopped. On failure,
notify management and have it restart source.
On success, management can kill source qemu.

> >On failure you return an error and migration fails
> >before destination is started. You can
> >
> 
> >>Only then I find out that the key was wrong. I guess any
> >>other handling would require blockdev.c's invocation of
> >>monitor_read_bdrv_key_start() to be 'somehow' extended and ... do
> >>what ? loop until the correct password was entered?
> >Return an error so vm start fails?
> >
> >>    Stefan
> >>>>In case of a QCoW2 encrypted VM image it's different. There I guess
> >>>>the intelligence of what is good and bad data is only inside the OS.
> >>>>
> >>>>    Stefan
>