Re: [Qemu-block] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Laszlo Ersek]

On 10/14/15 14:48, Paul Durrant wrote:
>> -----Original Message-----
>> From: Fabio Fantoni [mailto:address@hidden
>> Sent: 14 October 2015 12:12
>> To: Kevin Wolf; Stefano Stabellini
>> Cc: John Snow; Anthony Perard; address@hidden; xen-
>> address@hidden; address@hidden; Paul Durrant
>> Subject: Re: [Qemu-devel] Question about xen disk unplug support for ahci
>> missed in qemu
>>
>>
>>
>> Il 14/10/2015 11:47, Kevin Wolf ha scritto:
>>> [ CC qemu-block ]
>>>
>>> Am 13.10.2015 um 19:10 hat Stefano Stabellini geschrieben:
>>>> On Tue, 13 Oct 2015, John Snow wrote:
>>>>> On 10/13/2015 11:55 AM, Fabio Fantoni wrote:
>>>>>> I added ahci disk support in libxl and using it for week seems that was
>>>>>> ok, after a reply of Stefano Stabellini seems that xen disk unplug
>>>>>> support only ide disks:
>>>>>>
>> http://git.qemu.org/?p=qemu.git;a=commitdiff;h=679f4f8b178e7c66fbc2f39
>> c905374ee8663d5d8
>>>>>>
>>>>>> Today Paul Durrant told me that even if pv disk is ok also with ahci and
>>>>>> the emulated one is offline can be a risk:
>>>>>> http://lists.xenproject.org/archives/html/win-pv-devel/2015-
>> 10/msg00021.html
>>>>>>
>>>>>>
>>>>>> I tried to take a fast look in qemu code but I not understand the
>> needed
>>>>>> thing for add the xen disk unplug support also for ahci, can someone do
>>>>>> it or tell me useful information for do it please?
>>>>>>
>>>>>> Thanks for any reply and sorry for my bad english.
>>>>>>
>>>>> I'm not entirely sure what features you need AHCI to support in order
>>>>> for Xen to be happy.
>>>>>
>>>>> I'd guess hotplugging, but where I get confused is that IDE disks don't
>>>>> support hotplugging either, so I guess I'm not sure sure what you need.
>>>>>
>>>>> Stefano, can you help bridge my Xen knowledge gap?
>>>>
>>>> Hi John,
>>>>
>>>> we need something like hw/i386/xen/xen_platform.c:unplug_disks but
>> that
>>>> can unplug AHCI disk. And by unplug, I mean "make disappear" like
>>>> pci_piix3_xen_ide_unplug does for ide.
>>> Maybe this would be the right time to stop the craziness with your
>>> hybrid IDE/xendisk setup. It's a horrible thing that would never happen
>>> on real hardware.
> 
> Unfortunately, it's going to be difficult to remove such 'craziness' when you 
> don't know a priori whether the VM has PV drivers or not. 
> 
>>>
>>> Can't you just teach SeaBIOS how to deal with your PV disks and then
>>> only add that to your VM and forget about IDE/AHCI? I mean, that's how
>>> it's done for virtio-blk, and it doesn't involve any insanities like
>>> ripping out non-hotpluggable devices.
>>>
> 
> Does Windows have in-box virtio-blk drivers? If not, how does it boot?

The firmwares have virtio drivers (SeaBIOS and OVMF). The Windows
installer can be booted far enough, on top of the firmware services,
that Windows explicitly asks for a driver disk -- not only for the
installation *target* disk, but even for the CD-ROM that it is being
installed *from*.

In practice, you can install modern Windows (Windows 8 and later
definitely, I *think* maybe even Windows 7), from a virtio-scsi CD-ROM,
to, say, a virtio-block disk, and provide *only* the virtio drivers on a
separate IDE CD-ROM (which you can later remove completely, on the
device level).

The Windows installer's boot loader loads an initial ramdisk into
memory, using firmware services. Then its kernel starts and has access
to a number of basic drivers, like IDE CD-ROM drivers. That is
sufficient to load the virtio-scsi driver from the virtio-win ISO, and
then the installation can continue from the original virtio-scsi CD-ROM.

I always install windows guests like this; it is quite flexible. (In
brief: 1 virtio-block target disk, 1 virtio-scsi CD-ROM holding the
Windows ISO, 1 IDE CD-ROM (to be removed permanently from the guest
config, later) holding the virtio-win driver ISO.)

> 
>>> Hm... How does a reboot of a machine that had its IDE already removed
>>> actually work? Do you restart the qemu process on reboot?
>>>
> 
> The IDE disks are always present during boot, but before the OS
> enumerates them they are 'unplugged' and then PV drivers are used
> instead.

Supporting this in RHEL-5 and RHEL-6 guests was *incredible* pain.

Also, when you consider kexec / kdump in the guest, that was an unending
source of bug reports. If I recall correctly, the PV devices (net and
block) could not be used by the kdump kernel, because the main (crashed)
kernel may have left them in a bad state (and I'm not sure if it was
possible to reinit them.) And the emulated devices were slow... assuming
the kdump kernel didn't try to unplug them first!

(Maybe Vitaly (CC'd) knows more about the current status in this area;
I'm admittedly fuzzy, sorry. My intent is not to spread FUD.)

(I'll also admit that running kdump in the guest (that's already
crashed) is a bad idea in general, given that the host is there right
under it, capable of dumping the guests memory without issues. Maybe not
so flexible for some cloud setups. Whatever.)

> The only other way would be to somehow obscure them from OS
> enumeration so they could be left lying around but remain unused.
> That's feasible for Windows, but I don't know about other OS.

I think it can be solved if you remove the emulated devices completely
(on the guest configuration level), and teach the firmware(s) to boot
off the PV devices.

Thanks
Laszlo

> 
>   Paul
> 
>>> Kevin
>> I thinkthat would be a good idea, unfortunately I'm not able to do that
>> and I do not know if the developers of xen would agree to such modification.
>>
>> If will be done, for example having new disk type "xenpv" require the pv
>> drivers installed, with linux having drivers included should not be a
>> problem but on windows yes FWIK.
>> Like virtio driver should be installed before (or adding them in windows
>> install), I don't know if will be ok specifying them in install for with
>> xen driver, another possibility can be start domU with ide disk type,
>> install the xen pv drivers and reboot selecting xenpv disk type instead.
>> Doing it FWIK require not only xen and qemu changes but also pv drivers
>> change, I added on cc also Paul Durrant for see what think about.
>> With actual unplug based on my tests in some years I had many problem
>> with windows domUs (with both old and new pv drivers) resulting in
>> "windows boot blue screen error", I suppose that changing/improving
>> unplug method can decrease them, is it correct?
>>
>> About reboot xen do different from kvm recreating full domU each time
>> (and new qemu process), I don't know if is possible and good change the
>> method.
>>
>> Thanks for any reply and sorry for my bad english.
>