Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

Tried it again,

Important to understand that after modifying and saving configuration of xml with virsh edit $name_of_guest

when i reenter the xml i can't see the iothread configuration in there. don't understand why.

Also, after add what you suggested I checked with "info qtree" command on the guest looking for sign of data-plane is working,

this is the output for info qtree: 

      dev: virtio-blk-pci, id "virtio-disk1"

        class = 0 (0x0)

        ioeventfd = true

        vectors = 2 (0x2)

        virtio-pci-bus-master-bug-migration = false

        disable-legacy = false

        disable-modern = true

        migrate-extra = false

        modern-pio-notify = false

        x-disable-pcie = false

        addr = 07.0

        romfile = ""

        rombar = 1 (0x1)

        multifunction = false

        command_serr_enable = true

        class SCSI controller, addr 00:07.0, pci id 1af4:1001 (sub 1af4:0002)

        bar 0: i/o at 0xc0c0 [0xc0ff]

        bar 1: mem at 0xfebf3000 [0xfebf3fff]

        bus: virtio-bus

          type virtio-pci-bus

          dev: virtio-blk-device, id ""

            drive = "drive-virtio-disk1"

            logical_block_size = 512 (0x200)

            physical_block_size = 512 (0x200)

            min_io_size = 0 (0x0)

            opt_io_size = 0 (0x0)

            discard_granularity = 4294967295 (0xffffffff)

            cyls = 16383 (0x3fff)

            heads = 16 (0x10)

            secs = 63 (0x3f)

            serial = ""

            config-wce = true

            scsi = true

            request-merging = true

            indirect_desc = true

            event_idx = true

            notify_on_empty = true

            any_layout = false

      dev: virtio-blk-pci, id "virtio-disk0"

        class = 0 (0x0)

        ioeventfd = true

        vectors = 2 (0x2)

        virtio-pci-bus-master-bug-migration = false

        disable-legacy = false

        disable-modern = true

        migrate-extra = false

        modern-pio-notify = false

        x-disable-pcie = false

        addr = 04.0

        romfile = ""

        rombar = 1 (0x1)

        multifunction = false

        command_serr_enable = true

        class SCSI controller, addr 00:04.0, pci id 1af4:1001 (sub 1af4:0002)

        bar 0: i/o at 0xc040 [0xc07f]

        bar 1: mem at 0xfebf1000 [0xfebf1fff]

        bus: virtio-bus

          type virtio-pci-bus

          dev: virtio-blk-device, id ""

            drive = "drive-virtio-disk0"

            logical_block_size = 512 (0x200)

            physical_block_size = 512 (0x200)

            min_io_size = 0 (0x0)

            opt_io_size = 0 (0x0)

            discard_granularity = 4294967295 (0xffffffff)

            cyls = 16383 (0x3fff)

            heads = 16 (0x10)

            secs = 63 (0x3f)

            serial = ""

            config-wce = true

            scsi = false

            request-merging = true

            indirect_desc = true

            event_idx = true

            notify_on_empty = true

            any_layout = false

      dev: virtio-scsi-pci, id "scsi1"

        ioeventfd = true

        vectors = 4 (0x4)

        virtio-pci-bus-master-bug-migration = false

        disable-legacy = false

        disable-modern = true

        migrate-extra = false

        modern-pio-notify = false

        x-disable-pcie = false

        addr = 08.0

        romfile = ""

        rombar = 1 (0x1)

        multifunction = false

        command_serr_enable = true

        class SCSI controller, addr 00:08.0, pci id 1af4:1004 (sub 1af4:0008)

        bar 0: i/o at 0xc100 [0xc13f]

        bar 1: mem at 0xfebf4000 [0xfebf4fff]

        bus: virtio-bus

          type virtio-pci-bus

          dev: virtio-scsi-device, id ""

            num_queues = 1 (0x1)

            max_sectors = 65535 (0xffff)

            cmd_per_lun = 128 (0x80)

            hotplug = true

            param_change = true

            indirect_desc = true

            event_idx = true

            notify_on_empty = true

            any_layout = true

            bus: scsi1.0

              type SCSI

              dev: scsi-block, id "scsi1-0-0-0"

                drive = "drive-scsi1-0-0-0"

                channel = 0 (0x0)

                scsi-id = 0 (0x0)

                lun = 0 (0x0)

Thnaks,

Roy

On Wed, Jan 27, 2016 at 9:03 PM, Roy Shterman <address@hidden> wrote:

Hi,

First of all thank very much for your help,

Second, unfortunately data-plane didn't worked well, I tried to add threads from the instructions you gave me.

Here is my full xml file, maybe you can help me to understand why it didn't worked :

<domain type='kvm'>

  <name>gen-r-vrt-105-007-RH7.0x64</name>

  <uuid>8f79e97e-d452-4577-82bd-2ed903773026</uuid>

  <memory unit='KiB'>2097152</memory>

  <currentMemory unit='KiB'>2097152</currentMemory>

  <memtune>

    <hard_limit unit='KiB'>8388608</hard_limit>

  </memtune>

  <memoryBacking>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>2</vcpu>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>

    <boot dev='hd'/>

  </os>

  <features>

    <acpi/>

    <apic/>

    <pae/>

  </features>

  <clock offset='utc'/>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/.autodirect/mtrswgwork/roysh/git/qemu/x86_64-softmmu/qemu-system-x86_64</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='qcow2' cache='none'/>

      <source file='/images/gen-r-vrt-105-007/gen-r-vrt-105-007.img'/>

      <target dev='vda' bus='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='network' device='lun'>

      <driver name='qemu' type='raw'/>

      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1/iser'>

        <host name='11.212.32.52' port='3260'/>

      </source>

      <target dev='vdb' bus='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>

    </disk>

    <disk type='network' device='lun'>

      <driver name='qemu' type='raw'/>

      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1'>

        <host name='11.212.32.52' port='3260'/>

      </source>

      <target dev='sda' bus='scsi'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='network' device='lun'>

      <driver name='qemu' type='raw'/>

      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-2/1'>

        <host name='11.212.32.52' port='3260'/>

      </source>

      <target dev='sdb' bus='scsi'/>

      <address type='drive' controller='1' bus='0' target='0' unit='0'/>

    </disk>

    <controller type='scsi' index='0' model='virtio-scsi'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </controller>

    <controller type='scsi' index='1' model='virtio-scsi'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </controller>

    <controller type='usb' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <interface type='bridge'>

      <mac address='00:50:56:28:69:07'/>

      <source bridge='br0:'/>

      <model type='e1000'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <input type='tablet' bus='usb'/>

    <input type='mouse' bus='ps2'/>

    <graphics type='vnc' port='-1' autoport='yes'/>

    <video>

      <model type='cirrus' vram='9216' heads='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </video>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </memballoon>

  </devices>

</domain>

BTW, in RH 7.2 data-plane is default if one is choosing to work with virtio?

Thank you very much,

Roy

On Thu, Jan 21, 2016 at 11:01 AM, Paolo Bonzini <address@hidden> wrote:

On 20/01/2016 21:12, Roy Shterman wrote:
> Hi,
>
> I have two questions,
>
> First, I'm developing for Libiscsi and trying to work with virtio-scsi
> dataplane or even virtio-blk dataplane and it doesn't works well.
>
> I'm working with latest qemu and latest Libiscsi in RedHat 7 libvirt
> package.
>
> my iscsi xml part is :
>
> virtio-blk -
>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='vdb' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
>     </disk>
>
> virtio-scsi -
>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='sda' bus='scsi'/>
>       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>     </disk>
>     <controller type='scsi' index='0' model='virtio-scsi'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0'/>
>     </controller>

There is now support for dataplane in libvirt.  See
https://libvirt.org/formatdomain.html#elementsIOThreadsAllocation and
then you can add an iothread='NN' (NN is a number) to the <driver
name='qemu' type='raw'/> element.

> second thing, I'm trying to look for the code where QEMU allocate all
> guest memory (2 GB) in my case.

Start at memory_allocate_system_memory; ultimately you'll reach
qemu_anon_ram_alloc which is basically an mmap.

paolo