Re: [Qemu-devel] [PATCH 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Stefan Hajnoczi]

On Wed, Sep 9, 2015 at 11:19 AM, Deepak C Shetty <address@hidden> wrote:
> On 09/09/2015 03:16 PM, Stefan Hajnoczi wrote:
>> On Tue, Sep 08, 2015 at 08:40:58PM +0530, Deepak Shetty wrote:
>>> On Tue, Sep 8, 2015 at 8:32 PM, Daniel P. Berrange <address@hidden>
>>> wrote:
>>>> On Tue, Sep 08, 2015 at 06:34:09PM +0530, Prasanna Kumar Kalever wrote:
>>>>> This patch adds a way to specify multiple backup volfile servers to the
>>>>
>>>> gluster
>>>>>
>>>>> block backend of QEMU with both tcp and rdma transport types.
>>>>>
>>>>> Problem:
>>>>>
>>>>> Currenly VM Image on gluster volume is specified like this:
>>>>>
>>>>> file=gluster[+tcp]://server1:24007/testvol/a.img
>>>>>
>>>>> Assuming we have have three servers in trustred pool with replica 3
>>>>
>>>> volume
>>>>>
>>>>> in action and unfortunately server1 (mentioned in the command above)
>>>>
>>>> went down
>>>>>
>>>>> for some reason, since the volume is replica 3 we now have other 2
>>>>
>>>> servers
>>>>>
>>>>> active from which we can boot the VM.
>>>>>
>>>>> But currently there is no mechanism to pass the other 2 gluster server
>>>>> addresses to qemu.
>>>>>
>>>>> Solution:
>>>>>
>>>>> New way of specifying VM Image on gluster volume with backup volfile
>>>>
>>>> servers:
>>>>>
>>>>> file=gluster[+transport-type]://server1:24007/testvol/a.img\
>>>>>       ?backup-volfile-servers=server2&backup-volfile-servers=server3
>>>>
>>>> Comparison with RBD syntax:
>>>>
>>>>    file=rbd:pool/image:auth_supported=none:\
>>>>      mon_host=mon1.example.org\:6321\;mon2.example.org\:6322\;\
>>>>      mon3.example.org\:6322,if=virtio,format=raw
>>>>
>>>> As Peter already mentioned, you're missing port numbers.
>>>>
>>>> It is slightly unpleasant to have different ways of specifying the first
>>>> vs second, third, etc hosts. I wonder if it would be nicer to keep all
>>>> the hostnames in the host part of the URI. eg
>>>>
>>>>
>>>>
>>>> file=gluster[+transport-type]://server1:24007,server2:3553,server3:2423/testvol/a.img\
>>>>        ?backup-volfile-servers=server2&backup-volfile-servers=server3
>>>>
>>>> Of course it ceases to be a wellformed URI at that point, so another
>>>> option
>>>> would be to just allow the host part of the URI to be optional, and then
>>>> accept mutliple instances ofa  'server' arg, eg
>>>>
>>>>   file=gluster[+transport-type]:///testvol/a.img\
>>>>        ?server=server1:2424&server=server2:2423&sever=server3:34222
>>>>
>>>>
>>> Is it allowed to have this syntax and be a valid URI ? I admit i haven't
>>> looked at the
>>> URI rfc for a long time now, hence the Q. Also looking at rbd syntax, it
>>> looks
>>> to follow this model already is it ? Whats the difference between using
>>> ':'
>>> to
>>> separate key=value pairs Vs using '?" query syntax ? Should we look at
>>> having
>>> a uniform way of specifying URI be it rbd or gluster or sheepdog ... ? If
>>> yes
>>> what that uniform syntax be using ':" or '?" ?
>>
>> Instead of trying to make a gluster:// URI that accommodates multiple
>> volfile servers, perhaps the block driver can take a list of URIs.
>> Something like:
>>
>>    -drive
>> driver=gluster,uri[0]=gluster[+transport-type]://server1:24007/testvol/a.img,
>>
>> uri[1]=gluster[+transport-type]://server2:24008/testvol/a.img,
>>
>> uri[2]=gluster[+transport-type]://server3:24009/testvol/a.img
>
>
> Whats the adv of this Vs extending existing gluster URI ?

>From QEMU's perspective, it would be better to use separate fields
(that have type information) than to encode everything in an opaque
URI string.  Fields can do input validation in common code so that
block drivers don't need to check whether something is a valid number,
for example.  Fields can also be listed and their type information can
be displayed so the user knows the expected range of inputs
(self-documenting).

The other problem with URIs is that it becomes hard to encode
arbitrary data fields into them once you hit escaping.  It becomes
cumbersome for humans to construct these URIs.

> This actually sounds like a lot more typing, and making qemu cmd line a lot
> more lengthier :)

If you are typing a list of volfile backup servers each time then it's
already too long.  Anyone doing this would either use libvirt or a
shell script, so I'm not concerned about the length of the
command-line in this use case.

> Is -drive file=gluster://.... same as -drive driver=gluster,
> uri[0]=gluster..... ?

Yes, the code to make this possible is already in place today.  Block
drivers don't have to parse a single filename string, they can use a
dictionary of options passed to them.

> Also this would mean we need to change libvirt support for gluster based
> network disk completely
> and also maintain the old code for backward compat, right ?

Yes, libvirt would need to emit the new syntax.  The old syntax would
continue to work.

Stefan