Re: [Qemu-devel] [PATCH 1/1] Migration: libvirt live migration over RDMA of ipv6 addr failed

[David Z. Dai]

On Tue, 2017-01-24 at 17:36 +0000, Daniel P. Berrange wrote:
> On Tue, Jan 24, 2017 at 09:44:33AM -0600, David Dai wrote:
> > Using libvirt to do live migration over RDMA via ip v6 address failed. 
> > For example:
> > # virsh migrate  --live --migrateuri rdma://[deba::2222]:49152  \
> >               rhel73_host1_guest1 qemu+ssh://[deba::2222]/system --verbose
> > address@hidden::2222's password:
> > error: internal error: unable to execute QEMU command 'migrate': RDMA 
> > ERROR:  
> >                   could not rdma_getaddrinfo address deba
> > 
> > As we can see, the ip v6 address used by rdma_getaddrinfo() has only "deba" 
> > part. It should be "deba::2222".
> > 
> > 1) According to rfc 3986, a literal ip v6 address should be enclosed 
> > in '[' and ']'.
> > When using virsh command to do live migration via ip v6 addresss, user
> > will input the ip v6 address with brackets (i.e. rdma://[deba::2222]:49152).
> > libvirt will parse command line option by calling virURIParse(). 
> > Inside it calls virStringStripIPv6Brackets() to strip off the brackets.
> > The uri passed in to virURIParse()  is:
> >    "uri = rdma://[deba::2222]:49152"
> > Inside virURIParse() routine, it will strip off the bracket '[' and ']' if
> > it's ip v6 address. Then save the ip v6 address in this format "deba::2222" 
> > in the virURI->server field, and to be passed to qemu.
> > 
> > 2) At the beginning of migration, in qemu's qemu_rdma_data_init(host_port) 
> > routine, it calls inet_parse(host_port) routine to parse the ip v6 address 
> > and 
> > port string obtained from libvirt.
> > The input string host_port passed to qemu_rdma_data_init() can be:
> >     "hostname:port", or
> >     "ipv4address:port", or
> >     "[ipv6address]:port" (i.e "[deba::2222]:49152"), or
> >     "ipv6address:port" (i.e "deba::2222:49152").
> > Existing qemu api inet_parse() can handle the above first 3 cases properly, 
> >  
> > but didn't handle the last case ("ipv6address:port") correctly.
> > In this live migration over rdma via ip v6 address case, the server ip v6 
> > address obtained from libvirt doesn't contain the brackets '[' and ']' 
> > (i.e. "deba::2222:49152"). It caused inet_parse() to parse only "deba" 
> > part, 
> > and stopped at the 1st colon ':'. As the result, the subsequent 
> > rdma_getaddrinfo() with ip address "deba" will fail.
> > 
> > If we don't strip off brackets '[' and ']' for an ip v6 address in 
> > libvirt's 
> > virURIParse(), it will cause libvirt ipv6 ssh authentication failure.
> > 
> > NOTE:
> > If using libvirt to do live migration over TCP via ip v6 address:
> > # virsh migrate  --live --migrateuri tcp://[deba::2222]:49152  \
> >               rhel73_host1_guest1 qemu+ssh://[deba::2222]/system --verbose
> > It works fine.
> > In migrateuri of tcp case, libvirt will call virNetSocketNewConnectTCP()
> > directly to connect to remote "deba::2222:49152" after it strips off
> > the bracket '[' and ']' for an ip v6 address. 
> > On qemu side, fd_start_outgoing_migration() will be called to do migration.
> > It doesn't call inet_parse(). So we don't see issue in tcp case.
> > 
> > Solution:
> > I choose to fix the code in qemu's inet_parse() routine to parse the
> > ip v6 addresss w/o brackets properly (i.e. "deba::2222:49152" format).
> 
> That is not right - "deba::222:49152" is *not* a valid address. It
> is mandatory to use [] when providing a numeric IPv6 address. So
> there's nothing broken in QEMU here - libvirt needs fixing to pass
> correct data to QEMU
> 
> Regards,
> Daniel
Hi, Daniel:
   Just learned you are one of the libvirt maintainer from Dr. Gilbert.

   In Previous test, I changed libvirtd code in src/util/viruri.c file,
 virURIParse() routine to block
 "virStringStripIPv6Brackets(ret->server);" call. It caused ssh ipv6
authentication failure.
  
   You said libvirt needs to be changed to pass correct data to QEMU.
   
   As one of the solution on libvirt side: if I add this condition in
 virURIParse() routine:
+  if (strcmp(ret->scheme, "rdma")) {
        /* Strip square bracket from an IPv6 address.
         * The function modifies the string in-place. Even after such
         * modification, it is OK to free the URI with xmlFreeURI. */
        virStringStripIPv6Brackets(ret->server);
+  }

   Now both the virsh live migration with tcp and rdma via ipv6 address
work fine.
  
   Do you think this is the acceptable solution?

Thanks! - David Dai