Re: [Qemu-devel] [RFC PATCH v2 00/10] Add colo-proxy based on netfilter

[Dr. David Alan Gilbert]

* Li Zhijian (address@hidden) wrote:
> 
> 
> On 03/01/2016 04:04 AM, Dr. David Alan Gilbert wrote:
> >* Zhang Chen (address@hidden) wrote:
> >>From: zhangchen <address@hidden>
> >>
> >>Hi,all
> >>
> >>This patch add an colo-proxy object, COLO-Proxy is a part of COLO,
> >>based on qemu netfilter and it's a plugin for qemu netfilter. the function
> >>keep Secondary VM connect normal to Primary VM and compare packets
> >>sent by PVM to sent by SVM.if the packet difference,notify COLO do
> >>checkpoint and send all primary packet has queued.
> >
> >Hi Zhangchen,
> >   How are you dealing with the IP 'identification' field?
> >It's a very very random field, and not just the initial value in the 
> >connection.
> >I can't see how the kernel colo-proxy dealt with it either; but I think it's
> >comparison was after defragementation so probably ignored the identification
> >field
> You are right, most kernel colo-proxy code is working at mangle table (after 
> defrag).
> and colo proxy only compare the contents of L4(TCP/UDP) excluding IP 
> identification.
> 
> > - wouldn't that confuse a client at failover?
> Err..., instersting question.
> 
> for example, a COLO including primay(PVM) adn secondary(SVM)
> 1. primay is sending a compared P_IP packet(identification=0x12345, split to 
> IP_s1, IP_s2..IP_s100) to client
> 2. client is receiving the ip segment(but IP_s2, IP_s50, IP_s80..IP_s99 are 
> missing)
>    and primary host is down.
> 3. secondary VM takeover, and send a S_IP packet(ip contents is always same 
> as at PVM)
> 
> in step 3,
> if the S_IP identification isn't 0x12345, client will drop the ip segment at 
> step 2 because of defrag timeout.

So that triggers a timeout (30 second?) - hmm OK, a bit slow but OK.

> if the S_IP identification isn 0x12345, client may mix the segment from PVM 
> or SVM(just like segment come
> from different router), but that's Okay, because we have ensured the ip 
> contents is identical.

Could the S_IP identification match a later/earlier fragment?

Dave

> so IMO, it will not confuse the client at failover.
> 
> >
> >Dave
> >
> >>You can also get the series from:
> >>
> >>https://github.com/zhangckid/qemu/tree/colo-v2.2-periodic-mode-with-colo-proxyV2
> >>
> >>Usage:
> >>
> >>primary:
> >>-netdev tap,id=bn0 -device e1000,netdev=bn0
> >>-object colo-proxy,id=f0,netdev=bn0,queue=all,mode=primary,addr=host:port
> >>
> >>secondary:
> >>-netdev tap,id=bn0 -device e1000,netdev=bn0
> >>-object colo-proxy,id=f0,netdev=bn0,queue=all,mode=secondary,addr=host:port
> >>
> >>NOTE:
> >>queue must set "all". See enum NetFilterDirection for detail.
> >>colo-proxy need queue all packets
> >>colo-proxy V2 just can compare ip packet
> >>
> >>
> >>## Background
> >>
> >>COLO FT/HA (COarse-grain LOck-stepping Virtual Machines for Non-stop 
> >>Service)
> >>project is a high availability solution. Both Primary VM (PVM) and 
> >>Secondary VM
> >>(SVM) run in parallel. They receive the same request from client, and 
> >>generate
> >>responses in parallel too. If the response packets from PVM and SVM are
> >>identical, they are released immediately. Otherwise, a VM checkpoint (on
> >>demand)is conducted.
> >>
> >>Paper:
> >>http://www.socc2013.org/home/program/a3-dong.pdf?attredirects=0
> >>
> >>COLO on Xen:
> >>http://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping
> >>
> >>COLO on Qemu/KVM:
> >>http://wiki.qemu.org/Features/COLO
> >>
> >>By the needs of capturing response packets from PVM and SVM and finding out
> >>whether they are identical, we introduce a new module to qemu networking
> >>called colo-proxy.
> >>
> >>V2:
> >>   rebase colo-proxy with qemu-colo-v2.2-periodic-mode
> >>   fix dave's comments
> >>   fix wency's comments
> >>   fix zhanghailiang's comments
> >>
> >>v1:
> >>   initial patch.
> >>
> >>
> >>
> >>zhangchen (10):
> >>   Init colo-proxy object based on netfilter
> >>   Jhash: add linux kernel jhashtable in qemu
> >>   Colo-proxy: add colo-proxy framework
> >>   Colo-proxy: add data structure and jhash func
> >>   net/colo-proxy: Add colo interface to use proxy
> >>   net/colo-proxy: add socket used by forward func
> >>   net/colo-proxy: Add packet enqueue & handle func
> >>   net/colo-proxy: Handle packet and connection
> >>   net/colo-proxy: Compare pri pkt to sec pkt
> >>   net/colo-proxy: Colo-proxy do checkpoint and clear
> >>
> >>  include/qemu/jhash.h |  61 ++++
> >>  net/Makefile.objs    |   1 +
> >>  net/colo-proxy.c     | 939 
> >> +++++++++++++++++++++++++++++++++++++++++++++++++++
> >>  net/colo-proxy.h     |  24 ++
> >>  qemu-options.hx      |   6 +
> >>  trace-events         |   8 +
> >>  vl.c                 |   3 +-
> >>  7 files changed, 1041 insertions(+), 1 deletion(-)
> >>  create mode 100644 include/qemu/jhash.h
> >>  create mode 100644 net/colo-proxy.c
> >>  create mode 100644 net/colo-proxy.h
> >>
> >>--
> >>1.9.1
> >>
> >>
> >>
> >--
> >Dr. David Alan Gilbert / address@hidden / Manchester, UK
> >
> >
> >.
> >
> 
> -- 
> Best regards.
> Li Zhijian (8555)
> 
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK