Re: [Qemu-devel] [RFC PATCH V2 3/3] filter-rewriter: rewrite tcp packet to keep secondary connection

[Jason Wang]

On 2016年07月02日 14:22, Zhang Chen wrote:
> We will rewrite tcp packet secondary received and sent.
> When colo guest is a tcp server.
>
> Firstly, client start a tcp handshake. the packet's seq=client_seq,
> ack=0,flag=SYN. COLO primary guest get this pkt and mirror(filter-mirror)
> to secondary guest, secondary get it use filter-redirector.
> Then,primary guest response pkt
> (seq=primary_seq,ack=client_seq+1,flag=ACK|SYN).
> secondary guest response pkt
> (seq=secondary_seq,ack=client_seq+1,flag=ACK|SYN).
> In here,we use filter-rewriter save the secondary_seq to it's tcp connection.
> Finally handshake,client send pkt
> (seq=client_seq+1,ack=primary_seq+1,flag=ACK).
> Here,filter-rewriter can get primary_seq, and rewrite ack from primary_seq+1
> to secondary_seq+1, recalculate checksum. So the secondary tcp connection
> kept good.
>
> When we send/recv packet.
> client send pkt(seq=client_seq+1+data_len,ack=primary_seq+1,flag=ACK|PSH).
> filter-rewriter rewrite ack and send to secondary guest.
>
> primary guest response pkt
> (seq=primary_seq+1,ack=client_seq+1+data_len,flag=ACK)
> secondary guest response pkt
> (seq=secondary_seq+1,ack=client_seq+1+data_len,flag=ACK)
> we rewrite secondary guest seq from secondary_seq+1 to primary_seq+1.
> So tcp connection kept good.
>
> In code We use offset( = secondary_seq - primary_seq )
> to rewrite seq or ack.
> handle_primary_tcp_pkt: tcp_pkt->th_ack += offset;
> handle_secondary_tcp_pkt: tcp_pkt->th_seq -= offset;
>
> Signed-off-by: Zhang Chen <address@hidden>
> Signed-off-by: Li Zhijian <address@hidden>
> Signed-off-by: Wen Congyang <address@hidden>
> ---
>   net/colo-base.h       |   2 +
>   net/filter-rewriter.c | 110 +++++++++++++++++++++++++++++++++++++++++++++++++-
>   trace-events          |   5 +++
>   3 files changed, 115 insertions(+), 2 deletions(-)
>
> diff --git a/net/colo-base.h b/net/colo-base.h
> index 62460c5..7b32648 100644
> --- a/net/colo-base.h
> +++ b/net/colo-base.h
> @@ -71,6 +71,8 @@ typedef struct Connection {
>       uint8_t ip_proto;
>       /* be used by filter-rewriter */
>       colo_conn_state state;
> +    /* offset = secondary_seq - primary_seq */
> +    tcp_seq  offset;

Fail to find the definition of 'tcp_seq'.

>   } Connection;
>   
>   uint32_t connection_key_hash(const void *opaque);
> diff --git a/net/filter-rewriter.c b/net/filter-rewriter.c
> index c38ab24..9f63c75 100644
> --- a/net/filter-rewriter.c
> +++ b/net/filter-rewriter.c
> @@ -21,6 +21,7 @@
>   #include "qemu/main-loop.h"
>   #include "qemu/iov.h"
>   #include "net/checksum.h"
> +#include "trace.h"
>   
>   #define FILTER_COLO_REWRITER(obj) \
>       OBJECT_CHECK(RewriterState, (obj), TYPE_FILTER_REWRITER)
> @@ -64,6 +65,91 @@ static int is_tcp_packet(Packet *pkt)
>       }
>   }
>   
> +/* handle tcp packet from primary guest */
> +static int handle_primary_tcp_pkt(NetFilterState *nf,
> +                                  Connection *conn,
> +                                  Packet *pkt)
> +{
> +    struct tcphdr *tcp_pkt;
> +    static int syn_flag;
> +
> +    tcp_pkt = (struct tcphdr *)pkt->transport_layer;
> +    if (trace_event_get_state(TRACE_COLO_FILTER_REWRITER_DEBUG)) {
> +        char *sdebug, *ddebug;
> +        sdebug = strdup(inet_ntoa(pkt->ip->ip_src));
> +        ddebug = strdup(inet_ntoa(pkt->ip->ip_dst));
> +        trace_colo_filter_rewriter_pkt_info(__func__, sdebug, ddebug,
> +                    ntohl(tcp_pkt->th_seq), ntohl(tcp_pkt->th_ack),
> +                    tcp_pkt->th_flags);
> +        trace_colo_filter_rewriter_conn_offset(conn->offset);
> +        g_free(sdebug);
> +        g_free(ddebug);
> +    }
> +
> +    if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == TH_SYN)) {
> +        /*
> +         * this flag update offset func run oncs

typo?

> +         * in independent tcp connection
> +         */
> +        syn_flag = 1;

Does this really work if you have more than one tcp connections? You 
probably need a conn->syn_flag.

> +    }
> +
> +    if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == TH_ACK)) {
> +        if (syn_flag) {
> +            /* offset = secondary_seq - primary seq */
> +            conn->offset -= (ntohl(tcp_pkt->th_ack));
> +            syn_flag = 0;
> +
> +        }
> +        /* handle packets to the secondary from the primary */
> +        tcp_pkt->th_ack = htonl(ntohl(tcp_pkt->th_ack) + conn->offset + 1);

Maybe I miss something, but why +1 here?