[Gluster-devel] Memory leak(?) in GlusterFS TLA 2.5-patch-616

[Sam Douglas]

I have come across a fairly large memory leak(?) in glusterfsd using
the supplied configuration when doing many rewrites (read a block of
data, change it, write block back).  This results in one of the
glusterfsd processes consuming very large amounts of the machines
memory (95% ish).

It seems to occur when readahead and writebehind are loaded above
Unify on the client, which connects to two remote AFR volumes.

This occurs in glusterfs--main--2.5--patch-616.

I have been able to replicate the problem on Debian Etch machines,
using a fuse-2.7.0 compiled from source and a recent kernel and the
following configurations.

The attached benchmark program can be used to cause the problem
    rewrite_test --blocksize 8192 --count 8192 /mnt/foobar
should do it. Bonnie will also cause the problem when it does its rewrite test.

    Sam Douglas



-- Config Files --

--- Client ---

volume clientA
        type protocol/client
        option transport-type tcp/client
        option remote-host 10.0.0.19
        option remote-subvolume afr
end-volume

volume clientB
        type protocol/client
        option transport-type tcp/client
        option remote-host 10.0.0.20
        option remote-subvolume afr
end-volume


volume namespace
        type protocol/client
        option transport-type tcp/client
        option remote-host 10.0.0.19
        option remote-subvolume namespace
end-volume

volume unify
        type cluster/unify
        option namespace namespace
        option scheduler rr
        subvolumes clientA clientB
end-volume

volume unify-wb
        type performance/write-behind
        option aggregate-size 128K
        subvolumes unify
end-volume

volume unify-ra
        type performance/read-ahead
        option page-size 65536
        option page-count 32
        subvolumes unify-wb
end-volume


--- Server (10.0.0.19) ---

volume brick1
        type storage/posix
        option directory /stuff/test1
end-volume

volume brick2
        type storage/posix
        option directory /stuff/test2
end-volume

volume namespace
        type storage/posix
        option directory /stuff/test-namespace
end-volume

volume client1
        type protocol/client
        option transport-type tcp/client
        option remote-host 10.0.0.20
        option remote-subvolume brick1
end-volume

volume afr
        type cluster/afr
        subvolumes brick1 client1
end-volume

volume server
        type protocol/server
        option transport-type tcp/server
        option auth.ip.afr.allow *
        option auth.ip.brick1.allow *
        option auth.ip.brick2.allow *
        option auth.ip.namespace.allow *
        subvolumes afr brick1 brick2 namespace
end-volume


--- Server (10.0.0.20) ---

volume brick1
        type storage/posix
        option directory /stuff/test1
end-volume

volume brick2
        type storage/posix
        option directory /stuff/test2
end-volume

volume client2
        type protocol/client
        option transport-type tcp/client
        option remote-host 10.0.0.19
        option remote-subvolume brick2
end-volume

volume afr
        type cluster/afr
        subvolumes client2 brick2
end-volume

volume server
        type protocol/server
        option transport-type tcp/server
        option auth.ip.afr.allow *
        option auth.ip.brick1.allow *
        option auth.ip.brick2.allow *
        subvolumes afr brick1 brick2
end-volume