[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Gluster-devel] split brain with all-zero pending matrix
From: |
Emmanuel Dreyfus |
Subject: |
[Gluster-devel] split brain with all-zero pending matrix |
Date: |
Fri, 21 Jun 2013 02:53:31 +0200 |
User-agent: |
MacSOUP/2.7 (unregistered for 2344 days) |
On 3.4.0beta3, after using the volume for a while, I get split-brain errors
with an unhelpful pending matrix.
[2013-06-20 08:44:00.665731] E
[afr-self-heal-common.c:197:afr_sh_print_split_brain_log] 0-gfs34-replicate-1:
Unable to self-heal contents of
'/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o' (possible
split-brain). Please delete the file from all but the preferred subvolume.-
Pending matrix: [ [ 0 0 ] [ 0 0 ] ]
[2013-06-20 08:44:00.666431] E
[afr-self-heal-common.c:2212:afr_self_heal_completion_cbk] 0-gfs34-replicate-1:
background data self-heal failed on
/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o
[2013-06-20 08:44:00.667201] W [afr-open.c:213:afr_open] 0-gfs34-replicate-1:
failed to open as split brain seen, returning EIO
[2013-06-20 08:44:00.668193] W [fuse-bridge.c:875:fuse_fd_cbk]
0-glusterfs-fuse: 8711927: OPEN()
/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o => -1 (Input/output
error)
On the bricks (ls -l, md5 and first 32 bytes):
brick0
-rw-r--r-- 2 manu manu 7216 Jun 20 09:56
/export/wd3a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o
MD5 (/export/wd3a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o) =
24ee57aa8e2aeb6102ba170fb81bbf22
00000000 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 |.ELF............|
00000010 01 00 03 00 01 00 00 00 00 00 00 00 00 00 00 00 |................|
brick1
-rw-r--r-- 2 manu manu 7216 Jun 20 09:56
/export/wd1a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o
MD5 (/export/wd1a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o) =
24ee57aa8e2aeb6102ba170fb81bbf22
00000000 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 |.ELF............|
00000010 01 00 03 00 01 00 00 00 00 00 00 00 00 00 00 00 |................|
brick2
-rw-r--r-- 2 manu manu 6256 Jun 20 09:56
/export/wd3a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o
MD5 (/export/wd3a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o) =
58a4b8c5929cac2f799c60b3dd2acc2f
brick3
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
-rw-r--r-- 2 manu manu 7216 Jun 20 09:56
/export/wd1a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o
MD5 (/export/wd1a/manu/netbsd/usr/src/tools/gcc/obj/build/gcc/cfgbuild.o) =
2bc383747432c49793a2cf1a2c0e8cfa
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
That shows the offending copies without any doubt, but I still wonder how
we come to that situation. There are a lot of such files, making the glusterfs
volume just unusable at this point.
--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
address@hidden
- [Gluster-devel] split brain with all-zero pending matrix,
Emmanuel Dreyfus <=