[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] OSError: [Errno 24] Too many open files
From: |
Kenneth Loafman |
Subject: |
Re: [Duplicity-talk] OSError: [Errno 24] Too many open files |
Date: |
Tue, 12 May 2009 13:27:56 -0500 |
User-agent: |
Thunderbird 2.0.0.21 (X11/20090318) |
As far as I can tell, its just lots of incrementals. duplicity has to
open and process each signature file and, in some cases, the gpg process
is not being closed completely causing a zombie process.
This "seems" to be worse on a 64bit system, but I have been able to
partially reproduce it on a 32bit. The most zombie processes I've been
able to see are 5-7 and that's with about 60 incrementals. Even then,
all of them go away quickly.
...Ken
Jeff Singer wrote:
> I have a 64bit Ubuntu 9.04 system here that I'd be happy to attempt to
> reproduce it over the weekend. Is it basically a backup that involves
> lots of incrementals with many volumes that's causing this behavior?
>
> On Tue, May 12, 2009 at 1:51 PM, Ian Barton <address@hidden
> <mailto:address@hidden>> wrote:
>
> I have got a couple of 64 bit Ubuntu 9.1 (Jaunty) computers. I can
> try running a test on my system if you tell me what you would like
> me to do.
>
> FWIW my main computer for running is Debian Lenny 64bit and I have
> never seen the problem on it.
>
> Ian.
>
>
> Thanks for the help!
>
> I'm not sure at this point. I built a 64bit Ubuntu 9.04 system
> and it
> seems to be working fine. Debian may be different, but not much.
>
> As to what's broke, it could even be at the system level, but my
> bet is
> on gpg.
>
> ...Ken
>
> address@hidden <mailto:address@hidden> wrote:
>
> I am gonna install a 64bit debian in a virtual machine and gonna
> doublecheck that ... but what do you think is buggy? python,
> gpg ?
>
> ..ede
>
> The only real change to GnuPGInterface is a shift right
> instead of a
> shift left when printing the error code after an
> exception. He's not
> getting any exceptions, so he's not hitting the change.
> Other than
> that, its the same code from years ago.
>
> My best guess is still that its a 32-bit versus 64-bit
> issue.
>
> ...Ken
>
> Edgar Soldin wrote:
>
>
> true they close every time .. the only thing i see
> is that the amount of
> gpg processes grows with the amount of incrementals ...
>
> @yang: how many incrementals does your backup hold?
>
> the other thing is, maybe gpg itself is buggy ..
>
> @yang: could you try another gpg version?
>
> and the third thing
>
> @ken: is it really impossible for 0.5.17 to use an
> oldGnuPGInterface
> version, that might be laying around on the system?
> How could yang look
> for it?
>
> .. 2cents again :) ..ede
> --
>
>
> I tried this with tiny volume sizes of 1M over a
> 2GB backup (2K
> volumes)
> and did a count of the gpg processes (live,
> defunct) every 2 seconds
> and
> never had more than 5 (3 live, 2 defunct). This
> was on a very fast
> link
> between local VMs. Tried both backup and
> restore so it would test both
> directions. By the end, all were closed.
>
> It's possible to have some defunct while the
> task scheduler is
> updating,
> but Yang is showing hundreds that are holding
> files open. That's the
> confusing part.
>
> ...Ken
>
> Edgar Soldin wrote:
>
>
> i really have to be fast (a window of 10s
> let's say) and it helps to
> have a slow backup connection with lots of
> incrementals... but I
> may be
> seeing ghosts here. Can it that it is
> perfectly normal for
> duplicity to
> gpg decrypt something per each incremental
> but close all processes
> just
> a bit later? .. just a guess
> ..ede
>
>
> I cannot reproduce this on a 32bit
> machine at all. I'll be able
> to test
> on a 64bit machine sometime this week
> and, hopefully, find the
> problem.
>
> ...Ken
>
> address@hidden
> <mailto:address@hidden> wrote:
>
>
> i seem to get closer to it ...
> the more incrementals (regardless if
> file: or ftp: backend) the more
> defunct gpg processes appear .. but
> dissappear again while the
> backup is
> still running
>
> I did
> initial backup of 3GB
> touched a new test file to have a
> change & did a 2nd backup
> touched a new test2 file to have a
> change & did a 3nd backup
> ...
>
> and since the 3rd backup the defunct
> gpg processes keep appearing
>
> essentially I say: Take any more
> than 3 incrementals in size
> backup and
> do a new incremental and you will
> see those zombie gpg processes ..
>
> while I am not sure that this is
> what happens on the ulimit
> problem I
> thought I should report these
> observations ... ede
>
> PS:
>
> here a 'ps xf -u user' shortly after
> starting the sixth run .. the
> number of gpg zombies seems to be
> number of sets (full + incr's)
> minus 2 ..
>
> 9122 pts/0 S+ 0:00
> \_ /bin/bash
>
> /srv/www/user//release/ftplicity_1.4.2/ftplicity
> test backup
> 19128 pts/0 S+ 0:00
> \_ /bin/bash
>
> /srv/www/user//release/ftplicity_1.4.2/ftplicity
> backup
> 19193 pts/0 S+ 0:00
> \_ /bin/bash
>
> /srv/www/user//release/ftplicity_1.4.2/ftplicity
> backup
> 19194 pts/0 R+ 0:03
> \_ /usr/bin/python
>
> /srv/www/user/_apps/duplicity-0.5.10/bin/duplicity
> --verbosity 4
> --encrypt-key 00000000--sign-key
> 00000000--gpg-options=--always-trust
> --volsize 1 --exclude-globbing-filelist
> /srv/www/user//.ftplicity/test/excl
> 19196 pts/0 SL+ 0:00
> \_ gpg
> --logger-fd 4
> --passphrase-fd 8 --batch --no-tty
> --default-key 00000000--recipient
> 00000000--no-secmem-warning
> --always-trust --encrypt --sign
> 19197 pts/0 SL+ 0:00
> \_ gpg
> --status-fd 6
> --passphrase-fd 11 --logger-fd 5
> --batch --no-tty --default-key
> 00000000--no-secmem-warning
> --always-trust --decrypt
> 19199 pts/0 Z+ 0:00
> \_ [gpg]
> <defunct>
> 19200 pts/0 Z+ 0:00
> \_ [gpg]
> <defunct>
> 19201 pts/0 Z+ 0:00
> \_ [gpg]
> <defunct>
> 19202 pts/0 Z+ 0:00
> \_ [gpg]
> <defunct>
> 19203 pts/0 SL+ 0:00
> \_ gpg
> --logger-fd
> 23 --passphrase-fd 28 --batch
> --no-tty --default-key
> 00000000--recipient
> 00000000--no-secmem-warning
> --always-trust --encrypt --sign
>
>
>
>
> hmm .. you are right these
> processes are zombie (they appear in
> top as
> such) .. later on the processes
> disappear, while duplicity is still
> running ..
> but I don't feel this is
> normal.. or is it?
> just to be more sure, I'll try a
> 1MB chunk full backup of my 3GB
> home
> dir .. let's see with how many
> zombies I'll end up ...
>
> interestingly my ulimit =
> unlimited ... seems to be default in
> the old
> SUSE 10.2, btw also on my debian
> 5.0 box both 32bit
> can't remember to ever have
> touched this setting
>
> ..ede
> --
>
>
> These are not open
> processes, they are zombie
> processes, so no
> real
> resources taken. That
> simplifies it a bit. What
> normally
> causes this
> is the failure of a parent
> process to properly retrieve its
> exit status,
> but this is not the case or
> more people would have this
> problem.
>
> As far as I can tell, the
> only systems this is
> happening on are
> 64bit,
> and not even all of those.
> Some of my systems are
> 64bit and
> they don't
> show this, so I'm wondering
> if this could be limited to a
> particular
> version of GnuPG, or what.
>
> Edgar, from your note I
> could not tell, is this
> happening to
> you too?
>
> ...Ken
>
> address@hidden
> <mailto:address@hidden>
> wrote:
>
>
>
> just my quick
> observation .. a simple
> incr backup opens and
> leaves over
> 20 gpg processes open. I
> don't think this is
> healthy and also can
> imagine that this
> multiplies if the
> volumes get smaller
> (mine are
> 50MB).
>
> regards ede
>
>
> address@hidden:~> ps -u user
> xf
> PID TTY STAT
> TIME COMMAND
> 32731 ? S
> 0:00 sshd: address@hidden/0
> 32732 pts/0 Ss
> 0:00 \_ -bash
> 1167 pts/0 S
> 0:00 \_ /bin/bash
> /usr/local/bin/ftplicity
> profile backup
> 1168 pts/0 S
> 0:00 | \_ /bin/bash
>
> /srv/www//release/ftplicity_1.4.2/ftplicity
> profile backup
> 1174 pts/0 S
> 0:00 | \_
> /bin/bash
>
> /srv/www//release/ftplicity_1.4.2/ftplicity
> backup
> 1239 pts/0 S
> 0:00 |
> \_ /bin/bash
>
> /srv/www//release/ftplicity_1.4.2/ftplicity
> backup
> 1240 pts/0 S
> 0:05 |
> \_ /usr/bin/python
>
> /srv/www/user/_apps/duplicity-0.5.10/bin/duplicity
> --verbosity 4
> --encrypt-key XXXXXXX
> --sign-key B59ECD99
> --gpg-options=--always-trust
> --full-if-older-than 1M
> --volsize 50
> --exclude-globbing-filelist
> /srv/www/...
> 1249 pts/0 SL
> 0:00 |
> \_ gpg
> --logger-fd 4
> --passphrase-fd 8
> --batch --no-tty
> --default-key XXXXXXXX
> --recipient
> XXXXXXXX
> --no-secmem-warning
> --always-trust --encrypt
> --sign
> 1265 pts/0 SL
> 0:00 |
> \_ gpg
> --status-fd 6
> --passphrase-fd 11
> --logger-fd 5 --batch
> --no-tty --default-key
> XXXXXXXX
> --no-secmem-warning
> --always-trust --decrypt
> 1267 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1272 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1275 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1278 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1282 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1286 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1289 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1292 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1296 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1299 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1302 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1305 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1308 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1311 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1314 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1317 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1320 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1324 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1327 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1330 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1333 pts/0 Z
> 0:00 |
> \_ [gpg]
> <defunct>
> 1334 pts/1 Ss+
> 0:00 |
> \_
> /usr/bin/ncftp -u
> ******* backup.server.de
> <http://backup.server.de>
> 1335 pts/0 R+
> 0:00 \_ ps -u user xf
>
>
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
>
> ------------------------------------------------------------------------
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
> ------------------------------------------------------------------------
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
> ------------------------------------------------------------------------
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> <mailto:address@hidden>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden <mailto:address@hidden>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden <mailto:address@hidden>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
>
> ------------------------------------------------------------------------
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden <mailto:address@hidden>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden <mailto:address@hidden>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
signature.asc
Description: OpenPGP digital signature
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, (continued)
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Edgar Soldin, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Edgar Soldin, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Yang Zhang, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Ian Barton, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Jeff Singer, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files,
Kenneth Loafman <=
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Edgar Soldin, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Andrew Telford, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/13
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/13
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Yang Zhang, 2009/05/13
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Edgar Soldin, 2009/05/13
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Yang Zhang, 2009/05/13
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/13