Re: [rdiff-backup-users] rdiff-backup dies with ENOMEM due to requesting enormous amounts of memory

[Jeff White]

On 06/10/2016 04:34 AM, Patrik Dufresne wrote:

Hello Jeff,

You problem is interesting to me ! Rdiff-backup doesn't have the tendency to use alot of memory (it rarely reach 100MB on my systems). It is surprising to see an out or memory error like this in rdiff-backup.

May you provide some information revargin your current environment:

1. How many memory free you have

Should be plenty:

$ free -m
              total        used        free      shared  buff/cache   available
Mem:           1869         172        1169           0         527        1538
Swap:             0           0           0

2. May you describes how big it the stuff you are trying to backup (nb files, average file size, etc.)

I'm trying to back up the root filesystem of a generic CentOS 7 box.  It doesn't output that it even hit the filesystem yet though, it dies before that from what I can tell.  It dies in the same way even when I try to back up a single empty directory.

3. x86 or x86_64 ?

x86_64

4. What is you python version ?

$ python -V
Python 2.7.5

Can you get your hand on the server logs too ? Causes it fail when requesting stuff from remote server...

Nothing in any logs other than simply showing someone logged in via SSH.  The ENOMEM is /not/ on the remote side, it's on the side starting the backup and doing a pull from another server.

--

Patrik Dufresne Service Logiciel inc.

http://www.patrikdufresne.com/

514-971-6442

1-114 rue des Hautbois,

St-Colomban, QC J5K 2H6

On Thu, Jun 9, 2016 at 6:54 PM, Jeff White <address@hidden> wrote:

I'm trying to get rdiff-backup working for the first time across SSH (with sudo) on two CentOS 7 hosts.  I'm getting the following crash:

$ sudo rdiff-backup --restrict-read-only / -v5 --remote-schema 'ssh -l rdiff -i /home/rdiff/.ssh/id_rsa -t -t %s "sudo /usr/bin/rdiff-backup --server"' bacula-p1n01::/ /dumps/bacula-p1n01/
Thu Jun  9 15:34:42 2016  Using rdiff-backup version 1.2.8
Thu Jun  9 15:34:42 2016  Executing ssh -l rdiff -i /home/rdiff/.ssh/id_rsa -t -t bacula-p1n01 "sudo /usr/bin/rdiff-backup --server"
Thu Jun  9 15:34:42 2016  Client sending (0): ConnectionRequest: Globals.get with 1 arguments
Thu Jun  9 15:34:42 2016  Client sending (0): 'version'
Connection to bacula-p1n01 closed.
Thu Jun  9 15:34:43 2016  Exception '' raised of class '<type 'exceptions.MemoryError'>':
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/Main.py", line 321, in Main
    rps = map(SetConnections.cmdpair2rp, cmdpairs)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/SetConnections.py", line 76, in cmdpair2rp
    if cmd: conn = init_connection(cmd)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/SetConnections.py", line 150, in init_connection
    check_connection_version(conn, remote_cmd)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/SetConnections.py", line 158, in check_connection_version
    try: remote_version = conn.Globals.get('version')
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 368, in reval
    result = self.get_response(req_num)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 315, in get_response
    try: req_num, object = self._get()
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 240, in _get
    data = "">   File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 210, in _read
    try: return self.inpipe.read(length)

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/Main.py", line 321, in Main
    rps = map(SetConnections.cmdpair2rp, cmdpairs)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/SetConnections.py", line 76, in cmdpair2rp
    if cmd: conn = init_connection(cmd)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/SetConnections.py", line 150, in init_connection
    check_connection_version(conn, remote_cmd)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/SetConnections.py", line 158, in check_connection_version
    try: remote_version = conn.Globals.get('version')
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 450, in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 368, in reval
    result = self.get_response(req_num)
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 315, in get_response
    try: req_num, object = self._get()
  File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 240, in _get
    data = "">   File "/usr/lib64/python2.7/site-packages/rdiff_backup/connection.py", line 210, in _read
    try: return self.inpipe.read(length)
MemoryError


On that host when I strace the process I see:

[pid 11602] mmap(NULL, 18118029061677056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0 <unfinished ...>
...
[pid 11602] <... mmap resumed> )        = -1 ENOMEM (Cannot allocate memory)


... am I mistaken or is rdiff-backup requesting a ridiculous amount of memory?  Why would it do that?

--
Jeff White
HPC Systems Engineer
Information Technology Services - WSU


_______________________________________________
rdiff-backup-users mailing list at address@hidden
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki