[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[igraph] How to read in a large graph (and output a sparse matrix)
From: |
Raphael C |
Subject: |
[igraph] How to read in a large graph (and output a sparse matrix) |
Date: |
Mon, 1 Aug 2016 08:52:22 +0100 |
I have 8GB of RAM and I have a simple edge list text file of size
1.2GB. It was 62500000 edges and about half that many vertices. Each
line looks like
287111206 357850135
I would like to read in the graph and output a sparse adjacency
matrix. I am failing on all counts. I have tried
g = Graph.Read_Edgelist('edges.txt')
but this fails immediately with
MemoryError: Error at vector.pmt:439: cannot reserve space for vector,
Out of memory
This seems unrelated to the size of the graph is just a function of
the node ids being large.
So instead I tried
g = Graph.Read_Ncol('edges.txt')
This eats up all the RAM in my PC forcing me to kill the code.
I fact I tested g = Graph.Read_Ncol('edges.txt') with the first 1/5 of
the edges and have the same memory problem.
Each node id is a 32 bit integer so the graph should fit easily in 8GB of RAM.
What can I do?
Thanks very much for any help.
Raphael
- [igraph] How to read in a large graph (and output a sparse matrix),
Raphael C <=
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Tamas Nepusz, 2016/08/01
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Raphael C, 2016/08/01
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Tamas Nepusz, 2016/08/01
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Raphael Clifford, 2016/08/01
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Raphael C, 2016/08/01
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Tamas Nepusz, 2016/08/02
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Raphael C, 2016/08/02
- Re: [igraph] How to read in a large graph (and output a sparse matrix), Tamas Nepusz, 2016/08/05