igraph-help
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [igraph] reading large, sparse graph with read.graph


From: David Hunkins
Subject: Re: [igraph] reading large, sparse graph with read.graph
Date: Thu, 29 Apr 2010 23:20:45 -0700

Gabor, thanks for your help again. I am loading from an edgelist as you suggested, and I can get the whole graph into R in less than 2 minutes now. Great progress!

Dave Hunkins

On Apr 28, 2010, at 1:29 AM, Gábor Csárdi wrote:

On Wed, Apr 28, 2010 at 12:42 AM, David Hunkins <address@hidden> wrote:
So I am trying both of your suggestions (thanks):

Tamas: I created a test file that's a subset of the very large file I'm working with. I had already converted it to Pajek format. The read.graph function reads it into memory in about 15 seconds. But it is taking a very long time to write out the lgl file (now thirty minutes already). The memory footprint doesn't change, and the cpu is maxed at 100%. I would think that
the more 'compact' format could be written out quite fast, so I think
something is wrong.

You are right, this is very slow if you have edge weights. I am not
sure why, we just call back to R to convert the numbers to strings and
then print them, but the conversion is very slow. I'll check this.

system.time(g <- ba.game(1000, m=3))
  user  system elapsed
 0.000   0.000   0.002
system.time(write.graph(g, file="/tmp/ba.lgl", format="lgl"))
  user  system elapsed
 0.004   0.000   0.003
system.time(E(g)$weight <- runif(ecount(g)))
  user  system elapsed
 0.004   0.000   0.002
system.time(write.graph(g, file="/tmp/ba.lgl", format="lgl"))
  user  system elapsed
26.801   0.036  26.839

---------

g <- read.graph("email_1m.csv_dg", format="pajek")
write(proc.time(),file="")
write.graph(g, "email_1m_lgl", format="lgl")  <- Hangs here
----------

Gabor, I tried using graph.edgelist to create a graph from a matrix that was scanned from a file, and (consistent with the docs), edgelist doesn't want
any weights; it only accepts two-column input. What did I miss?

What I mean is something like:

el <- matrix(scan("/tmp/edgelist"), nc=3, byrow=TRUE)
g <- graph( t(el[,1:2]) )
E(g)$weight <- el[,3]

Where the "/tmp/edgelist" file looks like this:
0 1 1.5
1 2 2.2
2 1 0.1
3 4 0.4
...

If you have an edge list format in your Pajek files, then it is very
easy to convert it to a file that you can just read in with 'scan', as
above.

Don't forget that vertex ids start with zero in igraph, so you might
need to subtract one from first two columns of the 'el' matrix.

Best,
Gabor

[...]


--
Gabor Csardi <address@hidden>     UNIL DGM


_______________________________________________
igraph-help mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/igraph-help






reply via email to

[Prev in Thread] Current Thread [Next in Thread]