Re: [igraph] Saving Graphs

[Marco]

Hi Matthew,

thank you for your input!

I forgot to mention a little problem that makes me wonder if this
approach (which I thought of) is practicable.

The files (of the graphs) I write (in the number of thousands, as I
said, and totaling various gigabytes for each "species") have then to
be reread and analyzed several times (by another piece of code).

This makes me wonder if the overhead of compressing/decompressing so
much data each time is something I want to deal with: I'd say, having tried
just once, that the amount of time needed to do such
compression/decompression is longer than the actual time for the
analysis.

Being this true "always", would  it be worth the effort?

Thank you again!

marco

--
restituiremo il riso ai salici



On Thu, Dec 3, 2009 at 4:10 PM, Matthew Walker
<address@hidden> wrote:
> Hi Marco,
>
> I am no igraph nor python expert, but maybe you could try writing your
> graphs into one large XML file.  You can easily compress such a file once
> you've finished adding graphs.  I would even suspect that python would have
> the ability to compress the file on the fly.
>
> I think GraphML (an XML format) would be ideal for this.  You could write
> out a graph to a temporary file (using igraph_write_graph_graphml(), or its
> python equivalent) and then merge the temporary file into a main multi-graph
> file (also in GraphML).  Just how to best do this, I am not sure.  I imagine
> python already has strong abilities to manipulate XML files.  If that's not
> for you, then there is a useful command-line utility called xmlstar [1] that
> could help you do such a merge.
>
> I note that igraph already has the ability to select which graph to read
> given such a multi-graph file (see the documentation for the "index"
> parameter for igraph_read_graph_graphml()).
>
> I hope that helps,
>
> [1] http://xmlstar.sourceforge.net/
>
> Cheers,
>
> Matthew
>
> Marco wrote:
>>
>> Hi all!
>>
>> In the simulations I write I have to save very often the graph, with
>> all the attributes.
>> If I save it using gml or something like this, I end up with many
>> (say: thousands) of very small files, which are not very practical to
>> carry around and tend to full the filesystem very quickly.
>>
>> I am looking for a way to save graphs (even not single graphs, but
>> rather more graphs in the same file) which is a little more clever.
>>
>> So far I have been looking at pickling the whole thing, but I find it
>> rather unsatisfactory.
>> I have heard of hdf (or something like this). Looks to me that the
>> overhead to use it is rather big.
>>
>> Someone has highlights, remarks, advice?
>>
>> Thank you in advance,
>>
>> marco
>>
>> --
>> restituiremo il riso ai salici
>>
>>
>> _______________________________________________
>> igraph-help mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/igraph-help
>>
>>
>
>
> _______________________________________________
> igraph-help mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/igraph-help
>