Re: [Discuss-gnuradio] Question about GNURADIO File sink

[Marcus Müller]

Hi Sylvain,

"insane" is a relative term (and we're talking about things that /I'd/
do, so the reference insanity level is a bit raised to begin with). I
just tried this with

null source (complex, vlen = 100*1024*1024/8)  -> null sink

and asked "ipcs" about what happened. GNU Radio created exactly 400MB of
buffer, i.e. but four items worth of samples. I mean, sure, that's not
ideal, but on anything that is likely to store 100MBs of buffers to a
file probably not /that/ problematic (especially since you'd probably
want to rely on filesystem buffers, anyway). Of course, YMMV!

You're absolutely right, the right thing to do here would've been to
just write another file sink that actually takes items at
single-item-granularity, and closes and opens files as appropriate...
but, you know, beginners :)

I just had another idea (which probably won't use much less memory, but
at least not SHM):

If you're on some kind of Unixoid, create a named FIFO ("mkfifo
fifo_name") and use that in the file sink. Separately, run "split -b
100M --numeric-suffixes fifo_name samples".

Cheers,

Marcus


On 01.11.2016 15:34, Sylvain Munaut wrote:
> Hi,
>
>> You could write a python block that takes input which is a vector
>> of 100*1024*1024B/(8B/sample) = 13107200 complex numbers, and directly
>> writes each input to an own file using numpy. You can convert a stream
>> of (single) samples to a stream of vectors of that size using a
>> stream_to_vector block.
> I would _highly_ advise against that ...
>
> That would use an insane amount of memory since each buffer will try
> to store several of those vectors ...
>
> Cheers,
>
>    Sylvain