On Fri, Nov 25, 2011 at 8:55 AM, Martin Braun
<address@hidden> wrote:
Hi everyone,
we've recently been working one some GR projects where item sizes can
get really big. One example is cyclostationary spectral estimation code,
which will be released soon.
If you're not familiar with cyclostationary spectral estimation, here's
the problem: the result of the estimator is a matrix, which, depending
on your estimator resolution, can be quite big (at least a several
hundred thousand floats per estimate, which would correspond to one
logical item).
With GNU Radio, very large item sizes do not work that well since the
inter-block buffers are shared between threads and therefore, max size
is limited by the OS. I'm sure you know what I mean :)
Question is, what's a good way to handle this? For the cyclo stuff, we
had to fix things manually by introducing separate buffers, splitting
the estimate into many small items and then manually keeping track of
what already had been consumed.
This doesn't seem to be the ideal solution. The whole point of GNU Radio
is that someone else (i.e. the GR scheduler) takes care of all of this,
and I, the developer, can focus on the signal processing component.
I'm wondering if anyone has some experience with large items they would
like to share. In particular, is there a way to limit the number of
items in the buffers? Or is this possibly a shortcoming of GNU Radio?
Martin
Hi Martin,
Sorry for the delay in replying to this, as it's possibly a very important question. My reason for the delay is that I'm not sure about the answer and was hoping either I would think of something or someone else would :)
But here's a suggestion anyway to hopefully get the ball rolling again.
Thinking of the results as one large matrix item probably isn't going to work. As you said, there's a system limitation at play here. What if, instead, you passed the data along as though it were a stream of floats (or complex floats). Along side that, the block that creates the matrix sends a "start of matrix" and "end of matrix" tag, so the blocks down the line know what to look for and can reassemble the matrix from there.
I'm assuming that all of your blocks in the chain that deal with the data as a matrix are home-brewed and not current blocks, so you would only need to add the tag-handling capabilities to your blocks.
Have you come up with another solution in the meantime?
Tom