Re: [Discuss-gnuradio] How to analyse chunks of data in GNU Radio

[Marcus Müller]

Hi Chris,

don't worry, that's a very valid (and valuable) question, so welcome to the mailing list!

So, having read the GNU Radio guided tutorials (I presume you've read [1], but there's others, older ones, floating around the net), you're probably aware of the general_work() function of each block.
Now, the idea behind GNU Radio generally is that the scheduler calls your block's work with as many items (in your case, samples) as it can. Assuming the file source reads as many samples at once as fit in half its output buffer, roughly the following happens:

1. GNU Radio calls the source's work method. That work method writes the first 4096 samples from the file to its output buffer, and returns the number 4096, telling the runtime that it has produce 4096 items.
2. GNU Radio now calls your block's "forecast" method and asks it "to produce <size of your block's output buffer/2> items, how many items would you need?". Your block would then answer something like "10000"; since that is more than what is available, GNU Radio asks again: "to produce <size of output buf/4>...", your block still wants 10000, and so on. That's kind of a bad thing, because it makes the flow graph stall -- there's no way to produce 10000 input samples (the input buffer isn't even large enough), and your block can't work with less, so your block's general_work() is never called.

There's now two fundamental ways to go about this:
1. changing how GNU Radio sizes the buffers, and
2. changing your implementation to be able to adapt to smaller item "chunks".

Generally, 2. seems advisable -- it keeps the samples flowing more "constantly", and hence minimizes latency and maximizes parallelization.  However, not every algorithm lends itself to that, so here's an approach to do 1.:

You could either use the file source's "set_min_output_buffer(20000)" of the file source (if you're in the GNU Radio companion, look at the "advanced" tab), to instruct GNU Radio to reserve at least twice as much buffer as you'll need, or you could specify the input item size to be 10000*complex (assuming your input samples are complexes), and let the file source produce vectors of the same length. Both aren't really "nice" to the scheduler, but should work.

Now, I usually recommend a different approach to finding packets; that approach stems from the fact that your preamble is probably much shorter than 10000 samples, and until you find it, you can just throw away all input samples that you're sure don't contain a preamble. You can be sure of that for all but the (preamble length - 1) last samples, after correlation. GNU Radio has a feature called "history", which allows you to safe the last N items from last call to general_work as "prefix" to the new items. What I hence propose is a finite state machine that simply has the two states "looking for preamble", and "passing through input to output".

In the "looking" state, forecast always returns "to produce N items, I'll need at least (N + preamble length) items", whereas in the passing state, it returns "to produce N, I'll need N". Your general_work method would act based up on the current state, and consume() every item unless it finds the preamble, and then sets an internal counter "items_passed_through" (or similar) to as many items as it then can savely pass through to the output, copy those to the output buffer, change the state variable to "passing through", and return the number of passed items (to let GNU Radio know that it produced output).

In the "passing through" state, your block would just copy as many input items to the output as it can -- the upper limit either being the available items on input, or the remaining items to be passed through (hence the internal counter above). After that, it would just consume() all passed on items, return the items copied, update the counter, and if the number of items that should be passed have been passed, changes the state to "looking".

I hope this gave you some kind of architecture to consider;
happy hacking,
Marcus

[1] https://gnuradio.org/redmine/projects/gnuradio/wiki/Guided_Tutorials

On 04.10.2015 00:42, Chris Dandy wrote:

Hello,

I'm attempting to use GNU Radio to parse data packets and not having much luck. I'm a beginner at both GNU Radio and Python, so I apologize if the solution to my issue seems obvious to everyone else.

Currently, I have a block of standalone Python code which uses the numpy.fromfile() command to read in data contained in a .bin file. My code basically grabs 10,000 samples from the file at a time and stores them in an array, does some processing on that array to find the data packets, and then moves on to another 10,000 samples until the entire file has been read. It works great; no problems whatsoever.

The goal is to be able to use the code with a USRP to receive data in real-time, which is where GNU Radio comes in and I'm getting stuck. My first task is to get the code working with GNU Radio using a file source to read in the same .bin file I mentioned earlier. When that works, I'll move on to using the USRP.

I've done all the GNU Radio Python tutorials and read absolutely everything relevant I can find, but I'm having trouble understanding how to receive 10,000 samples at a time from the file source. At this point I've been successful creating OOT Python blocks that work; I'm just not sure how to make them operate on chunks of data instead of one sample at a time.

I know GNU Radio has some support for packets, but I haven't been successful using those modules. I think maybe the packets I'm trying to read aren't in the right format for them because they don't have a header? (My data packets have the form [Preamble with value 0xF1D90][60 data bits][16 bit CRC].)

I'm not really sure where to go from here. I've tried to do as much research as I can, but as a newbie the GNU Radio documentation is pretty overwhelming and there's a lot of it that I don't understand. If anyone has any hints or advice, I'd really appreciate it. I'd love to learn from someone who has more experience with this than me.

Thank you!

Chris Johnson

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio