Re: [Discuss-gnuradio] Data lost whe using big file sources

[Bogdan Diaconescu]

Ok, I got no so many replyies to this post and I thought I should present what 
I investigated so far.

I tried to trace the root of the problem and firstly I simplified the use case 
by filling the source files with just the same value (arbitrarily 0x18) and 
looking for different values into 
gnuradio-core/src/lib/runtime/gr_block_executor.cc:gr_block_executor::run_one_iteration()
 that is in the loop for the TPB.

One note, if I change the TPB to Single Threaded Schedulrer the problem is gone 
but the point is to use one thread for each block.

Looking for an error in the run_one_iteration() I see that the output of the 
file sinks is not corrupted, the corruption appears only in run_one_iteration() 
for the block I'm using, specifically d->input(i) has at a moment a corrupted 
value.

Speaking about corrupted value, the corruption I see is actually a zero value 
instead of expected data (0x18) but the funny thing is that if I print the 
value twice, on second print the value is the correct 0x18 (I guess here 
Heisenberg principle has it's part here :) )

From where the d->input(i) comes from? It is gr_block_detail that gets set-up 
when connecting blocks each other. This is probably next step to investigate. 

The results so far: moslty learning gnuradio internals, problem is still hidden.

Thanks,
Bogdan







> From: Bogdan Diaconescu
> Subject: Re: [Discuss-gnuradio] Data lost whe using big file sources
> To: address@hidden, "Martin Braun" <address@hidden>
> Date: Tuesday, April 10, 2012, 2:22 PM
> Hi Martin,
> 
> thanks for quick reply.
> 
> There are errors, values of the data that is different than
> the pattern.
> It is gr_block.
> 
> I never had time to look into this but I always speculated
> that the code that takes the data from the files does not
> expect the data to be available.
> 
> Thanks,
> Bogdan
> 
> 
> --- On Tue, 4/10/12, Martin Braun <address@hidden>
> wrote:
> 
> > From: Martin Braun <address@hidden>
> > Subject: Re: [Discuss-gnuradio] Data lost whe using big
> file sources
> > To: address@hidden
> > Date: Tuesday, April 10, 2012, 2:09 PM
> > Hi Bogdan,
> > 
> > On Tue, Apr 10, 2012 at 03:48:11AM -0700, Bogdan
> Diaconescu
> > wrote:
> > > Hello gnuradio fellows,
> > > 
> > > I have an issue that appears in all gnuradio
> versions I
> > used lately (I started with 3.3 and last week I updated
> to
> > latest from git) and I thought I should post here
> before
> > allocating the time to look into it by myself.
> > > 
> > > I'm modifying a gnuradio block that is connected
> from
> > python to 6 file sources. Everything works fine as long
> as
> > the files I'm using as source for data are relatively
> small
> > (30MB). When the files become large I see that the
> data
> > received in general_work() function is corrupted. It is
> not
> > massively corrupted but enough to screw my work.
> > 
> > I've seen this behaviour, too. My workaround was to use
> a
> > throttle block
> > after noticing that my code works with USRPs, but not
> with
> > files.
> > However, I never managed to trace the bug to the file
> > source. Thanks for
> > this!
> > 
> > > Investigating the problem, I did a small test
> with
> > having the 6 files filled with known patterns and
> printing
> > an error in the general_work() if what is received is
> > different. The result is that if I use files with sizes
> over
> > 100MB I see 50-80 errors in total.
> > 
> > Do you have errors, or are samples missing?
> > 
> > > Now, to unblock my work I did observe that if I
> insert
> > some printing in general_work() I will not get the
> errors.
> > Going further, inserting a boost delay of 100uS also
> solves
> > the problem.
> > > 
> > > some more data: 
> > > 1. I know the new blocks should use work() instead
> of
> > general_work() but is it still supposed to work as long
> as I
> > call consume_each(), right?
> > 
> > That should work since general_work() calls work() and
> then
> > consume_each(). Have a look at the code of gr_block (or
> was
> > it
> > gr_basic_block?)
> > 
> > MB
> > 
> > 
> > -- 
> > Karlsruhe Institute of Technology (KIT)
> > Communications Engineering Lab (CEL)
> > 
> > Dipl.-Ing. Martin Braun
> > Research Associate
> > 
> > Kaiserstraße 12
> > Building 05.01
> > 76131 Karlsruhe
> > 
> > Phone: +49 721 608-43790
> > Fax: +49 721 608-46071
> > www.cel.kit.edu
> > 
> > KIT -- University of the State of Baden-Württemberg
> and
> > National Laboratory of the Helmholtz Association
> > 
> > -----Inline Attachment Follows-----
> > 
> > _______________________________________________
> > Discuss-gnuradio mailing list
> > address@hidden
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> >
>