Re: [Discuss-gnuradio] [USRP-users] Gnu Radio apps freezes (locks up)

[Rickard]

On 31 mar 2012, at 15.43, Tom Rondeau wrote:

> On Thu, Mar 29, 2012 at 6:04 AM, Rickard Radio <address@hidden> wrote:
>> 
>> On Mar 29, 2012, at 3:26 AM, Tom Rondeau wrote:
>> 
>>> On Wed, Mar 28, 2012 at 10:02 AM, Rickard Radio <address@hidden> wrote:
>>>> Hi list,
>>>> 
>>>> After I upgraded to latest Gnu Radio 3.5.2, and latest UHD (and images), 
>>>> GR applications just freeze when running. No warnings, error messages or 
>>>> overflows etc. Just freeze.
>>>> A simple FFT plot directly on received samples from the USRPN210 just 
>>>> freezes after some seconds, or minutes (depending on the sample rate), 
>>>> although the load on the machine is not high. Need to kill GR-app and 
>>>> restart it, with the same problem occurring again.
>>>> 
>>>> This has never been a problem with earlier versions of GR/UHD, from about 
>>>> 6 months ago.
>>>> The freezing happens quicker with high sample rate setting but also with 
>>>> lower, eventually. No overflows happen (which was possible to get before 
>>>> with too high sample rates or load, etc.)
>>>> 
>>>> The USRPN210 stops sending samples to the computer at the same moment as 
>>>> GnuRadio freezes (as observed on the system monitor).
>>>> 
>>>> Same thing happen on two identical laptops running Ubuntu 10.10 (also 
>>>> upgraded it from 10.04). Not sure if its a strict GnuRadio problem (since 
>>>> it worked before), UHD, or some problem with the Ubuntu Linux 10.10. It 
>>>> work(ed) flawlessly with another machine on OSX (before I tried to upgrade 
>>>> GR on it but then got stuck...) with identical UHD version and images.
>>>> 
>>>> Installation of UHD+GnuRadio with the automatic linux script runs without 
>>>> any problems, as before, no errors or warnings.
>>>> 
>>>> Any de-freezing help or clues appreciated!
>>>> 
>>>> Rickard
>>> 
>>> Rickard,
>>> 
>>> Just to be clear. When you install 3.5.2 from the tarball, it freezes.
>>> When you use the build-gnuradio, everything works fine?
>>> 
>>> What's your machine?
>>> 
>>> Tom
>> 
>> Tom,
>> 
>> The installation with build-gnuradio script works just fine, as before (no 
>> tarballs).
>> Same result on both laptops, Acer Aspire TimelineX with i3 processors (2.26 
>> GHz),  running Ubuntu 10.10.
>> I did not have this problem earlier with Ubuntu 10.04. Or on a Mac with OS X 
>> (with the source from git).  Could Ubuntu 10.10 cause the problem somehow?
>> 
>> Note: Halting/freezing only happens when running an application (with the 
>> N210) as a receiver. (Not transmitter, see below.)
>> The flow of receiving samples just halts after a while and the application 
>> freezes/halts (as a consequence).
>> This happen sooner with high sampling rate (after a few seconds with 
>> 25MSPS), but eventually also with a bit lower sample rates. The CPU's are 
>> not overwhelmed (< 50%).
>> It happens even if the UHD usrp source is connected directly to a null sink 
>> only. I do not get any overflows before the halt.
>> In fact, I cannot even provoke a continuos stream of overflows since the 
>> reception just halts instead of producing overflows, which was the result 
>> earlier.
>>  GnuRadio itself does not freeze as a whole (like the grc in the 
>> background), just the running application, which I then need to abort.
>> 
>> Strangely enough, this "freezing/halting" does NOT happen when transmitting, 
>> correspondingly, even with high transmit sample rates such as 25 MSPS (or 
>> now possible even with 50MSPS with 8 bit/samples!). Then it works just fine 
>> - even without underruns (when using just a file source).
>> 
>> 
>> / Rickard
> 
> 
> Rickard,
> 
> Thanks for the data. Unfortunately, I have no idea what to make of
> this. There isn't that much difference between the last release
> (3.5.2.1) and what you get using the build-gnuradio script. That just
> grabs the latest master version from our git repo, and we haven't done
> all that much since the release. That really doesn't make a lot of
> sense.
> 
> How's your git? If you're comfortable doing so, can you check out the
> v3.5.2.1 tag on git and try that one instead of the tarball release?
> 
> Thanks,
> Tom



Thanks for your info.  My current take on this interrupt issue:

The problem may not be gnuradio or uhd's "fault", I now suspect. Instead 
somehow the network connection and its settings might cause the interrupts. 
However, I am no linux guru so I am learning at the same time as I am doing.

First, I've updated Ubuntu, from release 10.10 to 11.04, but then nothing 
worked (just got a completely blank screen without any gui), so fast-forward 
upgraded to 11.10 and then both laptops came right back on track. I then also 
updated the gnuradio+uhd to latest (3.6.0) version using the excellent 
build-gnuradio script (as before), which itself went just fine (also as 
before). 

Unfortunately, the exact same problem with interrupts (total halt) in the 
receiver at high sample rates persists. Note, as before, this happens (only?) 
with very high rates over the Ethernet (about 400 mbit/s or more, the higher 
rate the sooner it halts, typically just a few secs) although the computer 
display no overruns or other errors. 

Then by jacking around with the MTU setting (100,500, 1500,5000, etc), 
increasing the default too low (and initially also gnuradio complaining) buffer 
settings (net.core.rmem_max, net.core.wmem_max), disabling the Ubuntu network 
manager (via the menu) and removing the automatic network configuration when 
USRPN210 connects and instead setting up the network connection manually with 
"sudo ifconfig eth1 192.168.10.1" ,  I *sometimes* can get the Ethernet 
connection into a state to work with the N210 at high sampling rates without 
any interruptions at all !

In that case, a beautiful continuous flow of samples to the crunching computer 
(like a fft-plot), at highest possible rate 50 MSPS, 8 bit/samples over the 
wire. This *can* happen with a MTU above 1500 (or more), buffers increased to 
recommended settings by UHD, and when this works the Ubuntu system manager 
tells me that some 834 Mbit/samples flows through the Ethernet cable, and about 
50% load on the CPU-cores, very nice. Then it also works for repeated runs, not 
just one "lucky" one, and for a prolonged period of time (more than an hour).  
In the working network state I can also easily provoke nice expected overruns, 
strings of 'ooooooooooo':hs, which isn't possible when the Ethernet connection 
is in the "wrong" and "interrupted" state - since then it just halts/stops 
without further info.

However, finding this working network state is not just a matter of setting the 
particular network parameters as I hoped it to be...  I suspect some other 
things are happening behind the scenes, maybe some other settings etc (I do not 
yet have full knowledge of ethernets full functionality and behavior, there may 
be more influencing parameters ?) which prevent me finding the working network 
state in a consistent manner. Quite weird.

I have USRP-N210 rev2 (says sticker on the back) but now noticed that when I 
burned the latest fw and fpga images with the net burner tool it prints 
"Hardware type: n210_r3" although I selected the fpga version image for rev2 
corresponding to my version. Could this be related to my issue? Haven't noticed 
that inconsistent message earlier, though.

If some of this rings any bells, please give me some further advise. 
Sorry for long post.

Thanks,
Rickard