Re: [Discuss-gnuradio] Dropping samples "D" when CPU at 60%

[CEL]

Regarding your flow graphs:

energy_det:
###########

Moving average
==============

Your Decimating FIR is really just a moving average. Try that block; it
uses far less operations per sample.

Delay-Divide
============

This is a very nonlinear operation:

---------------->Divide
  -->Delay160----^

It's *probably* something like an f_sample/160 notch filter with
terrible, terrible spectral properties?

Are you *sure* this is what you want? Can you give a motivation for
this?

To motivate my doubts:

You get a peak at the output of your divider all 160 samples after a
very low power instant, 1/(x[t-160]). The actual energy at that point
plays a minor role, the fading depth is what makes the peak. Because
your filter only has a "memory" of 50 samples, what happens 160 samples
later is only linked through the channel's autocorrelation function. 

I recommend looking at plots of typical channel realizations[2] to see
why that's a bad idea. 

You'd need very low doppler spreads (in relation to your sampling
rater) for e.g. a Rayleigh model to make it likely enough that your
channel 160 samples (I approximate that in my head with a Gaussian with
variance ca doppler spread †) later is still correlated enough to just
avoid arbitrarily high peaks that really don't lend themselves to power
detection.

Division is a tool *very* rarely used with powers in signal processing,
exactly for its stability problems.


Delay-Subtract
==============

Your

-> Delay->Subtract
----------^

is really just a [1,-1] FIR Filter. Use that, and save some processing power. 
It's a half-band high pass filter, see the plot in [1].

Or don't -
 because the "threshold" before that will inherently give you an impulse that 
would pass the filter.

Threshold--High Pass--same Threshold
=====================================

So, the filter probably doesn't do anything useful, since you've definitely 
killed all high-frequency components with the moving average of length 64.
The thing here is that at frequencies where your delay-subtract passes signal,
you only see the (slowly decaying) side lobes of the sinc that is the frequency 
response of your moving average FIR.
And these side lobes have the unfortunate property of being f_sample/64 wide 
and having nulls in between!

So, what you're seeing at the output of that is totally dependent on 
*miniature* fluctuations of the periodicity of changes in receive power.
This makes little sense for an energy detector.

The second threshold is more than questionable. The rising output edge of the 
first threshold should inherently pass the second.

Overall
=======

This doesn't look like you've /designed/ it to fit a purpose; more like the 
result of a trial-and-error session.
Maybe sit down once more and think very closely about what the mechanism should 
be, and then implement that.


Best regards,
Marcus


†: frankly, please don't quote me on that one. I just looked really hard at the 
characteristic function of the Rayleigh distribution for 𝜎=1.8 and said "naaah" 
aloud.

[1]
[2] https://en.wikipedia.org/wiki/Rayleigh_fading

On Tue, 2018-03-27 at 18:00 +0000, Bakshi, Arjun wrote:
> Hi Marcus,
> 
> Thanks. I think it was just that ~18MSps is too much for my PC. After writing 
> a cpp block for detecting the trigger and piping some samples, the cpu 
> utilization jumped up to +90% on all cores, and I can now support 6.25MSps x 
> 3 streams. But not much more. For some reason I expected that my cpu could 
> handle the full 1Gbps connection.
> 
> The CPU is an older (Sandy Bridge/gen-2) address@hidden 4 cores, running on 
> Ubuntu 16. Memory wasn't an issue, as the memory usage for the flow hovers 
> around 100MB, with ~10GB to spare. 
> 
> The ethernet connection is able support higher sampling rates(8Msps x 3 
> streams) with direct USRP source to null sink connections. So the 
> ethernet/network should be fine.
> 
> I've attached screenshots of my flowgraphs, and some code for the blocks I 
> use. 
> 
> Only concern now is if its harmful to have 90%-100% cpu utilization for an 
> hour or so.
> 
> Thanks,
> 
> AB
> From: Müller, Marcus (CEL) <address@hidden>
> Sent: Tuesday, March 27, 2018 4:39:10 AM
> To: address@hidden; Bakshi, Arjun
> Subject: Re: [Discuss-gnuradio] Dropping samples "D" when CPU at 60%
>  
> Dear Bakshi,
> again, we'd need to get a holistic view of what you're doing. All we
> can do is speculate wildly based on your description.
> 
> > Is it just that the block I've written is not efficient enough? I
> would expect higher CPU utilization in that case.
> 
> Possibly. Don't know. You tell us nothing about your block, especially
> no code.
> 
> Please be more precise when asking, give your code, give actual
> throughput benchmark, give us hardware information (which I've asked of
> you before, but you never delivered). It's actually not easy to ask a
> good question, but you really need to get better at it!
> 
> Fair warning: a sum rate of 18.75MS/s is really not that little. It's
> absolutely possible you have shoddy network hardware or drivers that
> just can't deal with that much data. Again, pure speculation, since you
> tell us nothing about your hardware, or your OS, or anything about the
> environment you're working in.
> 
> All that we can say is: something in your overall system, be it
> hardware, your CPU, your RAM, your software or anything else, doesn't
> work as you want it to.
> 
> Best regards,
> Marcus
> 
> On Sat, 2018-03-24 at 19:30 +0000, Bakshi, Arjun wrote:
> > Hi all,
> > 
> > I have written a block in python to detect and decode some packet 
> > transmissions. I've noticed that when I have 3 rx streams running at 
> > 6.25Msps (total network traffic is 75MB/s), the host starts dropping 
> > samples after a while. The CPU utilization does not exceed 60%, and the 
> > Ethernet isn't being saturated. I've increased buffer sizes 
> > (net.core.w/rmem_max) to the largest values possible
> > 
> > A 'D' in the console means that the host isn't consuming samples fast 
> > enough.
> > 
> > Is it just that the block I've written is not efficient enough? I would 
> > expect higher CPU utilization in that case.
> > 
> > I think the most expensive thing I'm doing is looking at each sample in a 
> > stream of floats to find where the values goes above a threshold. This part 
> > is in my block, and triggers the decoding logic, which is in the same block.
> > 
> > I don't know how to do this more efficiently. I'm thinking of re-writing it 
> > in C++, but I don't know how much better it'll get.
> > 
> > Thank you,
> > 
> > AB
> > _______________________________________________
> > Discuss-gnuradio mailing list
> > address@hidden
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>   Discuss-gnuradio Info Page - lists.gnu.org lists.gnu.org
> Discuss-gnuradio -- GNU Radio, the Free & Open-Source Toolkit for Software 
> Radio About Discuss-gnuradio

smime.p7s
Description: S/MIME cryptographic signature