Re: [Discuss-gnuradio] signals/atsc question

[John Gilmore]

> (and while I do have access to an mc4020 board, it is
> not in a location that can receive ATSC signals and I'm not about to run
> coax all over the place).

I have to start by saying: either moving the mc4020 board to where the
antenna is, or running coax all over the place, will be MUCH easier
than to do what's needed to squirt this fat a signal over the USB bus.
The mc4020 board runs fine in the current GNU Radio 2.x infrastructure.

(Maybe Matt or Eric, or someone else on the list, has some better
ideas than what I discuss below.  I'm not a signal processing guru,
and they are.)

>                        I am using a 4937 frontend, so that gets the
> USRP a signal from 2.75 to 8.75MHz. The only way that I've been able to
> capture the signal reliably is by doing a DDC by -5.75MHz with a
> decimation factor of 8. This puts the signal nicely between -3 and +3
> MHz, at 8MS/s with complex samples. In order to convert this to 20MS/s
> real, I've tried two approaches:

There are about 10.76 Msymbols/sec in an ATSC signal, so you are going
to have Nyquist problems if you try to represent them in 8M complex
samples/sec.  You'll need at least 10.76 complex MS/s.  (Indeed you
may notice that we were skating inside the edge by using 20
non-complex MS/s on the old hardware.)  If we had a faster USB bus,
the USRP could sample at a nice high rate, like 32 Msamples/sec, then
convert those complex samples directly into shorts that the old code
will be happy with, without any further glop.

Unfortunately that would take about 120 Mbytes/sec over the USB bus.

If we can coax the USRP into providing shorts rather than complex, we
can transfer twice as much data thru our bottleneck.  I think it can,
but I'm not sure how to ask it to do that -- and 60 Mbytes/sec still
wouldn't suffice.

I'm a little surprised that the code in the USRP's FPGA doesn't do
arbitrary sample rate conversion, which would let us just say
"Whatever the hardware sample rate is, don't worry about it --
interpolate those samples to 20,000,000 samples per second and hand
them to me over the USB".  That would let us design the sample rate to
match the bandwidth available.  Instead, the current FPGA code will
only "decimate" by fixed integer ratios, e.g. it'll throw away all but
1 out of N samples for you, without doing interpolation, giving you
little control over sample rate and thus over USB bus data rate.

But even at 20 Ms/s, with 14-bit samples passed in 2-byte shorts,
we've exceeded the capacity of the USB bus.  It would be pretty easy
to build a sample-packer that would pass 16 samples in 14 bytes
(rather than in 16 bytes), which would save us 1/8th of the bandwidth.
That would get the data rate to 35 Mbytes/sec, right at the hairy edge
of what USB2 can do.  To get more headroom, we could round or truncate
the samples from 14 to 12 bits, again with FPGA code.  (Indeed, a nice
sample rate converter should probably also offer sample size
conversion, letting you grab 20ksamp/s of 7-bit samples, or 32.778403
Msamp/s of 8-bit samples, or 55.3 Msamp/s of 4-bit samples, or
whatever else you want.  It might even be able to give you 20 ksamp/s
of 16-bit or 32-bit samples, I don't know the math involved.)  (C++
code on the CPU side of the USB bus would unpack these 14- or 12-bit,
or shorter, samples into shorts again.)

So, if all this nonexistent VHDL code existed in the FPGA, then you
could do the sampling with the USRP.  This is why I recommend doing it
with the mc4020 board.

Now back to the old code that you're trying to validate before you
port it.  The argument parsing in the old code was kludged to only let
you select particular rates like 20,000,000 samples/sec (-s 20).
Sorry.  The bulk of the code is parameterized with the variable
"input_rate", which is a double.  I think the vast majority of the
code would work if you set that variable to any value that doesn't
mess up the Nyquist rules.  You can fix the arg parsing so that
"input_rate = atod(optarg) * 1e6;" and then feed it whatever -s value
matches your sample rate.

(As you later convert the old code to the new framework, you'll want to
notice and fix any part that isn't sample-rate-independent anyway...)

> exception of the pilot which isn't very pronounced. (Note that it
> *seems* as if the pilot is stronger on the original than on the
> converted -- see http://web.mit.edu/imirkin/www/gnuradio)

It took careful antenna aiming and agc gain twiddling to get those
nice pictures.  And even more care to get signals that the code liked.

> Both of these methods yield similar results -- the gnuradio-0.9 atsc_rx
> happily takes the input, and generates an output TS file, with an error
> rate of 1. However, I have plugged this very same antenna pointed in the
> same way into my pcHDTV 3000 card, and it has no problems with the
> signal (maybe an occasional glitch).

The pcHDTV card uses a chip that's specialized and engineered to do
this job very well in a variety of environments.  By contrast, you are
only about the fourth person to ever run the ATSC input code.  It's a
"dancing bear", which took more than a year of solid engineering just
to get it dancing at all.  It hasn't been optimized, either for CPU
usage or for non-perfect signal reception.

When I reproduced HDTV reception, I had a little script called
"point", which loops digitizing 8M samples (less than 1/2 sec) to
disk, and then running the receiver on them.  On my 2002 Athlon MP,
the loop ran every 20 seconds or so (we'd use fewer samples, which
would make it faster, but I think you need that many for the
equalizer).  If the receiver doesn't like the signal, then you change
something -- the AGC, the antenna direction, the HDTV station you're
tuning into, the phase of the moon, etc.  The receiver code was quite
finicky -- and my antenna looks at SF's Sutro Tower thru less than a
mile of clear air, where at least a dozen HDTV stations transmit.
Lather, rinse, and repeat until you are capturing signals that the
code likes to decode.  THEN stop the script and capture a longer
period of time to disk.  Then, run the decoder on that longer signal.

I'll attach my favorite point script below.  I thought it was in the 
source trees, but now I can't find it except on my hard drive.

Note that your disk will need to be able to keep up with your sample
rate.  At 20 Msamples x 2 bytes/sample we used Linux's "LVM" (logical
volume manager) support to stripe a pair of partitions (on different
IDE buses) to handle writing at 40 Mbytes/sec.  If you can't do that,
you'll need to buffer it all in RAM and write it out at the end.  RAM is
cheap, and a gigabyte will store 25 seconds of samples.

Also note that our southbridge had a bug in which pushing a lot of
disk traffic would starve the PCI bus for cycles, disrupting the
signal capture's DMA.  We ended up moving the disks onto a PCI IDE
disk controller, rather than using the motherboard IDE controller.  YMMV.

        John

#!/usr/bin/env python
# -*- python -*-
#
# Copyright 2002 Free Software Foundation, Inc.
# 
# This file is part of GNU Radio
# 
# GNU Radio is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
# 
# GNU Radio is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with GNU Radio; see the file COPYING.  If not, write to
# the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
# Boston, MA 02111-1307, USA.
# 

import os, os.path, math, sys, getopt, string

# set srcdir to the directory that contains Makefile.am

try:
    srcdir = os.environ['srcdir']
except KeyError, e:
    srcdir = "."
srcdir = srcdir + '/'

def system_chk (cmd):
    r = os.system (cmd)
    if (r != 0):
        print "FAILED (%d): %s" % (r >> 8, cmd)
        if ((r & 0xff) == 127) :          # killed by signal (i.e. interrupt)
            sys.exit (127)
        return 0
        if ((r & 0xff) != 0) :          # killed by signal
            sys.exit (127)
        else :
            sys.exit (r >> 8)
    return 0



dir="/mnt/acquire/d"

use_eq = 1
i = 0+int(sys.argv[1])

while 1:
    i=i+1
    if use_eq:
        print "S!",
        sys.stdout.flush()
        system_chk ("mc4020-read-adc -b -c 8000000 --ch3 --1v >%s/test%d.dat" % 
(dir,i))
        print "->D ",
        sys.stdout.flush()
        os.system("sync")
        print "\r%d: " % i,
        sys.stdout.flush()
        system_chk ("./run_rx -v -L -S 1252 -s 20 -f %s/test%d.dat -o 
%s/test%d.ts " % (dir,i,dir,i))
    else:
        system_chk ("mc4020-read-adc -b -c 4000000 --ch3 --1v >%s/test.dat" % 
dir)
        system_chk ("./run_rx -S 47 -s 20 -f %s/test.dat -o %s/test.ts 
2>/dev/null" % (dir,dir))