Re: Adding C++ Code generation support for c++ implemented blocks

[Daniel Estévez]

Hi Martin and Marcus,

Just seen this thread.

I would suggest that you base the demodulator part of the flowgraph on 
the current FSK demodulator that gr-satellites uses:

https://github.com/daniestevez/gr-satellites/blob/main/python/components/demodulators/fsk_demodulator.py

This is a Python hier block that can handle several different cases, but 
you can make a simpler C++ version adapted to S-NET.

This uses the Symbol Sync block with a Gardner TED instead of Mueller 
and Müller (mainly because I heard fred harris say that Mueller and 
Müller is not very good, though he didn't say which are the TEDs he 
prefers), and in general it is a better and more modern approach at FSK 
demodulation than the flowgraph for GNU Radio 3.7.

The Sync to PDU block is a Python hier block, but it uses only C++ 
blocks, so it would be really easy to port it to C++.

https://github.com/daniestevez/gr-satellites/blob/main/python/hier/sync_to_pdu.py

Regarding the S-NET deframer, as Marcus pointed out, this is Python (it 
makes heavy use of numpy) and it uses a BCH decoder in Python. The 
reason why this is done in Python (as all the other Python code in 
gr-satellites) is primarily because of the faster development, and also 
partly better maintainability due to the smaller and simpler code.

Performance-wise, the implementation of the BCH decoder in Python is a 
joke, but for these satellites that transmit at a few kpbs, performance 
is not a problem unless decoding on a really constrained embedded platform.

I would be happy to see C++ implementations of the BCH decoder and the 
S-NET deframer being upstreamed and replace the Python implementations 
in gr-satellites. Marcus was right on the money with his comment about 
giving away free puppies, but if the code quality is good and there are 
unit tests, I have no regrets about taking over maintaining the code 
when it is upstreamed.

I would definitely encourage upstreaming the resulting code. Specially 
with university student projects, otherwise it is very difficult to 
ensure the long-term maintenance of the software, since projects end and 
students go. The code then bit rots. As an example, I can mention the 
BEESAT GNU Radio out-of-tree module, which I think is still only 
available for GNU Radio <= 3.8 (this was also a TU-Berlin project).

Finally, I agree with Marcus on whether porting all this stuff to C++ is 
the best approach if the goal is just improving how the software is 
shipped and installed. The idea to build everything into a single static 
binary might sound appealing, but to me it seems more similar to how 
applications are bundled for embedded platforms than for the desktop, 
and I can tell you that it can be full of issues.

Would you link statically the GNU Radio libraries in the binary as well? 
This is definitely possible, and you can produce a binary that only 
requires libc, libstdc++ and friends (libpthread, libm, etc.). However, 
you'll face problems with libc and libstdc++ versions across different 
distributions, unless you build against an version of these that is 
older than the ones in all the distributions you want to support.

How about multi-platform support? Building just one of these static 
binaries is a bit of a pain (for embedded I tend to set up a Docker 
container that builds everything from scratch as required). If you want 
to support Linux x86_64, Linux arm, Linux aarch64, Windows x86_64, Mac 
x86_64 and Mac aarch64, then things start to look daunting. And don't 
forget that if you want people to be able to modify and rebuild the 
software, the build system needs to be easy to use and not break when 
software versions change (this is why I use a Docker container for 
building).

Also, coming back to what I said about bit rot, it's quite likely that 
no one will maintain this static binary build system long-term, so your 
binary will end up bundling an old version of GNU Radio and other 
dependencies. It will work in the same way as it did on release day, but 
it won't benefit from improvements in new versions of the dependencies.

All I'm saying with this is that building and shipping software is 
complicated. Marcus' suggestion of using conda is great because it 
already saves you much of the work and complications.

Some words about user friendliness. Unless you're going to embed the 
decoder in a GUI application that supports a wide range of SDR devices, 
software like this is never going to be trivial to use. You'll typically 
rely on users running your software as a "backend" that receives samples 
from a "frontend" SDR application in some way (audio interface, network 
protocols, pipe, etc.). Users will need to know the tricks of how to set 
up their SDR hardware and frontend application of choice and connect it 
to your backend (virtual audio cables in Windows are a common trick, 
resampling might need to be involved, etc.). I would say that, with this 
in mind, these days gr-satellites is reasonably easy to install and use, 
even on Windows.

Users that are familiar with all of this can conda install 
gr-satellites, and run something like

gr_satellites "S-NET A" --audio --samp_rate 48e3

or

gr_satellites "S-NET A" --udp --samp_rate 48e3

and setting up the connection to the SDR frontend software.

Where gr-satellite falls short is in providing a nicer user experience 
in terms of GUI (there's no GUI), such as having graphical means to 
assess signal quality (spectrum/waterfall display of received signal, 
constellation diagram / symbol time domain plot, SNR estimate plot, 
etc.), having a GUI to control options, and making it easier to 
troubleshoot why decoding is failing (there are many subtle ways to 
corrupt/distort a signal if the set up is not done properly). Tackling 
part or all of this would be orders of magnitude more work than porting 
a couple Python blocks to C++ and bundling the software as a binary.

Regarding signal transmission, gr-satellites is generally lacking in 
this respect. The main reason is that there are very few satellites to 
which anyone can uplink, and most of the few which support an open 
uplink use AX.25, for which there are many TNC applications available. 
Implementing the required GNU Radio blocks to encode and modulate uplink 
packets is a reasonably straightforward task, but where I'm lacking a 
good architectural concept is in interfacing this with an SDR. There is 
a varied degree of support of bursty transmission mode in the different 
SDRs in the market, and most of the common SDR "frontend" applications 
have no support for TX, let alone for getting TX samples from an 
external application. There can be other potential problems with GNU 
Radio implementations if things are not done right, such as large 
latency. Additionally, there is often the need for hardware control, 
such as PTT or TX/RX switches.

I think it would be great to open a discussion with the amateur 
satellite community to try to come up with good solutions for this problem.

Best,
Daniel.

On 18/05/2023 13:54, Marcus Müller wrote:
> Hey Martin,
>
> welcome to the community :)
> As you can imagine, with the GNU Radio project, code contributions are 
> always welcome, especially when it comes to adding code generation for 
> existing blocks written in C++. I'm pretty optimistic Daniel feels the 
> same way about gr-satellites.
>
> The S-NET flowgraph screenshot is already a bit older, it still uses GNU 
> Radio 3.7, but with the symbol sync block, it might be quite 
> straightforward to port it (Clock Recovery MM has been deprecated; you 
> can use Symbol Sync with a Mueller and Müller TED instead). You need 
> modern GNU Radio, because on 3.7, C++ generation was not there, and 
> also, the more modern the GNU Radio, the more blocks already come with 
> C++ generation definitions.
>
> I do have one piece of bad news for you, though: Many (most, I think) 
> blocks in gr-satellites, such as the S-NET deframer, are Python blocks. 
> So, the thing that's missing for creating C++ flowgraphs out of these is 
> not only the definition of how the C++ code needed to instantiate them 
> looks like, but also: a C++ implementation!
>
> And that is where it gets hairy: I don't think it's overly complicated 
> to re-implement snet_deframer.py from gr-satellites in C++. The question 
> is whether you *should*, on two levels: first, that's an effort you'd be 
> doing to recreate something that already exists, and second, you should 
> probably really work with upstream (so, Daniel) on whether he would 
> prefer your C++ implementation (he positively might! It's going to 
> faster, especially since you'll be implementing the BCH decodder in C++) 
> or his Python implementation (which is going to be easier to read and 
> maintain, probably). The thing with upstreaming is that you're giving 
> someone your code to take care of, and that's a bit like giving away 
> free puppies: A great gesture, but comes with some work.
>
> In this light, I'd recommend you go through each block in the flowgraphs 
> / applications you want to use and consider whether it's worth going 
> through the porting effort to avoid having to package Python.
>
> There's an alternative to having to package Python yourself, and all the 
> libraries (which are quite a few) that (the used subset of the in-tree 
> modules of) GNU Radio + gr-satellites would use:
> Make sure the conda recipes work flawlessly for the software you want to 
> ship, and build a conda installer akin to Ryan's radioconda for that 
> specific application. I guess you're mostly targeting Windows there – 
> making a nice installer should not be impossible, but I don't have any 
> personal experience with it. That would have the added benefit that, if 
> you decide that you want your target audience to also have access to 
> truly unportable Python-only software like GRC, you could, at nearly no 
> development cost, add that.
>
> Either way, what you set out to do is truly remarkable and I'm happy 
> you're tackling it!
>
> Best,
> Marcus
>
>
> On 18.05.23 11:25, Martin Hübner wrote:
>
> > Similarily, the C++ support for the rather easy quadrature demod block 
> > should look like this then, right?
> > https://github.com/Akira25/gnuradio/tree/add\_cpp\_generation-quadmod
>
> That link's private, but if you want us to have a look at it, why don't 
> you open a pull request against gnuradio/gnuradio? You can mark it as 
> "Draft" and write a nice comment what it signifies, and maybe even link 
> to your message in the discuss-gnuradio mailing list archives.

OpenPGP_signature
Description: OpenPGP digital signature