[Discuss-gnuradio] Proposed Offline Tools: Header Generation, File Manipulation and Format Conversion

[Garver, Paul W]

To use GNURadio as a data analysis platform, I've been developing a few tools 
which I think will be useful for the general community.  These are primarily 
Python/command line tools intended to manipulate prerecorded files.  I'm airing 
them out here for some feedback to ensure I'm not duplicating work and get 
helpful suggestions.  I'll make this a feature request on GR.org if people have 
sufficient interest.

Header Generation:
A user may record raw RF samples to disk but have metadata collected 
(frequency, sample rate, time of recording, etc) in a notebook, text file, etc. 
 There should be a way to generate headers to add metadata after the fact.  To 
this end, I've developed gr_mkheader which generates detached headers given 
some command line inputs.  This task could be extended to manipulating existing 
headers.

Data File Types:
Currently, either one must know a-priori the file type of raw data, or 
interpret three variables (Complex,item size, and type) if headers are 
available.  It is proposed these types be rolled up into a more succinct 
format, such as is used in UHD [1].  These file types should also be 
interpreted by gr_read_file_metadata.  For example, 
(Complex=true,itemsize=4,type=short) should be displayed as interleaved shorts 
(sc16). There should be a table in the documentation describing all (or common) 
combinations for easy reference. 

File Truncation with Headers:
For long records, it is often desirable to look at a subset of the data.  While 
there are certainly ways to do this in GRC/Python, to my knowledge there is no 
header support for this.  Additionally, a different GRC flowgraph is required 
for every data type.  The desired program would read a file with recorded 
samples and metadata and generate the subset with proper headers.  It would 
also take a output type and perform file conversion.  As a special case, this 
program can convert between file types by asking for the entire data set 
instead of a strict subset.  I'm in the process of developing a program to do 
this.

Do the file conversion routines currently support metadata?  E.g. do they 
update the data type/item size tag?  My understanding is that detached headers 
must start immediately in the file.  If we use a head/skip N block on a stream 
with metadata information, will the new headers be interpolated based on the 
closest header to the desired sample subset?






[1] 
https://github.com/EttusResearch/uhd/blob/master/host/lib/convert/convert_impl.cpp