Re: [Discuss-gnuradio] Raw sample storage format

[Dan Gisselquist]

Dave,

A standard header for data files makes a lot of sense.

I have struggled in times past with creating file formats whose useful lives  
haven't been as long as I've desired.  Ideally, you want a format that grows  
as your needs grow, without requiring you to rebuild all of your legacy  
software that already works with the header.  To that end, may I recommend an  
alternative header:

uint32          [4 byte magic key of ASCII, readable
                by any hex editor, such as 'RSSF'
                for raw sample storage format]
uint32          [4 byte file version.  Perhaps
                two bytes for a major and two
                bytes for a minor file version numbers]
uint32          A seek offset to skip past the header to the data in the file.

After this piece, it is easy to add optional information to the header while  
maintaining compatibility with any software that reads this piece.  Of course,  
there are still some other mandatory information requirements:

> uint32                   sample_rate
> uint8                    bits_per_sample

I'd personally place the name/description, date, and if_frequency information  
into optional header fields, call them "header packets", all having a a  
preamble such as:

uint16          Header packet type (such as 1 for name/description)
uint16          Header packet length
(whatever)      Header packet contents

That way, any program that processes the header can quickly scan through all  
of these packets to find any information it understands, while skipping any  
other information.  Better yet, extending this format is easy.

The one question I would have, however, is whether or not this format would  
support recording data from multiple antennas at the same time?  Would the  
format be limited to single data channels, or would it support multiple?  It's  
also worth asking how important word alignment is in this process--especially  
since compilers like to align things to word boundaries automatically.

Hope this helps ...

Dan




On 03/09/2005 12:03:45 PM, David Carr wrote:
> As it stands right now, a raw sample file is just that, raw values in a  
> file.  It seems that it might be wise to include some sort of header in the  
> file that describes its contents just a little.  An idea for a few fields is  
> below.  To ease implmentation the header could be a fixed length and if the  
> program was uninterested in the header info it could just seek to the  
> beginning of the data.
>
> example header:  [data could be stored in text or binary format - binary  
> shown]
> uint16                          [magic key to descriminate headerless files  
> from files with headers - for backwards compatibility]
> char[255]                    name/description
> sizeof(struct timeval)   date
> char[255]                    comments
> uint32                         sample_rate
> uint8                           bits_per_sample
> double                         if_freq
> ...
>
> I think that the implementation would be rather trivial to do (if I could do  
> it, it must be).  Whether to store the data in text or binary format is an  
> interesting question.  If the data is in text format its human readable but  
> the binary format is more compact and easier for the code to deal with.
>
> Lets hear some thoughts,
> -David Carr
>
>
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/discuss-gnuradio