Re: Precision of output

[Ben Pfaff]

John Darrington <address@hidden> writes:

> An issue which arose yesterday when I was trying to use pspp in my
> work:
>
> It seems that many procedures output their results with a fixed
> precision. I happened to need a greater precision than PSPP was
> displaying. 
>
> The relevant section of pspp code (which I wrote) uses tab_float with
> a literal interger for its W and D parameters.  Perhaps it should have
> used tab_value, and passed the variable's print member to  FMT_SPEC ;
> the problem is, in that particular routine, the variable is out of
> scope (however see below!).
>
> A quick browse through the code of other procedures shows that there
> are perhaps quite a lot of places where this problem has crept in.
> Perhaps we should do an audit of these sometime --- although I'd have
> to think hard about the general problem to know what the correct
> precision and width should be an all cases.  For example, given a
> bunch of integers, it might be necessary to know the mean value with
> better than integer precision ...
>
> Maybe it'd be better to modify tab_float so that the sentinel value
> -1, displays values with variable precision ? ie using %g.

What I really want to do in the end is to make PSPP produce its
output in a machine-readable format, such as HDF5, and then
derive human-readable output from that using translators.  Then
it's trivial to get whatever precision you want/need: just
extract the "double" value from the machine-readable output.

In the short term, we'd need something else, of course.  I don't
have a great idea, off-hand.
-- 
Ben Pfaff 
email: address@hidden
web: http://benpfaff.org