[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Precision of output
From: |
John Darrington |
Subject: |
Precision of output |
Date: |
Wed, 25 Jan 2006 10:22:27 +0800 |
User-agent: |
Mutt/1.5.9i |
An issue which arose yesterday when I was trying to use pspp in my
work:
It seems that many procedures output their results with a fixed
precision. I happened to need a greater precision than PSPP was
displaying.
The relevant section of pspp code (which I wrote) uses tab_float with
a literal interger for its W and D parameters. Perhaps it should have
used tab_value, and passed the variable's print member to FMT_SPEC ;
the problem is, in that particular routine, the variable is out of
scope (however see below!).
A quick browse through the code of other procedures shows that there
are perhaps quite a lot of places where this problem has crept in.
Perhaps we should do an audit of these sometime --- although I'd have
to think hard about the general problem to know what the correct
precision and width should be an all cases. For example, given a
bunch of integers, it might be necessary to know the mean value with
better than integer precision ...
Maybe it'd be better to modify tab_float so that the sentinel value
-1, displays values with variable precision ? ie using %g.
J'
--
PGP Public key ID: 1024D/2DE827B3
fingerprint = 8797 A26D 0854 2EAB 0285 A290 8A67 719C 2DE8 27B3
See http://pgp.mit.edu or any PGP keyserver for public key.
signature.asc
Description: Digital signature
- Precision of output,
John Darrington <=