Re: [Bug-apl] number precision in saved workspaces

[Juergen Sauermann]

Hi Xiao-Yong,

the problem with the *scanf() *(and also with the *printf()*) functions 
is that text of numbers
is different from APL, for example *-* vs. *¯*, complex numbers, etc.

I therefore believe it would be more useful to have a *⎕CR* variant that 
avoids full
parsing and execution of its argument (which *⍎* does). Actually *⎕TF* 
is already doing this
and should be much faster than *⍎*, for example:

*      ⍝ create numeric variable**
**      R←1.1 2J2 4 1R2.2 ¯5 6E¯2 **
****
**      ⍝ convert variable into text (to be written to a file)**
**      ⎕←TEXT←1 ⎕TF 'R'**
**NR 1 6 1.1000000000000001 2J2 4 
¯5.8850111725534582E¯1J8.0849640381959009E¯1 ¯5 **6E¯2**
****
**      )ERASE R*

***      ⍝ read text back from file and re-create variable**
**      1 ⎕TF TEXT**

****      R**
**1.1 2J2 4 ¯0.5885011173J0.8084964038 ¯5 0.06**
*
The *1 ⎕TF* record format is fairly simple (record type (N) followed by 
the variable name (R in the example) , rank, and shape of the value).
The inverse *1 ⎕TF* tokenizes the string but does, unlike *⍎*, neither 
parse nor execute it.

Please use *SVN 745* since I have fixed some *1 ⎕TF* bugs and improved 
its error reporting.

/// Jürgen


On 06/16/2016 12:35 AM, Xiao-Yong Jin wrote:
> Hi Jürgen,
>
> Just realized my script got much slower (⍕ on a few million numbers from a text 
> file) with these changes.  Since you have printf in ⎕FIO, can you please add 
> scanf, fscanf, and strto* functions to ⎕FIO, too?
>
> In addition, the tokenizer seems to be confused with exponents.
>
>        1E9
> 1000000000
>        1E10
> 1E10
>        1E9×1E9
> 1E18
>        ?1E10
> 3942470777
>        2?1E10
> DOMAIN ERROR
>        2?1E10
>        ^^
>
> Wouldn't anything below 9E18 be parsed as an integer?
>
> Best,
> Xiao-Yong
>
> > On Jun 13, 2016, at 9:24 AM, Xiao-Yong Jin <address@hidden> wrote:
> >
> > Yes, they work.  Thanks.
> >
> > > On Jun 11, 2016, at 9:29 AM, Juergen Sauermann <address@hidden> wrote:
> > >
> > > Hi Xiao-Yong,
> > >
> > > thanks, fixed in SVN 740. I haven't checked )DUMP and )OUT but I suppose
> > > they work now since the fault was in the tokenizer (which is also used by
> > > )LOAD and )IN).
> > >
> > > /// Jürgen
> > >
> > >
> > > On 06/10/2016 10:34 PM, Xiao-Yong Jin wrote:
> > > > Hi Jürgen,
> > > >
> > > > Thanks.  I found more floating point reproduction issues.
> > > >
> > > >       ⎕PP←17
> > > >       x-y←x←○3 4
> > > > 0 0
> > > >       x-⍎⍕x
> > > > 0 1.7763568394002505E¯15
> > > >       x-¯14⎕CR 14⎕CR x
> > > > 0 0
> > > >       2⎕TF 2⎕TF ‘y'
> > > > y
> > > >       y
> > > > 9.4247779607693793 12.566370614359172
> > > >       x
> > > > 9.4247779607693793 12.566370614359172
> > > >       x-○3 4
> > > > 0 0
> > > >       y-○3 4
> > > > 0 ¯1.7763568394002505E¯15
> > > >       x-y
> > > > 0 1.7763568394002505E¯15
> > > >       9.4247779607693793 12.566370614359172-○3 4
> > > > 0 ¯1.7763568394002505E¯15
> > > >
> > > >
> > > > The exact representation by 14⎕CR works great (as in one of the ISO CF 
> > > > implementations), but both ⎕TF and ⍎⍕ fail to reproduce the exact number.
> > > >
> > > > Since you last change, the )save )load does keep the exact values, and I looked 
> > > > at the saved workspace file, the numbers are
> > > > 146   <Ravel vid="1" cells="⁴9.4247779607693793⁴12.566370614359171"/>
> > > > 147   <Ravel vid="2" cells="⁴9.4247779607693793⁴12.566370614359172"/>
> > > > 185     <Symbol name="x" stack-size="1">
> > > > 186       <Variable vid="2"/>
> > > > 187     </Symbol>
> > > > 188
> > > > 189     <Symbol name="y" stack-size="1">
> > > > 190       <Variable vid="1"/>
> > > > 191     </Symbol>
> > > > Here, the last digit of y is 1 instead of 2, as shown in the REPL.
> > > >
> > > > So it seems the routine used to format numbers has some problem with the last 
> > > > digits, even if with ⎕PP=17?  Or it’s just the reader has some problem?
> > > >
> > > > In addition, )dump and )out still changes the floating point number, but 
> > > > perhaps that is the same issue as in ⎕TF and ⍎⍕?
> > > >
> > > >
> > > > > On Jun 7, 2016, at 3:29 PM, Juergen Sauermann <address@hidden>
> > > > > wrote:
> > > > >
> > > > > Hi Xiao-Yong,
> > > > >
> > > > > OK - SVN 739.
> > > > >
> > > > > /// Jürgen
> > > > >
> > > > >
> > > > > On 06/07/2016 08:16 PM, Xiao-Yong Jin wrote:
> > > > >
> > > > > > Hi Jürgen,
> > > > > >
> > > > > > Thanks, though the current behavior still does not give the exact number bit by bit.  
> > > > > > Can you change it to %.17g, or std::numeric_limits<double>::max_digits10?
> > > > > >
> > > > > > In addition, is it OK to raise the maximum allowed ⎕PP to 17?
> > > > > >
> > > > > > Best,
> > > > > > Xiao-Yong
> > > > > >
> > > > > >
> > > > > >
> > > > > > > On Jun 7, 2016, at 12:37 PM, Juergen Sauermann <address@hidden>
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > Hi Xiao-Yong,
> > > > > > >
> > > > > > > thanks, fixed in SVN 738.
> > > > > > >
> > > > > > > /// Jürgen
> > > > > > >
> > > > > > >
> > > > > > > On 06/07/2016 08:13 AM, Xiao-Yong Jin wrote:
> > > > > > >
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > Is the following behavior intended?  Note the x value after load.
> > > > > > > >
> > > > > > > >       )clear
> > > > > > > > CLEAR WS
> > > > > > > >       x←.12345678901234567890
> > > > > > > >       ⎕pp←16
> > > > > > > >       x
> > > > > > > > 0.1234567890123457
> > > > > > > >       )wsid tmp
> > > > > > > > WAS CLEAR WS
> > > > > > > >       )save tmp
> > > > > > > > 2016-06-07  01:09:15 (GMT-5)
> > > > > > > >       )clear
> > > > > > > > CLEAR WS
> > > > > > > >       )load tmp
> > > > > > > > SAVED 2016-06-07 01:09:15 (GMT-5)
> > > > > > > >       x
> > > > > > > > 0.123457
> > > > > > > >       ⎕pp
> > > > > > > > 16