Re: propose deprecation of generalized-vector-*

[Noah Lavine]

Hello,

On Wed, Feb 20, 2013 at 8:13 PM, Daniel Llorens <address@hidden> wrote:

On Feb 18, 2013, at 16:55, Andy Wingo wrote:

> It could make sense, yes.  What do others think?  What happens for
> array-set!?  Care to propose a patch?

Patch is attached. It looks a bit unwieldy because I am duplicating scm_array_handle_pos(), and I also had to fix the recent array_ref_1 optimization, but it shouldn't be slower for the old use.

Thanks for working on this!

 

...

---

It was interesting to read the other replies, thank you (plural). It was disappointing that nobody seems to be using arrays, but not very surprising. I use them a fair amount, but mostly for interfacing with C/C++. I do little with them in Guile since 1) the facilities are really not there or they are inconvenient to use, 2) when I need arrays I need speed and Guile isn't there either (yet!).

I agree about the speed issue, but I hope it will get better soon. The RTL VM will fix some of it, and native compilation will fix more.

 

More than getting this patch accepted (as it is or with different function names), my concern is that what this patch does cannot be done on the user side except through make-shared-array, which is too slow. So I think there's a good argument that scm_i_make_array(), SCM_I_ARRAY_V(), SCM_I_ARRAY_BASE() and SCM_I_ARRAY_DIMS() should be published.

Now some optional motivation:

In other languages that have arrays, rank-0 arrays are not differentiated from the scalar inside. This is the right approach if you want to treat arrays of any rank as values. Then you can say that a rank-n array is

-------------------
a rank-n array of rank-0 cells
a rank-(n-1) array of rank-1 cells
...
a rank-1 array of rank-(n-1) cells
a rank-0 array of rank-n cells = a rank-n cell.

Figure 1.
-------------------

The extended versions of array-ref and array-set! allow you to use a rank-n array as any of these.

I'm actually not very enthusiastic about this, not because you shouldn't be able to do this, but because in order to enable the automatic de-ranking, you have to have Guile assume which dimensions you want to map over. That's how C, C++ and Fortran do it because that's how arrays are actually stored in memory, so maybe that is the right way. It just seems too low-level for me - I'd rather see an array-slice function that can split along any dimensions.

 

Old Guile used to have something called ‘enclosed arrays’, they are described in the 1.6 manual. This is a mechanism inspired by APL that allows one to split an array as in Figure 1, so that if you have a function that operates on rank-k arrays, you can array-map! it on arrays of any rank >= k.

Example:

(define A #((1 2) (3 4) (5 6))
(define (sqr x) (* x x))
(define (norm v) (+ (sqr (array-ref v 0)) (sqr (array-ref v 1))))
(define out #(0 0 0))
(array-map! out norm (enclosed-array A 1))

out => #(5 25 61), hopefully (I don't have Guile 1.6 here to try).

In later versions of APL (and most obviously in J) people realized that this is a bad way to go about things, since the reason you want to split A in rank-1 cells is ‘norm’. That is, rank-1 is a property of the function ‘norm’. And you don't need to do any conversions to iterate over the rank-1 cells of A; you only need to move a pointer on each iteration, but to do this array-map! should be aware of the rank of the procedure in each of its arguments.

This gets at the heart of my issue with the array functionality. As far as I can tell, in Guile, there is no way to figure out what the rank of a function is. That's why you have to be explicit about what you're mapping over.

I suppose the Common Lisp-y approach would be to make an object property called 'rank', set it for all of the built-in arithmetic functions, and maybe have some way to infer the rank of new functions That might be interesting, but I'm skeptical.

 

...

Finally, to fuel the discussion of what kind of array operations Guile should provide, here is a list similar things that I have read about. I imagine there are many more. The software is all open source.

...

Thanks a lot for starting the conversation. I would like to see Guile provide enough array functionality for serious scientific computing, and it sounds like you want the same thing. I don't really know what's missing yet, though, because I haven't tried to write a program that would use it.

I think the idea of splitting arrays is great. My only concern is making it part of array-ref. I still think that's a really bad idea, because it introduces a new class of errors that are really easy to make - accidentally getting an array when you expected whatever was inside the array. I'm coming at this as a user of Matlab and Fortran. In those languages, this isn't a problem, because operations automatically map over arrays, so having an array where you expected a value doesn't lead to new errors. But in Scheme, operations *don't* automatically map, so getting an array could lead to an exception at some point later in a program when really the error was that you didn't give enough indices to array-ref.

Other than that, I'm excited about having this.

Best,

Noah