Re: [Gzz] A vob system based on structured graphics?

[Tuomas Lukka]

On Mon, Mar 31, 2003 at 12:02:20PM +0200, Benja Fallenstein wrote:
> Tuomas Lukka wrote:
> >On Sun, Mar 30, 2003 at 02:35:46PM +0200, Benja Fallenstein wrote:
> >>This is not completely thought-out; it's just a start. I hope to spark 
> >>discussion. *If* this system is viable, though, I hope that it can 
> >>simplify the vob system very much, both for its users and its developers.
> >
> >What I'm afraid is that the simplicity will force far greater
> >performance hits than the current system has. You'll need to convince
> >me it doesn't ;)
> 
> I can implement it for AWT. If someone else hooks it up with GL, we can 
> benchmark.

No, GL is exactly the *problem* here. With awt, there is never an expectation
of good performance.

I don't know how to hook it with GL with good performance.

> >However, this already seems to me to give away some flexibility: one
> >of the nicest features of the vob system with OpenGL is that you can
> >set the rendering state:
> >
> >     vs.map.put(GLCache.getCallList("""
> >             PushAttrib ...
> >             ... OpenGL state commans
> >     """))
> >
> >     ... render a bunch of stuff
> >
> >     vs.map.put(GLCache.getCallList("""
> >             PopAttrib ...
> >     """))
> >
> >How do you plan to achieve this?
> 
> With a modifier vob.
> 
> public class FooAttribVob extends ModifierVob {
>     public int putIntoListGL(...) {
>         i = GLCache.getCallList(...push...).putIntoListGL(...);
>         i = content.putIntoListGL(...);
>         i = GLCache.getCallList(...pop...).putIntoListGL(...);
>     }
> }
> 
> This is not really different from a vob setting e.g. the color of 
> another vob.

Needing another class for that makes the jython code much more convoluted.

Also, how will you do vob batching in this approach?

> >>Vobs are transformed by modifier vobs, like Translate, Rotate 
> >>or Fisheye vobs. Composite vobs include HBox and VBox (as in TeX) and 
> >>Overlay, which takes a list of vobs and renders them above each other.
> >
> >The problem here is that the current system gives you a lot of flexibility
> >by 
> >
> >     1) allowing vobs to have more than one coordinate system; for 
> >     instance,
> >        irregular quad has the contents and the edge described separately
> >     2) allowing coordinate systems to be defined by several others - 
> >        e.g. halfway between two points in different coordsyses you 
> >        could have a label.
> >
> >You seem to be going back to a tree model which I don't like.
> 
> Any vob can be *parametrized* by an arbitrary number of transforms. The 
> tree is for containment-- a vob in two places in the tree is shown in 
> two places on the screen. I explaned below,

Parametrized -- meaning that it can't be immutable any more because
you have to create a new instance for each case?

> >>Vobs like IrregularFrame would be done through parametrization, like
> >>connections: IrregularFrame would (in its constructor) get two
> >>transformations (Transform objects)
> >
> > Uhh, but this goes the wrong way around -- weren't you supposed to 
> construct
> > the transformations *AROUND* the transformed objects? That's what
> > the above stuff was mostly based on...
> 
> If the object is *graphically transformed*, yes. I.e., the object is a 
> graphic, and if you put it into a rotation it is rotated, if you put it 
> into a translation it is translated, and so on.
> 
> If an object takes two transforms, that already indicates these aren't 
> graphical transforms, but parameters to the object. Like with 
> connections: when put into two Translation transformations, the 
> connection isn't translated-- its endpoints are, but the connection as a 
> whole changes its shape in a more complex way. That's why this is a 
> parametrization, not putting the connection into two transformations.

Ok, so it's like the difference between the first coordsys and the other parents
given to current vobs.

> >>Positioning a number of objects in different places on the screen, as 
> >>our structure views do, can be handled by nesting each object in a 
> >>Translate vob and putting the Translate vobs into an Overlay. 
> >
> >What's an Overlay?
> 
> Explained in the previous paragraph:
> >>Composite vobs include HBox and VBox (as in TeX) and
> >>Overlay, which takes a list of vobs and renders them above each other.

Nasty term, implies something completely different. How about "sequence"?
"List"? Something like that.

Overlay implies to me that they are over something *else*.

> >However, it still requires too many
> >Java objects: creating a new wrappervob for each use does not sound good.
> >Of course, most of the wrappervobs would probably just contain
> >two fields: the real wrapping vob and the wrapped vob... That might 
> >be the angle to start from.
> >
> >I wonder if we could come up with a system that would work both ways...
> 
> Maybe, but I'm not yet convinced that this system will be too slow by 
> itself.

In Kaffe, new objects cost a lot.

> >>Vobs are more or less immutable. They are immutable in that you cannot 
> >>change the structure of a vob-- you cannot add or remove vobs inside it. 
> >>On the other hand, you may modify the parameters of a transformation 
> >>vob, to accomodate for continuous user actions like dragging.
> >
> >URRRRRRRRGH! 
> >
> >This is really bad, OTOH.
> >
> >Here you get all the disadvantages of both mutable and immutable objects.
> >You can't change them as you like but you can't trust them to remain the
> >same either.
> 
> You can trust them to remain the same in defined ways (structure) and 
> allow to be changed in defined ways (linear parameters).

Yes, but neither of those is a *USEFUL* invariant. 
Something needs to remain the same, period, or then it can change 
and it doesn't really matter how it changes.

One gives you optimization and caching opportunities and the other
flexibility.

Combining the two gives you neither.

> >>Each vob can be seen as being made of a tree vob sub-vobs (for primitive 
> >>vobs, the tree contains only that vob). Each vob in this tree has a 
> >>number. That is, when you have an Overlay vob containing Transform vobs 
> >>which contain RectVobs, then each of these vobs has an index inside the 
> >>Overlay vob. The Overlay vob itself has index 0.
> >>
> >>- Leaf vobs contain just themselves, at index 0.
> >>- Modificator vobs contain themselves, at index 0, and then the vob they 
> >>modify and the vobs contained by that vob, at indices 1+.
> >>- Container vobs like Overlay contain themselves, then the first vob in 
> >>them with all the vobs contained by it, then the second vob with all the 
> >>vobs contained in it, and so on.
> >>
> >>Simple arithmetic can be used: If vob U has index i in vob V, and vob V 
> >>has index j in vob W, then vob U has index i+j in vob W.
> >>
> >>These indices can also be seen as a 'path' into a vob. Vobs support 
> >>getVob(index) and getTransform(index) methods, where getTransform(index) 
> >>gives the cumulative transformation from the root vob down to that vob 
> >>in the tree. For example, if the vob is wrapped in a Translation which 
> >>is wrapped in a Rotation, getTransform() would return a transform 
> >>concatenating those other two transforms. [getTransform() does not 
> >>return a vob, but a Transform object.]
> >
> >Interesting idea, which forces the immutability...
> 
> Of structure. Linear parameter modification is not a problem.

What's "linear parameter modification"?

> >>Each vob (except the root) has one parent vob and inherits one parent 
> >>transform from this vob. Given that we see each vob as a graphic, this 
> >>makes sense. (Note: A vob doesn't know its parent; rather, the parent 
> >>transform is passed to the render() method of the child vob. This allows 
> >>us to use flyweighting, i.e. use the same vob in different places.)
> >
> >But we can't use it if, as you said above, the transforms are not fixed
> >inside a vob.
> 
> You can only use linear parameter modification in vobs you rendered 
> yourself, obviously.

Same q.

> On the other hand, we could also consider a copy-to-root approach to 
> linear modification. Maybe.

???

> >>The points it connects can be seen as *parametrizing* the connection-- 
> >>i.e., we create a line, but instead of telling it, "you are from (10,70) 
> >>to (44, 58)," we tell it, "you are from the right side of this vob to 
> >>the left side of that vob."
> >
> >This we do have already.
> 
> Yes. The difference is that we handle this through containment, not 
> parametrizing (passing a parameter to the constructor).
> 
> The point is that containment would be purely graphical.

No, we already do it by parametrizing - when you put a connection vob, you
give it two coordinate systems.

> >>We may need one efficiency hack here. For the simple structural RDF 
> >>views, we would like to avoid having to create a new object per node by 
> >>putting the node vob into a cache. However, with the approach above, we 
> >>would still need to create at least a new Translation vob for every 
> >>node-- except if we place the node in exactly the same place as before 
> >>(not very useful).
> >>
> >>To avoid this, as an efficiency hack, we may have a vob that is a mix of 
> >>Overlay and Translation: it would render a set of vobs and store a 
> >>translation (or even linear transform) for each. This wouldn't be as 
> >>clean as separating the concerns into two different vobs, but the 
> >>efficiency gains should justify it. (We can still have arbitrary 
> >>transforms by using the appropriate vobs, e.g. place a node into a 
> >>FisheyeVob).
> >
> >Unkh... no. This favors one type of transformation way too much. I want
> >this either for all transformations or none.
> 
> I don't see why it favors one transformation too much. It is an 
> efficiency hack and makes the common case more efficient. Efficiency 
> hacks for unusual cases are evil.

No, the point is that there is something wrong with the overall system
if it requires such an efficiency hack that will only work in a narrowly
fixed range of circumstances.

> >>During interpolation, all vobs are rendered where they would be rendered 
> >>outside interpolaion, except KeyVobs and the vobs contained in them. 
> >>(KeyVobs which have no matching equivalent in the target KeyPlane are 
> >>omitted from the interpolation, as usual, but vobs not wrapped in 
> >>KeyVobs are shown.) This means for example that we do not need to assign 
> >>keys to the contents of an RDF node, just to the node as a whole; the 
> >>contents of the node will just "jump" when we switch vob 'scenes' if 
> >>they are not assigned keys, following the rule above. Also, things like 
> >>the 'current view' indicator we've been placing in the upper right 
> >>corner of each window doesn't need a key; if it's not in the target vob 
> >>'scene,' it will simply disappear when we "jump."
> >>
> >>This gets us rid of a LOT of keys-- all the infamous 'role keys.' We 
> >>only assign keys to things that actually correspond to objects in the 
> >>underlying model.
> >
> >This we could do with the current system, I think also.
> >
> >But also has problems: when scrolling, new nodes would appear too early
> >at their final locations - not good.
> 
> Not if they are keyed.

Same in current system.

        Tuomas