Re: [Evms] EVMS Conference call (04/10/01) minutes

[Andrew Clausen]

Hi,

Please keep address@hidden cc'd.  There are a few ppl
there who will be interested in this stuff.

> First, thanks for starting to tackle the user space
> installation/configuration issues.  You have brought up
> several and we know there will be many more.  :)

;-)

> As a point of clarification, we are not envisioning keeping
> a list of commands to do.  Rather, the Engine just updates
> the in-memory structures.  When the commit command comes,
> each plug-in (disk manager, segment manager, feature, etc)
> figures out what it needs to do to configure the system to
> the new image represented in memory.

I don't think this can work for partitioning, though it
probably can for LVM style stuff.  (As I said before,
automatic partitioning is NP complete, and we believe
it is IN PRACTICE too hard)

> Naturally, the Engine needs to coordinate the order of how the
> commit changes are done.  Here is our current thinking of how
> to order the various components of commit should be executed.
> 
> Notify the kernel code of any deleted volumes.
> Run the "kill list", a list of sectors that should be wiped out.
>    (Plug-ins will notify the Engine of sectors they need
>     wiped out, for example, feature metadata that is being
>     removed.)
> Call the File System Interface Module for any volumes that are
>    to be shrunk.
> Commit segments (partitions)
> Commit containers (e.g., volume groups)
> Commit features
> Call the kernel code to rediscover the disk configuration
> Call the File System Interface Module for any volumes that are
>    to be expanded.

This algorithm cannot be adapted for partitioning.  Think
about swapping the on-disk order of two partitions, for example.

> > * handling references is a PITA.  If you delete two LV's that
> > where linked (linear raid), then how do you resurrect these
> > references?
>
> Hmmm.  Good question.

I think the answer here is:
* the UUID in the two LVs is saved in the inverse operation
of the two "delete LVs"
* the UUIDs of the two volumes that are being used by the
linking are remembered in the inverse operation of the
"remove linear RAID"

> > * the operations must be completely non-interactive (eg,
> > no questions like "do you want to convert from fat16 -> fat32"),
> > because you need to apply each operation twice (first
> > "in-memory", then on disk)
> >          This probably isn't a disaster, because stuff
> > like this can be passed as parameters.  (And if you NEED
> > to make a choice, then you can fail, and ask them to
> > provide better parameters)
> >
> 
> Option 1 is the one I had envisioned.  An undo would be an inverse
> operation of what was done.  More specifically, the Engine API
> would not be the one to provide the undo.  It would be the user
> interface that would figure out what Engine API(s) should be called
> to undo the previous action.

My big concern with this is the information that needs to be
stored, and restored.  The "Engine API" knows about the information,
not the front end.  This is my main objection.

I think it should be the Engine's responsibility to construct
a reverse operation, if this is the route we take.

> Of course, undo is more complex than simply executing an inverse
> operation.  For example, what if the user interface ends up calling
> a special purpose ioctl to one of the plug-ins?  How do you undo
> that?

Hang on... isn't "undo" before "commit".  Isn't calling the
ioctl part of "commit"?

> One idea that might help implement the undo function is the concept
> of "snapshots".  I'm not talking about LVM type snapshots.  (Maybe
> I need a better term.)  The idea is that the Engine provides the
> ability to take a snapshot of the in-memory data structures,
> essentially saving the current state.  The user could then roll back
> to a current level of snapshot.  The plug-ins would have to support
> snapshotting their own internal data.  The user interface could
> provide the snapshotting function to the user, or it could
> incorporate it into its own "undo" functionality -- it would take a
> snapshot when it doesn't know how to execute an inverse function for
> the undo.

This is quite similar to checkpoints.

> Doing intermediate commits sounds hard and risky to me.
> It sounds hard because, as you mentioned, you have to save
> the various check points and how to undo them.

I don't think saving the actual checkpoints is hard.  You
can basically do it by duplicating the existing state,
and saving it on a list.  (I have a proof-of-concept for
this in 1.5.x, which will be scrapped for now, due to the
reference issues)

The hard bit is undoing/keeping "references".

>  It also sounds
> hard because you have to determine the various checkpoints where
> you can guarantee that what you commit will leave the system in
> a usable state.

This is trivial.  2 obvious options:
(1) commit after each operation
(2) only parrelelize operations that touch non-overlapping parts
of the disk.

Further optimization is more tricky, but probably
unnessecary... unless you have millions of operations from
"hill climbing"... (which we might need to resort to!)

> Intermediate commits also end up doing unnecessary work if
> the user is just playing "what if?" with the configuration tool.

Nonono.  The user shouldn't commit unless he/she WANTS the disk
that way.

> The user may want to see how a new configuration might look and
> then quit without making changes.  If intermediate commits are
> made they all have to be undone.

No, the user should have 100% control over when commits occur.
The above design allows this.  (Why did you think otherwise?
Maybe I explained something badly)

> Intermediate commits sound risky to me because any time you make
> changes to the disk you run the risk of hosing thing up.
>
> <snip>

I don't see why - but I think you misunderstood what I was
saying.  Clearer now?

Andrew Clausen