Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Chris Marusich writes:
>
>>>>  - Initializing a data store.  For example, in dirvish I need to run
>>>>    a command to initialize a "vault" where I will be storing my data.
>>>>  - Manually invoking a garbage collection utility.
>>>>  - Manually invoking an integrity check utility.
>>>>  - Possibly some side effect involving querying the network.
>>>>  - Running schema migrations.
>>>>
>>>> All sorts of things!  Most of them (all?) involve state or side effects,
>>>> but plenty of our most important services will be "state overlords" of
>>>> some type.
>>
>> Why do those activities need to be represented as actions in Shepherd?
>> If we're running a daemon or service that already exposes a mechanism
>> for manually running tasks like these, then can't we just use "the usual
>> mechanism" for doing it?  For example, if we're running a daemon that
>> already provides a script to perform garbage collection, can't we just
>> invoke that script?  It isn't clear to me why we would need to model
>> domain-specific actions like that in Shepherd, although I can see why it
>> might be convenient.  Am I missing something?
>
> So sure, we can run "foo-db gc" occasionally (though system
> administrators sometimes have to run these kinds of commands by hand).
> But what about "foo-db dumpdb"?  That's not something we just run on a
> cronjob.  You need access to that command.  And in order for the command
> to do the right thing, it might need access to the config file.
>
> There are some other things where we can try to initialize or run
> migrations automatically, but that stuff can be very dangerous to just
> do implicitly.  Now note, I *do* think we want to handle some of this
> stuff behind the scenes as much as possible so that users don't have to
> think about it.  But have you ever done a *really big* database schema
> migration?
>
> We run into two challenges:
>  - Now we're trying to "idempotently manage state", which it turns out
>    is very hard (and the source of many bugs in devops tooling)
>  - Some commands either need to be run manually occasionally, or are
>    never automatically run (see the dumpdb example).
>
> Does this make sense?
>  - Chris

FWIW, I do agree that in most cases, we can put these in the background
and automate them so that users don't need to see them.  Possibly that's
true for the key things I want to do for dirvish. :)  So we can and
should when we can!

Where we can't (a dropdb type command, or something like "borg mount")
I do think it would be nice to expose Shepherd actions.

 - Chris