Re: [Fab-user] Greetings and suggestions

[Jeff Forcier]

<Niklas Lindström appears>

Holy feedback, Batman! Thanks for the detailed thoughts, much appreciated :)

Let's see if I can trudge through most of this, although it's been a
while since I had my head buried in the source code. I am looking at
my 'execution' branch of fabric.py while answering these, so answers
may differ slightly from the master Fabric branch. Plus, there may
well be things Christian has thought of during the design process that
I'm not seeing or thinking of, so he may step in to correct me.

> 1. Making "_private" names in scripts accessible to the functions
> (commands) you define.

I can see two questions here, one being the desire for "helper"
functions which are available inside commands but which don't become
commands themselves; and the other being a desire to have the entire
module-level namespace in your Fabfile, available to the commands. I
like the former; we could e.g. make a decorator one would use to flag
these helper functions so that load() knows not to make them actual
commands. The latter desire ties somewhat into your next question.

> 2. In "load", use the optional globals and locals that execfile
> supports to manage the names accessible in fabfiles.

This is a good idea and one I was actually thinking of a while ago
(but, of course, never got around to poking at). We would definitely
want to retain the stuff we expect fabfiles to use, like the decorator
mentioned above (and the other decorators already available that I've
written, namely @hosts and @mode). Retaining modules like 're', I
could go either way on -- yea, it's nice to have stuff already there,
but it would feel like magic. I've been learning Rails lately and
magic is hell when you're trying to debug or pick up someone else's
project.

Anyway, aside from stuff-in-fabric.py, there's also the earlier
question of what from the Fabfile(s) to include/exclude -- I think
that would require a higher level discussion of just what the
execution environment inside Fab commands, is expected to be.

> 3. One useful approach may be to create Command objects for all
> commands, which stores "private locals" from the file in which they
> were created.

I think Christian mentioned something along these lines at one point?
I know we've already class-ified the connection object which makes
certain tasks easier and more self-contained; doing this for commands
might also be a good idea. You're right that in the case where one
loads up >1 fabfile (a use-case I personally haven't had to think
about much, unfortunately) it'd be great to try and avoid inadvertent
shadowing of variables.

> 4. Only run "load" once per filename (by storing the loaded filenames
> to prevent from loading them twice).

+1 on this.

> 5. Add a "depend" operator.

Makes some sense, though I'm unsure what the best syntactical approach
is, the only thing I can think of that doesn't have issues is to do
something like depend('name_of_other_command'), which is slightly icky
but not horrible. (It's also where Ruby's symbols seem like a nice
idea, as they communicate the concept of something that's
not-quite-a-variable and not-quite-a-string)

I also don't think "depend" is the right name, given that you'd be
using this declaratively inside a command -- just doesn't sound verby
enough. Maybe "call_once" or something like that.

Alternately, we could set it up as a decorator, i.e.
@depends('other_command', 'other_command2') which would then execute
those other command(s) before actually executing the one being
decorated (and only if they haven't already been called). The problem
with this is that it doesn't allow you to call it declaratively, e.g.
in a situation where the current command wants to set something up
before 'depending' the other command.

> 6. Extend require to take a varargs instead of just one name, so you
> don't have to repeat calls to it if you have a bunch of variable names
> you need to get from the "provided_by" commands.

This would only work if it were made very clear that the optional
kwargs would then apply to _all_ of the varargs, i.e. that they are
all used for and/or provided by the same thing. If that's where you
were going with this, then I guess that makes sense -- you're
essentially refactoring aimed at the kwargs, so that a large pool of
variable names would end up 'collected' by any common
used_for/provided_by values.

> 7. Add decorator versions for require ("@requires(...)") and depend
> ("@depends(...)"), to be used for declaratively binding commands
> together.

I already mentioned part of this above, so obviously I agree to some
extent. Having things be used as either decorators OR commands feels a
little icky to me, however (not to mention that the actual
implementation of those functions would need to be pretty magical,
detecting how they were used and changing behavior appropriately). I
might prefer that they stick to being one or the other, or have
slightly different names for the two behaviors so they're not
literally the same function.

> 8. Support for filename globs (e.g. "components/*.py") in "load". This
> might be less useful if number 4 is implemented.

Globbing is almost always a welcome addition to programs that deal
with file paths; so +1 from me.

> 9. Extend prompt to repeat itself until "validate" is satisfied.

I forget if this is in main Fabric but I am pretty sure I have some
prompt repeating in the case of incorrect passwords, and having that
for other prompting methods would be a good idea. +1.

> 10. A regexp match complement to the current validate feature of
> prompt.

Not a bad idea; I actually think it might make the most sense to
overload the validate keyword: if a callable, it's called, if a
string, it's used as a regex match.

> 11. Optionally define variables on the commandline (e.g. "fab
> tag_release -v tagname=release-1.0").

How is this significantly different from the existing argument
functionality? (serious question)

> 12. Support for supplying the fabfile to use with a named commandline
> flag (e.g. "fab --use setup/vcs/tagging.py").

Can you provide a use case where this isn't solved by using (as you
mention) load() in your current fabfile? Or do you mean to use it in a
directory where no fabfile exists?

> 13. Support for changing working dir via operator(s).

Did you have an idea about how to implement this? Given that things
are executed in turn and the remote shell is reopened each time, the
only thing that really springs to mind is that Fabric would, in the
background, be prepending 'cd /foo/bar ; ' to every run/sudo/etc
command. Not terrifically clean; but not horrible either, and it would
give the end-user a feeling of a persistent shell session, which might
be nice.

> (Also note that this with-version would require e.g. a threadlocal for
> it to work concurrently (with e.g. "rolling strategy") since it'd
> alter a global state (current dir). Either that or binding the context
> with a name ("with directory(...) as dir_a"), but that'd make it a lot
> more like the non-with-version above.)

Check out my execution branch on Github, it would work with your
'with' version just fine if fab_mode is set to 'deep'. (I wrote it
specifically to enable fab commands that could meaningfully make use
of logic instead of just being a series of disconnected statements)

> 14. The name "set" shadows the built-in set type.

This is a minor peeve of mine too, but I can't think of any
alternative name that communicates what it does nearly as well, which
is why I haven't changed it. Pretty sure I've just dealt with sets by
using the __builtins__ trick you mention, although that is kind of
messy. Perhaps we could replace set() by exposing the mapping object
set() operates on (directly or indirectly) so users would just do
vars['foo'] = 'bar' instead of set(foo='bar').

> 15. Possibly adding Jython support. Take a look at
> <http://code.google.com/p/robotframework-sshlibrary/> for a way to go
> about that. I personally don't consider this important at the moment
> though, but it may be useful in the future.

Sadly Jython's not on my personal todo list at all, but once we get a
long-term ticket tracker set up, I think that'd be a good candidate
for a 'someday' milestone :)


Thanks again for all this input!

-Jeff