Re: Replacing Bower with "guix environment"

[David Thompson]

Christopher Allan Webber <address@hidden> writes:

> Hello all,
>
> I'm interested in "guix environment" as a universal solution for
> virtualenv/bower/etc.  There's good reasons to want this; aside from
> Guix being awesome, packaging for web applications is getting more and
> more insane and often involves many layers of package management, often
> *multiple language package managers at once*.  It's a sad state of
> affairs.

Agreed, it's terrible.  A modern web application typically requires
*four* separate package managers:

* The system package manager (apt, yum, guix, etc.)
* The server-side language package manager (pip, gem, etc.)
* The static asset package manager (bower)
* The server-side JavaScript package manager (npm)

> Why not have a universal solution?  "guix environment" could be that,
> I hope?  But there are some challenges.
>
> The virtualenv side (having libraries set up for development on the
> Python end) is really easy with "guix environment" assuming those files
> are there: you have your PYTHONPATH set up, done.
>
> But the "bower" side of things is not so easy.  Most web applications
> need to serve many assets to the browser: javascript like jquery, css
> like bootstrap, fonts like fontawesome or really any other
> fonts, not to mention any icon sets, etc.  Web applications need to
> serve these in both a "development" way, usually using some simple
> development server, in which case these have to be easily discovered and
> served.  For production, these need to be served by
> nginx/apache/whatever.  Describing to users how to deploy web
> applications tends to be pretty hard, so for this last one you want to
> make the configuration requirement as much of a one-liner as possible.
> This usually means then that the static assets (jquery, fonts, etc) are
> referenced from somewhere inside the package, usually in an extlib/
> directory or such.
>
> There are two common ways to do this:
>
>  - The "classic" method of doing this is pretty gross: just check all
>    those assets directly into your project, maybe in some sort of
>    "extlib/" directory.  I don't need to tell guix-devel just how bad of
>    a solution this is!  And yet, most web developers (including me
>    *shameface*) have done it, even in our free software projects.

If you take a look at the guix-web repo you'll see that this is what
I've been doing. :X

>  - The "hip new way" of doing things is to use Bower.  Bower is a
>    package manager, but it's made specifically for static assets served
>    to the user, such as css files, fonts, javascript like jquery, etc.
>    Bower also puts these in an extlib/ or whatever, but it puts them in
>    that place *for* you.
>
> It's possible in Guix or Debian packages or whatever to rely on a system
> installed jquery or etc, just a minor pain in the butt, linking in the
> jquery packages and etc manually into that extlib place during building
> the package/derivation.  Okay, not so bad.
>
> But what about "guix environment", where we're directly checked out
> inside the package?  What we want to do is this:
>
>   ~/devel/mediagoblin/$ guix environment -l package.scm
>
> And this will set up the python dependencies for sure, and maybe it can
> even install the jquery package and stuff, but if I run:
>
>   # put your dev server command here
>   ~/devel/mediagoblin/$ ./lazyserver.sh
>
> Oh no!  jquery is installed, but it's not linked into extlib/jquery/
> where I expected it.  So suddenly I'm looking at my development instance
> and none of my javascript is working and I'm very confused!
>
> It could be that we have jquery packaged, that it's in my
> ~/.guix-profile/ or etc, but my application doesn't know about it.  This
> is "guix environment" for development, so there's no munging the package
> during install, I'm hacking *in* the project.
>
> So what can we do?  I still think serving an extlib/ directory or some
> easily findable directory is the best way to keep deployment clean, and
> anyway, "guix package" can handle setting up the symlinks for that, it's
> just "guix environment" that has no way to play along.

To reiterate a bit of what I said on IRC for everyone else: I think this
problem is bigger than 'guix environment', which just sets some
environment variables and runs the command you tell it to run and
nothing more.  The real problem is that there's no notion of
$JAVASCRIPT_PATH or $CSS_PATH that build systems can use to find static
assets dependencies, thus deduplicating them system-wide.  Instead,
every web application just bundles all of the dependencies.  We all know
that bundling is bad for C libraries and such, and Chris and I think
it's clear that bundling is also bad for web assets.  Bower solves this
problem, at the expense of being yet another package manager and
duplicating the assets for each application on your system that uses
Bower.

> What can we do?  My first reaction on IRC is that "maybe there could be
> some sort of hook that's run after 'guix environment' that could link
> these things into the package".  This was not well received, and for
> good reason: it makes "guix environment" non-functional.

Not only is it imperative, but I think it's out of scope for 'guix
environment'.

> Dave Thompson has made what I think is probably a good suggestion: there
> should be some sort of environment variable set with something
> searchable.  This way, during the "./configure --link-extlib && make" or
> "python setup.py develop" or whatever such setup-for-development
> process, there's a way to see, aha here's the jquery, symlink that here;
> here's the font-awesome, symlink that here; here's the bootstrap,
> symlink that here...

Yes, something like that.  I think the most important part of any
solution is that it not be dependent upon Guix tools at all.  Just like
Guix isn't needed to find C libraries at configure time.  I wouldn't
want Guix integrated into web application build systems like Bower is.

Even if we come up with a good solution to this problem, Bower is still
deeply embedded into so many projects.  I wonder if we can be compatible
with it in any way.

> I don't know what this environment variable would be called.
> WEB_ASSETS_PATH?  The other tricky thing is, it's easy to say "put CSS
> files and jquery in that thing", but what about fonts?  Those have
> their own location already.  Should both be added to the path?

I need to think about this more.  I should take Bower for a spin and see
how people are integrating it into their build systems.  How do they
handle fonts?

> Anyay, I'd love to have a way to do this with Guix.  If someone can find
> a nice solution, I'll make it an option for development with GNU
> MediaGoblin instead of using virtualenv/bower. ;)  But we need to know
> the right path first!

Thanks for starting this discussion.  Let's keep brainstorming!

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate