Re: [Quilt-dev] [ANNOUNCE] Quian

[Jean Delvare]

Hi Andreas,

> >   http://jdelvare.net2.nerim.net/quian/2.6/
>
> Quian relly looks nice!

Thanks :')

> I'm eager to learn how much cpu and I/O bandwidth Quian consumes, and
> how fast it feels locally ;)

It's rather hard to evaluate the bandwidth, as it depends on many
factors, such as the number of users, the actual use (frequency and
nature of requests), the average size of source files, the number and
sizes of patches in the series, etc.

The setup I pointed you to is powered by my home server, that is, an old
Pentium III at 800 MHz with 256 MB memory and a 256 kB/s network
access. From your question, I can guess it felt a bit slow. This
remains viable for casual use and a limited number of users, methinks.

Locally, I just tried on a reasonably fact machine: an Athlon XP 2400+
with 512 MB memory and 100 Mb/s network access. In these conditions,
Quian use qualifies as smooth. With a 65 patch series (44.7 MB in bz2),
navigation is immediate, annotated file view is below 0.5 s (up to 1s
for really large files), highlight is almost immediate (thanks to
disk/system cache I presume), file diff is between 1 and 3 seconds
depending on the file and patch, files list is the slowest operation,
typically between 3 and 5 seconds.

I've made my best to keep the generated HTML code at a minimum size
without affecting readability (of both the rendered HTML and its
source). The pages tend to be large though, size frequently exceeds
100 KB. Again it really depends on the data you work with. Using
mod_gzip might help if the line is slow and the CPU is fast, but I
didn't bother trying.

Timing measurements show that the PHP overhead is almost always
neglectable compared with the time consumed by the underlying quilt
command, so any significant speed improvement would come from quilt -
although I'm not sure where anything can be saved without redesigning
the whole thing.

It should be possible to improve the performance by not using bz2
patches. Most commands seem to be unaffected but at least the files
list (files.php) should be. The actual benefit obviously depends on the
CPU/disk speed ratio of the server.

I have considered pre-computing a number of pages to make Quian faster,
but on second thought the required disk space and the initial computing
times looked frightening. If someone thinks it would still be better,
I'm sure that person will send me patches (after I have actually
released Quian, that is). Anyone having a large number of users and
wanting to improve performance could use a reverse proxy. I've made it
so that all pages have valid Last-Modified and Expire HTTP headers, so
caches and proxies should help much over time.

> I would have used Mercurial to produce a similar result, but Quian
> can present some things better than Mercurial.

I didn't want to learn yet another tool. I'm happy with quilt :)

> One thing we might want to add to quilt annotate is a view that
> displays the patch names side by side with the source code as Quian
> does. I so far didn't bother doing that because the code for figuring
> out how to properly align things scared me off. (Still shouldn't be
> more than a few lines of shell code...)

I agree it would be convenient to have that in quilt, maybe as an option
though. Beware that in Quian I convert the patch names to versions
(remember I wrote it with Linux version patches in mind), but in quilt
the annotations are likely to be much longer than that, so the annotated
source may end up being very wide. If quilt can do that, it will save
some code in Quian and, more importantly, it will let me generate the
HTML page on-the-fly rather than buffering the entire quilt annotate
output as is currently needed. It might improve performance a bit.

> Another would be to somehow get annotate to print which modifications
> are "underneath" the most recent modification, in case that's useful.
> Deletions might be interesting as well.

My approach to this problem has been the -p option of annotate. If the
user wants to know what happened before the version currently displayed
for a given line, he/she can ask for the annotated source as it was
right before that change happened. I think it's more realistic, as the
list of changes which have affected a given line, without the context
in which each change happened, is not such a valuable information IMHO.

> > 1* File moves are not handled. As moving files around happens
> > frequently in the lifetime of a large project, some history is not
> > visible using Quian. I have plans to improve quilt's annotate
> > command, but it sounds like a quite complex task, as patch files do
> > not clearly notify file moves.
> 
> You won't get that information with patch based tools, and I don't
> think it makes a lot of sense trying to reconstruct this information.

I think it does, simply because I have an immediate need for it ;)
Detecting the moves is not that complex, a few heuristics will do.
Doing it fast and reliably is more of a problem. Integrating this in
quilt's logic is the most difficult part. I'll give it a try eventually.

> > 2* I had to modify quilt's diff command not do print warnings on
> > stderr, else my Apache's logs were full of these. This explains a
> > recent post of mine [1] to this list. I hope to come up with a
> > solution that will make everyone happy before I can release Quian's
> > code to the public. I want people to be able to use Quian with a
> > stock quilt installation.
> 
> Isn't I/O redirection good enough?

Thats a possibility, but this would hide all potential problems for
this command, which might prevent diagnosing bogus code.

Additionally... I still believe that this error message belongs to
either quilt refresh or quilt add or quilt edit but not quilt diff.
Getting an error message when you asked quilt for something specific and
it returned exactly what you asked for is really confusing and beyond
logic IMHO. I need to understand how quilt supports the edition of
files in applied-but-not-top patches before I propose any change
though, as I am not familar with this. I never do it and realized only
recently that it was possible at all. It looks somewhat error-prone to
me, I have to say.

Thanks for your comments, BTW :)

-- 
Jean Delvare