Re: [lmi] PDF generation performance

[Vadim Zeitlin]

On Wed, 7 Feb 2018 15:34:53 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2018-02-07 15:12, Murphy, Kimberly wrote:
GC> [...]
GC> > Here are my results running the file in the manner similar as end 
GC> > users:
GC> > 
GC> > lmi_wx_shared.exe --ash_nazg --pyx=only_old_pdf
GC> > 39334 milliseconds
GC> > 
GC> > lmi_wx_shared.exe --ash_nazg --pyx=only_new_pdf
GC> > 19914 milliseconds
GC> Thanks. End users will see this project as doubling the speed of
GC> a noticeably time-consuming operation. It's not crucial to spend
GC> time looking for microscopic refinements that might secure a
GC> slightly greater improvement.

 I think it should be possible to obtain much more than microscopic
refinements. To put things in perspective, in the example I'm testing with,
we're generating roughly 0.5MiB of data on disk without really doing any
serious number crunching, so it wouldn't be unreasonable to expect the run
time to be limited by the speed of I/O. Yet, even with a simple mechanical
disk, let alone an SSD that I use, writing this amount of data shouldn't
take more than 0.1s, while it actually takes ~15s here (BTW, this is when
using --pyx option to disable the use of the old PDF generation code, it's
not exactly news that throwing exceptions and stack unwinding is very
expensive in C++, but I was yet again surprised by just how bad it is).

 I won't work on this right now if you think it's not worth it, but I
remain convinced that it should be possible to speed this up by a factor
of multiple times even without introducing parallel processing. Just for
the record, one idea which should almost certainly pay off would be to
rewrite Ledger::make_evaluator() to produce values on demand as this
functions takes more than half of the total time and creates a huge amount
of strings which are never used.

 But even without neither introducing threads nor making such architectural
changes, I _still_ think we could find big efficiency gains here. For
example, make_evaluator() seems to be so slow mostly because it uses
ledger_format() which creates a new std::locale object for each call to it
and this is a horrible expensive operation. Replacing it with a less
elegant but faster implementation should help a lot. Also, right now
wxPdfDocument::DrawText() takes ~20% of the total execution time which
definitely doesn't seem right and I suspect there must be optimization
opportunities there too.

 Are you sure you don't want me to look at doing at least this? Reducing
time from 40s to 20s is definitely nice, but reducing it down to 4s would
be much more impressive, and it would, IMHO, be a pity to not even try to
do it.

 Regards,
VZ