Re: reading jpegs faster

[Jaroslav Hajek]

On Fri, Jan 22, 2010 at 3:39 PM, David Grundberg <address@hidden> wrote:
> John W. Eaton wrote:
>>
>> On 22-Jan-2010, David Grundberg wrote:
>>
>> | I'm reading 10 megapixel jpeg files alot. Unfortunately it me takes 9 |
>> seconds to load such files with imread on the tip. I think that is too |
>> long time. Is someone experiencing the same problem as me? Or does it | have
>> to do with my configuration?
>> | | I've written a function file in C++ that loads the same file in about
>> 1 | second.
>>
>> Doesn't the GraphicsMagick library ultimately use libjpeg?  What
>> causes it to be slow?  Rather than writing a special case for jpeg
>> files, I think it would be better to find out why imread is slow and
>> fix that.  It's possible that would improve the performance of
>> Octave's imread function for all image formats, not just jpeg files.
>>
>> jwe
>>
>
> I would guess it the slowness is due to extra copying/conversion being done.
>
> For example,
>
>  for (int y = 0; y < rows; y++)
>   {
>     idx(0) = y;
>     for (int x = 0; x < columns; x++)
>       {
>         idx(1) = x;
>         idx(2) = 0;
>         im(idx) = scale_quantum_to_depth (pix[i].red, depth);
>         idx(2) = 1;
>         im(idx) = scale_quantum_to_depth (pix[i].green, depth);
>         idx(2) = 2;
>         im(idx) = scale_quantum_to_depth (pix[i].blue, depth);
>         i++;
>       }
>   }
>
> is slow and stands for the most of the time spent. I think it's a good idea
> to use a general-purpose library like GraphicsMagick++, but as it currently
> stands it's wasting a lot of cycles just converting between char, int and
> double, and modifying index vectors. And the end result is that we've
> converted uint8 to uint8. I did some ugly hacks to the above loop but I
> wasn't particularly successful; got down to disappointing 6 seconds.
>
> There's also the problem with the Octave image convention, a backwards way
> of storing images, where pixels are scattered throughout memory. Maybe the
> postprocessing to Octave image conventions could be done with more general
> purpose tools like Array::permute? Or is the current approach with
> rearranging directly better?
>

Array::permute is quite efficient in the general case; it attempts to
do the operation either as a sequence of block transposes or memcpy
operations. However, it requires an extra array for the result. If the
highest order dimension is small (like in this case), filling the
array directly may be a better idea. The best approach is to try both
and measure.

I think what accounts for the slowness is that the addresses for the 3
elements are computed from scratch. That means 3 hidden loops in the
loop. I see you've already discovered that. Nice job.

-- 
RNDr. Jaroslav Hajek, PhD
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz