Re: [Openexr-devel] Thread safety

[Peter Hillman]

In short: yes, if you are careful.

In long: you should be able to call readPixels/readTiles simultaneously from the same File object on two separate threads, but the calls will be serialised by the library, including the decompression/framebuffer copy. (This does assume that IlmThread's Lock mechanism is compatible with the way your threads are created). So, you gain no performance from the library by running your own threads.

If each thread wants to use a different framebuffer, you must yourself lock between setFrameBuffer and readPixels, and call setFrameBuffer before each call to readPixels. setFrameBuffer itself isn't threadsafe, and in any case both threads would end up reading into whichever FrameBuffer was called second.

This constraint also applies to multiple threads accessing the same part of a MultiPartInputFile object. Part objects are effectively 'smart' pointers to a single underlying File object. So two different part objects initialised on the same part of a MultiPartInputFile object behave as if they were the same InputFile object.

So if you were to do this:
MultiPartInputFile file(filename);
InputPart pt1(file,0);
InputPart pt2(file,0);
pt1.setFrameBuffer(fbuf1);
pt2.setFrameBuffer(fbuf2);
pt1.readPixels();

Then the pixels will be read into fbuf2 (because pt1.file==pt2.file, and the second setFrameBuffer overrides the first).
So different threads should still lock between setFrameBuffer and readPixels if they are accessing the same part of a MultiPartInputFile object, even if they have different InputPart objects.

Incidentally, this pointer mechanism does save you the bother of keeping hold of InputPart objects, as you can re-initialise them as needed with little overhead:

MultiPartInputFile file( filename );

for( int p=0 ; p < file.parts() ; ++p )
{
    InputPart( file , p ).setFrameBuffer( fbuf[p] );
}

for( int p=0 ; p < file.parts() ; ++p )
{
    InputPart( file , p ).readPixels();
}

There is a mechanism for DeepScanlineInputFile/Part that allows two separate threads to read scanlines more efficiently. Instead of calling setFrameBuffer, you can call rawPixelData from each thread (locking to ensure the calls serialise) then call the versions of readPixelSampleCounts and readPixels that take a FrameBuffer and the rawdata. This allows simultaneous decompression/pixel copying of data. It has to be used with care, and is probably only useful as a last resort for maximising performance.

It is safe to read different parts from the same file simultaneously, and if you have your threads set up suitably can be quite efficient. The setFrameBuffer restriction doesn't apply to different parts within the same file.

As always, the fastest way to read EXRs is to read as much of the file as possible in a single call, and let the EXR library manage its own threads.

Peter

On 04/03/2013 06:14 AM, Larry Gritz wrote:

I know that libIlmImf is reentrant/threadsafe with two threads reading from separate open files.  But is it known to be safe, or not, for two threads to be simultaneously do a readTiles() or readPixels() from the same open inputPart?

--
Larry Gritz
address@hidden



_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel