Re: [Openexr-devel] Thread safety

[Piotr Stanczyk]

I'd like to echo the TBB sentiments. It really is a pleasure to work with, but it is not part of the standard as you mention.

I would like to get a better sense from the developer community about the range of versions of compilers supported as well as the platforms targeted.  Would C++11 be a viable default option?

Piotr

---

From: Christopher Horvath address@hidden
Sent: 03 April 2013 09:17
To: Piotr Stanczyk
Cc: Peter Hillman; address@hidden
Subject: Re: [Openexr-devel] Thread safety

I think it should be a config-level decision. People who (god-forbid) need to maintain different thread-capable installations could then have their packaging system deal with "exr_omp", "exr_posix", "exr_cxx11", "exr_tbb".

I have seen astonishing performance from the TBB library, but it is not part of a standard, and it is controlled entirely by Intel, which makes it a risky choice for the default implementation.  However, OpenVDB depends entirely upon it, so there is a precedent. OpenMP is natively supported in both C and C++, and is implemented across a large number of compilers by way of compiler directives.  It is a more restrictive standard, and has a lot of limitations. Plus, it's existence in #pragma form seems kludgy to me.

My vote for the default would be to use the std::thread library, as part of the C++11 standard. 

http://en.cppreference.com/w/cpp/thread

I would then support TBB, OMP, and other implementations as optional extensions.

Chris

On Wed, Apr 3, 2013 at 9:06 AM, Piotr Stanczyk <address@hidden> wrote:

I was just having the same conversation yesterday...

I think I would like to see an abstraction where you can sub in your own implementation with a fallback to the current one.

Anyone have strong thoughts / opinions on this? Is this something that people would like to be able to configure at build time or runrtime?

---

From: openexr-devel-bounces+pstanczyk=address@hidden [openexr-devel-bounces+pstanczyk=address@hidden] on behalf of Christopher Horvath [address@hidden]
Sent: 03 April 2013 08:44
To: Peter Hillman
Cc: address@hidden
Subject: Re: [Openexr-devel] Thread safety

Is there any hope for IlmThread being retired in favor of OpenMP, TBB, or the C++ 11 std threads library?

On Tue, Apr 2, 2013 at 6:23 PM, Peter Hillman <address@hidden> wrote:

On 04/03/2013 12:25 PM, Larry Gritz wrote:

(This does assume that IlmThread's Lock mechanism is compatible with the way your threads are created).

I'm not sure I followed that statement. Huh?

I'd understood that there are some high performance threading libraries which aren't pthread compatible. On Posix systems, for example, IlmThread::Mutex::lock() calls pthread_mutex_lock(). This might not actually lock the thread if it is some exotic thread implementation unrelated to pthread. If you are using pthread, windows threads, or better still the IlmThread library, all will be well.

My threads do each set up separate frame buffers.  It's too bad the framebuffer isn't an argument to readPixels/readTiles rather than set by a separate call to setFrameBuffer

There'd be little point in passing a framebuffer to readpixels unless the file read is separated from decoding the pixel data, in the same way that the rawpixel approach does in deepscanlines.
Under the hood, it might as well just do a lock, setframebuffer, readpixels, unlock. It might be worth extending the deepscanline rawpixel approach to regular images

As always, the fastest way to read EXRs is to read as much of the file as possible in a single call, and let the EXR library manage its own threads.

Ha ha, yeah, until you have 2 TB of image files you're accessing incoherently.

Oh dear.  Perhaps it might be worth the effort of implementing some super system with a single thread that handles calls to readPixels/readTiles, with other threads post requests to that and wait for the reply. That thread can reorder the requests to reduce the amount of file seeking that takes place. I'm guessing that file seeks are far more expensive than decoding pixels.

Rather you than me :-)

Peter

_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel

--
I think this situation absolutely requires that a really futile and stupid gesture be done on somebody's part. And we're just the guys to do it.

--
I think this situation absolutely requires that a really futile and stupid gesture be done on somebody's part. And we're just the guys to do it.