Re: [Openexr-devel] EXR texture memory overhead

[Karl Rasche]

Do you have any control over the decode thread pool size?

If so, you might try twisting that knob and see if it makes a difference. I suppose you could be seeing differences due to the exr concurrency model.

(Is it disconcering to be spending nearly a half-hour opening files?)

Karl

On Friday, September 16, 2016, Søren Ragsdale <address@hidden> wrote:

Hello, OpenEXR devs. I've been doing some comparative rendering tests I've found something a bit surprising. 

TIFF and EXR texture access *times* seems more or less the same, which is fine because the underlying data is equivalent. (Same data type, compression, tile size, etc.) But the RAM overhead seems much higher for EXRs. We've got a 9GB render using TIFFs and a 13GB render using EXRs.

Does anyone have some theories why EXR texture access is requiring 4GB more memory?

Prman-20.11, OSL shaders, OIIO/TIFF textures:

real 00:21:46

VmRSS 9,063.45 MB

OpenImageIO ImageCache statistics (shared) ver 1.7.3dev

  Options:  max_memory_MB=4000.0 max_open_files=100 autotile=64

            autoscanline=0 automip=1 forcefloat=0 accept_untiled=1

            accept_unmipped=1 read_before_insert=0 deduplicate=1

            unassociatedalpha=0 failure_retries=0 

  Images : 1957 unique

    ImageInputs : 136432 created, 100 current, 796 peak

    Total size of all images referenced : 166.0 GB

    Read from disk : 55.5 GB

    File I/O time : 7h 2m 33.9s (16m 54.2s average per thread)

    File open time only : 27m 44.0s

Prman-20.11, OSL shaders, OIIO/EXR textures:

real 00:21:14

VmRSS 12,938.83 MB

OpenImageIO ImageCache statistics (shared) ver 1.7.3dev

  Options:  max_memory_MB=4000.0 max_open_files=100 autotile=64

            autoscanline=0 automip=1 forcefloat=0 accept_untiled=1

            accept_unmipped=1 read_before_insert=0 deduplicate=1

            unassociatedalpha=0 failure_retries=0 

  Images : 1957 unique

    ImageInputs : 133168 created, 100 current, 771 peak

    Total size of all images referenced : 166.0 GB

    Read from disk : 55.5 GB

    File I/O time : 6h 15m 42.1s (15m 1.7s average per thread)

    File open time only : 1m 22.5s