Re: [Openexr-devel] Loss of layer names in GetChannelsInMultiPartFile()

[Peter Hillman]

Hi Phil,

GetChannelsInMultiPartFile() is something I wrote, based around an assumed standard for how channels would be split across multi-part files.

Unfortunately Nuke doesn't use that convention. It also doesn't follow the OpenEXR standard, writing red, green and blue as "R","G","B" unless they are from Nuke's "rgba" layer. The RgbaFile API will not read any data from that file.

That largely makes GetChannelsInMultiPartFile rather redundant. You'll need to roll your own scheme.

Perhaps a good algorithm would be to check if the same channel name appears multiple times in the file. If it does, then treat the part name as the layer name, extracting the view name from the part name (plus any surrounding characters that look like separators). You'll also need to rename any variants of red, green and blue to RGB. If each view has uniquely named channels, use GetChannelsInMultiPartFile

For the record, the "standard" I was assuming writing GetChannelsInMultiPartFile is:

• red, green, blue, alpha, depth are stored as R,G,B,A,Z respectively (always single upper case letter) as the channel name suffix, regardless of the layer
  
• the layer name prefixes the channel name, so "layer1.R"
  
• If there is a "view" attribute, the view name is omitted from the channel name - instead the view is stored in the "view" attribute
• Single part, "multiview" files are not deprecated - these do have the view in the channel name, and have a "multiview" attribute. GetChannelsInMultiPartFile() understands that scheme and decodes the channel and view names for you
  
• a part may contain multiple layers, and the channels of a layer may be stored in different parts (*)
  
• a part cannot mix channels from different views, unless there is exactly one part in the file and it has a "multiview" attribute.
  
• within a file sequence, a channel should always appear in the same named part, but that part may change index (if, for example, the list of channels changes throughout the sequence)
• a channel should be read from the first part of the required view that stores that channel: if "layer1.R" appears in part 0 and part 3, both of which have view "main", use part 0 not part 3

Secondary occurrences of the same channel would be alternative representations or "non-standard" schemes. For example, you might store the same data in both scanline and tiled format in separate parts within the same file, or you might store a multilayer composited image as well as the individual elements within the same file. Those schemes would generally be used "in-house" - I wouldn't expect commercial software to understand how to handle those extra channels unless it created the file itself. By default tools would only read the first occurrence of a channel. It may be necessary to offer an "advanced" UI which allows the user to select which EXR channels to read from which (named) parts and how to map those to the internal channels within the software.

I regret not enforcing the GetChannelsInMultiPartFile convention more deeply in the code - we wanted to allow for flexibility but ended up leaving too much room for confusion.

Peter

p.s. (*) disconnected parts and layer like this is a particularly powerful feature for performance and quality: you can store just RGB in part 0, A in part 1, then all your 'rarely used' layers in part 2. This means playback performance critical systems (like Baselight!) can skip the extra layers if they aren't needed. It will often not need the alpha channel, which might be stored using a different compression scheme than RGB to make the file smaller, or to allow lossy compression on RGB but not A. Storing all the 'rarely used' layers together might help to compress them smaller as the compression can exploit similarities between the channels. I've seen good compromise between file size and performance by storing RGB uncompressed in part 0, then compressing everything else including A in part 1. The approach of using part names for layer names doesn't allow for this.

On 14/09/16 02:25, Phil Barrett wrote:

Hi there

I'm the developer responsible for OpenEXR reading and writing in Baselight.

It looks like layer names are being lost in the output of GetChannelsInMultiPartFile() when reading multi-part files (at least, those written by Nuke).

exrheader shows

  part 0:
channels (type chlist):
     blue, 16-bit floating-point, sampling 1 1
     green, 16-bit floating-point, sampling 1 1
     red, 16-bit floating-point, sampling 1 1
name (type string): "layer1.main"
view (type string): "main"


  part 1:
channels (type chlist):
     blue, 16-bit floating-point, sampling 1 1
     green, 16-bit floating-point, sampling 1 1
     red, 16-bit floating-point, sampling 1 1
name (type string): "layer2.main"
view (type string): "main"


  part 2:
channels (type chlist):
     blue, 16-bit floating-point, sampling 1 1
     green, 16-bit floating-point, sampling 1 1
     red, 16-bit floating-point, sampling 1 1
name (type string): "layer3.main"
view (type string): "main"

but GetChannelsInMultiPartFile() gives me

Channel 'blue' view 'main' part 0 internal 'blue' layer ''
Channel 'green' view 'main' part 0 internal 'green' layer ''
Channel 'red' view 'main' part 0 internal 'red' layer ''
Channel 'blue' view 'main' part 1 internal 'blue' layer ''
Channel 'green' view 'main' part 1 internal 'green' layer ''
Channel 'red' view 'main' part 1 internal 'red' layer ''
Channel 'blue' view 'main' part 2 internal 'blue' layer ''
Channel 'green' view 'main' part 2 internal 'green' layer ''
Channel 'red' view 'main' part 2 internal 'red' layer ''

Any suggestions?

Thanks

Phil