Re: [Openexr-user] [Openexr-devel] Per-pixel Object Identifiers?

[Florian Kainz]

Hi Chris,

Unless OpenEXR was modified, object identification with masks would be 
prohibitively expensive for images that contain a large number of objects,
such as an aerial view of a city or a battle scene with thousands of
soldiers and their equipment.

Per-object masks could be implemented if the image allowed a variable number
of data samples per pixel.  An image might contain two channels for object
identification.  One, called objectMask stores a vector of HALFs per pixel;
the other called objectID stores a vector of UINTs per pixel.  objectID[i]
would identify an object, and objectMask[i] would specify how much that
object contributes to the final pixel color.

The scheme I proposed would be equivalent to binary per-object masks,
and it could be implemented in the current version of OpenEXR, which does
not support variable-length per-pixel data.  The scheme would be sufficient
for ILM's use case.

Florian


Chris Cox wrote:
> Per pixel metadata is probably overkill, but some people have specified
> object/surface identification with masks -- using one mask for each surface
> or object.  When querying a pixel for information, you look in the matching
> list of masks for non-zero values and add to the list that should be
> associated with that pixel.
>
> Chris
>
>
>
> On 4/14/11 3:13 PM, "Florian Kainz" <address@hidden> wrote:
>
> > Hi,
> >
> > At ILM we want to implement a workflow where a computer graphics artist
> > can bring up an OpenEXR image of, say, a scene from Rango on his or her
> > screen, point to a pixel, and be find out that the object seen at that
> > pixel is called "Beans/dress/button3."
> >
> > This will require storing per-pixel object identifiers in an OpenEXR file.
> > In order to avoid re-inventing the wheel, I would like to find out if any
> > OpenEXR user have done something like this already. If you have done it,
> > would you be willing to share how you did it?
> >
> > Also, is there any interest in per-pixel object identifiers outside ILM?
> >
> > Florian
> >
> >
> > Unless somebody has a better idea, we'll probably do something like this:
> >
> >      Add a channel called objectID, of type UINT, to the image.
> >      If the image has multiple views, then add an objectID channel
> >      to every view.
> >
> >      Add an attribute called objectID to the header.  The type of the
> >      attribute is a map from unsigned integers to lists of strings.
> >      If the image has multiple views, then add multiple attributes
> >      with names of the form <view name>.objectID.
> >
> >      For each unsigned integer value that occurs in one or more pixels
> >      in the objectID channel, a corresponding map entry in the objectID
> >      attribute contains a list of all objects that are visible in those
> >      pixels.  More than one object may be visible in a given pixel because
> >      of transparency, motion blur, reflections, or anti-aliasing.
> >
> >      To find out which object or objects cover a given pixel, application
> >      software first looks up the value stored in the objectID channel for
> >      that pixel, then it looks up the corresponding list of object names
> >      in the objectID attribute.
> >
> >      In images with lots of transparency or motion blur any given object
> >      name may occur multiple times in the map from unsigned integers to
> >      string lists.  In order to save disk space the objectID attribute
> >      could be compressed by using a two-stage lookup, where the attribute
> >      contains two maps, one from pixel values lists of integer object
> >      identifiers, and one from object identifiers to object names.
> >
> >      With 32-bit UINT pixel values this scheme could run out of object
> >      identifiers for images with more than four Gigapixels, but in VFX
> >      production images that large are rare.
> >
> > _______________________________________________
> > Openexr-devel mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/openexr-devel