Re: [pdf-devel] Object layer API

[Michael Gold]

On Wed, Feb 18, 2009 at 07:18:01 -0800, address@hidden wrote:
>  > Date: Wed, 18 Feb 2009 01:38:57 -0500
>  > From: Michael Gold <address@hidden>
>  > 
>  > So the object layer needs to know the file type anyway (not just the
>  > header string); and given this, it can choose an appropriate header when
>  > saving, or detect an appropriate header when opening.

> BTW, I believe this header content should be analyzed when we're opening an
> existent file, we don't want to rely on file extensions.

Agreed.

>  > I definitely wouldn't want to require all data to be stored in RAM.
>  > The callback idea I suggested in my first mail would be OK for low-
>  > memory systems, and would work as follows:
>  >  - when creating a stream object (pdf_obj_stream_new), the client would
>  >    provide a callback function instead of a pdf_stm_t
>  >  - when saving a file, the object layer would execute this callback when
>  >    it needed to write a stream
>  >      - the callback would return an open pdf_stm_t
>  >      - the object layer would read this until EOF, writing the filtered
>  >        data to the output file; then it would close the stream
> 
> But, in the meantime (i.e. before the save), isn't all in RAM anyways ?
> If it is, I can't see the benefit of this idea in terms of RAM consumption.
> I'd still define this feature as optional, we could use temp files (the
> legacy method) or use the method you propose (a more secure method).

The data could be anywhere, not necessarily in RAM.  In addition to the
function pointer the client would supply a void* to store some
parameters.  If the data was coming from a file, the filename or FD
could be stored here; then when the callback was executed it would
create and return a pdf_stm_t (e.g. using pdf_stm_file_new, or some
other function defined in the future).  We might need to have a separate
destructor callback too.

The memory usage would be the same as a temp file, since the file stream
would be read on demand (and wouldn't actually be opened until we needed
to write its data).

We could provide a wrapper around pdf_obj_stream_new that immediately
writes the data to a temp file, and reads it back when saving.
Internally it could still use the callback mechanism.  The documentation
should warn the user about the risk of leaking FDs in certain cases (and
we should try to prevent it where possible).  Another wrapper could
allow the client to include an existing file (like a JPEG) as a stream.

The other option I mentioned was basically the same, but encapsulated in
the pdf_stm_t interface (instead of a callback, pdf_obj_stream_new would
take a pdf_stm_t -- and internally, all the pdf_stm_t functions would
just execute callbacks; we could make the first pdf_stm_read call open a
file, for example).  I think some way for the client to create a custom
pdf_stm_t implementation would be useful anyway.

-- Michael

signature.asc
Description: Digital signature