Re: [pdf-devel] DCT filter: suspending or buffering

[gerel]

 > Date: Mon, 15 Dec 2008 22:51:19 +0100
 > From: address@hidden
 > 
 > Hi Yang.
 > 
 >    Normally, the data feeding to libjpeg decompressing is done in a
 >    callback function (in which way I think pdf might take...). That means
 >    there is no chance to return a ENINPUT to outer loop to get more input
 >    data so we need a internal buffer to cache all compressed data and put
 >    them into libjpeg.I don't know wether it's acceptable.I mean, if there
 >    is very huge jpeg pic...
 > 
 >    Or, there is a another mode of I/O of libjpeg, suspending in which the
 >    decompressing will return if it needs more input.But the document says
 >    there may be a performance issue in this mode.No promise of the running
 >    time.
 > 
 >    I'm proceeding along to the first choice.I think maybe we can adjust pdf
 >    I/O interfaces or implement both of the modes of libjpeg without
 >    changing our interface.
 > 
 > We definitely want to use the suspended mode, using jpeg_read_header,
 > jpeg_start_decompress, etc. We should avoid to store the entire
 > contents of a PDF stream in memory: these streams can become really
 > big!.
 > 
 > Don't worry about the performance issue.

Hi hackers,

Just a note for the record.
Generally, encoding/decoding libraries have 2 different implementations in order
to encode/decode an existing bunch of data.
One, is generally a one-call decoding/encoding method, given a large enough
buffer to put the decoded/encoded data. This method is normally used for local
files.
The other is implemented as an _incremental_ encoding/decoding method on which
the size of the given buffer hasn't to be the total encoded/decoded data size.

I think we always should use the incremental method for the filters.
As jemarch noted we're dealing with streams on which both ends can have an
arbitrary large data size and a given filter could be an unnecessary bottleneck
(think of remotely located backends).

cheers,

-gerel