[pdf-devel] Modifications on pdf_token_read to get token boundaries

[jemarch]

Hi Michael.

I am adapting the type 4 functions parser to use the new token
reader. It seems to work properly but for a little detail that may
require a change in the API of the tokerniser.

We need to be able to determine the boundaries of a token that has
been read, for error reporting. We cannot rely on the stm used by the
token reader to determine the beginning position of a read token,
since it is skipping white characters.

We would need to expand the pdf_token_read to communicate both the
beginning position and the end position in the stm of the last read
token. It could be done using two extra parameters:

pdf_status_t pdf_token_read (pdf_token_reader_t reader,
                             pdf_u32_t flags,
                             pdf_size_t *beginning_pos,
                             pdf_size_t *end_pos,
                             pdf_token_t *token);

If NULLs are passed then the parameters are not filled.

An alternative would be to expand the pdf_token_t TAD to include such
information, but I think it would not be quite appropriate, since it
is not part of the semantics of the token.

Would this modification be ok with you?

-- 
Jose E. Marchesi  <address@hidden>
                  http://www.jemarch.net
GNU Project       http://www.gnu.org