[pdf-devel] Initial API for the tokeniser module

[jemarch]

Hi Michael.

Based on your patch I just committed some early work on the API of the
tokeniser module. You can find it in the reference manual
(doc/gnupdf.texi).

I would like to discuss it if you like.

As it is now the prefix used for the module is (pdf_token*). The
tokeniser module would provide support for:

  - Read PDF tokens from a reading stream.
  - Write PDF tokens into a writing stream.

The data types provided by the module would be:

  - pdf_token_t

    Data type representing a typed PDF token. Each pdf_token_t can
    have a number of attributes that may influence how the token is
    written (such as the use of the hex representation for strings).

    NOTE: an alternative would be to use flags to pdf_token_write().

  - enum pdf_token_type_e

    An enumeration containing all the possible token types:

    @item PDF_TOKEN_BOOLEAN
    A PDF boolean value.
    @item PDF_TOKEN_NUMBER
    A PDF number value.
    @item PDF_TOKEN_NULL
    The null PDF object.
    @item PDF_TOKEN_STRING
    A PDF string value.
    @item PDF_TOKEN_NAME
    A PDF name value.
    @item PDF_TOKEN_INDIRECT
    A PDF indirect reference.
    @item PDF_TOKEN_COMMENT
    A PDF comment.
    ...

  - pdf_token_reader_t

    A token reader operating in a reading stream (i.e. a tokeniser).

  - pdf_token_writer_t

    A token writer operating in a writing stream.

The module would provide several sets of functions:

  - Functions to create/destroy readers and writers

    pdf_token_reader_new (stm, &reader)
    pdf_token_writer_new (stm, &writer)
    pdf_token_reader_destroy (reader)
    pdf_token_writer_destroy (writer)
    ...

  - Functions for reading and writing streams

    pdf_token_read (reader, &token)
    pdf_token_write (writer, token)

  - Functions to manipulate token variables

    type = pdf_token_get_type (token)
    pdf_token_set_type (token, type)
    pdf_token_set_attribute (token, attribute)
    ...

The idea of this module is to make it independent from the parser that
will be implemented in the object layer. Also, it will be used by the
type 4 functions implementation in pdf-fp-func.[ch]. It would be quite
useful for the user, also.

At this point it is critical to identify the needed token types.

-- 
Jose E. Marchesi
address@hidden

GNU Project
http://www.gnu.org