Re: How to make Bison disregard tokens?

[henrik . sorensen]

On Friday 02 March 2007 13:39, Edsko de Vries wrote:
> On Fri, Mar 02, 2007 at 01:17:04PM +0100, address@hidden wrote:
> > On Friday 02 March 2007 13:01, Edsko de Vries wrote:
> > > Well, no, that's the point. Blank lines should be recorded as NOPs
> > > where possible, so that we know where they are and unparse them.
> >
> > Can you reveal some details regarding what you are trying to do ?
>
> I am working on the front end of phc, an open source framework for
> working on PHP scripts (and hopefully ultimately compiling PHP to native
> machine code-we're getting there).
thanks for explaining this.
> One such program is an unparser that takes a PHP script in our abstract
> representation, and dumps it back to PHP syntax (a pretty printer). This
> is useful because after transforming your PHP script (for example, doing
> some refactoring), you may want to output it again and run it.
in this case you should try to look at location tracking features for each 
token. 
I think this will solve your problems, and further will not clutter up your 
grammar file with white-space handling.

If you are interested I have implemented location tracking across multiple 
files, and with exact offsets in pl1gcc.sourceforge.net

Basically for each token returned to the parser beside the value of the token, 
you also get a location token, that for pl1gcc looks like this:
 I had to extend the YYLTYPE structure:
 typedef struct pl1ltype
       {
       int first_file;
       int first_line;
       int first_column;
       int first_offset;
       int last_file;
       int last_line;
       int last_column;
       int last_offset;
       } pl1ltype;



Henrik