Re: Recovering token values

[Hans Aberg]

At 12:35 +0100 2001/04/30, address@hidden wrote:
>I try to get the token values using the $n variables. I am working with
>strings, but when I need to get the value of a token , bison return something
>completely different. I.e:
>
>I have a rule  like this:
>
>TK_LT NAME ATTRIBUTES TK_GT
>This rule is to recognize a start tag in a xml document. When the rule is
>matched and the action is executed, I use something like this:
>
>STag_Name = $2
>
>but bison returns, not only the value of name, but all the token values until
>the end of the rule. I.e:
>
>If the start tag is <Ingredients number = "12" price = "$12">, and I use
>       STag_Name = $2
>to recover the name of the tag, I only recover a surprise when I notice that
>what I have in the varible STag_Name is 'Ingredients number = "12" price
>= "$12"> '
>
>Can anybody suggest something to find out a solution?

This kind of problem has been discussed several times in this list.

Typically when this happens they use a lexer produced by Flex. When Flex
identifies a lexeme, it hands over a char* pointer yytext, and a number
yyleng, telling how long it is, and sets yytext[yyleng] temporarily to '\0'
in the buffer. When so the lexer continues to scan for the next lexeme,
yytext[yyleng] is first changed back to its original character value.

So it means that if one does not make a copy of the identified lexeme,
yytext will look as a C string up to the point where the latest read lexeme
ends if you are lucky, and at nothing if you are unlucky because Flex may
have changed its buffer.

So make a copy of the lexeme strings before using them.

I do it under C++ (so I do not have to woryy about deleting them and memory
leaks) by:

In file X.h:

#include <string>

class parser_type {
public:
  std::string text;
};

#define YYSTYPE parser_type

In file X.l:

%{
#include <X.h>
#define get_text yylval.text = std::string(yytext, yyleng)
%}

%%
[[:alpha:]]+   { get_text; return identifier; }
...


  Hans Aberg