Re: Block comments

[John W. Eaton]

On  6-Jun-2006, William Poetra Yoga Hadisoeseno wrote:

| On 6/6/06, John W. Eaton <address@hidden> wrote:
| > On  5-Jun-2006, William Poetra Yoga Hadisoeseno wrote:
| >
| > | I think we should only match for "^%{$" and "^%}$", because in Matlab,
| > | a block comment starter and ender is only valid if the two characters
| > | %{ or %} are the only characters on the line. So for example this one
| > | doesn't start a block comment:
| > |
| > |   myvar = 1; %{ this is not a block comment
| > |   %} and this line should generate a warning
| >
| > What about whitespace (TAB or SPC)?
| >
| 
| From
| http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_env/edit_d14.html
| and
| 
http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/f0-44984.html
| it says:
| 
| "The lines that contain %{ and %} cannot contain any other text."
| "The %{ and %} operators must appear alone on the lines that
| immediately precede and follow the block of help text. Do not include
| any other text on these lines."
| 
| It's not clear whether tabs or spaces constitute "other text" here,
| but I think they do. From the examples given on the page, it seems
| that they don't encourage/don't want to use indentation in block
| comments (the examples for block comment nesting).

I think whitespace is allowed, but nothing else.

Here is a rough patch (relative to the current CVS) that attempts to
implement block comments that you could start with.  I'm not sure how
this will interact with functions like those that try to extract the
help text or skip the copyright notice.  It might be nice to convert
the current method of handling comments in lex.l to use a method
similar to the one used in the patch below for block comments.  But
again, I don't know what changes will need to be made to accomodate
the functions that look for help text and copyright notices.

I have no more time to work on this right now, but perhaps someone
else can work through the details and submit a more complete patch and
ChangeLog entry (I think the changes below work, but I did not do much
testing).

jwe


Index: src/lex.h
===================================================================
RCS file: /cvs/octave/src/lex.h,v
retrieving revision 1.43
diff -u -u -r1.43 lex.h
--- src/lex.h   24 Apr 2006 19:13:08 -0000      1.43
+++ src/lex.h   20 Jun 2006 17:57:07 -0000
@@ -122,6 +122,12 @@
   // Return transpose or start a string?
   bool quote_is_transpose;
 
+  // Count of the depth of block comment nesting.
+  int block_comment_nesting_level;
+
+  // Flex start state.
+  int saved_start_state;
+
 private:
 
   lexical_feedback (const lexical_feedback&);
Index: src/lex.l
===================================================================
RCS file: /cvs/octave/src/lex.l,v
retrieving revision 1.236
diff -u -u -r1.236 lex.l
--- src/lex.l   8 Jun 2006 20:37:29 -0000       1.236
+++ src/lex.l   20 Jun 2006 17:57:08 -0000
@@ -25,7 +25,7 @@
 
 %s COMMAND_START
 %s MATRIX_START
-
+%x BLOCK_COMMENT_START
 %x NESTED_FUNCTION_END
 %x NESTED_FUNCTION_BEGIN
 
@@ -234,6 +234,8 @@
 
 static unsigned int Vtoken_count = 0;
 
+static std::string block_comment_text;
+
 // Forward declarations for functions defined at the bottom of this
 // file.
 
@@ -267,15 +269,15 @@
 
 %}
 
-D      [0-9]
-S      [ \t]
+D      ([0-9])
+S      ([ \t])
 NL     ((\n)|(\r)|(\r\n))
 SNL    ({S}|{NL})
 EL     (\.\.\.)
 BS     (\\)
 CONT   ({EL}|{BS})
-Im     [iIjJ]
-CCHAR  [#%]
+Im     ([iIjJ])
+CCHAR  ([#%])
 COMMENT        ({CCHAR}.*{NL})
 SNLCMT ({SNL}|{COMMENT})
 NOT    ((\~)|(\!))
@@ -284,6 +286,10 @@
 IDENT  ([_$a-zA-Z][_$a-zA-Z0-9]*)
 EXPON  ([DdEe][+-]?{D}+)
 NUMBER (({D}+\.?{D}*{EXPON}?)|(\.{D}+{EXPON}?)|(0[xX][0-9a-fA-F]+))
+
+BLOCK_COMMENT_BEGIN ^({S}*{CCHAR}\{{S}*{NL})
+BLOCK_COMMENT_END   ^({S}*{CCHAR}\}{S}*{NL})
+
 %%
 
 <NESTED_FUNCTION_END>. {
@@ -471,6 +477,53 @@
       }
   }
 
+{BLOCK_COMMENT_BEGIN} {
+    block_comment_text = yytext;
+
+    lexer_flags.block_comment_nesting_level++;
+
+    lexer_flags.saved_start_state = YY_START;
+
+    input_line_number++;
+    current_input_column = 0;
+
+    BEGIN (BLOCK_COMMENT_START);
+  }
+
+<BLOCK_COMMENT_START>. {
+    block_comment_text += yytext;
+  }
+
+<BLOCK_COMMENT_START>{NL} {
+    block_comment_text += yytext;
+
+    input_line_number++;
+    current_input_column = 0;
+  }
+
+<BLOCK_COMMENT_START>{BLOCK_COMMENT_BEGIN} {
+    block_comment_text += yytext;
+
+    lexer_flags.block_comment_nesting_level++;
+  }
+
+<BLOCK_COMMENT_START>{BLOCK_COMMENT_END} {
+    block_comment_text += yytext;
+
+    input_line_number++;
+    current_input_column = 0;
+
+    if (--lexer_flags.block_comment_nesting_level == 0)
+      {
+       octave_comment_buffer::append (block_comment_text,
+                                      octave_comment_elt::block);
+
+       block_comment_text = "";
+
+        BEGIN (lexer_flags.saved_start_state);
+      }
+  }
+
 \[{S}* {
     nesting_level.bracket ();
 
@@ -2406,6 +2459,12 @@
 
   // Quote marks strings intially.
   quote_is_transpose = false;
+
+  // Count of the depth of block comment nesting.
+  block_comment_nesting_level = 0;
+
+  // Flex start state.
+  saved_start_state = INITIAL;
 }
 
 bool