Re: Buffer overrun when syntax fragment contains no terminating newline.

[Ben Pfaff]

Sorry about that.  I'll fix it as soon as I can.

On Sun, Sep 30, 2018 at 08:21:48AM +0200, John Darrington wrote:
> This fix seems to be causing test 1075 to fail.
> 
> On Sat, Sep 29, 2018 at 03:47:25PM -0700, Ben Pfaff wrote:
>      On Sat, Sep 29, 2018 at 04:26:28PM +0200, John Darrington wrote:
>      > I've just pushed a change fixing some sporadic crashes in the gui.   
>      > 
>      > The bug (which took a bit of tracking down) turned out to be caused by
>      > a buffer overrun in lexer.c (lex_source_get_).   In particular, we have
>      > the code:
>      > 
>      >  const char *newline = rawmemchr (line, '\n');
>      > 
>      > But the documentation for rawmemchr says that it's unpredictable if
>      > line does not contain a '\n'.
>      > 
>      > So this means our syntax parser can crash if we present it with a
>      > fragment which is not newline terminated.   I wasn't aware that we
>      > had such a limitation.     Does this need to be fixed, or at least
>      > explicitly documented ?
>      
>      Until recently, the lexer and its lower level infrastructure required
>      source files to end in \n.  Because of this limitation, all the code
>      that read source files added a trailing newline if one wasn't already
>      present.  I fixed the limitation in commit e0f9210e814d ("lexer: Add
>      support for embedded \0 bytes and missing trailing new-line.") because
>      it made null bytes hard to handle properly.  At the same time, I removed
>      the code to automatically add a trailing newline, because it was no
>      longer necessary.
>      
>      In my code review, I missed this code that still assumed a trailing
>      newline, and none of the tests caught it for me.  I pushed what I
>      believe to be a fix now; I don't have enough time right at the moment to
>      add some more tests, but I'll try to go back and add them later.
> 
> -- 
> Avoid eavesdropping.  Send strong encrypted email.
> PGP Public key ID: 1024D/2DE827B3 
> fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
> See http://sks-keyservers.net or any PGP keyserver for public key.
>